Sync rules

image-20240506-092529.png

Partial sync

Synchronization time interval

Specify the time interval (in seconds) for the App to synchronize data with connected instances. The minimum value is 1 second, and the maximum is 10 seconds.

Updates processing control

Partial Sync mechanism is a background process designed to update the status of tasks in BigPicture boxes, based on changes made by users on the Jira side.

Change of the ‘Updates processing control’ settings affects:

Platform

Sync speed between BigPicture and Jira

Jira performance

Platform

Sync speed between BigPicture and Jira

Jira performance

Jira Cloud

affected

unaffected

Jira Data Center

affected

affected

High priority queue - changes related to tasks in active boxes (recently visited). A high-priority queue is handled quickly to ensure that when you make a change in Jira you see the corresponding changes in the tasks of the just-opened BigPicture box.

Low priority queue - changes related to tasks in boxes that haven’t been recently visited. A low-priority queue ensures that boxes (that are not visited) are updated on ongoing basis (regular 15 minute intervals). Such an interval of syncs reduces the load on Jira (CPU and memory utilization) and mitigates the risk of causing a slowdown. From a user perspective this change is barely noticeable - when a user enters a box it becomes a “recently visited box” and the updates are moved to the high-priority queue for quick handling.

option

description

option

description

Optimized updates processing

High and low priority queue enabled

We suggest choosing the "Optimized updates processing" option for fast updates of frequently used boxes with low risk of high CPU load.

Simple updates processing

One queue for processing all updates

The "Simple updates processing" option does process updates quickly, but could be inefficient and lead to a risk of high CPU load when handling large volumes of data.

With a large number of changes on the Jira side (and a large number of BigPicture boxes in Open or In Progress statuses), there is a risk that with only one queue enabled it will run continuously, in practice never emptying. This can cause a continuous load on CPU threads, affecting the performance of the entire server instance.

In such a situation, it is first recommended to close unused boxes by moving them to a closed or archived status (closed and archived boxes are not updated and the the low-priority queue load is decreased).

Additionally, all boxes with scope based on complex filters may run slower than boxes based on Jira projects - it is recommended to limit the number of filter-based boxes if possible.

If there is still a performance problem after performing the mentioned optimization steps, it is recommended to reduce the number of CPU threads supporting the low-priority queue, or disable it completely. However, if the operation of the low-priority queue does not significantly affect the load on the node, then there is no need to disable it.

Limited updates processing

Only high priority queue enabled

Not recommended - use as a last resort when the ‘optimized’ mode still results in high CPU utilization and you have confirmed the issues are caused by BigPicture partial sync mechanism.

Opting for the "Limited updates processing" will reduce the CPU load, although it may increase the synchronization time for less frequently visited boxes. It can have a significant negative impact on BigPicture user experience and extend the time from entering a box to data being fully up to date.

Adjust database parameters

The above-mentioned mechanisms are controlled by the following parameters set in the BigPicture database.

Active boxes are defined based on the following parameters:

  • box activity threshold

  • active box limit

Parameter

PartialSyncRecentActivityHourThreshold

Parameter

PartialSyncRecentActivityHourThreshold

Description

If a box has been visited within the last X hours, it is considered as a “box visited recently” (changes that affect it will be processed by the high-priority queue). Other boxes will be considered as a “box not visited recently” (changes that affect them will be processed by the low-priority queue).

Values

Default = 12 hours

Min = 1 hour

Max = 144 hours

Parameter

PartialSyncHighPriorityGroupMaxSize

Parameter

PartialSyncHighPriorityGroupMaxSize

Description

Limit of boxes supported by the high-priority queue.

If BigPicture detects that the high-priority queue is to handle too many boxes, then to improve its performance only the last X boxes will be left in the queue (the remaining boxes will be handled by the low-priority queue).

Values

Default = 30 boxes

Min = 10 boxes

Max = 100 boxes

Specify what falls into the high priority queue and adjust the timeout threshold for box synchronization:

Parameter

PartialSyncExecutionTimeMillisThreshold

Parameter

PartialSyncExecutionTimeMillisThreshold

Description

If the box synchronization time exceeds X milliseconds (for the last 2 days) then the box will be handled by the low-priority queue. This parameter is intended to improve the operation of the high-priority queue, by removing changes related to the BigPicture box that is in full synchronization process (so the high-priority queue would
unnecessarily waste resources on synchronizing it).

Values

Default = 2000 millisec.

Min = 500 millisec.

Max = 10000 millisec.

Enables high and low priority queues:

Parameter

PartialSyncPrioritizedQueueEnabled

Parameter

PartialSyncPrioritizedQueueEnabled

Description

"true" enables the division into high-priority and low-priority queues. “false” disables the new mechanism (Partial Sync process works as in previous versions).

Values

Default = true

Min = true

Max = false

Parameter

PartialSyncLowPriorityQueueDisabled

Parameter

PartialSyncLowPriorityQueueDisabled

Description

“false” causes the low-priority queue to be enabled.

“true” causes the low-priority queue to be disabled. In such a situation, boxes considered as “boxes not visited recently” will not have a regularly updated structure. The structure update will be performed when the user enters such a box (this may take some time).

Values

Default = false

Min = true

Max = false

Example of a database entry:

insert into "AO_0456E7_PROPERTY" ("CONVERTER_NAME", "SCOPE_LEVEL1", "TECH_UNIQUENESS", "VALUE") values ('StringConverter', 'PartialSyncPrioritizedQueueEnabled', 'PartialSyncPrioritizedQueueEnabled', 'true'), ('StringConverter', 'PartialSyncLowPriorityQueueDisabled', 'PartialSyncLowPriorityQueueDisabled', 'false'), ('StringConverter', 'PartialSyncExecutionTimeMillisThreshold', 'PartialSyncExecutionTimeMillisThreshold', '2000'), ('StringConverter', 'PartialSyncHighPriorityGroupMaxSize', 'PartialSyncHighPriorityGroupMaxSize', '30'), ('StringConverter', 'PartialSyncRecentActivityHourThreshold', 'PartialSyncRecentActivityHourThreshold', '12');

CPU threads

There are two parameters controlling the number of CPU threads allocated to high-priority and low-priority queues. These parameters are defined in the bigpicture.properties file in the “plugins” directory (located in the shared folder of the selected instance).

Parameter

jiraServerExtTaskQuerySingleJobExecutorMaxThreadPoolSize

Parameter

jiraServerExtTaskQuerySingleJobExecutorMaxThreadPoolSize

Option a)

defines the number of CPU threads for the high-priority queue

(values from 4 to 1, default = 3), when the new queue division is enabled
(PartialSyncPrioritizedQueueEnabled parameter is set to “true”)

Option b)

defines the number of CPU threads for the Partial Sync mechanism (values from 5 to 1, default = 5), when the new queue division is disabled (PartialSyncPrioritizedQueueEnabled parameter is set to “false”)

Parameter

jiraServerExtTaskQueryLowPrioritySingleJobExecutorMaxThreadPoolSize

Parameter

jiraServerExtTaskQueryLowPrioritySingleJobExecutorMaxThreadPoolSize

Option a)

defines the number of CPU threads for the low-priority queue

(values from 4 to 1, default = 2), when the new queue division is enabled
(PartialSyncPrioritizedQueueEnabled parameter is set to “true”)

Option b)

ignored when the new queue division is disabled

(PartialSyncPrioritizedQueueEnabled parameter is set to “false”)

Additional note - the sum of threads divided into queues cannot exceed 5 threads, otherwise the default values will be used (3 CPU threads for the high-priority queue and 2 CPU threads for the low-priority queue). Examples of correct configurations:

jiraServerExtTaskQuerySingleJobExecutorMaxThreadPoolSize = 4
jiraServerExtTaskQueryLowPrioritySingleJobExecutorMaxThreadPoolSize = 1

or

jiraServerExtTaskQuerySingleJobExecutorMaxThreadPoolSize = 2
jiraServerExtTaskQueryLowPrioritySingleJobExecutorMaxThreadPoolSize = 2

If the low-priority queue is disabled due to PartialSyncLowPriorityQueueDisabled set to “true”, CPU threads allocated to this queue will not be used (i.e. only 3 CPU threads allocated to the high-priority queue will be used).

Additional box sync

Scheduled synchronization

Jira does not always notify the BigPicture App about changes (for example, no updates are triggered when a project is removed) - the scheduled synchronization mechanism assures that the scope of your work is always up to date. 

Task limit

Maximum number of tasks in a box

Set the maximum number of tasks that a single Box can hold to limit BigPicture's performance impact on your Jira instance. The default value is 100,000 tasks, and there is no maximum limit.

For more information, see our BigPicture Sizing Guide.

The performance can also be improved by increasing the synchronization time interval.

Task grouping limit

Set a limit on how many tasks a box can handle to allow grouping. The maximum number is set to 50000 tasks.