Exporting and Importing App Data

As a Dataplane administrator you can export all Dataplane app data to a human-readable XML file on the Jira server, and then import that data into a new Jira instance.

While the export capability is intended primarily for migrating Dataplane to a new Jira instance, it can also be used for examining Dataplane data for app management or debug purposes, sharing report or other configuration information with Arsenale Support, or even making surgical changes to Dataplane report or user data on the existing Jira instance without having to run SQL commands against the Jira database.

It is highly recommended that you read this page in its entirety before experimenting with Dataplane data imports.

Importing Dataplane data into a Jira instance can get rather complicated when remapping object IDs and keys is necessary.

Enabling Exporting and Importing

For security purposes the export and import capabilities are disabled by default and must be enabled via the following JVM properties. These properties are typically set in the Jira setenv.sh file.

To enable data exports:

-Dcom.arsenalesystems.dataplane.enableDataExport=true

To enable data imports:

-Dcom.arsenalesystems.dataplane.enableDataImport=true

Access and Authentication

You must be a Jira administrator authenticated through websudo in order to export or import Dataplane app data.

An easy way to enable an administrator websudo session is to visit the Dataplane Reports > Administration page, which will prompt for authentication.

Exporting Data

Export Basics

1) First make sure you are authenticated as a Jira administrator through websudo.

2) To initiate a full export of Dataplane app data, in your web browser, request the web page at the following URL:

http(s)://<JIRA_ROOT_DOMAIN>/rest/dataplane/latest/admin/exportData

For example, if your JIRA root/domain is "jira.mydomain.com/staging" then request page "http(s)://jira.mydomain.com/staging/rest/dataplane/latest/admin/exportData".

3) Upon successful export, your browser will be redirected to a simple webpage with a JSON object confirming the operation completed successfully:

4) On the server where you performed the data export, go to the "<JIRA_DATA_DIR>/export" directory. You will see a new XML file in that directory, with the following name scheme, containing the entire exported app data for Dataplane:

arsenale-dataplane-data-<current date/time>.xml

Export File Format

The Dataplane data export XML file contains the following app data:

all saved reports for each user, including their ownership, configurations and sharing permissions
all saved reports collections for each user (e.g. "My Reports", "Favorites")
all report subscriptions

The export file is divided into a few key sections:

<data>
    <exportVersion>2</exportVersion>
    <reportVersion>8</reportVersion>
    <appVersion>2.4-dev</appVersion>
    <info>
        <reports>121</reports>
        <collections>4</collections>
        <schedules>3</schedules>
        <references>
            <users>5</users>
            <groups>1</groups>
            <customfields>13</customfields>
            <projects>6</projects>
            <filters>2</filters>
            <boards>1</boards>
        </references>
    </info>
    <reports>
	    ...
    </reports>
    <collections>
        ....
    </collections>
    <schedules>
        ...
    </schedules>
    <references>
        ...
    </references>
</data>

The <info> block is for information purposes only. None of the data in this block is validated on import so, while there should be no need, if you decide to edit the XML file to add or remove items, you do not need to be concerned about updating these numbers to match.

The <reports>, <collections> and <schedules> blocks contain the actual Dataplane app data.

The <references> block contains a list of all the internal Jira objects—users, groups, projects, custom fields, filters and boards—that the Dataplane app data refer to.

Items in the <references> block are NOT themselves imported by the Dataplane import operation.

They are only used to resolve references within the imported Dataplane app data.

Importing Data

Import Basics

1) On the Jira server into which you will be importing Dataplane app data, copy an exported Dataplane app data XML file to directory "<JIRA_DATA_DIR>/import".

When importing, Dataplane automatically chooses the first "arsenale-dataplane-data-xxxxx.xml" file it locates in the import directory. So be sure you only place one Dataplane export XML file in this directory at a time so you know which is being used for the import.

2) Go to the Dataplane Reports > Administration > Support page and enable Dataplane debug logging.

3) In your web browser, to check that the basic import functionality is available, request the web page at the following URL:

http(s)://<JIRA_ROOT_DOMAIN>/rest/dataplane/latest/admin/importData

For example, if your JIRA root/domain is "jira.mydomain.com/staging" then request page "http(s)://jira.mydomain.com/staging/rest/dataplane/latest/admin/importData".

This will initiate a PREVIEW of an import just to test that the basic import functionality is available.

No data is actually imported yet, so it's safe to do this.

4) Upon successful import preview, your browser will be redirected to a simple webpage with a JSON object confirming the operation completed successfully:

Reviewing Import Results

All status and other debug output for app data imports is piped to the main Jira log file. This is usually found in the following location:

<JIRA_INSTALL_DIR>/logs/catalina.out

The bottom of the import operation log output contains useful stats on success and issues encountered:

Import Complete! [mode=IMPORT (items with import="true")]
    Reports (121): processed=2 (ok=2 errors=0 duplicates=0) ignored=119 imported=2
    Collections (4): processed=1 (ok=0 errors=1) ignored=3 imported=0
    Schedules (3): processed=1 (ok=0 errors=1 duplicates=0) ignored=2 imported=0

ERROR messages indicate an error preventing a specific item from being imported, or a general application error
WARN messages indicate a potential problem, but it may or may not block items from being imported

Import Options

The following REST options can be added to the "importData" URL to affect import functionality:

Property	Options	Default	Description
import	true/false	false	false: the import operation runs in PREVIEW mode only. All validations are performed on the data to be imported, however no actual data is imported. true: the import operation runs in IMPORT mode. Validations are performed on the data to be imported, and the data is then imported into Jira.
all	true/false	false	false: import only those "<report>", "<collection>" and "<schedule>" items in the XML file that have an attribute 'import="true"' true: import all "<report>", "<collection>" and "<schedule>" items in the XML file, except for those that have an attribute 'import="false"'
overwrite	true/false	false	false: when importing a "<report>" or "<schedule>" item, if find one with the same ID in the importing Jira instance, then do not import it again true: when importing a "<report>" or "<schedule>" item, if find one with the same ID in the importing Jira instance, replace it
own	true/false	false	false: imported "<report>", "<collection>" and "<schedule>" items retain their specified owners true: ownership on all imported "<report>", "<collection>" and "<schedule>" items is overridden and assigned to the current user doing the import. This can be useful for initial import testing, for handling items that require special tweaking after import, or for reassigning reports owned by users no longer with the organization.

The import REST call is designed such that the default value of all properties results in the least possible impact on existing data in the instance, and the scope of what actions are taken during import increases only as property values are explicitly set to "true".

For instance, an actual data import is only performed if you include 'import=true' as a REST URL parameter.

REST Examples

Example #1

https://jira.mydomain.com/staging/rest/dataplane/latest/admin/importData

import=false (default): run in PREVIEW mode only. No data is actually imported.
all=false (default): only consider for import those XML items where the user has explicitly added attribute 'import="true"'
overwrite=false (default): if this wasn't only a PREVIEW, items for import that already exist in the instance would be skipped

Example #2

https://jira.mydomain.com/staging/rest/dataplane/latest/admin/importData?all=true&overwrite=true

import=false (default): run in PREVIEW mode only. No data is actually imported.
all=true: consider for import all XML items, except for those where the user has explicitly added attribute 'import="false"'
overwrite=true: if this wasn't only a PREVIEW, items for import that already exist in the instance would be replaced

Example #3

https://jira.mydomain.com/staging/rest/dataplane/latest/admin/importData?import=true

import=true: data will be imported
all=false (default): only consider for import those XML items where the user has explicitly added attribute 'import="true"'
overwrite=false (default): items for import that already exist in the instance are skipped

Example #4

https://jira.mydomain.com/staging/rest/dataplane/latest/admin/importData?all=true&overwrite=true&import=true

import=true: data will be imported
all=true: consider for import all XML items, except for those where the user has explicitly added attribute 'import="false"'
overwrite=true: items for import that already exist in the instance are replaced

Import Strategy

The import operation validates all imported Dataplane app data before any import is performed.

If there is a problem importing a particular report, report collection or report subscription schedule, errors are output to the Jira log, that item is skipped over and the import operation continues with the next item to import.

Here are some useful importing tips:

Tips on Importing

Start by running in PREVIEW mode with "all=true".
- The PREVIEW mode does everything a regular data import would do, including all data validation, except no data is actually added to the importing Jira instance.
- Using the "all=true" REST property in your URL request will process all items in the XML file and give you a summary of issues to address across the full import.
When trying to resolve issues with a specific import item, use the "all=false" REST property and set 'import="true"' for that one item in the XML file.
- Whether running in PREVIEW mode or in actual IMPORT mode, this allows you to surgically work on one import item (or a small handful) at a time.
If you make a mistake importing a specific item, redo the import on that item using the "all=false&overwrite=true" REST properties and set 'import="true"' for that one item in the XML file.
To exclude an item from import, set its attribute to 'import="false"' rather than removing it from the XML file.
Pay special attention to the "Checking references..." section of the log output for any ID, key value or data type issues you might need to address.

Customizing the Import

The Dataplane data export file can be manually edited in a number of different ways in order to customize the data import operation.

Manual edits you make to the export XML file will not be reflected in the export file of any subsequent Dataplane export.

You will likely want to create a script of your own in which you itemize all your Dataplane export XML file customizations, so you may easily re-apply those changes to future export files.

Including and Excluding Specific Items

By default (or when REST property 'all=false'), the import operation will only validate and look to import items in the XML file that have attribute 'import="true"'. All other items in the file will be ignored by the import operation.

This applies to all "<report>", "<collection>" and "<schedule>" items.

For example, if REST property 'all=false':

*** Will be imported ***
<report import="true">
    <id>7b778342-4dd8-4dd4-a7a9-0fe5b693bc9c</id>
    <key>reportKey.historicalSnapshotReport</key>
    <name>Z Denali Backlog</name>
	...
</report>

*** Will NOT be imported ***
<report>
    <id>7b778342-4dd8-4dd4-a7a9-0fe5b693bc9c</id>
    <key>reportKey.historicalSnapshotReport</key>
    <name>Z Denali Backlog</name>
	...
</report>

*** Will NOT be imported ***
<schedule import="false">
    <owner>david</owner>
    <reportId>cfd4ed7e-9797-40cf-83b9-392b13acd5f3</reportId>
    <frequency>weekly</frequency>
    <time>9:00</time>
	...
</schedule>

If REST property 'all=true', then the import operation will validate and look to import all "<report>", "<collection>" and "<schedule>" items in the XML file, except those that have attribute 'import="false"'.

For example, if REST property 'all=true':

*** Will be imported ***
<report import="true">
    <id>7b778342-4dd8-4dd4-a7a9-0fe5b693bc9c</id>
    <key>reportKey.historicalSnapshotReport</key>
    <name>Z Denali Backlog</name>
	...
</report>

*** Will be imported ***
<report>
    <id>7b778342-4dd8-4dd4-a7a9-0fe5b693bc9c</id>
    <key>reportKey.historicalSnapshotReport</key>
    <name>Z Denali Backlog</name>
	...
</report>

*** Will NOT be imported ***
<schedule import="false">
    <owner>jeff</owner>
    <reportId>cfd4ed7e-9797-40cf-83b9-392b13acd5f3</reportId>
    <frequency>weekly</frequency>
    <time>9:00</time>
	...
</schedule>

Remapping IDs and Keys

It is likely that, in moving data from one Jira instance to another, various custom fields, saved filters, users, etc in the exporting Jira instance will have different ID or key values in the importing Jira instance. By modifying items in the "<references>" section of the XML file, you can remap IDs and keys to match the corresponding items in your importing instance.

To remap an item's identifier in the XML file, change the value of its 'importXXX="YYYY"' attribute.

For example:

*** Dataplane references to filter id=10032 will be changed to filter id=10743 on import ***
<filter importId="10743">
    <id>10032</id>
    <name>Assigned to Jeff</name>
    <owner>jeff</owner>
</filter>
 
*** Dataplane references to filter id=10032 will NOT be changed on import ***
<filter importId="10032">
    <id>10032</id>
    <name>Assigned to Greg</name>
    <owner>greg</owner>
</filter>
 
*** Dataplane references to custom field id=10715 will be changed to custom field id=11525 on import ***
<customfield importId="11525">
    <id>10715</id>
    <name>Payment Date</name>
    <type>com.atlassian.jira.plugin.system.customfieldtypes:datepicker</type>
</customfield>
 
*** Dataplane references to user 'jeff' will be changed to user 'greg' on import ***
<user importName="greg">
    <name>jeff</name>
    <key>jeff</key>
    <email>jeffrey@mycompany.com</email>
    <displayName>Jeffrey Smith</displayName>
</user>

Remapping Limitations

There are some limitations to Dataplane's remapping capability during data import that are important to understand:

The import operation does NOT inspect or transform any JQL statements.
- Reports where the user has provided a custom JQL statement for the Search option ((instead of dropdown-selected projects/filters/boards) will need to be inspected manually for any project/filter/field/status/user/sprint/etc references that might need to be remapped. On import, the JQL statement is imported verbatim. The import operation will output a WARN log message whenever a report is found with a JQL statement, to alert you to the need for manual inspection. You can make any necessary changes to the JQL in the appropriate XML file "<report>" items before import, or you can make a change to the report's JQL after import using the regular Dataplane UI.
The import operation does NOT inspect or transform any custom issue Statuses.
- Some reports allow the user to select from various issue Statuses (e.g. "Open", "Closed"). If you have any custom Statuses in your exporting Jira instance, be sure those same Statuses exist in the importing Jira instance before doing any importing. Otherwise you may encounter problems after import, with reports referencing issue Statuses that aren't present in the instance.

Page Contents