How to test a Confluence instance for macro errors and attachment link problems

Description

When upgrading Confluence or plugins, how do you know if macros are still working properly? This is a way to test all the pages in a space or set of spaces to see if any rendered pages have macro errors like Unknown macro or macro generated errors. This provides some assurances that you have installed all the macros that are needed and they do not exhibit any obvious errors.

This is done using a Gint test. Each testcase represents a page. A testcase fails if that page has any of the indicated errors, otherwise it is successful.

This is a general approach that can be used for other similar situations by customizing the script. For instance, adding additional failure indicator text or regex patterns.

Requirements

Installation of ACLI
Gint is a Gradle plugin and therefore, needs Gradle. Most projects that use Gradle, use the Gradle Wrapper that supports zero install requirements for running in pipelines. Refer to the Gradle documentation for information.

Running the test

You need a user with authority to render all pages - this usually means a confluence administrator level user
You need a ACLI site configuration for your Confluence instance using the administrator account
Run the following gradle script or customize it to your own needs

macroerrors.gradle

// Assumes acli is on your path and myconfluence is your defined confluence site configuration
// Tested with Gradle wrapper 7.3.3 and 8.5
// ./gradlew -b macroerrors.gradle -Pcli="acli myconfluence"

plugins { id("org.gint.atlassian") version "3.8.5" }

gint.verbose = true
def info = gint.getConfluenceHelper().getServerInfoWithVerify()  // verify server is available, otherwise end test

def space = gint.helper.getParameterValue('space', 'Examples')
def file = 'pageList.txt'

boolean result = gint.helper.runCmd(cmd: info.cli + " -a getPageList --space ${space} -f ${file}")
if (!result) {
    gint.addFailedMessage('getPageList failed!!!')
    gint.stopNow = true
}
def pageList = gint.helper.readFileToList(file)
// gint.helper.log('pageList', pageList)

// Define a task for each page in the list. 
// This makes it easy to run a test for a single page and to report on each page that fails individually.
pageList.each { entry ->
    def name = gint.helper.cleanTaskName(entry)
    gint.taskHelper.add('entry', [
        [action: 'renderPage', name: name,
            file: gint.getOutputFile(name),
            parameters: [
                space: space,
                title: entry,
            ],
            output: [
                failData: [
                    'placeholder/unknown-attachment',       // missing attachment like: src="/wiki/plugins/servlet/confluence/placeholder/unknown-attachment
                    'class="unresolved"',                   // missing link like: <a class="unresolved" href="#">missing-link</a>
                    'The included page could not be found', // include macro problem

                    'Unknown macro:',                       // missing macro
                    '<span class="error">',                 // general error
                    ~/(?:(?:<div class="error)|(?:<div class="[^"]*? aui-message-error))/, // covers both older style and newer style errors
                ],
            ]
        ]
    ])
}

...
> Task :macroerror
    [start] macroerror
      [out] Rendered data written to file: .../macroerror.txt
     [info] Found: The included page could not be found
     [info] Context: data-macro-name="include"><span class="error">Unable to render {include}</span> The included page could not be found.</div></p><p class="enh-settings hidden hidden" style="display: none !important
   [failed] macroerror: Unexpected data 'placeholder/unknown-attachment', 'class="unresolved"', 'The included page could not be found', 'Unknown macro:', '<span class="error">', (?:(?:<div class="error)|(?:<div class="[^"]*? ...
[reproduce] acli myconfluence --action renderPage    --file ".../macroerror.txt"   --space "Examples"  --title "macroerror"   
 [complete] macroerror - 2.185 secs

> Task :macroerror FAILED
...