How to test a Confluence instance for macro errors

Description

When upgrading Confluence or plugins, how do you know if macros are still working properly? This is a way to test all the pages in a space or set of spaces to see if any rendered pages have macro errors like Unknown macro or macro generated errors. This provides some assurances that you have installed all the macros that are needed and they do not exhibit any obvious errors.

This is done using a GINT test. Each testcase represents a page. A testcase fails if that page has any of the indicated errors, otherwise it is successful.

This is a general approach that can be used for other similar situations by customizing the script. For instance, adding additional failure indicator text or regex patterns (in groovy that is something like: ~/The page.*does not exist/) or modifying the sql selection criteria to include or exclude content.

Requirements

1. Installation of Confluence Command Line Interface (CLI) or Appfire Command Line Interface (CLI) release 2.2.0 or above
2. GINT, specifically Dependencies #1-3.
3. Downloaded copy of macroError.gant

Running the test

1. You need a user with authority to render all pages - this usually means a confluence administrator level user
2. You need read access to the Confluence database
3. You need a database specific JDBC driver in your client classpath
4. See the comments in macroError.gant for parameters

gant -f macroError.gant -Dverbose -Ddatabase=confluence-3.5.13 -Dcli="/sde/atlassian-cli-2.2.0/atlassian confluence-3.5.13" -Dspace=zconfluencecli

...
     [info] Successful testcases  . . . . . . . . . . : 19    <<< TEST SUCCESSFUL
     [info] Total testcases . . . . . . . . . . . . . : 19
     [info] Elapsed run time  . . . . . . . . . . . . : 5.844 secs

= = = = = =   macroError completed at Thu Oct 20 21:30:08 CDT 2011 = = = = = =

BUILD SUCCESSFUL
Total time: 7.04 seconds

...
   [failed] experiment:checklist: Unexpected data [Unknown macro:, <span class="error">] found in file target/output/macroerror/experiment:checklist.txt
...
     [info] Failed testcases  . . . . . . . . . . . . : 10    <<< TEST FAILED
     [info] Successful testcases  . . . . . . . . . . : 144
     [info] Total testcases . . . . . . . . . . . . . : 154
     [info] Elapsed run time  . . . . . . . . . . . . : 134.534 secs (2.2 mins)

= = = = = =   macroError completed at Thu Oct 20 22:05:42 CDT 2011 = = = = = =
: Test failed.

BUILD FAILED
Total time: 2 minutes 15.85 second