--- /dev/null
+
+Bacula Regression Suite and CTest
+==================================================================
+
+The Bacula regression scripts have now been modified to use the ctest component
+of cmake. The major gain from this, since Bacula already had a working test
+framework in place, is the ability to have the results of each test submitted
+to a centralized dashboard system. All of the test results are aggregated and
+summarized, where all of the developers can quickly see how the regression
+tests are running.
+
+How to Use CTest
+==================================================================
+
+For more complete documentation on ctest, go to:
+
+http://www.cmake.org/Wiki/CMake#CTest
+
+The first step is to install the cmake package, which includes ctest. If your
+distribution does already package it, you can download it directly from the
+source:
+
+http://www.cmake.org
+
+Next, you must edit your regression config file and add a paramter called
+SITE_NAME to identify the machine running the tests. Ideally, it should
+contain something to identify yourself to whoever is viewing the test results
+as well as something to allow you to identify which machine is running the
+tests, ie
+
+SITE_NAME=kern-bacula-gumbie
+
+Once you have cmake installed, you can perform one of two different kinds of
+runs to submit test results. The most common kind will be Nightly runs. A
+Nightly CTest backup will update the source directory (as defined by the
+BACULA_SOURCE setting in your config file) to the current version, run the
+specified list of tests, and submit all of the results to the server. Note
+that all of the results in a given 24 hour period (starting at 9pm EST) are
+lumped together to appear as a single block, rather than each test showing up
+as a different run.
+
+The simplest way to trigger a nightly run is to use one of the two provided
+scripts. The nightly-all script will run all non root tests, both tape and
+disk based, while the nightly-disk script will run only the disk based tests.
+
+Periodically, however, you may want to submit a single test separately from a
+weekly run. This may be a test of a particular patch you're working on, or
+perhaps a new OS patch. For these one-shot tests, you will want to manually
+run ctest in Experimental mode, something like:
+
+REGRESS_DEBUG=1 ctest -D Experimental -R all-non-root:auto-label-test
+
+The '-D Experimental' option tells ctest to submit the test results as
+Experimental instead of Nightly. We reccomend you use the REGRESS_DEBUG
+environmental variable to ensure that any errors from the test are logged in
+the dashboard (all of the ctest wrapper scripts set it). The '-R <pattern>'
+option gives ctest a regular expression. Any tests with a name as defined in
+DartTestfile.txt that matches the pattern will be run.
+
+Note that you must have run ./scripts/do_sed at least once already in order to
+use Experimental mode.
+
+Updating and Building Within CTest
+==================================================================
+
+Before each Nightly run, ctest will automatically update the BACULA_SOURCE
+directory, and submit these updates along with the test results. Any
+Experimental runs will not.
+
+Before either type of run actually begins running tests, ctest will run the
+script scripts/update-ctest. This script first compares the svn version of
+BUILD_SOURCE with that of the build/ directory. If the two versions differ, or
+if the build/ directory does not exist, it will automatically run 'make setup'
+for you.
+
+Viewing the Dashboard
+==================================================================
+
+You can view the dashboard at:
+
+http://regress.bacula.org:8081
+
+Results will not be visible as soon as they are submitted to the server.
+Processing is currently done every 10 minutes, so you may have to wait up to 15
+minutes or so before your results show up.
+
projects / bacula / bacula / commitdiff