X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=regress%2FREADME.ctest;h=87361f58b40a2e3e5a07a2d50a274ccdbcf52d2a;hb=7a31fe9348a77c27f05cabcd5d74ea775d474789;hp=9b8986ad8ef0ce8e5b3dddb703862fc1a0b2c4e3;hpb=d02ddf707df81e992c68942800cf37978d030c18;p=bacula%2Fbacula diff --git a/regress/README.ctest b/regress/README.ctest index 9b8986ad8e..87361f58b4 100644 --- a/regress/README.ctest +++ b/regress/README.ctest @@ -2,12 +2,12 @@ Bacula Regression Suite and CTest ================================================================== -The Bacula regression scripts have now been modified to use the ctest component -of cmake. The major gain from this, since Bacula already had a working test -framework in place, is the ability to have the results of each test submitted -to a centralized dashboard system. All of the test results are aggregated and -summarized, where all of the developers can quickly see how the regression -tests are running. +Thanks to Frank Sweetser, the Bacula regression scripts have now been modified +to use the ctest component of cmake. The major gain from this, since Bacula +already had a working test framework in place, is the ability to have the +results of each test submitted to a centralized dashboard system. All of the +test results are aggregated and summarized, where all of the developers can +quickly see how the regression tests are running. How to Use CTest ================================================================== @@ -42,6 +42,26 @@ as a different run. The simplest way to trigger a nightly run is to use one of the two provided scripts. The nightly-all script will run all non root tests, both tape and disk based, while the nightly-disk script will run only the disk based tests. +So, you can choose between the following scripts: + + cd + ./nightly-all # does disk and tape testing + ./nightly-disk # disk only tests + + ./experimental-all # experimental disk and tape testing + ./experimental-disk # experimental disk testing + +If you are a developer and you have modified your local SVN repository, you +should be running the experimental tests -- they are designed for developers. +If you do modify your local repository and commit it, then run a nightly +test, the local repository may be reverted to a prior version so that the +nightly tests all have a consistent cutoff time. + +If you are just doing testing on a nightly basis (no development in your +source repository), then please use the nightly tests. + +All the old scripts (./do_all, do_file, all-non-root-tests, ...) manually +run the tests outside of ctest. Periodically, however, you may want to submit a single test separately from a weekly run. This may be a test of a particular patch you're working on, or @@ -165,3 +185,30 @@ BuildName parameter was misnamed, and intended to be treated more as a build platform name, rather than the name of the build being tested. Rather than create a hook to tag the version being tested, everyone as far as I can tell just seems to rely on the timestamp of the test. +====== + +NOTE !!!!!!!!! ctest can actually back out changes that have been made to +your local source repository. As a consequence, it is probably better not to +use a directory in which you are developing code for Nightly tests. Seee the +below explanation given by Frank Sweetser. + +When a Nightly run is done, the timestamp is set to the last occurring +instance of the time defined by the NightlyStartTime parameter. The piece +that I missed is that, in addition to using that timestamp for reporting to +the dashboard, the update stage also uses that point in time to determine +exactly which version of the repository to check out. + +So if you make commit changes at 10PM EST, and then run a Nightly test run, +the NightlyStartTime of 9PM EST will back out those changes in the local +repository. Any subsequent runs that are started at 9PM EST the following day +or later will include them. This implies to me that NightlyStartTime should +be set such that you don't expect any developers to commit any changes in +between NightlyStartTime and the time at which the ctest run actually starts. + +The alternative is to make use of the Experimental track. While it normally +just uses the local source tree as is, you can manually have it update: + +ctest -D ExperimentalUpdate + +Unlike Nightly, this will update to whatever the latest version of the +repository is.