Kern;;;30 October 2011 at 14:50 CET We have released Bacula Version 5.2.1 to Source Forge Release Notes for Bacula 5.2.1 Bacula code: Total files = 1,110 Total lines = 231,439 (Using SLOCCount) General: -------- This 5.2.1 version is a major release since the last version 5.0.3. It includes well over 299,387 lines of changes made by 20 contributors, lots of bug fixes (see below), 1,851 software commits,and a number of important new features: Version 5.2.0 ------------- Version 5.2.0 was short lived, because Marco (thanks) found a number of last minute bugs. So the first official release of 5.2 is 5.2.1. Compatibility: -------------- As always, both the Director and Storage daemon must be upgraded at the same time. Older 5.0.x and 3.0.x File Daemons are compatible with the 5.2.1 Director and Storage daemons. There should be no need to upgrade older File Daemons. Main New Features: ------------------ - LZO Compression - New Tray Monitor - Purge Migration Job - Changes in Bvfs (Bacula Virtual FileSystem) - Changes in the Pruning Algorithm - Ability to Verify any specified Job - Additions to RunScript variables - Additions to the Plugin API - ACL enhancements - XATTR enhancements - Class Based Database Backend Drivers - Hash List Enhancements Some of the above are described below; all the above are described in more detail in the New Features chapter of the manual. Closed Bugs since 5.0.3: --------------------- 1389 1444 1448 1466 1467 1468 1476 1481 1486 1488 1493 1494 1497 1499 1501 1502 1504 1509 1511 1513 1516 1524 1526 1527 1532 1536 1538 1541 1542 1549 1551 1553 1554 1558 1559 1560 1564 1567 1568 1569 1571 1574 1577 1581 1582 1584 1587 1594 1595 1600 1601 1602 1603 1604 1606 1608 1610 1612 1623 1624 1633 1643 1648 1655 1661 1664 1666 1669 1672 1675 1684 1685 1695 1696 1699 1700 1703 1735 1741 1749 1751 1761 1762 1764 1768 1770 1771 2710 ================== Warning !!!!!! ========================== The new pruning algorithm will not allow pruning of jobs that are essential to restore since the last Full backup. This is, of course, correct and desirable, but it is different from older Bacula versions that could inappropriately prune jobs. Pruning of volumes is not affected. Building Bat: ------------- To build Bat, you need Qt4 version 4.6.2. If you build with any other version it is likely not to work. If you do not have version 4.6.2, on your system, you can download the Bacula depkgs_qt version 16Dec10. Packaging the SQL backend: -------------------------- The main Bacula Director code is independent of the SQL backend in this version. This means that the Bacula Director can be packaged by itself, then each of the different SQL backends supported can be packaged separately. It is possible to build all the DB backends at the same time. - configure can be run with multiple database configure options. --with-sqlite3 --with-mysql --with-postgresql Order of testing for database is - postgresql - mysql - sqlite3 Each configured backend generates a libbaccats--.so A dummy catalog library is created named libbaccats-version.so At configure time the first detected backend is used as the so called default backend and at install time the dummy libbaccats-.so is replaced with the default backend type. If you configure all 3 backends you get 3 backend libraries and the postgresql gets installed as the default. When you want to switch the default you can copy one of the 3 backend libraries over the libbaccats-.so e.g. cp libbaccats-postgresql-.so libbaccats-.so And update the default backend in the following files: create_bacula_database drop_bacula_database drop_bacula_tables grant_bacula_privileges make_bacula_tables make_catalog_backup update_bacula_tables New Catalog format in version 5.2.0 and greater ----------------------------------------------- This release of Bacula uses a new catalog format. We provide a set of scripts that permit to convert a 5.0.x (version 12) catalog to 5.2.1 (version 14). The Bacula project rpm spec files have been modified to detect the database change and do the upgrade for you. We strongly recommend that you save a copy of your existing database before upgrading using rpms. The simplest way to upgrade the database assuming you are building from source (i.e. manually): 1. Stop any current version of Bacula from running. 2. Save a copy of your existing database. 3. Configure, build, and install the 5.2.1 version of Bacula Note the new ./configure options in the manuals see the link at the bottom of this file. 4. cd 5. ./update_bacula_tables 6. If you have multiple catalogs, edit the update_bacula_tables script and put the name of another database, and then execute the script again. Repeat this until all Bacula catalogs have been updated. 7. Start the new Bacula. If everything worked, it should not print any error messages. If you start from scratch, you don't need to run the update_bacula_tables script because the create_bacula_tables script automatically creates the new table format. However, if you are using a version of Bacula older than 5.0.0 (e.g. 3.0.3) then you need to run the update_bacula_tables script that will be found in the /src/cats directory after you run the ./configure command. As mentioned above, before running this script, please backup your catalog database, and when you do running it, be sure shutdown Bacula and be aware that running the script can take some time depending on your database size. ================================================================ Changes: Speed and Memory Usage Enhancements: - Implement LZO compression algorithm (need lzo2 headers to activate this option) - New Pruning algoritm -- safer and faster - Use of PostgreSQL cursor when doing large queries, should reduce the memory usage during restore and accurate backups. - limit command in some list commands - Bacula Virtual File System for fast catalog access - Faster crc32 algorithm Security Enhancements: - Additional security against injection of illegal characters - Fix possible fnmatch problem Features Enhancements: - Improvements in ACL and xattr handling on additional platforms: AIX, Darwind, FreeBSD, HPUX, IRIX, Solaris, Tru64 - Tray monitor for Windows - New SQL backend and ./configure that permits building multiple catalog backends in one build, and has the main Bacula core code totally independent of the SQL backend, which should simplify packaging (only the libbacsql.so/dll needs to be changed to switch from one database backend to another (e.g. MySQL to PostgreSQL). - Indent lsmark during restore - Disable batch insert config item - Director selection option in bconsole - restorejob command in bconsole - Many additional features for bat such as "brestore" panel - The old bat version browser has been turned off since it does not work correctly and the brestore panel provides the same functionality - Improved hash table for hard links - Additional script variable editing such as %b (JobBytes), %F (JobFiles) and %h (Client Address) Other Enhancements: - Includes better handling for Windows repares points, mount points, and junction points. - New database format - Many new plugin API features - Improved memory handling for restores - Significant stability enhancements to bat - xattr/ACL restore failure are now printed as Warning - A few path name length limits in Windows have been removed. =========================================================================== ;;;; Kern;;;06 August 2010 at 19:15 CET We have released Bacula Version 5.0.3 to Source Forge This 5.0.3 version is significant bug fix update to version 5.0.2. It includes new code and some new features. There is no database change since prior versions of 5.0. Note, if you are running 5.0.0, read the 5.0.1 and 5.0.2 release notes below in the 5.0.1 release section. Important!!! Compatibility: -------------- As always, both the Director and Storage daemon must be upgraded at the same time. Older 5.0.x and 3.0.x File Daemons are compatible with the 5.0.3 Director and Storage daemons. There should be no need to upgrade older File Daemons. Changes and New Features: - If a VSS (Windows) snapshot fails, the whole Job is failed, previously it printed a warning message and continued. - New editing codes %b = Job Bytes and %f = Job Files for runscripts - Any non-Bacula project plugins (i.e. third party plugins) must be upgraded to specify AGPLv3 in place of the GPLv2 used previously for the PLUGIN_LICENSE. - The bat Media page text columns can be sorted by clicking on the column. - If possible, the SD will automatically update the catalog Volume size if it does not match the Volume size. The mismatch between the Volume sizes typically occurs when the FD has a comm line drop or the FD crashes. Bug fixes 1538 1554 1564 1567 1568 1569 1571 1577 1581 1582 1587 1594 1595 1606 1610 - Change license from GPLv2 to AGPLv3 - Make Win32 no filesystem change more explicit - Backport Branch-4.0 release to Branch-5.0 - Massive backport from Branch-5.1 to Branch-5.0 - Fix postgresql catalog creation when version is not on the first line - Fix bug #1610 handle empty xattr values on Linux and xBSD. - Garbage collect memory pool at end of job and before sm_dump - Add xattr seg fault protection suggested by Marco for bug #1610 - Add archlinux to os.m4 - More fixes for 32/64 bit problems in smartall - Fix 32/64 bit problem in smartalloc dump routine - Fix bad copy/paste in commit c88dccb88 prably a seg fault - Add .dump and .exit commands for daemons - Make SD automatically fix the Volume size in the Catalog when out of sync - Use Pmsg in smartall.c rather than printf for tracing Windows - Up maximum block size to 20M - Fix for TLS bugs #1568 and #1599 - Improve comm line error handling for TLS, may fix bug #1568 and #1599 - Fix problem when sending Cancel event to plugin - Convert all Jmsg on the watchdog thread to Qmsg - Fix SD crash due to mismatched lock/unlock in error condition - Fix bug #1587 if you have clients with different catalogs configured, both catalogs contains all clients after a reload or restart of the dir - Fix bug #1577 During migration: ERROR in block.c:950 Failed ASSERT: dev->is_open() by moving and commenting out the ASSERT() - Skip HB kill in FD if HB thread terminated - Fix crash from unequal volume_lock/unlock calls - New prunning algorithm -- should be more accurate - Save any dumps during regression to dumps directory - Fix crash from rw_lock/unlock miss match - Attempt to fix duplicate job kill seg fault - Replace pthread_kill by my_thread_send_signal() in jcr.c - fix #1595 about batch mode detection problem for postgresql with non standard install dir - Fix #1594 about prune copy jobs - Apply literal string fix in printf submitted by Luca Berra - Fix bug #1582 Restore from multiple storage daemons breaks subsequent backups - Fix segfault on "cancel jobid=" command - Fix bug #1554 Windows installer doesn't honor /S flag - Remove pool zap code that breaks 2drive-concurrent-test - Fix #1571 and install libraries with 755 perms - Fix bug #1581 ASSERTs in src/dird/jobq.c can never trigger - Fix problem with BaseJob and Accurate FileSet options on client side - Fix for bug #1569 deadlock/crash in Dir - Fix segfault when loading Plugins - Fix #1567 about display of long volume names truncated during restore - Added new bacula-libs package. - Fix bug #1538 Start Scripts return incorrect value - Fix bug #1564 init scripts not LSB compliant See ChangeLog for more details. ;;;; Kern;;;01 August 2010 at 17:23 CET Welcome to our new bacula.org server This is our new bacula.org server, graciously provided by UKFast. Our previous server, also hosted by UKFast, was getting to be a bit old, so they furnished us a new one, and we are very pleased. ;;;; Kern;;;28 April 2010 at 19:15 CET Eric has released Bacula Version 5.0.2 to Source Forge This 5.0.2 version is primarily a important bug fix update to version 5.0.1. Note, if you are running 5.0.0, read the 5.0.1 release notes below. Important!!! Compatibility: -------------- As always, both the Director and Storage daemon must be upgraded at the same time. Older 5.0.x and 3.0.x File Daemons are compatible with the 5.0.2 Director and Storage daemons. There should be no need to upgrade older File Daemons. Changes since 5.0.1 ------------------- Bug fixes 1502 1511 1517 1524 1527 1532 1536 1541 1549 1551 1553 1559 1560 - Probable fix for SD crash bug #1553 - Fix #1559 problem when restoring pruned jobs with a regexp - Fix for bug #1560 bcopy cannot find Volume - Fix cancel crash bug #1551 - Check if sql backend is thread-safe - Correct Pool display in SD status. Fixes bug #1541 - Fix cancel crash reported by Stephen Thompson - Rewind on close to fix #1549 - Remove closelog() in bpipe fixes bug #1536 - Fix #1517 about missing Base level in .level command - Replace ASSERT in block.c with fail Job - Fix database locking calling db_lock and returning from function without calling db_unlock. - Add missing db_unlock to bvfs_update_cache. - Fix #1532 about permission on binaries - Fix #1527 about deadlock during migration - Another fix for OpenSSLv1 - Add -lrt to Solaris links - Fix tls.c for OpenSSLv1 - Fix #1511 when trying to insert more than 50.000 directories in bvfs - Fix plugin load not to stop if one plugin bad -- pointed out by James - Remove --without-qwt from configure statement. - Second correct fix to bug #1524 verify fails after adding or removing files - Fix bug #1524 verify fails after adding or removing files - Apply fix suggested by Andreas in bug #1502 for mediaview column sort problem - Fix OpenSSL 1.x problem in crypto.c on Fedora 12 - Display AllowCompress warning message only if compression used in FileSet ;;;; Kern;;;25 February 2010 at 15:35 Bacula Version 5.0.1 has been released to Source Forge Bacula version 5.0.1 source code and Windows (32/64 bit) binaries have been released to Source Forge (thanks Eric). This is a major bug fix release including a few directives that have been rewritten, one new directive, and some different directive behavior (see the release notes below). As is usual for a patch release (last digit changes by one), this version is compatible with the 5.0.0 database and with prior clients. However, you *must* upgrade all components that are on any one machine (that is you must upgrade your Director, Storage daemon, and File daemon at the same time, if they reside on the same machine). Note, Bacula does not normally uninstall previous versions, and we have changed the shared object naming convention, so you might want to first save your configuration files then uninstall the old Bacula using the old Bacula uninstall prior to installing the new one. If you do not, it should not be serious, but you may be left with some older Bacula shared objects that are not used and hence wasting a small amount of disk space. If you are upgrading from version 3.0.x or prior, please see the full release notes as you must do a database upgrade. When updating from 5.0.0 to this release there is no database upgrade needed. Scott has made a number of changes and improvements in the rpm packaging over the past few weeks since version 5.0.0 was released, so he will probably be releasing the 5.0.1 rpms quite soon. Thanks for using Bacula :-) Best regards, Kern ============= Performance Note ================== Some of you have encountered performance problems with your database (mainly with MySQL) with Bacula version 5.0.0. This is mainly because we've changed the SQL query used for restore, accurate jobs and base jobs. We have extensively tested this change, and though it should be a little bit slower than the previous versions, on a well configured database it should run extremely well. We strongly recommend to avoid the temptation to add new indexes. In general, these will cause very significant performance problems in other areas. A better approch is to carefully check that all your MySQL memory configuation parameters are are suitable for the size of your installation. If you backup millions of files, you need to adapt the database memory configuration parameters concerning sorting, joining and global memory. By default, sort and join parameters are very small (sometimes 8Kb), and having sufficient memory specified by those parameters is extremely important to run fast. If adjusting your MySQL memory configuration values does not solve your problem, you can also consider switching to PostgreSQL, which performs much better with Bacula on big installations (many millions of files per Job). However for large installations, you will also need to adjust the default PostgreSQL memory configuration parameters. ========================================== Release Notes for Bacula 5.0.1 Bacula code: Total files = 1,081 Total lines = 217,272 (Using SLOCCount) !!!!!!!!!!!!!!!!!!!!! NOTE !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! The Allow Duplicate Jobs directive has been significantly reworked, and the default value has changed. See below. Truncate On Purge has been totally rewritten. See the new features section of the manual. When Volume Poll Interval is set in the SD DEVICE configuration, (default 5 mins), after a certain number of polling tries (approx 10) polling will stop and the operator will be asked to resolve the problem. Previously there was no limit, and an error message could be produced at each poll attempt. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Changes since 5.0.0 ------------------- - We believe that we have resolved most of the problems concerning canceled or failed jobs being "stuck" in the Director. There is one outstanding problem in the SD when canceling jobs that we will fix in the next major release. If you see jobs that seem to be stuck, in general issuing a cancel command in bconsole should now make them go away. Directives: - The default for "Allow Duplicate Jobs" has been changed from no to yes. If you use this directive, please check your conf file, and note the next two items !!!!!!!!!!!!!!!!!!! - AllowHigherDuplicates disabled. It did not work as documented and was confusing. - New directive "CancelLowerLevelDuplicates" See New Features section in the manual. - Truncate on Purge rewritten. See New Features section in the manual. Bug fixes: 1448 1466 1467 1468 1476 1481 1486 1488 1494 1497 1499 1501 1505 1509 1513 - Ensure SD asks for help when looping even if poll set. Fixes bug #1513. - Fix three-pool regress bug - Modify bacula.spec fixes bug #1505 - This version fixes an issue where the console window would start out docked. It is fixed by initiating the variables in the Pages class wi constructor. - Fix make_catalog_backup.pl fails when catalog db is on other host - Apply MacOSX installer patch from bug #1509 - Apply fix to previous fix of Copy problem. Fix proposed by reporter o #1476 - Fix bug #1501 -t does not print errors - Apply SQLite3 update fix from bug #1497 - Apply bashism fix for diskchanger.in script from bug #1499 - Apply rpm fix for Sci Linux from bug #1494 - Take most recent Ukranian po from bug #1448 - Probable fix for Copy/Migration bug #1476 - Fix bug #1488 -- avoid recursion and race conditions in messages.c - Upgrade cats library also to 5.0.0 - Fix missing console page in bat - Add bat help files to Window install - Improve Windows upgrade to ensure old FD is shutdown - Fix bug #1481 -- bat consumes all console file descriptors - Backport truncate on purge from 5.1.x - Fix bug #1486 -- bat doesn't show any errors on command-line - Update the bsock error URL - Correct .my.cnf umask in make_catalog_backup.pl - Apply fix for dbcheck use by make_catalog_backup.pl - Fix seg fault in bscan from new comment field - Allow multiple CNs when using TLS - Fix seg fault in SQlite driver - Make shared libs version the same as the Bacula release version - Remove file_index sequential check - Fix #1466 about Bogus pruning message For Packagers: 1. The default query.sql file is now, except for some comments, empty. The old file, which we no longer support (it is impossible or difficult to make it work on every backend, and the queries are mostly contributed) can be found in /examples/sample-query.sql. The sample file is not installed by the Makefiles 2. When you install the mtx-changer script, you must also install mtx-changer.conf if it does not exist. This new file (mtx-changer.conf) is required for mtx-changer to work, but it is a user configurable file, so on any update, any existing file should not be overwritten. 3. Bat should be built on every platform that is capabable of running Qt. However, the Qt code is changing rather quickly and is not always compatible from version to version. We have built and verified bat on Qt 4.3.4. We strongly recommend that you do not build and distribute bat with any other version of Qt unless you personally test it. To build against Qt 4.3.4, download the depkgs-qt package from the Bacula Source Forge download location, read the README file and follow the instructions. If you are building for Bacula version 5.0.0, please ensure that you do not have qmake-qt4 loaded on your system. If you do, either remove it or rename it before trying to build bat. If you do not, bat will probably be built using the shared objects on your system. For Bacula 5.0.1 and later, this problem (bug) does not exist. depkgs-qt does not install Qt on your system, nor does it interfere with you having any other version of Qt installed on your system. Once you build bat with depkgs-qt, it should *not* use the Qt shared objects, but rather they will be linked into the program. After fully installing bat (make install), you can run "ldd bat" to see what shared objects it will use. If any Qt shared objects are referenced, something has gone wrong. 4. Unless absolutely necessary, we recommend that you do not define any special library environment variables that apply to the ./configure -- for example: LIBDIR=/... ./configure is strongly discouraged. Doing so, could potentially cause Bacula to be linked against the wrong shared objects. 5. The Bacula project strongly recommends that you install Bacula into a single directory, with a few minor exceptions such as the MySQL or PostgreSQL databases. Preferrably this should be /opt/bacula. The full recommendation is: #!/bin/sh # Recommended configure script for Bacula prefix=/opt/bacula email=xxx@yyy.zz CFLAGS="-g -O2 -Wall" \ ./configure \ --sbindir=${prefix}/bin \ --sysconfdir=${prefix}/etc \ --docdir=${prefix}/html \ --htmldir=${prefix}/html \ --with-working-dir=${prefix}/working \ --with-pid-dir=${prefix}/working \ --with-subsys-dir=${prefix}/working \ --with-scriptdir=${prefix}/scripts \ --with-plugindir=${prefix}/plugins \ --libdir=${prefix}/lib \ --enable-smartalloc \ --enable-tray-monitor \ --enable-bat \ --with-mysql \ --with-dump-email=${email} \ --with-job-email=${email} \ --with-smtp-host=localhost \ --with-baseport=9101 Obviously, the email, and some of the minor options (mysql, postgresql, ...) can be changed to suit your distribution, but the directory names defined above are strongly recommended, and over time the default values in the bacula-dir.conf and bacula-sd.conf will reflect these choices. If you have any questions about this or would like a detailed document describing our recommendations including packaging requirements, please send an email to the bacula-devel list. 6. Starting with Bacula version 3.0.0 up to Bacula 5.0.0, the shared libraries that Bacula uses by default are named xxx-1.0.0. Starting with Bacula 5.0.1, we are going to name the libraries using the Bacula version. So in Bacula 5.0.1, the libraries will be named xxx-5.0.1. With future versions, the last digit may or may not change when we distribute patch updates (i.e. the last digit of the version changes). This will depend on whether or not we have changed something in the library. Hopefully this new procedure will resolve some of the incompatibility problems between different versions of the shared objects. 7. The default build option for bconsole is conio (my own little console routines). I did this because some years ago, readline was very difficult to maintain -- it and where it was found seemed to change on every release. This generated at the time a number of support problems. It seems to me that since then there have been very few problems with readline. As a consequence, I have no problem if you want to make bconsole with readline enabled. It will actually give some very nice new bconsole command completion functionality that Eric has written. Bottom line: feel free to use readline or not as you please. ;;;; Eric;;;25 January 2010 at 10:00 Bacula Version 5.0.0 has been released to Source Forge Release Numbering: ----------------------------------------------------------- You might be wondering why this release jumps from 3.0.x to 5.0.0 thus skipping version 4.0.0. We have done this for several reasons: first, we wanted a way of distinguishing the numbering system for the Bacula System Enterprise version and the Bacula Project version. So, to do so, we have decided that the first number of the Bacula Project version will always be odd, and the first number of the Enterprise version will always be even. Thus the Bacula Project now moves from 3.0.x to 5.0.x. In addition, we want to keep the Bacula Project version larger than the Enterprise version to indicate that the Bacula Project version is more advanced or has more features than the Enterprise version. For memory, the current Enterprise version is 2.6.1, and the next release (in a few months -- before June 2010) will be version 4.0.0. Warning: -------------------------------------------------------------- Please note the three Warnings below before doing any install or upgrade. Changes since 3.0.3a (the last Bacula Project release): ------------------------------------------------------- Bug fixes: 1337 1351 1352 1354 1357 1361 1367 1369 1370 1373 1376 1391 1403 1409 1420 1422 1427 1428 1431 1450 1454 1455 1458 1456 Fixed: - Problem that locks the Director when the SMTP server wasn't responding to the bsmtp program - Restore's dir command shows incorrect file sizes - Fix various problems with the reload command - Fix problems with conio - Eliminate several deadlocks or potential race conditions from SD Enhancements: - Project 5: Truncate volume after purge - Project 6: File Deduplication using Base Jobs - Project 10: Restore from Multiple Storage Daemons - Project 11: AllowCompression per Device - Project 23: Add Maximum Concurrent Jobs for Devices to balance load between drives - Add Accurate Fileset Options to configure accurate detection. Can use checksum verification for example. - Allow FD to keep root read but drop write permission (CAP) - Secure handling of passwords for catalog backup - Add Tab-completion for Bconsole when using Readline - Add Bvfs API to query the catalog without building a memory tree - Add new speed test to btape program - Add new Bat screens (Autochanger content, Job view, Media view, ...) - Windows version of Bat - Added Spanish Bacula translation - New bconsole -u nnn option to have timeout on commands - Allow Migrate, Copy, and Virtual Full to read and write to the same Pool - show disabled -- shows disabled Jobs - Add Pool File and Job retention periods (take precedence over Client) - Many ACL improvements - Level added to FD status Job entry - Allow turning on/off Block Checksum per device - Set Device Poll interval to 5 minutes -- previously did not poll - Implement lock manager with bad order protection Compatibility: As always, both the Director and Storage daemon must be upgraded at the same time, and on any given machine, you must run only one version of Bacula. This means that on the Director and Storage daemon machines, you *must* upgrade your File daemon as well. Older 3.0.x and possibly 2.4.x File Daemons are compatible with the 5.0.0 Director and Storage daemons. There should be no need to upgrade older File Daemons immediately. However, we recommend that you do so as soon as is reasonable for your situation. ================== Warning !!!!!! ========================== New Catalog format in version 5.0.0 (3.1.9 or later) ----------------------------------------------------- This release of Bacula uses a new catalog format. We provide a set of scripts that permit to convert a 3.0.x (version 11) catalog to 5.0.0 (version 12). The Bacula project rpms distributed on Source Forge will automatically detect the database change and do the upgrade for you. We strongly recommend that you save a copy of your existing database before upgrading using rpms. The simplest way to upgrade the database assuming you are building from source: 1. Stop any current version of Bacula from running. 2. Save a copy of your existing database. 3. Configure, build, and install the 5.0.0 version of Bacula Note the new ./configure options in the manuals see the link at the bottom of this file. 4. cd 5. ./update_bacula_tables 6. If you have multiple catalogs, edit the update_bacula_tables script and put the name of another database, and then execute the script again. Repeat this until all Bacula catalogs have been updated. 7. Start the new Bacula. If everything worked, it should not print any error messages. If you start from scratch, you don't need to run the update_bacula_tables script because the create_bacula_tables script automatically creates the new table format. However, if you are using a version of Bacula older than 3.1.9 (e.g. 3.0.3) then you need to run the update_bacula_tables script that will be found in the /src/cats directory after you run the ./configure command. As mentioned above, before running this script, please backup your catalog database, and when you do running it, be sure shutdown Bacula and be aware that running the script can take some time depending on your database size. ================================================================ ================== Warning !!!!!! ========================== If you upgrade a previous Win32 client installation, you must first stop the File daemon, then uninstall it before attempting the upgrade. If you do not do so, the upgrade will probably fail and you will need to manually find and run the Uninstall.exe file, before you can upgrade to the new version. This is because the file locations are different. If you want to save your previous bacula-fd.conf file, please do so before uninstalling it. ================================================================ ================== Warning !!!!!! ========================== Due to permissions problems on Windows Vista and later systems, we no longer require SYSTEM or Administrator permissions to access the binaries and configuation files. If you wish to restrict access to these files, please see the New Features section of the manual for how to set access control lists. ================================================================ Note, the Win32 Installer no longer installs the Bacula Servers (Director and Storage daemon). See the New Features section of the manual for more details. ============ Documentation improvements needed ================== Note!!!!! The 5.0.0+ documentation has a number of known problems: 1. We have reduced the number of manuals, which has fixed many of the broken links found in the old 3.0.x version, but there are still some that need fixing. 2. Integration of the 3.0.x features into the main manuals has not yet been done. The documentation will not be completed in the near future due to lack manpower. If you are unhappy with this situation as we are, we suggest you help by submitting patches to the documentation. Thanks. ===== This release includes large number of new features and changes. It should be more or less compatible with previous xx.conf files and File daemons providing that you are not using any of the new features. Please see the New Features chapter of the new manual, which documents some but not all the new features (we are still working on it): http://www.bacula.org/5.0.x-manuals/en/main/main/New_Features_in_5_0_0.html The ChangeLog has more details. ;;;; Kern;;;18 October 2009 at 21:00 Bacula Version 3.0.3 has been released to Source Forge Note, if you are not already running 3.0.x, read the 3.0.x release notes below. Important!!! Version 3.0.3 is primarily a important bug fix update to version 3.0.2. Changes since 3.0.2: Bug fixes: 1391, 1288, 1346, 1352, 1359, 1355, 1363, 1364, 1365, 1366, 1368, 1369, 1370, 1371, 1382 Fixes: Apply James' fix for some Exchange plugin crashes Integrate patch for building dmg on OSX from Lorenz Schori Fix bug #1391 Job status improperly set due to subtle variable overload problem Fix #1352 about double free with regexp and big filenames on windows Apply Graham's idea for recovering from disk full by recursing when doing fixup_write_error ... Make fix for VirtualFull changing device based on suggestion by Nicolae Mihalache Fix concurrent Job recycle bug #1288 Backport part of one possible fix from bug #1346 Fix bug #1382 newly created disk volumes -> file not found warning Fix Alpha ifdefing -- should fix bug #1359 Use old code for selecting file during restore. Cherry-pick 3.1.x commits tweak space on projects file Fix seg fault in ignoredir code Apply Frank's slash patch Fix seg fault in SD bug #1371 3.0.2-Fix-seg-fault-in-SD-bug-1371.patch Add more info when SD connection refused 3.0.2-Add-more-info-when-SD-connection-refused.patch Fix bug #1355 Director crashes with double free in Accurate SQL query 3.0.2-accurate.patch Fixes bug #1368 ASSERT Failure on MacOS. 3.0.2-mac-path-len.patch Fix #1364 and #1363 about compression buffer error. 3.0.2-zlib-buffer.patch Fixes bug #1365 and #1366 about MacOSX resource forks. 3.0.2-bug-1365.patch and 3.0.2-bug-1366.patch Fix "Exclude Dir Containing" fileset option 3.0.2-exclude-dir-containing-fd.patch 3.0.2-exclude-dir-containing.patch 3.0.2-exclude-dir-containing-top_level.patch Compatibility: As always, both the Director and Storage daemon must be upgraded at the same time. Older 3.0.x and possibly 2.4.x File Daemons are compatible with the 3.0.3 Director and Storage daemons. There should be no need to upgrade older File Daemons. ;;;; Kern;;;20 July 2009 at 14:49 Bacula Version 3.0.2 has been released to Source Forge Note, if you are not already running 3.0.x, read the 3.0.x release notes below. Important!!! Version 3.0.2 is primarily a important bug fix update to version 3.0.1 with some enhancements (see below). Changes since 3.0.1: Bug fixes: 1029, 1285, 1298, 1303, 1304, 1305, 1306, 1307, 1308, 1309, 1311, 1315 1317, 1318, 1323, 1325, 1331, 1333, 1335 Fixed: - Fix problem with the Postgresql driver that sometime displays values - Cleanup old job records when starting the Director - Fix bugs in xattr code - Fix encoding problem with Postgresql - Fix that prevents restore by file selection from using Copy job Enhancements: - Security enhancement for the File Daemon - New directive to specify source address for outgoing connexions - Show volume availability in autochanger when doing restore. This is project 31 in the projects file. - Add accurate=yes/no option to estimate command - Add --with-hostname, --with-bsrdir, --with-logdir to ./configure - Add a restore menu that contructs a full restore when entering a JobId. This is project 2 in the projects file. - Add new Nagios plugin Compatibility: Like always, both the Director and Storage daemon must be upgraded at the same time. Older 3.0.x and possibly 2.4.x File Daemons are compatible with the 3.0.2 Director and Storage daemons. It should be no need to upgrade older File Daemon. ;;;; Kern;;;30 April 2009 at 17:49 Bacula Version 3.0.1 has been released to Source Forge. Note, if you are not already running 3.0.0, please read the 3.0.0 release notes. Important!!! Version 3.0.1 is primarily a minor bug fix update to version 3.0.0 with one enhancement (see below). Changes since 3.0.0: Bug fixes: 1276, 1274, 1275, 1272 1246, 1268, 1281, 1282. Fixed: - Check for job_canceled() in fd_plugin code. - Update Win32 table creation to have new DB version 11 format - Remove illegal Options in Exclude of default Win32/64 bacula-dir.conf - Remove 'Reposition' messages when restoring - Updated projects file - Tweak version string to display versionid field at the end and keep fields order. - Add additional mysql connection debug code submitted by: Chandranshu Enhancement: - Modify insertion of read Volumes in SD to be done *before* the drive reservation. This ensures that a Volume to be read will not be reserved for writing. Significant improvement in eliminating tape deadlock situations. Note, both the Director and Storage daemons must be upgraded at the same time. The 3.0.1 File daemons are compatible with 3.0.0 file daemons and as noted below in the 3.0.0 release notes the Director and Storage daemon should be compatible with 2.4.x File daemons. In general, we do not expect any of the 3.0.x File daemons to work with older Directors and Storage daemons (as has always been the case for Bacula releases). ============ Documentation improvements needed ================== Note!!!!! The 3.0.0+ documentation needs rather extensive modifications in the following areas: 1. Missing documentation. 2. Correction of links within various manuals -- most fail. 3. Integration of the new features into the main documents. 4. Much more. The documentation will not be completed in the near future due to lack manpower. If you are unhappy with this situation as we are, we suggest you help by submitting patches to the documentation. Thanks. ===== ;;; Kern;;;09 April 2009 at 22:30 Bacula Version 3.0.0 has been released to Source Forge. This is to inform you that we have uploaded the Bacula version 3.0.0 source tar files and the Win32/64 installer files to the Bacula Source Forge download location. There are quite a number of new features in this release, and upgrading to it requires a database upgrade, so please read the release notes carefully (included below) and the documentation for the new features at: http://www.bacula.org/manuals/en/concepts/concepts/New_Features.html There is still a good deal of work to do to finish the release process (mainly reorganizing the web site manuals ...), but at least the source files are released. The rpms should follow shortly (probably this weekend). If you are currently using 2.4.4, please note that within a few months that version will no longer be supported, so please consider the possibility of upgrading. This release probably has the most new features that we have ever released, and it has taken a long time to produce it. The basic code is very well tested, and if you stick to 2.4.4 features, you are unlikely to experience problems. However, many of the new features have not been well tested, so your help will be appreciated. The following issues with 3.0.0 are known and we are working on them: 1. The documentation is not 100% complete (there are so many new features that we missed documenting some). 2. All the new documentation is in the New Features chapter, but not all of it has yet been integrated back into the appropriate chapters. 3. James has reported that the Exchange plugin may crash if you cancel a job while it is running. He is working on this. 4. The SD has many new features that will make it more stable. One in particular is that it knows what Volumes will be read during a migration, copy, or Virtual backup job, and will ensure that they are not selected for writing (something easy to do if you are using disk files and the same pool for reading an writing). However, the day after the release was cut, I found a problem with this code that prevents it from working properly during certain pre-allocations of the write tape. 5. There is still an outstanding bug report on VSS failing (it seems to be a bit of a rare case). As I previously announced, we will now make 3.0.0 our main release source code stream, and we hope to release new version every six months and bug fix updates even sooner. To be able to do this, we will support version 2.4.4 only for a few more months. There after the project will only support the most current release ... This release comprises a lot of work by a lot of different people and many contributions. Normally I hesitate to mention particular people or contributions because I always forget someone, but since this was such a big release, I would like to mention at least the big contributions: Sponsor Company(s) - Programmer - Project Bacula Systems - Eric Bollengier - Accurate Backup Bacula Systems - Eric Bollengier - Catalog format enhancements Bacula Systems - Eric Bollengier - bextract non-portable Win32 data Bacula Systems - Eric Bollengier - Win64 File daemon Bacula Systems - Eric Bollengier - SD deadlock detection + postmortum dump Bacula Systems - Eric Bollengier - Faster Attribute despooling if DIR & FD on same machine Bacula Systems - Eric Bollengier - Virtual tape emulation Bacula Systems - Kern Sibbald - USB rescue key Equiinet funded development - James Harper - MS Exchange plugin Personal contributions: Kjetil Torgrim Homme - Regex expressions for restore in bootstrap file Marco van Wieringen - ACL code overhaul Marco van Wieringen - Solaris ZFS/NFSv4 ACL support Marco van Wieringen - Extended Attribute support Marco van Wieringen - Shared Objects Dirk Bartley - Bat speedup + stability improvement + many enhancements Joao Henrique Freitas - libdbi catalog driver Scott Barninger - rpm packaging Kern Sibbald - many big and little projects ... Many people -- see the AUTHORS file. Many thanks to everyone for making Bacula what it is today. === List of new features === - Accurate Backup - Copy Jobs - ACL code overhaul - Solaris ZFS/NFSv4 ACL support - Extended Attribute support --disable-xattr (default=auto) - Shared Objects --enable-libtool (default) --disable-libtool - Virtual Backup - Catalog Format Enhancements - Support for more than 4 Billion file entries - New stats tables - Duplicate Job Control - TLS Authentication - bextract non-portable Win32 data - State File updated at Job Termination - Bacula Plugins - FD bpipe plugin - FD Exchange plugin - libdbi DB driver - Console Enhancements - status slots - list joblog - command separator - Bootstrap regex - Restore seek optimization - Virtual Tape Emulation - Bat Enhancements - RunScript Enhancements - Status enhancements - FD connect timeout default to 3 minutes - ftruncate implemented for NFS Volumes - FD Version compatibily flag - Statistics Enhancements - Faster Attribute despooling - Document split into 6 manuals - GPLv2 license now compatible with OpenSSL - Win64 File daemon - Postmortum dump improvements in SD - Deadlock mutex detection in SD - Improved Volume management in SD to avoid deadlocks. - New configure install options --docdir= (default=/usr/share/doc/bacula-VERSION) --htmldir= (default=/usr/share/doc/bacula-VERSION/html -- bat help files) --plugindir= (default=sysconfdir, where plugins will be installed) - New Bare Metal Recovery with USB key. See rescue/linux/usb New Directives: - Max Full Interval - Max Diff Interval - Honor No Dump Flag - Exclude Dirs Containing - Recycle Pool - Max Run Sched Time - Max Wait Time - Full/Diff/Incr Max Run Time - Scratch Pool - Max Console Connections - Spool Size (in Job resource) - Allow Mixed Priorities - Allow Duplicate Jobs - Allow Higher Duplicates - Cancel Queued Duplicates - Cancel Running Duplicates - TLS Authenticate - Console (in RunScript) New features from Project Items Completed for version 3.0.0 also listed above: Item 1: Accurate restoration of renamed/deleted files Item 3: Merge multiple backups (Synthetic Backup or Consolidation) Item 4: Implement Catalog directive for Pool resource in Director Item 5: Add an item to the restore option where you can select a Pool Item 8: Implement Copy pools Item 12: Add Plug-ins to the FileSet Include statements. Item 13: Restore only file attributes (permissions, ACL, owner, group...) Item 18: Better control over Job execution Item 26: Store and restore extended attributes, especially selinux file contexts Item 27: make changing "spooldata=yes|no" possible for Item 28: Implement an option to modify the last written date for volumes Item n: Split documentation into several books ================================ Release Notes for Bacula 3.0.0 Bacula code: Total files = 520 Total lines = 205,459 (*.h *.c *.in) The diff between 2.4.4 and 2.5.x is 208,340 lines, so this release represents an enormous effort. This Director and Storage daemon must be upgraded at the same time, but they should be compatible with all 2.4.x File daemons, unless you use some of the new features that affect the FD. In other words, you should not have to upgrade all your File daemons when you upgrade. However, any release of version 2.5.40 or greater requires a database upgrade. ================== Warning !!!!!! ========================== New Catalog format in version 3.0.0 (2.5.40 or later) ----------------------------------------------------- This release of Bacula uses a new catalog format. We provide a set of scripts that permit to convert a 2.4.x (version 10) catalog to 3.0.0 (version 11). The Bacula project rpms distributed on Source Forge will automatically detect the database change and do the upgrade for you. We strongly recommend that you save a copy of your existing database before upgrading using rpms. The simplest way to upgrad the database assuming you are building from source: 1. Stop any current version of Bacula from running. 2. Save a copy of your existing database. 3. Configure, build, and install the 3.0.0 version of Bacula Note the new ./configure options in the manuals see the link at the bottom of this file. 4. cd 5. ./update_bacula_tables 6. If you have multiple catalogs, edit the update_bacula_tables script and put the name of another database, and then execute the script again. Repeat this until all Bacula catalogs have been updated. 7. Start the new Bacula. If everything worked, it should not print any error messages. The upgrade operation will convert the FileId index field of the File table from 32 bits to 64 bits. This operation will take TIME and will *temporarily* DOUBLE THE SIZE of your catalog. Bacula must be shutdown during this update, so, won't be able to run jobs during this period. For example, a 3 million files catalog will take 2 mins to upgrade on a normal machine. Don't forget to backup it before executing the script. If you start from scratch, you don't need to run the update_bacula_tables script because the create_bacula_tables script automatically creates the new table format. However, if you are using a version of Bacula older than 2.5.40 (e.g. 2.4.4) then you need to run the update_bacula_tables script that will be found in the /src/cats directory after you run the ./configure command. As mentioned above, before running this script, please backup your catalog database, and when you do running it, be sure shutdown Bacula and be aware that running the script can take some time depending on your database size. Important New configure options : ---------------------- This release of Bacula 3.0.0 uses libtool by default to generate the Bacula libraries as shared objects rather than being directly linked in as in prior versions. This means that the Bacula shared objects, must either be in a shared object directory known to the loader or they must be in the directory where the Bacula binaries are installed. It is also possible to explicitly specify the libraries directory on the ./configure line (--libdir=/full-path/dir), but this should not be necessary. ./configure --libdir= If you have problems with libtool or you wish to use the old way of building libraries and linking them into Bacula, you can do so by disabling libtool on the configure command line with: ./configure --disable-libtool During the installation of Bacula 3.0.0, it will now install bat help files (if you have configured bat). These files are placed in /usr/share/doc/bacula/html by default. If you want to put them elsewhere use: ./configure --htmldir= During the installation of Bacula, it will now install the ReleaseNotes, License, technotes, ChangeLog, and similar files in /usr/share/doc/bacula by default. If you want to put them elsewhere use: ./configure --docdir= During the installation of Bacula, any plugins will be installed in the plugindir, which is by default the same as libdir (/usr/lib). You can specify an alternate location with: ./configure --with-plugindir= ============================================================== This release includes large number of new features and changes. It should be more or less compatible with previous xx.conf files and File daemons providing that you are not using any of the new features. Please see the New Features chapter of the new manual, which documents some but not all the new features (we are still working on it): http://www.bacula.org/manuals/en/concepts/concepts/New_Features.html The ChangeLog has more details. ;;; Kern;;;03 Nobember 2008 at 17:35 Critical 2.4.3-orphaned-jobs.patch released We recently found and corrected a serious bug in Bacula that causes jobs to be orphaned or "stuck" in the director during the pruning process. Depending on what jobs get stuck and how many there are, it could prevent additional jobs from running. As best we can tell, this problem occurs only during what could be called extreme pruning conditions -- i.e. there are insufficient Volumes available for normal operations. In any case, we recommend that everyone apply the 2.4.3-orphaned-jobs.patch, which you can find in the bacula-patches section of the Source Forge download area as well as in the directory patches of both the trunk and Branch-2.4 SVN. ;;; Kern;;;12 October 2008 at 10:30am Creation of Bacula Systems SA Dear Members of the Bacula Community, I'm pleased to announce the creation of Bacula Systems S.A., a new company to provide professional support and service for Bacula. I believe you will be pleased to see how we have married a strong commitment to the ethos of honest open source software with the availability of services desired by a wide range of Bacula users. As many of you know, I began work on Bacula in 2000 and released the first public version in April 2002 to Source Forge. Although I have personally written a very substantial part of the code, I offer sincere thanks to the many members of the community who have provided ideas, fixes, testing, and contributions of code, without which Bacula would not be what it is today. It has been a labor of love and I am pleased that so many people have found Bacula useful. Before telling you more about Bacula Systems, let me reiterate my unfaltering, ongoing commitment to open source software and the Bacula open source project. The code for Bacula and all future releases will remain available as free open source software under the GPL license that I transferred to the protection of the Free Software Foundation of Europe. Unlike some other open source projects, there will be no separate "enterprise" or proprietary version of the Bacula source code. www.bacula.org will continue to be the home for Bacula project development. So, why create Bacula Systems? The company was created to respond to requests from users who want professional support with guaranteed levels of service and response times. To ensure continuity and the highest level of knowledge transfer, I will take the position of Chairman of the Board and Chief Technical Officer of Bacula Systems S.A.. Several members of the Bacula community have joined me to start the company. The other team members are also strongly dedicated to the spirit of keeping Bacula open and free while simultaneously offering commercial services to those for whom it is essential in making the decision to deploy Bacula. Bacula Systems is self-financed -- the initial capital comes from the founders; no outside venture capital was sought or accepted. This means we have the power to establish the company's priorities and we can remain dedicated to "honest open source". Bacula Systems S.A. is a Swiss company with offices in Yverdon-les-Bains, and with branches in France and Germany initially. We will open additional branches as business requires. I will continue to support development of Bacula via bacula.org and the Bacula community. Future developments will be in the "community" code base which will remain free. Bacula Systems will offer "funded development" to customers who wish to accelerate the addition of certain features to Bacula. Code developed this way will be contributed to the free distribution and will be available to the entire community at no charge. Bacula Systems will provide additional fee-based services, including high quality consulting, training, documentation and certified binaries. Our goal in establishing Bacula Systems is to be inclusive of the community, and expand it by creating opportunities for members of the Bacula community through a well-defined Partner Program. If you provide IT services or consulting, the Bacula Systems Partner Program can provide you with the additional back-up (pardon the pun) support you need to serve your customers more effectively. Our plan is to develop an international network of partners who offer high value added services to Bacula users -- installing, configuring and optimizing Bacula for their customers, as well as Level 3 and 4 support, with guaranteed service levels. We will also provide training and tools to our partners. The Bacula project website, www.bacula.org, and Source Forge will be unchanged in the mission and ideals of being free sources of information, mail lists and forum-based support for and by the community. Both Bacula source code and Bacula binaries will continue to be available for free, as in the past, and new releases will also be available for free. The new commercial website, www.baculasystems.com will provide information on commercial subscriptions for those users who want guaranteed levels of service, support, documentation and certified binaries. All members of the Bacula community are encouraged to register for free at www.baculasystems.com, so that you will receive announcements and news about our commercial activities, and so that you can send feedback to our new company, Bacula Systems. Thank you for your interest and participation in the Bacula community. Sincerely, Kern Sibbald ;;; Kern;;;11 October 2008 at 11:30am Bacula version 2.4.3 Released This is to let you know that the source tar files and the Win32 binaries for Bacula version 2.4.3 have been released to the Bacula Source Forge download area. Release 2.4.3 This is a bug fix to version 2.4.2. All daemons are compatible with other 2.4.x versions. Bugs Fixed: 1159, 1162. 1149, 1128, 1143, 1140, 1129 Change Summary: - Apply 2.4.2-cancel-non-running-jobs.patch to be able to cancel a non created job faster. - This should correct bug #1159 where Migration does not properly respect the Migration Low Bytes directive. - Fix typo in configure.in that breaks --with-db-password option - This code should fix the race condition that leads to a Director crash at job end time when the job list is updated. This was reported in bug #1162. - Remove all double quotes from SQLite creating script and replace by single quotes as suggested by John Huttley. - Fix SQL case problem that may cause the failure of DiskToCatalog in bug #1149. - Copy missing storage name into edit buffer. As far as I can tell this never caused a bug. - Remove catalog dependency for bcopy - Modify catalog scripts to have an easier packaging integration, using default variables. Make difference between SQLite3 and SQLite in db_get_type() - Attempt to fix bug #1128 InChanger flag cleared during Migration job when reading from one autochanger and writing to another. - Add more information to SD acquire.c INFO messages. - Fix NULL Volume name error when reading and the drive must be switched. - Fix a Verify InitCatalog problem where in certain cases a garbage filename may be entered in the verification database. This fixes bug #1143. - Add space after version before date in bat about dialog. - Fix seg fault in Dir during estimate command with no level value given. This fixes bug #1140. - Add message to migration job when the target job is already migrated. This closes bug #1129. Unless some really serious new bug shows up, this will be the last release until the next major version (3.0.0), which we hope will be ready around the end of the year. ;;; Kern;;;26 July 2008 at 10:30am Bacula version 2.4.2 Released This is to inform you that we have released the source and Win32 binaries of Bacula version 2.4.2 to the Bacula Source Forge download area. This is an important bug fix release since version 2.4.1. Bugs Fixed: 1034, 1125, 1118, 1124, 1107, 1129, 1126 Change Summary: 26Jul08 - When a migration job actually runs, re-check the Job record and skip if the job is already migrated. This should significantly reduce the problems with bug #1129. - Break the do_swapping into do_unload, do_swapping, and do_load. It is much more logical that way. - Implement a set_dcr_from_vol subroutine in acquire.c for reading volumes. This allows the dcr to be refreshed after being zapped when the wrong volume is mounted. This should fix bug #1126 - - During multiple tape restore, bacula does not ask for physical tape change, but rereads same tape - Apply patch submitted for bug #1107 with a small modification. This fixes a bug where bcopy copied too many records. - Make some tweaks to bsmtp based on patch submitted in bug #1124. This fixes bug #1124. - Make the default bat restore Pool be Any. This fixes bug #1118. - Fix bug #1125 which reports a SD crash at the end of a tape during restore. It was a debug statement that should not be in a release. - Fix #1034 by setting big timeout on mysql connections - Do partial integration of the Win32 bat build created by Eric. - Ensure that SD tried to mount a volume not in an autochanger at least once before asking for operator intervention. - Fix a pthreads bug in the Win32 pthreads emulation code affects only version 2.5.x and above. ;;; Kern;;;7 July 2008 at 10:30am Bacula version 2.4.1 Released This is to inform you that we have released the source and Win32 binaries of Bacula version 2.4.1 to the Bacula Source Forge download area. Bugs Fixed: 1103, 1100, 1105, 1094, 1109, 1106, 1101, 1102, 1112 1117, 1116, 1097. Change summary: - Clear in-use bit on vol when unused while swapping. - Remove sleep at end of ./bacula - Clear hash packet for hardlinked files correctly. This corrects a long standing bug where hardlinked files selected individually would occasionally not be restored. - Apply Eric's patch to ensure that autoprune does not return a Volume not in the autochanger unless requested to do so. - Correct bat restore display of multiple drives. This fixes bug #1117. - Do find_a_volume() each time an unload is done. This fixes the failure of maxvol2-test. - Use Qmsg() in job.c watchdog callback. - Release main control rwlock if ABORTing. - Make watchdog connect timeout queue messages rather than sending directly to avoid lock conflicts with the real thread. - Remove const char that causes problems with Python, which has older 'incorrect' headers. - Add const char in dbd.c to avoid compiler warnings. - Fix mtx-changer to detect both versions of Ubuntu (Debian) mt. This fixes bug #1116. - Fix failure of 2drive-concurrent-test. Always read label; clear_unload() only after drive is defined in acquire_for_read(). - Generally clean up the manual tape loading code. The main conceptial change is that when a volume is marked to be unloaded, its volume name is retained, and it is only marked as unloaded when either the autoloader says it is unloaded or another tape is read on that drive. - Fix a recycling problem with two autochangers reported in bug #1106. This may also (unlikely) fix a second recycling bug as reported in #1103. - Fix a SD lock volumes deadlock problem reported in bug #1100. - Fix format problem in bscan output reported in bug #1105. - Copy more data when restarting a job so that run overrides are kept. This should fix bug #1094. - Backport updates to jobq.c where possible. - Fix bug where SD did not ask operator if the device could not be opened. Reported by Eric. - Add dbuser to DIR conf file (replaces user). - Add --with-db-password to ./configure - Fix regress to handle db_password. - Fix a couple of pedantic compiler warnings. - Fix bug when starting a restore on a storage that doesn't have the requested MediaType. - More work on word alignments -- eliminate all ints from the configuration routines. - Fix bug reported by Scott Barninger where the bacula script refers to scripts in the wrong directory. Needed to meet the requirements of recent FHS location changes. - Fix word alignment problem on non-Intel 64 bit machines such as Solaris. This should fix bug #1097 -- bus error in SD. - Add missing win32/patches/wxWidgets.patch to 2.4 repository. ;;; Kern;;;4 Jun 2008 at 10:30am Bacula version 2.4.0 Released This is to inform you that we have released the source and Win32 binaries of Bacula version 2.4.0 to the Bacula Source Forge download area. This release consists mainly of a number of bug fixes since 2.2.x but most importantly, a rewrite of significant portions of the Storage daemon reservation and autochanger code. This rewritten code is simpler, more stable, and works much better, particularly with multiple drive autochangers than the prior code did. That said, there are still a number of issues to be resolved. Note, the new code has not been tested in manual volume mount situations. As usual, we strongly recommend when installing to keep the Director and the Storage daemon on the same version. However, since this release only contains refactored code and bug fixes rather than new features, the clients (File daemons) should function perfectly with any 2.2.x version, and there is no database upgrade needed. ===================================================== Release Notes for Bacula 2.4.0 Bacula code: Total files = 500 Total lines = 188,654 (*.h *.c *.in) This Director and Storage daemon must be upgraded at the same time, but they should be compatible with all 2.0.x File daemons, unless you use some of the new features that affect the FD. In other words, you should not have to upgrade all your File daemons when you upgrade. There is no database upgrade needed from version 2.0.x to 2.2.0 nor from version 2.2.x to version 2.4.0 ================== Warning !!!!!! ========================== As of Version 2.4.0, the Win32 FD should not descend into any reparse point directory or reparse mount points unless the specified directory is explicitly mentioned at the top level of FileSet (as with Unix). A file that is linked to another file will be backed up, much like Unix hardlinked files. If you want reparse directories backed up, you must explicitly include them in your FileSet. ====================================================== This release consists mainly of a number of bug fixes since 2.2.x but most importantly, a rewrite of significant portions of the Storage daemon reservation and autochanger code. This rewritten code is simpler, more stable, and works much better, particularly with multiple drive autochangers than the prior code did. That said, there are still a number of issues to be resolved. Note, the new code has not been tested in manual volume mount situations. Bugs fixed: 1095, 1086, 1091, 1070, 1041, 1083, 1078, 1066, 1068, 1038 1062, 1065, 1046, 1047, 1031 New features: - You can now build bat without the QWT libraries. Use --without-qwt as a ./configure option. Release Version 2.4.0 - Update version to 2.4.0 to reflect the magnitude of the SD changes. - Make sure to clear JS_WaitMedia when operator mounts a Volume. This fixes bug #1095. - Add create_postgresql_database.sql to Win32 installer. Should fix problem reported by Juilio Monteiro. - Get correct slot when auto unloading a device. This fixes bug #1086. - Fix possible seg fault if SQL error. - Fix renaming a Volume, which used the wrong Volume name. - If operator has rewind tape, print warning, release tape and try once more. If tape is positioned somewhere, something went wrong, so mark the tape in error and try once more. Previously this error was fatal, now it produces an error message. - Ensure correct volume name displayed during restore - Fix a few more Coverity reported problems. - Fix #1091 about bad output in estimate command. - Modify autochanger locking to attempt to avoid race conditions. - Make sure device not busy before doing label command. - Display open() errors except when polling. Previously too many were suppressed. This should fix bug #1070. - Fix Win32 reparse points. Bacula will not recurse into any reparse point directory, including mount points, unless the directory is explicitly mentioned at the top level (same as with Unix). A file that is linked to another file will be backed up -- much as Unix does for hardlinked files. This fixes bug #1041. - Remove StorageId test when pruning and recycling (Eric's changes). - Fix buffer overruns detected by Coverity. - Implement --without-qwt - Fix layout of restore tree dialog - Add configure bat QWT libraries, so that bat can be built with or without the QWT libaries. - Implement regression that explicitly tests swapping a Volume from one drive to another. - Enhance disk-changer to detect most error conditions. - Fix SD code so that it properly swaps a Volume between drives. This fixes bug #1083. - Prevent a Volume that is being swapped from being freed from the volume list. This will most likely fix, at least partially, bug #1083. - Fix strippath so that it does not get a buffer overrun and crash FD. This fixes bug #1078. - Make inability to change owner/group when creating a dir only a warning rather than an error. - Fix SQL query in migration code - Fix bat seg fault at termination. - Add Bacula generated version to bat about box. - Backport development stream SD reservation system changes. - When wrong volume is mounted during read, unload_autochanger. - Stop searching for Volumes in SD askdir if DIR returns the same volume name twice in a row. - Rework class structures for VOLRES, DCR, and DEVICE to make the method names a bit more logical, and for more logically handling the responsibilities. - Remove redundant code in terminating the scheduler that just causes a seg fault in many cases. - Improve algorithm for detecting pre-reserved volume and swapping volumes. - Prevent volume from being released while being swapped. - Refactor parts of stored/mount.c - Add sanity checks for VolWriteTime and VolReadTime - Take care of bad clock changes while computing VolWriteTime and VolReadTime. This should fix or limit #1066 - Correct error string numbers in dird/catreq.c - Restructure reserving, acquiring, and mounting volumes. Calls to autochanger are deferred for mount.c -- simplifes the code. - Do not prune any running job. It just fails the job. - Lock the volumes when changing dev->reserved_device and marking the volume unused otherwise the device can get reserved by another job before the volume is released, thus blocking it. - Correct some SD catalog request error messages. - Turn off code in read_record that causes a seg fault in the SD when reading past an EOS_LABEL. - Turn off unloading the autochanger in reserve.c as it just causes problems. - Lock volumes when unreserving a device. - Do not mark volume unused when recycling. - Fix bug #1068 fixes a SD crash when using Virtual autochanger. - Generate correct JobMedia records during spooling/despooling when running concurrent jobs. Thanks to Tom Ivar Helbekkmo for excellent analysis and testing. - *Massive* commit of all fixes and back ports to create 2.2.9 Mostly SD reservations enhancements. - Fix Win32 FD backup/restore memory leak due to improper termination of BackupRead/Write. This fixes bug# 1038. - Fixes a StorageId problem with volumes after migration. - Implements autostart scripts for Debian - Fix mtx-changer.in for broken Debian mt program. - Apply doc fix from bug #1062. - Resolve crash and improper restore wx-console Win32 restore GUI. Fixes bug #1065. - Update Win32 wxWidgets to latest version. - Attempt to do correct handling of Win32 mount points. Should fix bug #1046. - Add const to AIX prototype for initgroups() - Fix to the JobMedia patch, which introduced a new problem. - Fix creating first JobMedia record during Migration to include proper index. This caused slow restores of migrated jobs. - Fix bug #1047, do not strip paths on symbolic links. - Set catalog backup database and user name from values specified on the ./configure line. - Apply patch to correct bug #1031, about wrong pool source information in job report. ;;;; Kern;;;27 Jan 2008 at 10:30am Bacula version 2.2.8 Released Bacula version 2.2.8 source tar files as well as the Win32 binaries are available in the bacula and Win32 release areas of Source Forge. I have hidden the previous BETA release Version 2.2.8 is a bug fix to version 2.2.7 and has a few additional bugs fixed than the BETA 2.2.8 version previously released: - It fixes bugs: 1036, 1033(doc), 1028, 1040, 1045(doc), 1030, 1042(partially), and possibly bugs 1018 and 1032. Bugs 1018 and 1032 involve problems with multiple-drive autochangers, and are difficult to reproduce. See the ChangeLog for more details. Note, if you previously loaded version 2.2.8 BETA, you will probably want to upgrade, but it is not urgent ... ;;; Kern;;;9 Jan 2008 at 20:30 Bacula version BETA 2.2.8 Released Bacula version 2.2.8 source tar files as well as the Win32 binaries are available in the bacula-beta and Win32-beta release areas of Source Forge Version BETA 2.2.8 is a bug fix to version 2.2.7: - It fixes bugs: 1036, 1033(doc), 1028, and possibly 1018 and 1032. Bugs 1018 and 1032 involve problems with multiple-drive autochangers. ;;; Kern;;;24 Dec 2007;;20:30 Bacula version 2.2.7 Released Bacula version 2.2.7 source tar files as well as the Win32 binaries are available in the Bacula release area of Source Forge Version 2.2.7 has several new features and several important bug fixes since version 2.2.6: - It fixes bugs: 1009, 1022, 1024, 1019, 1012, 1021, 1020 1007, 1008. For more details, please see the technotes-2.1 file. - Fixed a seg fault reported by Frank Sweetser that depended on exact path lengths, but the problem was in bsnprintf.c - Double quoting include filenames (@xxx) in Bacula conf files is now possible, thanks to a patch by Michael Stapelberg. - You can pipe input to include filenames (@|prog) by using a vertical bar, thanks to the above patch. - A daylight savings time bug in the Win32 bsmpt was fixed by a patch from Nerijus Baliunas. - Marc Cousins submitted a patch that permits building the PostgreSQL driver with version 8.3. - The configure option --archivedir has been changed to --with-archivedir ;;; Kern;;;9 Nov 2007;;15:25 Bacula version 2.2.6 Released Bacula version 2.2.6 source tar files as well as the Win32 binaries are available in the Bacula release area of Source Forge. This release is a minor fix upgrade to version 2.2.5, and we recommend that all users upgrade when they can. Version 2.2.6 is a minor bug fix realease to version 2.2.5 - It fixes bugs: #1003, 942, 982, 990(response only), 991, 993, 986, 976. - Fix bat crash wen it cannot connect to Director. - Fix joblist failure bug in bat. ;;; Kern;;;9 Oct 2007;;15:25 Bacula version 2.2.5 Released Despite the fact that the Release Notes are rather short, the bug fixes represent quite a lot of work: Version 2.2.5 is a major bug fix release to version 2.2.4 - It fixes the following bugs: #961, 962, 963, 969, 968, 960, 964, (possibly 935 and 903), 953, 953, 967, 966, 965, 954, 957, 908, 958, and 955. - It also improves listing performance problems in bat pointed out by Chris Howells. ;;; Kern;;;29 Sept 2007;;20:30 A serious data loss bug in Bacula version 2.2.4 found and fixed This bug was very elusive and time consuming to track down. It turned out to be a race condition that can lose the last one or two blocks of a Job, which can happen in rare cases only when running multiple simultaneous jobs when a volume fills and one Job finishes receiving the last data from the FD and at the same time a second job detects the end of the Volume. The bug was introduced in version 2.0.0 and did not exist in version 1.38.x. If the bug is triggered, the data is lost (not written to the Volume), and it can occur for any backup job of any level under the conditions noted above. The bug is reported in the bugs database as bug #964, and just this morning I found and tested a fix, which is attached as a patch to the bug report. The patch (2.2.4-lost-block.patch) is also uploaded to the bacula-patches 2.2.x release area. This bug may also be part of the problems reported in bug #935 and possibly #903. Also in the bacula-patches 2.2.x area, you will find other patches that fixes bugs #953, 966, 967, 965, 958, 908, and 955. Please refer to bugs.bacula.org for the details. Though this bug quite serious (data loss), it should be rare. However, we recommend everyone to apply the patch. ;;; Kern;;;2007/9/14;;;14:30 Bacula version 2.2.4 Released This version of Bacula is a minor bug release to version 2.2.3. It contains the following fixes: - Possible fix for authorization problems bug #953. - Possible fix for bug #908. - Add waits to multiple exit detection code to try to force pid file to always be deleted. - Restore good dev.tar.gz to rescue set appropriate binary property. This fixes bug #950. - Fix seg fault in error exit of acquire_for_read after unsuccessfully trying to switch drives by checking for blocking before unblocking. Fixes bug #906. - Cancel storage daemon in all cases where FD reports error. This should fix virtually all cases of bug #920 and will ensure that Devices are released as soon as possible. - Fix error message that was clobbered when Dir tells SD it does not have write permission on Volume. This should fix a minor point in bug #942, but not the main problem. - Fix migration code to get correct Volume name with multiple volumes by skipping |. Fixes bug #936. - Implement patch supplied by Landon to fix bug #944 where using TLS with bconsole uses 99+% of the CPU. - Fix bug #946 about "bacula-dir -t" which doesn't works as expected. - Using "m" in bconsole will show messages as in prior versions and not memory usage. - Note, you need GTK >= 2.10 to be able to link the Tray Monitor program. ;;; Kern;;;2007/9/9;;;14:30 I regret to announce that there is a rather serious bug in Bacula. Bacula bug #935 reports that during a restore, a large number of files are missing and thus not restored. This is really quite surprising because we have a fairly extensive regression test suite that explicitly tests for this kind of problem many times. Despite our testing, there is indeed a bug in Bacula that has the following characteristics: 1. It happens only when multiple simultaneous Jobs are run (regardless of whether or not data spooling is enabled), and happens only when the Storage daemon is changing from one Volume to another -- i.e. the backups span multiple volumes, and it only happens for Jobs writing to the same volume. 2. It has only been observed on disk based backup, but not on tape. 3. Under the right circumstances (timing), it could and probably does happen on tape backups. 4. It seems to be timing dependent, and requires multiple clients to reproduce, although under the right circumstances, it should be reproducible with a single client doing multiple simultaneous backups. 5. Analysis indicates that it happens most often when the clients are slow (e.g. doing Incremental backups). 6. It has been verified to exist in versions 2.0.x and 2.2.x. 7. It should also be in version 1.38, but could not be reproduced in testing, perhaps due to timing considerations or the fact that the test FD daemons were version 2.2.2. 8. The data is correctly stored on the Volume, but incorrect index (JobMedia) records are stored in the database. (the JobMedia record generated during the Volume change contains the index of the new Volume rather than the previous Volume). This will be described in more detail below. 9. You can prevent the problem from occurring by either turning off multiple simultaneous Jobs or by ensuring that while running multiple simultaneous Jobs that those Jobs do not span Volumes. E.g. you could manually mark Volumes as full when they are sufficiently large. 10. If you are not running multiple simultaneous Jobs, you will not be affected by this bug. 11. If you are running multiple simultaneous Jobs to tapes, I believe there is a reasonable probability that this problem could show up when Jobs are split across tapes. 12. If you are running multiple simultaneous Jobs to disks, I believe there is a high probability that this problem will show up when Jobs are split across disks Volumes. 13. The bug concerns only the Storage daemon so there is no need to update the clients, though I do recommend updating the Director when installing an updated Storage daemon. I have uploaded patches to bug #935 (bugs.bacula.org) that will correct version 2.2.0, 2.2.1, and 2.2.2. The patch has been tested only on version 2.2.2 and passes all regression tests as well as the specific test that reproduced the problem. The patch has now been confirmed to fix the problem reported, and Bacula version 2.2.3 has been released to Source Forge. For the technical details of the bug, please see: http://www.bacula.org/downloads/bug-935.txt ;;; Kern;;;2007/8/11;;;14:30 Bacula Version 2.2.0 has been released to Source Forge. Release Notes for Bacula 2.2.0 Bacula code: Total files = 520 Total lines = 195,550 (*.h *.c *.in) 82 new files, 41,221 new lines of code, 208,380 lines of change from 2.0.3 This Director and Storage daemon must be upgraded at the same time, but they should be compatible with all 2.0.x File daemons, unless you use some of the new features that affect the FD. In other words, you should not have to upgrade all your File daemons when you upgrade. There is no database upgrade needed from version 2.0.x to 2.2.0. Areas requiring caution or testing: - You must have the thread safe version of MySQL client libraries loaded to build with MySQL enabled. - Volumes are pruned only when absolutely necessary -- this may cause your database to grow compared to prior Bacula versions. - Solaris door and even port files are no longer restored (equivalent to what we do with sockets). - SQLite is no longer supported for Solaris -- it fails too often with bus errors. SQLite3 should work on Solaris. - The default setting for SQLite3 is now 'PRAGMA synchronous=OFF'. This makes it run 30 times faster, but increases the possiblity of a corrupted database if your server shuts down unexpectedly. The default behavior can be changed in src/version.h - Restore on Win32, and in particular on Vista is untested. Please test before relying on it. It should backup and restore reparse points. - Win32 servers are untested, and very likely not to work. The major new features are: - Much faster insertion of attributes (somewhere around 10 times), many thanks to Eric Bollengier and Marc Cousin. - First release of bat (Bacula Administration Tool). Note to build bat you must explicitly enable it on the ./configure line, you must have the Qt4 version 4.2 or later libraries loaded, the qwt (Qt Graphics) package loaded, and qmake and the other Qt4 tools must be available. Most of the implementation was done by Dirk Bartley. - Red/Black restore in memory tree (500 times faster loading). - The Regex Where code to allow easier relocation of restored files thanks to Eric. - Socket level heartbeat for all connections (untested). - posix_fadvise() use in the FD to improve performance reduces swapping due to opening/reading lots of files. Win32 equivalent implemented. - Much improved Volume reservation code that should eliminate most conflicts experienced in multiple drive autochangers. - Simpler locking in the SD in the reservation system. - Detection of file size/date change during backup if enabled. - New Recycle Pool feature -- thanks to Eric. - Efficient implementation of very large include/exclude lists. - Volumes are no longer pruned during 'status dir' - Pruning is now more efficient, and if a Volume is purged, during pruning, it is immediately discovered. - License is now GPL v2 without modifications, fix a few copyright mistakes made when adding FSFE copyright notice. New ./configure options: - --enable-bwx-console - --enable-bat - --with-qwt= - --with-db-name= - --with-db-user= - --enable-batch-insert New bconsole commands: - exec - memory - update recyclepool - .sql - .api - .pwd (in restore tree) - restoreclient (keyword) - backupclient (keyword) - regexwhere (keyword) - update jobid (new command) - recyclepool (keyword) New directives: - Heartbeat Interval (bconsole.conf) - Heartbeat Interval (bacula-dir.conf in Director, Client, Storage) - TLS Allowed CN (bacula-dir.conf in Client) - Regex Where (bacula-dir.conf in Job) - Strip Prefix (bacula-dir.conf in Job) - Add Prefix (bacula-dir.conf in Job) - Add Suffex (bacula-dir.conf in Job) - Recycle Pool (bacula-dir.conf in Pool) - FailJobOnError (bacula-dir.conf in RunScript) - CheckFileChanges (bacula-dir.conf in FileSet) - StripPath (bacula-dir.conf in FileSet) Other features or bug fixes: - Fixed bugs: 916, 910, 917, 914, 906, 907, 842, 830, 893, 861, 888, 886, 807, 877, 872 885, 887, 864, 874, 882, 881, 863, 859, 856, 854, 845, 847, 846, 809, 772, 822, 825, 824, 808, 802, 797, 798, 795, 789, 791, 788, 780, 763, 782, 612, 767, 775, 621, 772, 764, 766, 621, 761, 755, 750, 741, 747, 742, 744, and 743. - Configure bat with --enable-bat. Define qwt libraries with --with-qwt=. See Installation chapter of manual for details. - Create a depkgs-qt package that has both Qt4 and qwt, which are needed to build bat. Most modern Linux systems will have both these packages in the distro. - Storage overrides delete all previous storage definitions instead of prepending. - One should be able to mount and unmount removable devices if the Device resource has Removeable Media set and the mount and unmount directives are defined. - ./configure will do a better job of searching for qwt libraries. - The Win32 version can no longer be shutdown from the tray monitor. Use the command line or the Services panne. - Keep prune_volumes() from pruning the whole Scratch pool. - More debug output in status storage. - Correct moving a Scratch volume from pool to pool (some critical columns were lost). - Different locking in reservations and despooling systems, which means more micro-locking and less macro-locking, which should give a lot more concurrency at the expense of slightly (<0.1%) more overhead due to more locking/unlocking, but concurrent jobs should run much faster. - Additional drive reservation algorithm that should solve a lot of the problems experienced with multiple drive autochangers. - Storage daemon status command enhanced to more clearly show Volume, pool and media type when a job is waiting. - Made bsmtp work with more strict SMTP servers. - Detect doubly freed buffers in smartall.c - wx-console renamed to bwx-console (Fedora request) - gnome-console renamed to bgnome-console (Fedora request) - Migration preserves original job's FileSetId - Spooling implemented for migration jobs. - Config files can be read through a pipe, by specifying a leading | in front of the configuration path/filename. - New memory command (bconsole) that will print the current Director's memory usage, and an in use buffer dump. - Console name changed from *Console* to -Console- to accomodate Win32 filename restrictions. - Corrected the Win32 ftruncate bug. - Additional version information added to Job reports. - New -8 option for bsmtp to specify UTF-8 encoding type. - bsmtp will no longer add < > to target (from, to, cc) if there is already a < in the target. - Prefer Mounted Volumes extended to mean a volume that is being mounted by another job. - Runscript timing set to same as version 1.38.11 - New TLS Allowed CN directive(s) permitted. - New bconsole 'exec' command to run external script/command. - Turn on FreeBSD/OpenBSD code to set EOT model on tape. - Drop from root done before database is opened. - Database user and name can be configured with: --with-db-name=name --with-db-user=user - Turn on wait_for_drive in mtx-changer script. - Rework how bar codes are handled in mtx-changer script. This appears to have been an unreported bug. - Updated nagios plugin supplied by Christian Masopust - Better restricted console ACL checking. - New Client Connect Wait directive in Storage daemon so that users can configure how long SD waits for FD connection. - Bacula will no longer permit pruning of the currently running job (note, if other jobs are running, they may be pruned). - Sockets are no longer restored. - dbcheck works in 300K chunks so will typically run faster. - Include/Exclude lists can now be efficiently handled for quite large sizes -- tens of thousands of entries. - Several memory leaks in migration and database usage were fixed. - New console keyword restoreclient, which specifies the client to which the restore will be sent. The client keyword specifies the backup client. The restoreclient keyword is optional if it is not specified, the backup client will also be the restore client. - The disk file size is now checked to ensure that it agrees with the catalog value before Bacula will append to the disk (same as tape and DVD). ;;; Kern;;;2007/3/6;;;14:30 Bacula Version 2.0.3 has been released to Source Forge. This is a bug fix release to version 2.0.2. If you are upgrading from a version older than 2.0.0, please be sure to read the important notes below. Also, please check the patches directory in the current SVN or the patches release section on Source Forge for important bug fixes to this version. 5Mar07 kes File migrate bug with Pool Occupancy using mediaid instead of jobids. Fixes bug #795. kes Fix orphaned buffers in filed backup and verify due to crypto buffers not freed during errors. Fixes bug #789. 04Mar07 kes Add smartctl call to bacula-sd.conf as an example of getting tape alert info. 02Mar07 kes Add Client Connect Wait to Storage daemon to permit users to modify the time the SD waits for a FD connection. 28Feb07 kes Apply Command ACL filter to JobId list in restore command. kes Correct typeo in var.c patch. 27Feb07 kes Don't let Bacula prune File or Job records for the current Job. kes Fix variable substitution pad + inc bug reported (with patch) in bug #791. 26Feb07 kes Correct SQLite log table index as reported by Luca Berra. 24Feb07 ebl Fix FIFO stuff, bacula tries to rewind the FIFO... Thanks to Andreas 22Feb07 kes Fix a few places in lib/message.c where the open fd may not be zeroed. 21Feb07 kes Add LANG=C to autoconf/randpass so it works with languages other than English. Fixes bug #788. 20Feb07 ebl Revert ClientRunBeforeJob as it was in 1.38.x This fixes bug #780 You will not be able to generate Include/Exclude list any more. If you want to use this, you can apply trunk/bacula/patches/testing/clientrunbeforejob_can_generate_include_exclude_list.patch 19Feb07 kes Restore of sockets created false error messages because Bacula no longer restores sockets, but the code was still trying to set the attributes on a non-existent file. Reported by a user. 16Feb07 kes Fix encryption deblocking bug, which caused some restored files to be truncated. This fixes bug #763. This is a CRITICAL bug fix. kes Add FD event sequence order prepared by Eric -- for RunScripts. kes Fix 12am/pm bug as reported in bug #782. 15Feb07 kes Add quick disconnect FD code from 2.1.4 to 2.0.3. This code causes the SD to release the FD as soon as the FD has sent all the data to the SD. After that the SD will do any final despooling (data and attributes) that are necessary. This allows laptops to disconnect much quicker from the network after a backup. 13Feb07 kes Apply Eric's ClientRunScriptAfter patch to 2.0 and 2.1. 11Feb07 kes Optimize the use of the database a bit in the Status dir command. Only open it when needed, ensure that if any previous database was opened, it is closed. 10Feb07 kes Modify dbcheck to handle orphaned JobMedia, Path, Filename, and File records in 300K chunks to be more efficient. This idea came from Juan Luis Frances (if I remember right). 09Feb07 kes Update projects list. 08Feb07 kes Fix dird/ua_cmds.c so that a cancel command checks if the console is authorized to cancel the job. This fixes bug #767. kes Modify SD so that the VolCatJobs medium record is updated at the beginning of a Job rather than the end. This fixes bug #775 where exceeding MaxVolJobs caused jobs to fail. kes Added a mutex around getting and setting Volume information so that multiple simultaneous jobs will single thread. 07Feb07 Switch to using Subversion kes Remove src/pygtk-console/ from configure 06Feb07 kes Delete src/lib/btree.c from win32 build, then add rblist.c plus the entrypoints. kes Apply patch supplied that corrects debug print in canceling jobs for max run time. Supplied as part of bug #621, which was previously fixed. 05Feb07 kes Test on job_canceled() and sd_msg_thread_done inside loop starting the message thread to avoid a race condition. Fixes bug #771. kes Remove rl_catch_signal from console.c as it conflicted with the header definition. Fixes bug #765. ebl Fixes bug #766 about RunsOnClient = Yes (case sensitive) 04Feb07 kes Clarify some stored/acquire messages to indicate if the problem is with read or append. 02Feb07 kes Fix memory leak with storage ids in cats/sql_get.c kes Terminate watchdog earlier to avoid reference to released memory -- reported by Jason Austin. kes Move closing the database from jobq.c to the director daemon termination routine. This fixes memory leaks for shadow jobs (i.e. migration jobs). kes Free up the unique jobid chain items in migrate.c. This fixes a memory leak problem. kes Convert some ugly looking for statements to use foreach_alist in findlib/find.c. This will facilitate converting the structures to use dlist (for large include/exclude lists). kes Fix a bug in the btree.c and btree.h routines, then rename them rblist and add them to be built in src/lib. Include some new methods written by Rudolf Cejka that make the code more readable (hides some of the ugly casting). 26Jan07 (back port) kes Implement item #12 on project list -- quick release of FD by the SD. This is noted in more detail above. ;;; Kern;;;2007/1/28;;;14:30 Bacula Version 2.0.2 released: 28Jan08 kes Fix maxruntime bug #621. 27Jan07 kes Get current main CVS .specs for RedHat, SuSE, Mandrake. 26Jan07 ebl Implement the include JobID in spool file name project. kes Reorder projects file in order determined by Jan 2007 vote. kes Fix open of SQLite3 db where user does not have write permission so that DIR does not crash. Fixes bug #761. 25Jan07 kes Change 'Device not configured to autolabel' from INFO to WARNING. kes Fix jobq.c to restart failed jobs when Reschedule Times = 0 which should restart an indefinite # of times. kes Fix configure --help to print --with-mysql[=DIR]. Same for other DIR specifications. 23Jan07 rn Fix msvc build problems. Fix bacula.dll exports for msvc build. Fix path quoting in SQLite scripts. Fix problems with SHGetFolderPath. 19Jan07 kes Create patches/2.0.1-restart.patch. Fixes bug #755. 18Jan07 kes Fix Job restart on error bug that promotes an Inc to a Full backup. This should fix bug #755. kes Add qt-console and first cut of code. ;;; Kern;;;2007/1/13;;;14:30 Bacula Version 2.0.1 has been released to Source Forge. This is bug fix update to version 2.0.0 and contains the following fixes: - Fix Bacula->Documentation link on Win32 to point to index.html instead of bacula.html. Fixed bug #750. - Return JobId in db_get_job_record() when JobId==0. This should fix bug #741. - Do not release source pointers when restarting a failed job. - Add dynamic dll entry point for SHGetFolderPath to Win32 code. This *should* fix bug #747. - Modify winbacula.nsi to substitute with g bin_dir_cmd. Should fix bug #742. - Modify USTORE constructor to set an empty store_source string, and don't copy the store_source string in a cancel. Hopefully this will fix Arno's seg fault, bug #744. - Add back code to disable conio in configure. Fixes bug #743. - Correct the Options scanner in the FD to correctly handle SHA1 option, which was eating the next option. Reported by Sebastien Guilbaud. - Add code to indicate when the SD is spooling, spool wait, and despooling as requested by Alan Brown. ;;; Kern;;;2007/1/4;;;14:30 Bacula Version 2.0.0 has been released to Source Forge. There is an English Press Kit, written by Dan Langille at: http://www.bacula.org/about/press/presskit200.html.en and a German version, translated by Arno Lehmann at: http://www.bacula.org/about/press/presskit200.html.de ;;; Kern;;;2007/1/4;;;14:31 Release Notes for Bacula 2.0.0 Bacula code: Total files = 438 Total lines = 154,329 (*.h *.c *.in) ==== IMPORTANT Catalog update required ===== - The database format has been updated from what was used in Bacula 1.38.x. You must manually update your database before running Bacula 1.39.x or higher. If you are using Bacula supplied RPMs this is not necessary as the RPM does it automatically. Please backup your previous version of the database before running the update. The update script will be automatically installed in your scripts directory, or can also be found in /src/cats. It is called: ./update_bacula_tables It is necessary to run it only once the first time you move to a 2.0.0. Upgrading the Bacula version thereafter does not require updating the database again. Depending on the size of your database the script make take a bit of time, to run, but in general, it should be very fast. ==== IMPORTANT new Win32 install procedure ===== For Win32 migrations from versions prior to 1.39.0 nothing special needs to be done to upgrade. Everything should be taken care of automatically. The only thing not done is to delete the old C:\bacula directory mostly out of paranoia. ==== IMPORTANT miscellaneous ==== - The Gnome console program (gconsole) no longer functions as it should.More importantly, in restore mode, the restore tree is no longer shown in the left pane. I suspect this is due to incompatible changes in the GTK+ API, and hence have given up on gnome and gtk+, as this has already happened several times previously. At some point there will be a new GUI console. - The bacula-dir.conf directive Accept Any Volume has been removed because it was never implemented. You must delete all occurrence of this directive for the Director to run. The Storage daemon will automatically accept any valid Volume that you mount. - The --mandir ./configure option now points to the top level man directory. The man files will be installed under mandir/man8 and mandir/man1 as appropriate. - You *should* be able to use 1.38.x FDs with version 2.0.0 Director and SD providing you do not use any of the new features (runscript, data encryption). It seems to work here, but we do not guarantee it. - Your Director and SD must be simultaneously upgraded. - The restore command no longer uses the MediaType as the primary method of finding a suitable Storage device. Normally it will select the last device used to write a Volume. If no storage device is defined, it will use the old algorithm which selects the first Storage resource with the correct MediaType. - The MD5/SHA1 hash codes kept in the database are now kept in a binary format compatible with the rest of the world. If you are running verify jobs, you *must* do an InitCatalog or all files will show an MD5 difference. Also, authentication uses the new algorithm by default, but *should* accept connections from older components (FD) using the old non-compatible algorithm. This has been tested, but more more testing is still needed. - A stock SuSE 10.1 kernel may crash when Bacula runs and accesses the tape drive. Workaround, load the SuSE 10.1 version 2.6.16.21-0.25 kernel or later. The SuSE 10.2 Alpha 5 or later kernel also works. This problem may also be present in certain Fedora FC5 kernels. - If you have been using the data encryption feature of 1.39.x, please be aware that certain combinations of encryption and other options with version 1.39.0 through 1.39.26 created Volume data that cannot be restored. We strongly recommend that anyone using encryption carefully review his/her backups and at a minium do a Full backup with 1.39.28 of all encrypted data. Robert Nelson has identified and fixed it as follows: As of 1.39.27: No filters = Works fine Sparse = Works fine Compression = Works fine Encryption = Works fine Sparse + Compression = Works fine Sparse + Encryption = Restore broken Sparse + Compression + Encryption = Restore broken Compression + Encryption = Restore broken As of 1.39.28: Most combinations work, but some encrypted files are not properly restored. As of 1.39.30: All combinations work fine except Sparse+Encryption. Bottom line, if you are using data encryption, please test restoring data to be sure it all works correctly. - The current Volume format written by 1.39.22 is different from the format written by previous versions. The two formats are not compatible. Thus any DVDs written prior to 1.39.22 will be unreadable by version 1.39.22. - If you previously run a Bacula version prior to 1.39.30, the JobMedia data may not be correct for disk Volumes. This can cause incorrect seeking in versions after 1.39.30. If you experience what appears to be data I/O or integrity errors during restores, please add the following directive to your disk DEVICE resource in the Storage daemon conf file: Block Positioning = no This will turn off all seek requests during restores and avoid this problem. - VSS for Windows clients is now enabled by default. - Do not unload autochanger when doing "update slots" - Implement mount command for autochanger, see manual. New Features in 2.0.0: - Turn on disk seek code for restores. - There is a partial but reasonable translation of the Bacula messages for French. To install it, first configure and build Bacula, then as root run: cd /po make install If you have your locale set properly you will get the translated messages. - Bacula now support Migration jobs that are documented in a new Migration chapter in the manual http://www.bacula.org/dev-manual/Migration.html - Data encryption is now implemented and is documented in a new chapter of the manual at: http://www.bacula.org/dev-manual/Bacula_Data_Encryption.html - Additional support for removable devices. See the Requires Mount, Mount Point, Mount Command, and Unmount Commands in the Storage daemon configuration chapter: http://www.bacula.org/dev-manual/Storage_Daemon_Configuratio.html Also see the Edit Codes for Mount and Unmount Directives in the same chapter. - Switch the Win32 build from using Microsoft C++ to using mingw32 cross-compiling. The initial work was done by Howard Thomson, then tweaked by me. Robert Nelson then totally reworked the cross-compiling code so that it not only cross-compiles, but also compiles on Visual Studio, and at the same time, he added all the current Unix features to the FD, such as selection on drives, encryption support, building *all* the tools, ... - The Director and Storage daemon have now been ported to Win32. This code is working but should still be tested carefully before putting into production. - Bacula restore and bextract can now extract non-portable Win32 data to any client (including Unix/Linux clients). Of course, in doing so, the Microsoft specific permissions and ACLs will be lost. Thanks to Thorsten Engel for this code. - The 260 character limitation for Win32 paths name lengths is now eliminated thanks to Thorsten Engel. However, if you are using Volume Shadow Copy, please be careful to specify all the paths in the bacula-fd.conf file using a full path notation including the drive letter. - Eric Bollengier wrote new RunScript directives that includes the old RunBefore/AfterJob and ClientRunBefore/AfterJob features plus a *lot* more, allowing you to control just about every aspect of running scripts. See the manual for detailed documentation. http://www.bacula.org/dev-manual/Configuring_Director.html#5227 - SunOS ACLs should now work thanks to a patch from David Duchscher. - Apply days keyword patch from Alexander.Bergolth at wu-wien.ac.at If this patch is applied, the number of days can be specified with "list nextvol days=xx" or "status dir days=xx" This can be used to preview the next scheduled job (and the next tape to be used) on Fridays if there are no scheduled jobs during the weekend. - From Eric Bollengier. One can now using the bconsole wait command do: wait (wait for all jobs to stop) wait jobid=nn wait jobuid=unique id wait job=job-name - Volumes can now be set to Enable, Disable, or Archive. If they are not enabled, Volumes will not be mounted. Implement update volume enable=(on|off|true|false|archived|0|1|2) - Add Catalog message destination in Messages resource that puts the job report in the Log database table. - Writing/reading DVD Volumes is much more stable -- to the point of being useful. Thanks to Richard Mortimer. - Add enable/disable job=. This command prevents the specified job from being scheduled. Even when disabled, the job can be manually started from the console. - The database Id records should be 32/64 bit independent now. 64 bits can be enabled by changing one define and changing the appropriate table variable. Normally, you need 64 bits only for FileId. - Relative path specifications (i.e. ../xxx) are now permitted in the restore cd command. - When running multiple simultaneous jobs, most jobs that use spooling will now finish faster due to a mutex optimization made by Eric Bollengier. - Conf files containing UTF-8 marker at the head of the file as well as conf files containing Window cr/lf and Mac cr line termination characters are now accepted thanks to Robert Nelson. - Windows tray status windows are scrollable and resizable. - Win32 external script execution is much more flexible -- handles spaces in names better, ... - Lots of DVD fixes -- writing DVDs is now reported to work. - Fix opening of database in a restricted console to respect any Catalog ACL. - Much better automatic handling of multiple database catalogs in the restore command. - Permit multiple console/director resources in bconsole.conf. patch from Carsten Paeth calle@calle.in-berlin.de - Character substitution in Job/JobDefs WriteBootStrap. from Eric Bollengier. - Apply patch supplied in bug #656 to pass priority field in the run dialog to the Director in gnome console. - Add support of encrypted data stream to bscan from Eric. display data_len instead of data content (may be binary). - Add Enabled=xxx on update slots command. - Add host:port to connect failure messages to FD and SD from Dir/ - Add WhereACL to console ACL list. If nothing is specified, only the default is permitted for restore. Otherwise, *all* allows any path, or you can specify permitted paths. This should allow control over where users can restore files. This is untested. - Install man pages with 'make install'. - Add Media.Enabled flag to client backups for dotcmds.c - Enforce Media.Enabled=1 for a current restore to work - Require restore case 3 to have sqlquery permission to work. - Add -n option to bconsole to turn off conio -- used in bweb. - The bytes field in the terminated jobs part of the status command now reports in KB, MB, ... units. - When not descending into a directory, print the File= name that triggered it -- makes why not descending a bit clearer - Do not unload autochanger when doing "update slots" - Implement mount command for autochanger - Fix bug #462 incorrect error message printed when client script called from File= was not found. - Fix bug #558 (waiting for feedback) where Bacula needs too much time to do a rewind on Solaris when no tape is in the drive (Solaris does not have the detailed errno found on Linux). Added Solaris specific code. Note, this may apply to other OSes as well. - The examples directory has a new bacula_mail_summary.sh file that creates a single email summary of any number of jobs. Submitted by Andrew J. Millar. - Add nagios plugin to the examples directory. Submitted by Christian Masopust. - Modify most restore error messages to be queued so that they appear at the end of the job rather than mixed with the restore listing where they could be "lost". - Apply patch supplied by user (slightly modified) to fix correct detection of holes in block devices and FIFOs. Bug # 506. - Added a report.pl program to the examples directory from Jonas Bjorklund. - Add two new queries to query.sql provided by Arno. One list volumes known to the Storage device, and the other lists volumes possibly needing replacement (error, ...). - Implement new code for changing userid and group at startup. This should get Bacula into the correct groups. - Implement support for removable filesystems -- device type directive and mount, unmount directives. - Transfer rates are now presented in a more readable format thanks to a user submission. - SD is now aware of what volumes are mounted. More information is printed in the Status report. You must take much more care now to unmount devices prior to removing tapes that Bacula has open or prior to changing a magazine. Don't forget to do a mount afterwards or the device will be blocked. - The Pool Maximum Volumes directive is now respected in all places. - A Storage device can now be specified in a Pool resource. It will override all other Storage specifications. - Most but not all directives accept true/false in place of yes/no. - A patch from Karl Hakimian that reads JobIds, FileIndexes from a database table for restore. - There are a number of new tables. Some such as the Location table are designed for user use in doing Volume Management software. - Maximum Changer Wait, Maximum Open Wait, Maximum Rewind Wait to accept time qualifiers. - Implement jobuid to replace old usage of job in keywords as suggested by Eric Bollengier. - Implement write variables for Python to set Priority (anytime), and Job Level, only during JobInit event. - Use the keyword ujobid to mean the unique job id; job or jobname to mean the Job name given on the Name directive, and jobid to be the numeric (non-unique) job id. - Allow the SD to use multiple drives during a backup/restore (only one at a time). - Integrate addition of line count limitation to bsmtp -l from Sebastian Stark - Split the bacula start/start script into four files: bacula -- starts and stops calling other scripts bacula-ctl-dir -- starts/stops the director bacula-ctl-fd -- starts/stops the File daemon bacula-ctl-sd -- starts/stops the Storage daemon - Remove automatic case folding on Windows FDs. You must explicitly use the 'Ignore Case = yes' option. - Implement wild program in tools directory for testing wild-cards. Almost identical to the regex program. - Use the new bregex.c to implement Regex expressions on Win32. - Apply patch from Christopher Hull - Allow multiple connections to database with different parameters. - Invalidate the scheduler when doing a reload. Fixes seg fault, but still 60 second window. - Additional info in Reschedule message. - Use set_jcr_job_status() everywhere to prevent loss of cancel, error. - Display peer IP in FD if error from connecting DIR. - Don't increment file count for DIRBEGIN. - Replace illegal characters in Win32 filename by _. - Add SE_CREATE_PERMANENT_NAME privilege in Win32. - Hash hard link filenames rather than linked list (performance). - Fix for security failure in chdir on Win32. - Add CreateDirectoryA/W win32 API entry points. - programs to duplicate Bacula's base64 algorithm using standard routines. This fixes bugs #296, and 565. Patch submitted by author of bug #565. - Fixes to reloading the Dir conf file from Eric Bollengier and Christopher Hull. - Modify LICENSE to correct some problems pointed out by Debian. - Apply a patch submitted by cesarb in bug #606 to implement O_NOATIME support. O_NOATIME is a open() flag which makes it possible to read a file without updating the inode atime (and also without the inode ctime update which happens if you try to set the atime back to its previous value). It also prevents a race condition when two programs are reading the same file, but only one does not want to change the atime. It's most useful for backup programs and file integrity checkers (and bacula can fit on both categories). You enable it in the Bacula FileSet Options resource by setting: noatime = yes The effect of this option is similar to the keepatime option except it is more efficient and avoids modifying ctime. - Implement a pile of new man pages contributed by Jose Tallon. - Modify the database format for handling Migration jobs: Add PriorJobId, RealEndTime to Job table Delete MAC table Remove Stripe from JobMedia record (not used, wasting space) Add ScratchPoolId, RecyclePoolId, Enabled to Media record Add Cost to Location table. Enabled to Media table and Location table. - Security: harden authentication failure in FD by single threading errors and forcing a 6 second wait. - If using GCC (actually g++) add the following compiler flags -fno-strict-aliasing -fno-exceptions -fno-rtti - Turn on new bsnprintf() code. The reason for this code is to eliminate the security problems associated with using the system libraries print routines. - Implement job report that indicates where Storage and Pool came from -- with overrides and Pool storage and NextPool, it is all very complicated. - Add more detail (Storage, Device) to list of volumes printed for restore. - Add new VOLMGMT message class. No messages are yet sent with this class. - Improved Bacula rescue procedures -- see the Disaster Recovery chapter of the manual - Add spooling/despooling info in status output of SD. - Add Comment field to llist of a volume. - Allow true/false in many but not all yes/no directives. - The Bacula source code is now copyrighted by the Free Software Foundation Europe. The Developer's Guide documents the new procedures, and the LICENSE file has been updated. - Apply Jaime Ventura's patch that implements the Messages resource Mail On Success directive. - The Client returns its Version string, which is printed in the Job report. - The Client returns whether or not VSS and Encryption are used, which are printed in the Job report. Additional Features Added for Win32: - Added DriveType directive to the Director's Include Option FileSet resource. Allowed values are: fixed, removable, cdrom, and remote. There is only an implementation for Windows because it is the only platform that has the concept of drives. - Adds EnhancedWild directive to the Director's Include Option FileSet resource. Allowed values are: yes and no. When EnhancedWild is enabled then the processing of the Wild, WildDir and WildFile is changed in the following ways. Patterns conform to Posix \ is not a special character in character classification [] To match a - it must be the first or last character To match a ] it must be the first character fnmatch option FNM_FILE_NAME is specified * doesn't match a / so it won't match multiple directory levels in a path - Relative WildFile patterns (ones without a leading /) match against the filename portion. This in combination with the FNM_FILE_NAME fnmatch() flag makes directives such as WildFile = abc*.def work as expected. - Adds support for the shell's feature of brace expansion. Here is an example where braces allow 24 lines to be expressed in 5. # Exclude directories full of lots and lots of useless little files WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Cookies" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Recent" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/History" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temp" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temporary Internet Files" ;;;