Release Notes for Bacula 2.4.0 Bacula code: Total files = 500 Total lines = 188,654 (*.h *.c *.in) This Director and Storage daemon must be upgraded at the same time, but they should be compatible with all 2.0.x File daemons, unless you use some of the new features that affect the FD. In other words, you should not have to upgrade all your File daemons when you upgrade. There is no database upgrade needed from version 2.0.x to 2.2.0 nor from version 2.2.x to version 2.4.0 ================== Warning !!!!!! ========================== As of Version 2.4.0, the Win32 FD should not descend into any reparse point directory or reparse mount points unless the specified directory is explicitly mentioned at the top level of FileSet (as with Unix). A file that is linked to another file will be backed up, much like Unix hardlinked files. If you want reparse directories backed up, you must explicitly include them in your FileSet. ============================================================== This release consists mainly of a number of bug fixes since 2.2.x but most importantly, a rewrite of significant portions of the Storage daemon reservation and autochanger code. This rewritten code is simpler, more stable, and works much better, particularly with multiple drive autochangers than the prior code did. That said, there are still a number of issues to be resolved. Note, the new code has not been tested in manual volume mount situations. Bugs fixed: 1095, 1086, 1091, 1070, 1041, 1083, 1078, 1066, 1068, 1038 1062, 1065, 1046, 1047, 1031 New features: - You can now build bat without the QWT libraries. Use --without-qwt as a ./configure option. Release Version 2.4.0 - Update version to 2.4.0 to reflect the magnitude of the SD changes. - Make sure to clear JS_WaitMedia when operator mounts a Volume. This fixes bug #1095. - Add create_postgresql_database.sql to Win32 installer. Should fix problem reported by Juilio Monteiro. - Get correct slot when auto unloading a device. This fixes bug #1086. - Fix possible seg fault if SQL error. - Fix renaming a Volume, which used the wrong Volume name. - If operator has rewind tape, print warning, release tape and try once more. If tape is positioned somewhere, something went wrong, so mark the tape in error and try once more. Previously this error was fatal, now it produces an error message. - Ensure correct volume name displayed during restore - Fix a few more Coverity reported problems. - Fix #1091 about bad output in estimate command. - Modify autochanger locking to attempt to avoid race conditions. - Make sure device not busy before doing label command. - Display open() errors except when polling. Previously too many were suppressed. This should fix bug #1070. - Fix Win32 reparse points. Bacula will not recurse into any reparse point directory, including mount points, unless the directory is explicitly mentioned at the top level (same as with Unix). A file that is linked to another file will be backed up -- much as Unix does for hardlinked files. This fixes bug #1041. - Remove StorageId test when pruning and recycling (Eric's changes). - Fix buffer overruns detected by Coverity. - Implement --without-qwt - Fix layout of restore tree dialog - Add configure bat QWT libraries, so that bat can be built with or without the QWT libaries. - Implement regression that explicitly tests swapping a Volume from one drive to another. - Enhance disk-changer to detect most error conditions. - Fix SD code so that it properly swaps a Volume between drives. This fixes bug #1083. - Prevent a Volume that is being swapped from being freed from the volume list. This will most likely fix, at least partially, bug #1083. - Fix strippath so that it does not get a buffer overrun and crash FD. This fixes bug #1078. - Make inability to change owner/group when creating a dir only a warning rather than an error. - Fix SQL query in migration code - Fix bat seg fault at termination. - Add Bacula generated version to bat about box. - Backport development stream SD reservation system changes. - When wrong volume is mounted during read, unload_autochanger. - Stop searching for Volumes in SD askdir if DIR returns the same volume name twice in a row. - Rework class structures for VOLRES, DCR, and DEVICE to make the method names a bit more logical, and for more logically handling the responsibilities. - Remove redundant code in terminating the scheduler that just causes a seg fault in many cases. - Improve algorithm for detecting pre-reserved volume and swapping volumes. - Prevent volume from being released while being swapped. - Refactor parts of stored/mount.c - Add sanity checks for VolWriteTime and VolReadTime - Take care of bad clock changes while computing VolWriteTime and VolReadTime. This should fix or limit #1066 - Correct error string numbers in dird/catreq.c - Restructure reserving, acquiring, and mounting volumes. Calls to autochanger are deferred for mount.c -- simplifes the code. - Do not prune any running job. It just fails the job. - Lock the volumes when changing dev->reserved_device and marking the volume unused otherwise the device can get reserved by another job before the volume is released, thus blocking it. - Correct some SD catalog request error messages. - Turn off code in read_record that causes a seg fault in the SD when reading past an EOS_LABEL. - Turn off unloading the autochanger in reserve.c as it just causes problems. - Lock volumes when unreserving a device. - Do not mark volume unused when recycling. - Fix bug #1068 fixes a SD crash when using Virtual autochanger. - Generate correct JobMedia records during spooling/despooling when running concurrent jobs. Thanks to Tom Ivar Helbekkmo for excellent analysis and testing. - *Massive* commit of all fixes and back ports to create 2.2.9 Mostly SD reservations enhancements. - Fix Win32 FD backup/restore memory leak due to improper termination of BackupRead/Write. This fixes bug# 1038. - Fixes a StorageId problem with volumes after migration. - Implements autostart scripts for Debian - Fix mtx-changer.in for broken Debian mt program. - Apply doc fix from bug #1062. - Resolve crash and improper restore wx-console Win32 restore GUI. Fixes bug #1065. - Update Win32 wxWidgets to latest version. - Attempt to do correct handling of Win32 mount points. Should fix bug #1046. - Add const to AIX prototype for initgroups() - Fix to the JobMedia patch, which introduced a new problem. - Fix creating first JobMedia record during Migration to include proper index. This caused slow restores of migrated jobs. - Fix bug #1047, do not strip paths on symbolic links. - Set catalog backup database and user name from values specified on the ./configure line. - Apply patch to correct bug #1031, about wrong pool source information in job report. ========================================================================= Version 2.2.8 is a bug fix to version 2.2.7: - It fixes bugs: 1036, 1033(doc), 1028, 1040, 1045(doc), 1030, 1042(partially), and possibly bugs 1018 and 1032. Bugs 1018 and 1032 involve problems with multiple-drive autochangers, and are difficult to reproduce. See the ChangeLog for more details. Version 2.2.7 has several new features and several important bug fixes since version 2.2.6: - It fixes bugs: 1009, 1022, 1024, 1019, 1012, 1021, 1020 1007, 1008. For more details, please see the technotes-2.1 file. - Fixed a seg fault reported by Frank Sweetser that depended on exact path lengths, but the problem was in bsnprintf.c - Double quoting include filenames (@xxx) in Bacula conf files is now possible, thanks to a patch by Michael Stapelberg. - You can pipe input to include filenames (@|prog) by using a vertical bar, thanks to the above patch. - A daylight savings time bug in the Win32 bsmpt was fixed by a patch from Nerijus Baliunas. - Marc Cousins submitted a patch that permits building the PostgreSQL driver with version 8.3. - The configure option --archivedir has been changed to --with-archivedir Version 2.2.6 is a minor bug fix release to version 2.2.5 - It fixes bugs: #1003, 942, 982, 990(response only), 991, 993, 986, 976. - Fix bat crash wen it cannot connect to Director. - Fix joblist failure bug in bat. Version 2.2.5 is a major bug fix release to version 2.2.4 - It fixes the following bugs: #961, 962, 963, 969, 968, 960, 964, (possibly 935 and 903), 953, 953, 967, 966, 965, 954, 957, 908, 958, and 955. - It also improves listing performance problems in bat pointed out by Chris Howells. Version 2.2.4 is a minor bug fix release to version 2.2.3 - Possible fix for authorization problems bug #953. - Possible fix for bug #908. - Add waits to multiple exit detection code to try to force pid file to always be deleted. - Restore good dev.tar.gz to rescue set appropriate binary property. This fixes bug #950. - Fix seg fault in error exit of acquire_for_read after unsuccessfully trying to switch drives by checking for blocking before unblocking. Fixes bug #906. - Cancel storage daemon in all cases where FD reports error. This should fix virtually all cases of bug #920 and will ensure that Devices are released as soon as possible. - Fix error message that was clobbered when Dir tells SD it does not have write permission on Volume. This should fix a minor point in bug #942, but not the main problem. - Fix migration code to get correct Volume name with multiple volumes by skipping |. Fixes bug #936. - Implement patch supplied by Landon to fix bug #944 where using TLS with bconsole uses 99+% of the CPU. - Fix bug #946 about 'bacula-dir -t' which doesn't works as expected. - Using 'm' in bconsole will show messages as in prior versions and not memory usage. - Note, you need GTK >= 2.10 to be able to link the Tray Monitor program. Version 2.2.3 is a critical bug fix release to version 2.2.2 - Fix bug #935, and possibly also bug #903 where files were not restored. MediaId was not properly set in JobMedia record after a Volume change. This bug is known to affect all versions from 2.0.0 on, and probably versions prior to that. A detailed analysis of the bug can be found at: http://www.bacula.org/downloads/bug-395.txt - Add ./configure search in qwt-qt4 for qwt package - Apply Martin Simmons patch that should turn off the new API usage when batch insert is turned off allowing building on older PostgreSQLs. - Note, you need GTK >= 2.10 to be able to link the Tray Monitor program. Version 2.2.2 is a minor bug fix release to 2.2.1 - Detect if new PosgreSQL batch insert API is present. - Correct incorrect mempool call causing Director crash on older systems that do not have va_copy(). - Update spec files for 2.2.1 release Version 2.2.1 is mainly a minor bug fix release to version 2.2.0, the main changes since 2.2.0 are: - Fixed bugs #921, 930, and 923. - More portable timezone offset in bsmtp thaks to Attila Fülöp. - Fix several bat issues. - Use GTK+ calls instead of egg files in tray-monitor. - Win32 service routines rewritten - Updated projects file Please see the ChangeLog for details. Notes on 2.2.0 release follow: Areas requiring caution or testing: - You must have the thread safe version of MySQL client libraries loaded to build with MySQL enabled. - Volumes are pruned only when absolutely necessary -- this may cause your database to grow compared to prior Bacula versions. - Solaris door and even port files are no longer restored (equivalent to what we do with sockets). - SQLite is no longer supported for Solaris -- it fails too often with bus errors. SQLite3 should work on Solaris. - The default setting for SQLite3 is now 'PRAGMA synchronous=OFF'. This makes it run 30 times faster, but increases the possiblity of a corrupted database if your server shuts down unexpectedly. The default behavior can be changed in src/version.h - Restore on Win32, and in particular on Vista is untested. Please test before relying on it. It should backup and restore reparse points. - Win32 servers are untested, and very likely not to work. The major new features are: - Much faster insertion of attributes (somewhere around 10 times), many thanks to Eric Bollengier and Marc Cousin. - First release of bat (Bacula Administration Tool). Note to build bat you must explicitly enable it on the ./configure line, you must have the Qt4 version 4.2 or later libraries loaded, the qwt (Qt Graphics) package loaded, and qmake and the other Qt4 tools must be available. Most of the implementation was done by Dirk Bartley. - Red/Black restore in memory tree (500 times faster loading). - The Regex Where code to allow easier relocation of restored files thanks to Eric. - Socket level heartbeat for all connections (untested). - posix_fadvise() use in the FD to improve performance reduces swapping due to opening/reading lots of files. Win32 equivalent implemented. - Much improved Volume reservation code that should eliminate most conflicts experienced in multiple drive autochangers. - Simpler locking in the SD in the reservation system. - Detection of file size/date change during backup if enabled. - New Recycle Pool feature -- thanks to Eric. - Efficient implementation of very large include/exclude lists. - Volumes are no longer pruned during 'status dir' - Pruning is now more efficient, and if a Volume is purged, during pruning, it is immediately discovered. - License is now GPL v2 without modifications, fix a few copyright mistakes made when adding FSFE copyright notice. New ./configure options: - --enable-bwx-console - --enable-bat - --with-qwt= - --with-db-name= - --with-db-user= - --enable-batch-insert New bconsole commands: - exec - memory - update recyclepool - .sql - .api - .pwd (in restore tree) - restoreclient (keyword) - backupclient (keyword) - regexwhere (keyword) - update jobid (new command) - recyclepool (keyword) New directives: - Heartbeat Interval (bconsole.conf) - Heartbeat Interval (bacula-dir.conf in Director, Client, Storage) - TLS Allowed CN (bacula-dir.conf in Client) - Regex Where (bacula-dir.conf in Job) - Strip Prefix (bacula-dir.conf in Job) - Add Prefix (bacula-dir.conf in Job) - Add Suffex (bacula-dir.conf in Job) - Recycle Pool (bacula-dir.conf in Pool) - FailJobOnError (bacula-dir.conf in RunScript) - CheckFileChanges (bacula-dir.conf in FileSet) - StripPath (bacula-dir.conf in FileSet) Other features or bug fixes: - Fixed bugs: 916, 910, 917, 914, 906, 907, 842, 830, 893, 861, 888, 886, 807, 877, 872 885, 887, 864, 874, 882, 881, 863, 859, 856, 854, 845, 847, 846, 809, 772, 822, 825, 824, 808, 802, 797, 798, 795, 789, 791, 788, 780, 763, 782, 612, 767, 775, 621, 772, 764, 766, 621, 761, 755, 750, 741, 747, 742, 744, and 743. - Configure bat with --enable-bat. Define qwt libraries with --with-qwt=. See Installation chapter of manual for details. - Create a depkgs-qt package that has both Qt4 and qwt, which are needed to build bat. Most modern Linux systems will have both these packages in the distro. - Storage overrides delete all previous storage definitions instead of prepending. - One should be able to mount and unmount removable devices if the Device resource has Removeable Media set and the mount and unmount directives are defined. - ./configure will do a better job of searching for qwt libraries. - The Win32 version can no longer be shutdown from the tray monitor. Use the command line or the Services panne. - Keep prune_volumes() from pruning the whole Scratch pool. - More debug output in status storage. - Correct moving a Scratch volume from pool to pool (some critical columns were lost). - Different locking in reservations and despooling systems, which means more micro-locking and less macro-locking, which should give a lot more concurrency at the expense of slightly (<0.1%) more overhead due to more locking/unlocking, but concurrent jobs should run much faster. - Additional drive reservation algorithm that should solve a lot of the problems experienced with multiple drive autochangers. - Storage daemon status command enhanced to more clearly show Volume, pool and media type when a job is waiting. - Made bsmtp work with more strict SMTP servers. - Detect doubly freed buffers in smartall.c - wx-console renamed to bwx-console (Fedora request) - gnome-console renamed to bgnome-console (Fedora request) - Migration preserves original job's FileSetId - Spooling implemented for migration jobs. - Config files can be read through a pipe, by specifying a leading | in front of the configuration path/filename. - New memory command (bconsole) that will print the current Director's memory usage, and an in use buffer dump. - Console name changed from *Console* to -Console- to accomodate Win32 filename restrictions. - Corrected the Win32 ftruncate bug. - Additional version information added to Job reports. - New -8 option for bsmtp to specify UTF-8 encoding type. - bsmtp will no longer add < > to target (from, to, cc) if there is already a < in the target. - Prefer Mounted Volumes extended to mean a volume that is being mounted by another job. - Runscript timing set to same as version 1.38.11 - New TLS Allowed CN directive(s) permitted. - New bconsole 'exec' command to run external script/command. - Turn on FreeBSD/OpenBSD code to set EOT model on tape. - Drop from root done before database is opened. - Database user and name can be configured with: --with-db-name=name --with-db-user=user - Turn on wait_for_drive in mtx-changer script. - Rework how bar codes are handled in mtx-changer script. This appears to have been an unreported bug. - Updated nagios plugin supplied by Christian Masopust - Better restricted console ACL checking. - New Client Connect Wait directive in Storage daemon so that users can configure how long SD waits for FD connection. - Bacula will no longer permit pruning of the currently running job (note, if other jobs are running, they may be pruned). - Sockets are no longer restored. - dbcheck works in 300K chunks so will typically run faster. - Include/Exclude lists can now be efficiently handled for quite large sizes -- tens of thousands of entries. - Several memory leaks in migration and database usage were fixed. - New console keyword restoreclient, which specifies the client to which the restore will be sent. The client keyword specifies the backup client. The restoreclient keyword is optional if it is not specified, the backup client will also be the restore client. - The disk file size is now checked to ensure that it agrees with the catalog value before Bacula will append to the disk (same as tape and DVD).