Release Notes for Bacula 1.39.20 Bacula code: Total files = 448 Total lines = 150,330 (*.h *.c *.in) Version 1.39.20: - The basic features of Migration are now complete. - There is a Migration chapter in the doc. - There is an Encryption chapter in the doc. New Features in 1.39.20: - Add -n option to bconsole to turn off conio. - The bytes field in the terminated jobs part of the status command now reports in KB, MB, ... units. - When not descending into a directory, print the File= name that triggered it -- makes why not descending a bit clearer - Do not unload autochanger when doing "update slots" - Implement mount command for autochanger Fixes for 1.39.20: - Writing DVDs should work much better thans to Richard Mortimer. - Fix bug #462 incorrect error message printed when client script called from File= was not found. - Fix bug #558 (waiting for feedback) where Bacula needs too much time to do a rewind on Solaris when no tape is in the drive (Solaris does not have the detailed errno found on Linux). Added Solaris specific code. Note, this may apply to other OSes as well. - Many cleanups/enhancements for Win32 systems. - Correct improperly formated list command output reported by Dan. - Eric fixed RunScripts, which I (Kern) had broken in the last beta. - Correct Client migration SQL as pointed out by Marc. ========================================== Warning for version 1.39.18 and greater: - The documentation is almost non-existent for all new features except RunScripts (written and documented by Eric Bollengier). - If you have an old pre 1.39.18 database, you must upgrade it using the upgrade_bacula_tables script. Please backup your database before upgrading just in case. - You *should* be able to use 1.38.x FDs with version 1.39.18 Director and SD providing you do not use any of the new features (runscript, data encryption). It seems to work here, but we do not guarantee it. - Your Director and SD must be simultaneously upgraded. - The restore command no longer uses the MediaType as the primary method of finding a suitable Storage device. Normally it will select the last device used to write a Volume. If no storage device is defined, it will use the old algorithm which selects the first Storage resource with the correct MediaType. - The MD5/SHA1 hash codes kept in the database are now kept in a binary format compatible with the rest of the world. If you are running verify jobs, you *must* do an InitCatalog or all files will show an MD5 difference. Also, authentication uses the new algorithm by default, but *should* accept connections from older components (FD) using the old non-compatible algorithm. This has been tested, but more more testing is still needed. - The --mandir ./configure option now points to the top level man directory. The man files will be installed under mandir/man8 and mandir/man1 as appropriate. - The "Accept Any Volume" directive has been removed. It was never implemented. Bacula will always accept any valid tape that is in the drive for appending. - To install the 1.39.18 Win32 FD, you *must* first stop any previous version that is running in a DOS shell with: net stop bacula or select the 'Close Bacula' menu item from the Bacula tray icon, you should then save your bacula-fd.conf file, and either uninstall (using Uninstall.bat in the c:\bacula directory) or remove the old installation by manually in a DOS shell in the Bacula directory enter: bacula-fd /remove after that you can proceed with the installation. My experience is that it does not properly create the new bacula-fd.conf file, so copy the one saved to the directory you chose to install Bacula, or to the directory chosen by the Installer to put the bacula-fd.conf file. Then you can start Bacula in a DOS shell with: net start baculafd or use the Windows Control Panel->Administrative Tools->Component Services menu item. A Lot of New features for 1.39.18: - Switch the Win32 build from using Microsoft C++ to using mingw32 cross-compiling. The inital work was done by Howard Thomson, then tweaked by me. Robert Nelson then totally reworked the cross-compiling code so that it not only cross-compiles, but also compiles on Visual Studio, and at the same time, he added all the current Unix features to the FD, such as selection on drives, encryption support, building *all* the tools, ... Finally, he also ported the Directory and the Storage daemon to Win32. - Data encryption done in the Client is now supported due to code submitted by Landon Fuller. - Bacula restore and bextract can now extract non-portable Win32 data to any client (including Unix/Linux clients). Of course, in doing so, the Microsoft specific permissions and ACLs will be lost. Thanks to Thorsten Engel for this code. - The 260 character limitation for Win32 paths name lengths is now eliminated thanks to Thorsted Engel. - Eric Bollengier wrote new RunScript directives that includes the old RunBefore/AfterJob and ClientRunBefore/AfterJob features plus a *lot* more, allowing you to control just about every aspect of running scripts. See the manual for detailed documentation. - The examples directory has a new bacula_mail_summary.sh file that creates a single email summary of any number of jobs. Submitted by Andrew J. Millar. - SunOS ACLs should now work thanks to a patch from David Duchscher. - The database Id records should be 32/64 bit independent now. 64 bits can be enabled by changing one define, but this has never been tested. - Apply days keyword patch from Alexander.Bergolth at wu-wien.ac.at If this patch is applied, the number of days can be specified with "list nextvol days=xx" or "status dir days=xx" My use case is to be able to preview the next scheduled job (and the next tape to be used) on fridays if there are no scheduled jobs during the weekend. This patch was probably back ported to 1.38.x - Add nagios plugin to the examples directory. Submitted by Christian Masopust. - Modify most restore error messages to be queued so that they appear at the end of the job rather than mixted with the restore listing where they could be "lost". - Apply patch supplied by user (slightly modified) to fix correct detection of holes in block devices and FIFOs. Bug # 506. - Added a report.pl program to the examples directory from Jonas Bjorklund. - Add enable/disable job=. This command prevents the specified job from being scheduled. Even when disabled, the job can be manually started from the console. - Add two new queries to query.sql provided by Arno. One list volumes known to the Storage device, and the other lists volumes possibly needing replacement (error, ...). - Implement new code for changing userid and group at startup. This should get Bacula into the correct groups. - Implement support for removable filesystems. - Transfer rates are now presented in a more readable format thanks to a user submission. - SD is now aware of what volumes are mounted. More information is printed in the Status report. - The Pool Maximum Volumes directive is now respected in all places. - A Storage device can now be specified in a Pool resource. It will override all other Storage specifications. - Most but not all directives accept true/false in place of yes/no. - A patch from Karl Hakimian that reads JobIds, FileIndexes from a database table for restore. - There are a number of new tables. Some such as the Location table are designed for user use in doing Volume Management software. - Maximum Changer Wait, Maximum Open Wait, Maximum Rewind Wait to accept time qualifiers. - Implement jobuid to replace old usage of job in keywords as suggested by Eric Bollengier. - Apply patch for enhancing wait from Eric Bollengier. One can now: wait (wait for all jobs to stop) wait jobid=nn wait jobuid=unique id wait job=job-name - Implement write variables for Python to set Priority (anytime), and Job Level, only during JobInit event. - Use the keyword ujobid to mean the unique job id; job or jobname to mean the Job name given on the Name directive, and jobid to be the numeric (non-unique) job id. - Allow the SD to use multiple drives during a backup/restore (only one at a time). This is not well tested. - Integrate addition of line count limitation to bsmtp -l from Sebastian Stark - Split the bacula start/start script into four files: bacula -- starts and stops calling other scripts bacula-ctl-dir -- starts/stops the director bacula-ctl-fd -- starts/stops the File daemon bacula-ctl-sd -- starts/stops the Storage daemon - Remove automatic case folding on Windows FDs. You must explictly use the 'Ignore Case = yes' option. - Implement wild program in tools directory for testing wild-cards. Almost identical to the regex program. - Use the new bregex.c to implement Regex expressions on Win32. - Apply patch from Christopher Hull - Allow multiple connections to database with different parameters. - Invalidate the scheduler when doing a reload. Fixes seg fault, but still 60 second window. - Additional info in Reschedule message. - Use set_jcr_job_status() everywhere to prevent loss of cancel, error. - Display peer IP in FD if error from connecting DIR. - Don't increment file count for DIRBEGIN. - Replace illegal characters in Win32 filename by _. - Add SE_CREATE_PERMANENT_NAME privilege in Win32. - Hash hard link filenames rather than linked list (performance). - Fix for security failure in chdir on Win32. - Add CreateDirectoryA/W win32 API entry points. - Add /silent option to Win32 FD for Install/Remove service. programs to duplicate Bacula's base64 algorithm using standard routines. This fixes bugs #296, and 565. Patch submitted by author of bug #565. - Fixes to reloading the Dir conf file from Eric Bollengier and Christopher Hull. - Modify LICENSE to correct some problems pointed out by Debian. - Apply a patch submitted by cesarb in bug #606 to implement O_NOATIME support. O_NOATIME is a open() flag which makes it possible to read a file without updating the inode atime (and also without the inode ctime update which happens if you try to set the atime back to its previous value). It also prevents a race condition when two programs are reading the same file, but only one does not want to change the atime. It's most useful for backup programs and file integrity checkers (and bacula can fit on both categories). You enable it in the Bacula FileSet Options resource by setting: noatime = yes The effect of this option is similar to the keepatime option except it is more efficient and avoids modifying ctime. - Implement a pile of new man pages contributed by Jose Tallon. - Implement a number of user supplied patches for DVD writing. - Modify the database format for handling Migration jobs: Add PriorJobId, RealEndTime to Job table Delete MAC table Remove Stripe from JobMedia record (not used, wasting space) Add ScratchPoolId, RecyclePoolId, Enabled to Media record Add Cost to Location table. Enabled to Media table and Location table. - Security: harden authentication failure in FD by single threading errors and forcing a 6 second wait. - If using GCC (actually g++) add the following compiler flags -fno-strict-aliasing -fno-exceptions -fno-rtti - Turn on new bsnprintf() code. This could cause print output to look different. - Implement job report that indicates where Storage and Pool came from -- with overrides and Pool storage and NextPool, it is all very complicated. - Add more detail (Storage, Device) to list of volumes printed for restore. - Add new VOLMGMT message class. No messages are yet sent with this class. - Add Catalog message destination in Messages resource that puts the job report in the Log database table. - Spend a lot of time integrating mkcdrec with the rescue disk. - Add spooling/despooling info in status output of SD. - Add Comment field to llist of a volume. - Allow true/false in some yes/no questions. - Implement update volume enable=(on|off|true|false|archived|0|1|2) Additional Features Added for Win32: Added DriveType directive to the Director's Include Option FileSet resource. Allowed values are: fixed, removable, cdrom, and remote. There is only an implementation for Windows because it is the only platform that has the concept of drives. Adds EnhancedWild directive to the Director's Include Option FileSet resource. Allowed values are: yes and no. When EnhancedWild is enabled then the processing of the Wild, WildDir and WildFile is changed in the following ways. Patterns conform to Posix \ is not a special character in character classification [] To match a - it must be the first or last character To match a ] it must be the first character fnmatch option FNM_FILE_NAME is specified * doesn't match a / so it won't match multiple directory levels in a path Relative WildFile patterns (ones without a leading /) match against the filename portion. This in combination with the FNM_FILE_NAME fnmatch() flag makes directives such as WildFile = abc*.def work as expected. Adds support for the shell's feature of brace expansion. Here is an example where braces allow 24 lines to be expressed in 5. # Exclude directories full of lots and lots of useless little files WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Cookies" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Recent" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/History" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temp" WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temporary Internet Files" Here are some hints if you want to play with the Windows Director and/or Storage daemon. Changer and Tape device names in Windows are Changer0, Changer1, etc and Tape0, Tape1, etc. If there isn't a device driver loaded for the Changer then you need to use the address :::. Port is the SCSI Adapter Number, Bus is the Bus Number on the adapter (usually 0 since most adapters only have one bus), Target is the device's Target Device ID, Lun is the Logical Unit Number. You must specify DeviceType = tape in the Device resource in bacula-sd.conf since detection doesn't currently work.