Create 2.2.10 by backporting trunk SD reservation changes

[bacula/bacula] / bacula / ReleaseNotes

          Release Notes for Bacula 2.2.9 

  Bacula code: Total files = 498 Total lines = 167,596 (*.h *.c *.in)
  82 new files, 41,221 new lines of code, 208,380 lines of change from 2.0.3

This Director and Storage daemon must be upgraded at the same time,
but they should be compatible with all 2.0.x File daemons, unless you
use some of the new features that affect the FD.  In other words, you should
not have to upgrade all your File daemons when you upgrade. There is
no database upgrade needed from version 2.0.x to 2.2.0.

Version 2.2.10 is primarily a Storage daemon reservations system
  enhancement to 2.2.9-beta
16Apr08
kes  Permit multiple simultaneous restores -- experiment.
15Apr08
kes  Backport development stream SD reservation system changes.
     See below:
===
16Apr08
kes  Move final volume swapping code to DCR method and
     call it from acquire_for_read().
kes  When wrong volume is mounted during read, unload_autochanger.
14Apr08
kes  Stop searching for Volumes in SD askdir if DIR returns the
     same volume name twice in a row.
13Apr08
kes  Tweak the swap Volume from one drive to another code so that
     it now seems to work.
kes  Resolve several problems with recycling that broke the new
     code (these problems are in 2.2.9 as well).
kes  Remove the volume lock when calling the sysop code.   
kes  Don't mark a reserved volume with no writers as unused otherwise
     jobs fail.
12Apr08
kes  Rework class structures for VOLRES, DCR, and DEVICE to make
     the method names a bit more logical, and for more logically
     handling the responsibilities.
11Apr08
kes  Remove redundant code in terminating the scheduler that just
     causes a seg fault in many cases.
kes  Cleanup releasing a volume and make it a DCR method.
kes  Improve algorithm for detecting pre-reserved volume and
     swapping volumes.
10Apr08
kes  Tweak volume swapping code so it works.
kes  Correct name overloading in mount.c
kes  Replace released flag in VOLRES with reserved and add access
     methods for the VOLRES class.
kes  Prevent volume from being released while being swapped.
kes  Refactor parts of stored/mount.c
09Apr08
ebl  Add sanity checks for VolWriteTime and VolReadTime
ebl  Take care of bad clock changes while computing VolWriteTime
     and VolReadTime. This should fix or limit #1066
kes  Correct error string numbers in dird/catreq.c
kes  Restructure reserving, acquiring, and mounting volumes. Calls to
     autochanger are deferred for mount.c -- simplifes the code.
08Apr08
kes  Do not prune any running job. It just fails the job.
kes  Lock the volumes when changing dev->reserved_device and marking
     the volume unused otherwise the device can get reserved by
     another job before the volume is released, thus blocking it.
===                                        

Version 2.2.9 is a bug fix to version 2.2.8:
- Close bat console windows first to eliminate error message
  from the notifier.
- Remove useless code in Scheduler that causes seg fault on
  termination.
- Correct some SD catalog request error messages.
- Don't allow pruning of any running job.
- Turn off code in read_record that causes a seg fault in the SD
  when reading past an EOS_LABEL.
- Turn off unloading the autochanger in reserve.c as it just
   causes problems.
- Lock volumes when unreserving a device.
- Do not mark volume unused when recycling.
- Fix bug #1068 fixes a SD crash when using Virtual autochanger.
- Generate correct JobMedia records during spooling/despooling when
  running concurrent jobs.  Thanks to Tom Ivar Helbekkmo
  <tih@hamartun.priv.no> for excellent analysis and testing.
- *Massive* commit of all fixes and back ports to create 2.2.9
  Mostly SD reservations enhancements.
- Fix Win32 FD backup/restore memory leak due to improper termination
  of BackupRead/Write. This fixes bug# 1038.
- Fixes a StorageId problem with volumes after migration.
- Implements autostart scripts for Debian 
- Fix mtx-changer.in for broken Debian mt program.
- Apply doc fix from bug #1062.
- Resolve crash and improper restore wx-console Win32 restore
  GUI. Fixes bug #1065.
- Update Win32 wxWidgets to latest version.
- Attempt to do correct handling of Win32 mount points. Should
  fix bug #1046.
- Add const to AIX prototype for initgroups()
- Undo bug #1047 heap overrun fix as it is not correct.
- Fix to the JobMedia patch, which introduced a new problem.
- Fix creating first JobMedia record during Migration to include
  proper index. This caused slow restores of migrated jobs.
- Fix bug #1047, do not strip paths on symbolic links.
- Set catalog backup database and user name from values specified on 
  the ./configure line.
- Apply patch to correct bug #1031, about wrong pool source information
  in job report.

Version 2.2.8 is a bug fix to version 2.2.7:
- It fixes bugs: 1036, 1033(doc), 1028, 1040, 1045(doc), 1030,
  1042(partially), and possibly bugs 1018 and 1032.
  Bugs 1018 and 1032 involve problems with multiple-drive autochangers,
  and are difficult to reproduce.         
  See the ChangeLog for more details.

Version 2.2.7 has several new features and several important bug fixes
  since version 2.2.6:
- It fixes bugs: 1009, 1022, 1024, 1019, 1012, 1021, 1020
  1007, 1008.  For more details, please see the technotes-2.1 file.
- Fixed a seg fault reported by Frank Sweetser that depended on
  exact path lengths, but the problem was in bsnprintf.c
- Double quoting include filenames (@xxx) in Bacula conf files is now
  possible, thanks to a patch by Michael Stapelberg.
- You can pipe input to include filenames (@|prog) by using a vertical
  bar, thanks to the above patch.
- A daylight savings time bug in the Win32 bsmpt was fixed by a patch from 
  Nerijus Baliunas.
- Marc Cousins submitted a patch that permits building the PostgreSQL
  driver with version 8.3.
- The configure option --archivedir has been changed to --with-archivedir

Version 2.2.6 is a minor bug fix release to version 2.2.5
- It fixes bugs: #1003, 942, 982, 990(response only), 991,   
  993, 986, 976.
- Fix bat crash wen it cannot connect to Director.
- Fix joblist failure bug in bat.

Version 2.2.5 is a major bug fix release to version 2.2.4 
- It fixes the following bugs: #961, 962, 963, 969, 968, 960,
  964, (possibly 935 and 903), 953, 953, 967, 966, 965, 954, 
  957, 908, 958, and 955.
- It also improves listing performance problems in bat pointed
  out by Chris Howells.

Version 2.2.4 is a minor bug fix release to version 2.2.3
- Possible fix for authorization problems bug #953.
- Possible fix for bug #908.
- Add waits to multiple exit detection code to try to force pid
  file to always be deleted.
- Restore good dev.tar.gz to rescue set appropriate binary property.
  This fixes bug #950.
- Fix seg fault in error exit of acquire_for_read after unsuccessfully
  trying to switch drives by checking for blocking before unblocking.
  Fixes bug #906.
- Cancel storage daemon in all cases where FD reports error. This
  should fix virtually all cases of bug #920 and will ensure that Devices
  are released as soon as possible.
- Fix error message that was clobbered when Dir tells SD it does not
  have write permission on Volume.  This should fix a minor point
  in bug #942, but not the main problem.
- Fix migration code to get correct Volume name with multiple volumes 
  by skipping |.  Fixes bug #936.
- Implement patch supplied by Landon to fix bug #944 where using
  TLS with bconsole uses 99+% of the CPU.
- Fix bug #946 about "bacula-dir -t" which doesn't works
  as expected.
- Using "m" in bconsole will show messages as in prior versions
  and not memory usage.

- Note, you need GTK >= 2.10 to be able to link the Tray Monitor
  program.

Version 2.2.3 is a critical bug fix release to version 2.2.2
- Fix bug #935, and possibly also bug #903 where files were not
  restored. MediaId was not properly set in JobMedia record after 
  a Volume change. This bug is known to affect all versions from 
  2.0.0 on, and probably versions prior to that.  A detailed analysis
  of the bug can be found at:
    
      http://www.bacula.org/downloads/bug-395.txt
      
- Add ./configure search in qwt-qt4 for qwt package
- Apply Martin Simmons patch that should turn off the new API usage
  when batch insert is turned off allowing building on older
  PostgreSQLs.
- Note, you need GTK >= 2.10 to be able to link the Tray Monitor 
  program.

Version 2.2.2 is a minor bug fix release to 2.2.1
- Detect if new PosgreSQL batch insert API is present.
- Correct incorrect mempool call causing Director crash on older
  systems that do not have va_copy().
- Update spec files for 2.2.1 release

Version 2.2.1 is mainly a minor bug fix release to version 2.2.0,
  the main changes since 2.2.0 are:
- Fixed bugs #921, 930, and 923.
- More portable timezone offset in bsmtp thaks to Attila Fülöp.
- Fix several bat issues.                           
- Use GTK+ calls instead of egg files in tray-monitor.
- Win32 service routines rewritten
- Updated projects file
Please see the ChangeLog for details.

Notes on 2.2.0 release follow:

Areas requiring caution or testing:
- You must have the thread safe version of MySQL client libraries loaded
  to build with MySQL enabled.
- Volumes are pruned only when absolutely necessary -- this may cause
  your database to grow compared to prior Bacula versions.
- Solaris door and even port files are no longer restored (equivalent
    to what we do with sockets).
- SQLite is no longer supported for Solaris -- it fails too often with
  bus errors.  SQLite3 should work on Solaris.
- The default setting for SQLite3 is now 'PRAGMA synchronous=OFF'.
  This makes it run 30 times faster, but increases the possiblity
  of a corrupted database if your server shuts down unexpectedly.
  The default behavior can be changed in src/version.h
- Restore on Win32, and in particular on Vista is untested. Please
  test before relying on it.  It should backup and restore reparse
  points.
- Win32 servers are untested, and very likely not to work.

The major new features are:
- Much faster insertion of attributes (somewhere around 10 times),
  many thanks to Eric Bollengier and Marc Cousin. 
- First release of bat (Bacula Administration Tool).  Note to build
  bat you must explicitly enable it on the ./configure line, you
  must have the Qt4 version 4.2 or later libraries loaded, the qwt
  (Qt Graphics) package loaded, and qmake and the other Qt4 tools 
  must be available. Most of the implementation was done by Dirk Bartley.
- Red/Black restore in memory tree (500 times faster loading).
- The Regex Where code to allow easier relocation of restored files 
  thanks to Eric.
- Socket level heartbeat for all connections (untested).
- posix_fadvise() use in the FD to improve performance reduces
  swapping due to opening/reading lots of files. Win32 equivalent
  implemented.
- Much improved Volume reservation code that should eliminate most
  conflicts experienced in multiple drive autochangers.
- Simpler locking in the SD in the reservation system.
- Detection of file size/date change during backup if enabled.
- New Recycle Pool feature -- thanks to Eric.
- Efficient implementation of very large include/exclude lists.
- Volumes are no longer pruned during 'status dir'
- Pruning is now more efficient, and if a Volume is purged,
  during pruning, it is immediately discovered.
- License is now GPL v2 without modifications, fix a few copyright
  mistakes made when adding FSFE copyright notice.

New ./configure options:
- --enable-bwx-console    
- --enable-bat
- --with-qwt=
- --with-db-name=
- --with-db-user=
- --enable-batch-insert

New bconsole commands:
- exec
- memory
- update recyclepool
- .sql
- .api
- .pwd (in restore tree)
- restoreclient (keyword)
- backupclient  (keyword)
- regexwhere    (keyword)
- update jobid  (new command)
- recyclepool   (keyword) 

New directives:
- Heartbeat Interval (bconsole.conf)
- Heartbeat Interval (bacula-dir.conf in Director, Client, Storage)
- TLS Allowed CN     (bacula-dir.conf in Client)
- Regex Where        (bacula-dir.conf in Job)
- Strip Prefix       (bacula-dir.conf in Job)
- Add Prefix         (bacula-dir.conf in Job)
- Add Suffex         (bacula-dir.conf in Job)
- Recycle Pool       (bacula-dir.conf in Pool)
- FailJobOnError     (bacula-dir.conf in RunScript)
- CheckFileChanges   (bacula-dir.conf in FileSet)
- StripPath          (bacula-dir.conf in FileSet)

Other features or bug fixes:
- Fixed bugs: 916, 910, 917, 914, 906, 907, 
  842, 830, 893, 861, 888, 886, 807, 877, 872
  885, 887, 864, 874, 882, 881, 863, 859, 
  856, 854, 845, 847, 846, 809, 772, 
  822, 825, 824, 808, 802, 797, 798, 795, 789, 791, 
  788, 780, 763, 782, 612, 767, 775, 621, 772, 764, 
  766, 621, 761, 755, 750, 741, 747, 742, 744, and 743.
- Configure bat with --enable-bat. Define qwt libraries with
  --with-qwt=<dir>. See Installation chapter of manual for details.
- Create a depkgs-qt package that has both Qt4 and qwt, which are
  needed to build bat. Most modern Linux systems will have both
  these packages in the distro.
- Storage overrides delete all previous storage definitions instead
  of prepending.
- One should be able to mount and unmount removable devices if the
  Device resource has Removeable Media set and the mount and unmount
  directives are defined.
- ./configure will do a better job of searching for qwt libraries.
- The Win32 version can no longer be shutdown from the tray monitor.
  Use the command line or the Services panne.
- Keep prune_volumes() from pruning the whole Scratch pool.
- More debug output in status storage.
- Correct moving a Scratch volume from pool to pool (some
  critical columns were lost).
- Different locking in reservations and despooling systems,
  which means more micro-locking and less macro-locking, which
  should give a lot more concurrency at the expense of slightly
  (<0.1%) more overhead due to more locking/unlocking, but     
  concurrent jobs should run much faster.
- Additional drive reservation algorithm that should solve a lot of
  the problems experienced with multiple drive autochangers.
- Storage daemon status command enhanced to more clearly show Volume,
  pool and media type when a job is waiting.
- Made bsmtp work with more strict SMTP servers.
- Detect doubly freed buffers in smartall.c
- wx-console renamed to bwx-console (Fedora request)
- gnome-console renamed to bgnome-console (Fedora request)
- Migration preserves original job's FileSetId
- Spooling implemented for migration jobs.
- Config files can be read through a pipe, by specifying a leading |
  in front of the configuration path/filename.
- New memory command (bconsole) that will print the current Director's
  memory usage, and an in use buffer dump.
- Console name changed from *Console* to -Console- to accomodate Win32
  filename restrictions.  
- Corrected the Win32 ftruncate bug.
- Additional version information added to Job reports.
- New -8 option for bsmtp to specify UTF-8 encoding type.
- bsmtp will no longer add < > to target (from, to, cc) if there
  is already a < in the target.
- Prefer Mounted Volumes extended to mean a volume that is being
  mounted by another job.
- Runscript timing set to same as version 1.38.11
- New TLS Allowed CN directive(s) permitted.
- New bconsole 'exec' command to run external script/command.
- Turn on FreeBSD/OpenBSD code to set EOT model on tape.
- Drop from root done before database is opened.
- Database user and name can be configured with:
  --with-db-name=name --with-db-user=user
- Turn on wait_for_drive in mtx-changer script.
- Rework how bar codes are handled in mtx-changer script.
  This appears to have been an unreported bug.
- Updated nagios plugin supplied by Christian Masopust
- Better restricted console ACL checking.
- New Client Connect Wait directive in Storage daemon so that
  users can configure how long SD waits for FD connection.
- Bacula will no longer permit pruning of the currently running
  job (note, if other jobs are running, they may be pruned).
- Sockets are no longer restored.
- dbcheck works in 300K chunks so will typically run faster.
- Include/Exclude lists can now be efficiently handled for quite
  large sizes -- tens of thousands of entries.
- Several memory leaks in migration and database usage were fixed.
- New console keyword restoreclient, which specifies the client to
  which the restore will be sent. The client keyword specifies 
  the backup client. The restoreclient keyword is optional if it is
  not specified, the backup client will also be the restore client.
- The disk file size is now checked to ensure that it agrees with the
  catalog value before Bacula will append to the disk (same as tape
  and DVD).