- Fix scheduler to handle time skew (eg daylight savings).

[bacula/bacula] / bacula / kes-1.39

              Technical notes on version 1.39  
                        Kern Sibbald

General:
20Apr06
- Fix scheduler to handle time skew (eg daylight savings).
- Fix scheduler to use lock_jobs() to avoid most problems
  with reload. Window is now milliseconds.
19Apr06
- Apply patch from Christopher Hull
  - Allow multiple connections to database with different
    parameters.
  - Invalidate the scheduler when doing a reload. Fixes seg
    fault, but still 60 second window.
  - Additional info in Reschedule message.
  - Use set_jcr_job_status() everywhere to prevent loss of
    cancel, error.
  - Display peer IP in FD if error from connecting DIR.
  - Don't increment file count for DIRBEGIN.
  - Replace illegal characters in Win32 filename by _.
  - Add SE_CREATE_PERMANENT_NAME privilege in Win32.
  - Hash hard link filenames rather than linked list.
  - Fix for security failure in chdir on Win32.
  - Add CreateDirectoryA/W win32 API entry points.
- Add /silent option to Win32 FD for Install/Remove service.
- Always print Heap statistics in FD.
18Apr06
- Remove the -f option from the chown in Makefile.in for more
  portability.
- Change setting the group in Makefile.in to use chgrp for
  more portability.
- Implement a write_store and read_store to replace
  dirstore in reserve.c
- Implement a Bacula read/write lock for Python rather
  than using the Python lock to avoid recursive problems.
- Correct the uninstall directory names in filed/Makefile.in
  as reported by a user.
17Apr06
- Correct some problems with database creation (new tables).
- Replace a bunch of old 0x%x by %p.
- Get first cut of Migration with Job selection working.
16Apr06
- Change store_bit() to store_bool() for the Spooling in 
  dird_conf.c -- bug reported by Robert Nelson.
15Apr06
- Correct Makefile bug found by Dan Langille.
14Apr06
- Correct Makefile for Solaris /bin/sh
- Correct mtx-changer.in for Solaris /bin/sh
- Abort if a conf resource does not have a Name =
12Apr06
- Change the name of the regex program to bregex.
- Add the bwild program to the tools directory. It is similar
  to the bregex program.
- Implement create bregex.h and bregex.c in src/lib from the
  Python regexp program.  
- Use the new bregex.c to implement Regex expressions on Win32.
11Apr06
- More work on migration.
- Implement wild program in tools directory for testing
  wild-cards. Almost identical to the regex program.
- Up port VSS single thread locking code.
- Replace N_(..) by NT_(...) to inhibit translation of
  commands.
- Modify Makefile to change the permissions on Working Directory
  to 770 if the directory is created.
- Do not fail the Makefile if changing the permissions or 
  owner/group on WorkingDir fails.
- Remove early selection of Scratch Volumes in the recycling
  algorithm.
- Correct the old recycling algorithm so that Scratch Volumes
  are selected when looking for a Volume in the changer.
- Correct a typo in the Verify SQL reported by Joe Park.
10Apr06
- Remove automatic case folding on Windows FDs. You must
  explictly use the 'Ignore Case = yes' option.
- Remove the code added to 1.38.6 and 1.38.7 that pulls a
  scratch volume in an Autochanger early in the 'recycling'
  algorithm.  
09Apr06
- Make weof() and clrerror() methods of class DEVICE.
08Apr06
- Tweak license to include Microsoft restrictions.
- Move mysql.reconnect to after real_connect().  Thanks to
  Frank Sweetser for the patch.
- Disallow a backslash in a File = directive (Windows junk)
  unless the string is quoted.                     
- Apply Eric's patch to ua_label.c so that daemon protocol
  is not translated.
- Add NT_ definition for strings that should not be translated.
- Apply NT_() to ua_label.c

- Remove timed wait for VSS on Win2K3 as it is not yet
  implemented.
- Correct bacula.in script to reference bacula-ctl-xx in the
  sysconfig directory rather than the bin directory.
26Mar06
- Fix a long standing bug in the bacula start/stop scripts.
- Attempt to add the new bacula-ctl-* files to the rpm.
- Switch to using typeof() for list traversing to avoid
  as many FC5 g++ compiler warnings as possible.
25Mar06
- Split the bacula start/start script into four files:
   bacula         -- starts and stops calling other scripts
   bacula-ctl-dir -- starts/stops the director
   bacula-ctl-fd  -- starts/stops the File daemon
   bacula-ctl-sd  -- starts/stops the Storage daemon
24Mar06
- Create datestyle fix for PostgreSQL. Fixes bug #574.
- Correct editing of JobId from int to int64 in fd_cmds.c
- Eliminate FileSet name race with bash_spaces() and multiple
  threads by bashing in a local.
- Fix error return from 'use storage' to print a correct error
  message rather than nothing.
- Correct false re-read last block error message when two jobs 
  are simultaneously writing at the end of a tape.
- Simplify exit conditions in the reserve.c code to avoid  
  possible non-release of reservation_lock().
- Suffle lock order in reserve to avoid deadlock between
  reservation lock and device mutex.
21Mar06
- Initialize jcr mutex before first use. Thanks to Thorsten for
  tracking this down for me !!!! as it broke the Win32 build.
20Mar06
- Integrate addition of line count limitation to bsmtp -l from
  Sebastian Stark <stark at tuebingen.mpg.de>
17Mar06
- Implement regex test program in tools directory.
- Attempt to fix time problem with bsmtp with foreign langs.
- Add strip_trailing_newline() submitted by user.
- Implement regex matching in migrate.c
16Mar06
- Fix bug #537 to allow arbitrary time to mount a volume for
  restore, if polling is turned on.     
- If dir_user or dir-group is specified in ./configure apply it to
  the working-dir. Fixes bug #533.
- If rescheduling a job cancel the previous incarnation with the SD.
  Fixes bugs #566 and 557.
- Fix bug #567 do_message() definition type conflict.

14Mar06
- Add more jcr methods and make mutex and use_count private.
- Create lock/unlock methods for jcr.
- Fix PostgreSQL bug doing sql_data_seeek() by explicitly reading
  records to get to seek position.
- Integrate patch from bug #561 to correct conio.c signal definitions.
- Fix Rescheduling failed Jobs. Ensure that SD message thread 
  terminates correctly by doing pthread_kill(). Do not destroy
  SD cond wait variable between executions of the job. Use local
  mutex for cond variable to avoid blocking jcr chain. Fix poor 
  use of jcr use count in jobq.c for restarted jobs.
- Fix obsolete usage of foreach_dlist() to use foreach_jcr() in
  lib/jcr.c -- prevents locking the jcr chaing.
- Apply patch from bug #564, which corrects listing volumes with
  multiple autochangers. Apply same fix to next volume list.
- Fix bug #562 where restore bootstrap file is not unique.
- More details in implementing David's migration syntax proposal.
- Save and restore dcr when swapping drives so that if we have
  a write dcr it is not lost.
- Use new routine lock_reservations() to lock the reservations
  system, and call it while looking for a volume in askdir.c.
  This could possibly fix bug #543.
- Stop SD command loop if job is canceled.

08Mar06
- Remove old code from findlib/create_file.c
- Rename mac.c migrate.c
- Add user friendly display of VolBytes in job report.
- Rename target... to previous... to make it a bit easier to
  understand.
- Add selection type and selection pattern to Migration (idea
  given by David Boyes). 

04Mar06
- The po files should now be current.
- Fix new sql_use_result() code to properly release the
  buffers in all cases.
- Use the keyword ujobid to mean the unique job id; job or jobname
  to mean the Job name given on the Name directive, and jobid to
  be the numeric (non-unique) job id.
- Allow listing by any of the above.
- Add the user friendly job report code for reporting job elapsed time
  and rates with suffexes from John Kodis <kodis at comcast.net>.
- Add Priority and JobLevel as Python settable items.
- Use TEMPORARY table creation where the table is created by
  Bacula.
- Add new code submitted by Eric for waiting on specific jobid.
- Add ACL checking for the dot commands.
- Fix restore of writable FIFOs.
- Fix a bug in bpipe where the string was freed too early.
27Feb06
- Modify the Python class examples to inherit object -- new way
  of defining classes. Patch from Felix Schwarz.
- Implement jobuid to replace old usage of job in keywords as
  suggested by Eric Bollengier.
- Apply patch for enhancing wait from Eric Bollengier. On can now:
   wait  (wait for all jobs to stop)
   wait jobid=nn
   wait jobuid=unique id
   wait job=job-name
- Implement write variables for Python to set Priority (anytime), and
  Job Level, only during JobInit event.
26Feb06
- Fix the block listing bug pointed out by Arno.
- Update the po files at Eric's request.
24Feb06
- Fix Maximum Changer Wait, Maximum Open Wait, Maximum Rewind Wait to 
  accept time qualifiers.
- Make catalog respect ACL.
- Add recycle count to Media record.

Changes to 1.39.6-beta1 24Feb04
23Feb06
- Add Rudolf Cejka's new rc-chio-changer.
- Implement pulling Volume from Scratch Pool if the
  Volume is in the autochanger.
- Implement additional command arguments for update Volume.

Changes to 1.39.5 
22Feb06
- Back port changes to 1.38.5
- Fix recycle SQL for StorageId.
- Fix misplaced quote in mtx-changer script.
20Feb06
- Note. Your database must be updated, or you must create 
  a new database. I have not yet incremented the database level.
- Add Location table. 
- Add LocationId, DeviceId, and MediaTypeId to Media record.
18Feb06
- Implement create/get mac record in database for adding extended
  Migration data to the job record.
- Add new MAC table to update/make database scripts.
- Return Storage name used when getting VolumeNames for a job.
- Change bsr file keyword Storage to Device, which is more accurate.
- Ensure that Mac records are pruned/purged.
- Tweak SD tools to deal with changing media type.
- Integrate more dev.c subroutines as methods (e.g. strerror, bsr, ...)
- Pass pointer to dcr pointer to acquire_device_for_read() so
  that the subroutine can switch devices, and hence dcrs.
- Modify the multiple MediaType read code to re-use the same
  dcr when switching devices. This makes the code much more
  robust.
- Integrate patch from Karl Hakimian that reads JobIds, FileIndexes
  from a table for restore.
- Add Storage name to VolParams, but it really should be Device.
14Feb06
- Add disk-changer to scripts directory + configure/Makefile
- Eliminate PoolId from jcr -- it is in jcr->jr.PoolId
- Implement store_bit scanner to replace store_yesno.  Mostly done.
- Implement new store_bool that stores in a bool. 
- Add true/false to yes/no conf directives.
- Make first cut at changing appropriate store_yesno to store_bool.
- Complete implementation of Pool storage devices.
- Move starting clones to job.c
- Move create_restore_bootstrap_file() to job.c
- Make copy_storage() more general to be able to handle
  Pool storage.
- Cleanup a lot of migration code for manual running, including
  using Pool storage.
- Move getting a scratch Volume into a subroutine.
- Make all places a Volume that is added to a pool to 
  respect max vols.
- Fix bug in autochanger recycle code (improper edit 64 bit).
- Fix segfault in restore command when no value specified.
- Start adding code to handle multiple MediaTypes in restore.
- Eliminate race condition in getting Volume name for
  two drive autochanger.
- More debug code in autochanger.
- Add storage keyword to bootstrap file, add parsing.
- Move slot in bsr file into Volume record as there will be
  a different slot for each Volume.
- Create reserve.h
07Feb06
- Implement Pool storage overrides.
06Feb06
- Implement first cut of Migration.
- Implement mysql_use_result() from patch by Karl Hakimian.
  This reduces significantly the memory consumption during
  the restore tree building, and hence runs faster too.
- Implement StorageId patch supplied by user (reggie) in bug
  #536.  This should permit Bacula to work correctly with two
  autochangers.
- Implement Job listing variations suggested by a user.
- Move updating bootstrap code in backup.c to subroutine
  update_bootstrap_file().
- Add new job status elapsed time and bytes written user
  friendly job report output patch sent by John Kodis
  <kodis at comcast.ne>.
- Implement a storage list in Pools.
- Separate out setup_job() code from run_job().
- Get migration working -- lots of changes in mac.c in both
  DIR and SD.
- Apply patch from user (Eric Bollinger I think) that fixes a  
  DIR crash when no arguments are supplied to a dot command.
- Fix typo (strcpy->strcmp) in ua_prune.c as reported by Martin.
- Fix command arg name->volume in label command of gnome-console.
- Fix SD acquire.c to release correct DCR by explicitly testing
  on the dcr address rather than trying to devine if it is a read
  or write dcr.  This failed in error conditions when the device was
  not fully setup.
 
30Jan06
- Apply user supplied patch for more readable rate output
  in job report.
- Continue implementing migration.
- Implement support for removable filesystems in SD.
- Ensure that btraceback scripts can be read by anyone.
- Replace dvd-freespace and dvd-writepart by dvd-handler.
- Correct bug where canceling restore before the FD contacts
  the SD causes the drive to be left in read mode.
- Move ofline_or_rewind into DEVICE::close().
- Eliminate close_device.
- Convert several dev subroutines to methods (e.g. bsf,
  eod, ...)
- Eliminate force_close_device().
- Implement Device Type directive in Device resource that
  can have values File, Tape, Fifo, DVD, or Prog.
- Add has_cap() method to Device.
Changes to 1.39.4
17Jan06
- Add patch from bug #527 to allow RedHat user to specify
  options/user/group for starting each daemon in      
  /etc/sysconf/bacula.
16Jan06
- Add two new queries to query.sql provided by Arno. One
  list volumes known to the Storage device, and the other
  lists volumes possibly needing replacement (error, ...).
15Jan06
- Add periodic (every 24 hours) garbage collection of memory 
  pool by releasing free buffers.
14Jan06
- Correct bug counting sized (for display only) in smartall.c
- Print FD mempool stats if debug > 0 rather than 5.
12Jan06
- Make db_lock() mutex error fail the job rather than abort
  Bacula.  Canceling the job caused the mutex to fail.
- Correct bug in alist.c that re-allocated the list if the
  number of items goes to zero.
- Move the reservation system thread locking to the top level
  so that one job at a time tries all possible drives before
  waiting.
- Implement a reservation 'fail' message queue that is built         
  and destroyed on each pass through the reservation system.
  These messages are displayed in a 'Jobs waiting to reserve
  a drive' list during a 'status storage='.  Note, multiple
  messages will generally print for each JobId because they
  represent the different problems with either the same drive
  or different drives.  If this output proves too confusing
  of voluminous, I will display it only when debug level 1
  or greater is enabled in the SD.
11Jan06
- Add enable/disable job=<job-name>.  This command prevents
  the specified job from being scheduled. Even when disabled,
  the job can be manually started from the console.
- During 'update slots' clear all InChanger flags where the
  StorageId is zero (old Media records).

Beta release 1.38.4:
09Jan06
- Fix autochanger code to strip leading spaces from returned
  slots number. Remove bc from chio-changer.
- Back port a bit of 1.39 crypto code to reduce diffs.
- Fix first call to autochanger that missed close()ing the
  drive. Put close() just before each run_program().  Fixes
  Arno's changer bug.
07Jan06
- Add PoolId to Job record when updating it at job start time.
06Jan06
- Pull in more code from 1.39 so that there are fewer file
  differences (the new ua_dotcmds.c, base64.h, crypto.h
  hmac.c jcr.c (dird and lib) lib.h md5.h parse_conf.c 
  util.c. Aside from ua_dotcmds.c these are mostly crypto
  upgrades.
- Implement new method of walking the jcr chain. The
  incr/dec of the use_count is done within the walking
  routines.  This should prevent a jcr from being freed
  from under the walk routines.


Changes to 1.39.3:
04Jan06
- Start implementing Verify list output.
- Move the suitable_drive flag to a better place to prevent
  premature termination of the reservation if all drives
  are busy -- should fix Arno's diff/inc pool failures.
26Dec05
- Add mutex to single thread VSS code in Win32.
22Dec05
- Simplify code in askdir.c that waits for creating an appendable
  volume so that it can handle multiple returns from the wait
  code.
- Modify the wait code to permit multiple returns.
- Return a zero when "autochanger drives" is called and
  it is not an autochanger.
- Make rewind_dev() a method taking a DCR as an argument.
  This permits closing and reopening the drive if the
  rewind fails as happens if the drive was loaded while the
  file descriptor was open. This refreshes the file descriptor.
- Remove the ST_OPENED flag and always rely on fd < 0 for knowing
  if the device is open or not.  This should eliminate
  Arnos problem.
- Return error if reserve cannot find at least one suitable device.
- Make wait_for_sysop() return correct state information.
- Fix Win32 state file problem. write was not using compat
  code. This should fix bug #500.
21Dec05
- Modify gui on command to set only GUI mode and not batch.
- Modify .messages command to always print messages regardless
  of the mode.
- If GUI mode is on, suppress automatic printing of 
  You have messages. 
- Delete old bnet packet code.
- Ignore new BNET_START_SELECT and BNET_END_SELECT signals in
  wx-console.
- Modify restore command in wx-console to set gui on and to use
  only .messages instead of messages.  Hopefully this fixes bug
  #514.
- Fix seg fault in exit of acquire when canceling a job --
  reported by Wolfgang Denk
- Pull in latest reservation system changes from 1.38
- Make .messages command always print messages regardless
  of the automessages flag.
17Dec05
- Fix seg fault if user labels a drive directory bug #513
- Remove quotes around Version as it breaks things.
16Dec05
- Merge in Aleksandar Milivojevic's mods to the spec file.
- Apply sparse code fix for raw drives and fifos. Bug 506
- Thorsten fixed Unicode cd problem with wx-console bug 505.
14Dec05
- Correct reservation system to do a last ditch try
  for any mounted volume, then anyone anywhere.
- Add quotes around table Version because of
  error in MySQL 4.1.15 -- bug report submitted.
- Correct some minor problems with btape in the fill
  command.
- Updates to ssh-tunnel from Joshua Kugler.
- Added a report.pl program from Jonas Bjorklund.            
- Simplify the O_NONBLOCK open() code for tape drives,
  and always open nonblocking.
- Do not wait for open() if EIO returned (shouldn't happen).
- Eliminate 3 argument to tape open().
- Correct the slot # edited in the 3995 Bad autochanger unload
  message.
- With -S on bscan (show progress) do not divide by zero.
13Dec05
- Make cancel pthread_cond_signal() pthread_cond_broadcast().
- When dcr is freed, also broadcast dev->wait_next_vol signal.
- Remove unused code in wait_for_device.  
- Make wait_for_device() always return after 60 seconds of wait.

Changes to 1.39.2:
13Dec05
- Add stubs for non-crypto build.
12Dec05
- Use localhost if no network configured
11Dec05
- Eliminated duplicate MaxVolBytes in cat update -- bug 509.
- Remove debug print.
- Add bail_out in error during state file reading.
10Dec05
- Merge changes made to 1.38.3 into HEAD
- Add stubs for pygtk-console code
- Create Makefile.in for pygtk-console code
09Dec05
- Merge updates into 1.38 branch
- Update specs to include mysql4 define.
- Fix when attributes are sent, must be after binit().
- Stop read_record() if status not ok in second loop.
- Return rec->FileIndex in dcr->VolLastIndex for normal
  and partial records in read_record().  This allows bscan
  to get FileIndex at EOT correct.
- Fix butil.c to correctly set dcr -- fixes seg fault in bls.
08Dec05
- Fix Win32 built to work with new crypto code.
- Apply patch supplied by user (slightly modified) to fix
  correct detection of holes in block devices and FIFOs. 
  Bug # 506.
- Apply patch supplied by user (slightly modified) 
  to fix SD hang with multiple pools and bad client
  IP. Fixes bug # 508.
07Dec05
- Add nagios plugin to the examples directory. Submitted by
  Christian Masopust.
- Remove warning message about multiple saves of hardlinked files
  from find_one.c as it can generate too many warning messages.
- Modify most restore error messages to be queued so that they
  appear at the end of the job rather than mixted with the restore
  listing where they could be "lost".
06Dec05
- Reset timeout values before select() per patch from 
  Frank Sweetser for problems with non-blocking sockets.
- Unlink the state file if either reading or writing it gets
  errors.  Hopefully this will fix Win32 exit problems.
- Add sanity check in append.c to ensure that dcr is not NULL.
  This can happen if multiple drive autochanger SCSI control
  channel and drive indicies do not correspond.
05Dec05
- Get next volume from Scratch pool before creating a volume.
- Set new Pool defaults in Vol when moved from Scratch Pool.
- Remove argument from create_bacula_database for SQLite as it
  caused an error.
- Add back index code so that two drive autochangers can get
  a second tape.
- Change a bunch of debug levels to aid debugging autochangers.
- Fix reservation so that mutexes are properly applied.
- Rework reservation algorithm so that two drives can be used
  at the same time.
04Dec05
- Landon merged his data encription changes into the HEAD
- Apply days keyword patch from Alexander.Bergolth at wu-wien.ac.at 
  If this patch is applied, the number of days can be specified with
  "list nextvol days=xx"
  or
  "status dir days=xx"
  My use case is to be able to preview the next scheduled job (and the 
  next tape to be used) on fridays if there are no scheduled jobs during 
  the weekend.
Changes to 1.39.1:
03Dec05
- Fix font code in gnome2 console user patch. Fixes bug #501.
- Fix malformatted bnet error message that caused seg fault
  fixes bug 502
- Applied user patch to improve README.vc8 in src/win32.
29Nov05
- Add Migrate, Copy, Archive Job types (some where there)
- Correct some more editing of JobId's (for 64 bit compatibility).
- Ensure that StorageId is stored in Media record when ever possible.
- Add Migration Job to Job.
- Add Migration Time, Migration High Bytes, Migration Low Bytes
  Next Pool to Pool resource.
- Add more code to mac.c (migration archive copy).
- Change Start Storage daemon job to require read and write storage
  pointers.
- Pass read storage data to SD as well as write storage data.
- Remove old code from winservice.cpp
- Break on error in scan.
- Fix typo in signal.c
- Separate read/write DCR in SD.  Add jcr->read_dcr.
- Cleanup how find_device() works.
- Add read output to Status in SD.
Changes to 1.39.0:
23Nov05
- Add red-black btree routines
21Nov05
- Remove abs() in bfile.c so that it compiles on Solaris. 
  Bug #491.
20Nov05
- Fix crash in tray-monitor when daemon disconnects. Bug #479.
- Fix bnet-server bug found on OpenBSD. Bug #486
- Fix cancel failure bug. Bug #481
- Fix failure when Pool name has spaces. Bug #487   
- Fix SD crash in autochanger code. Mutex failure. Bug #488
- Fix a couple of free()s in src/filed/acl.c
- Fix memory overrun in bfile.c in building OS X resource
  fork filename. Bug #489 
- Add Pool name to SD status output.
14Nov05
- Apply SunOS patch for ACLs submitted by David Duchscher.                  
- Make sure to set storage before trying to set drive.
- Add bacula_mail_summary.sh to examples directory. It makes
  a single email summary of any number of jobs. Submitted
  by Adrew J. Millar.
- Make sure when we do a mount to unblock the device even
  if the drive could not be opened.  
13Nov05
- Remove the USE_WIN32STREAMEXTRACTION #defines (always on)
  and correct a few minor problems to make it build on Linux.
10Nov05
- Remove delete of CVS from all Makefiles
- Fix seg fault when clicking on Add button in wx-console
  restore panel.  Bug #470.
- Fix copyright date and URL typo -- bug #468.
- Change autostart install for FreeBSD to look for rc.conf  
  rather than rc.local as suggested fix for bug #466.
- Apply patch supplied by Eric Bollinger to fix PostgreSQL    
  grant on status. Bug #465
- Apply patch supplied by Eric Bollinger to fix PostgreSQL
  update script. Bug #464
- Tweak #ifdefing a bit in new Win32 stream code.
- Fix #ifdeffing for FD_NO_SEND_TEST.
- Add documentation of performance #defines