Initial revision

[bacula/docs] / docs / techlogs / 2003 / kes-1.30

              Technical notes on version 1.30 28Apr03
                      28 April 2003
                       Kern Sibbald

General:

Changes submitted this submission:
28Apr03
- Fix command history for gnome-console (must malloc).
- Add two cancel points in acquire_resources(). The job will not
  be immediately terminated in all cases.
24Apr03
- The new AutoMount code broke File archives -- fixed.
22Apr03
- Correct watchdog mutex race introduced yesterday.
- Add JobLevel to Admin run started by UA.
- Add -v to daemon startup scripts
21Apr03
- Implement bmicrosleep(sec, msec);
- Implement When for "run" command.
18Apr03
- Remove unsigned from socklen_t definition in hopes it will work better
  with older systems.
- Remove code that attempts to set PID dir and SUBSYS dir if they
  do not exist because it set them to sbindir!
- Modify mount to initially attempt to read the volume if Bacula
  wants a tape, none is mounted, and "Automatic Mount = yes" is set.
17Apr03
- Prohibit setting --sbindir and --with-subsys-dir the same.
- Fix missing argument that cause Kaboom in update slots.
16Apr03
- Fix orphaned bsock when UA gets error contacting FD or SD
- Nearly full implementation of Win Backup API, but it does not
  work due to lack of permission! Arggg!
14Apr03
- Implemented restoring directory permissions as they should be.
- Implemented reasonable regression script.  It found a number of
  restore errors.
- Implemented the following @ commands in the console that work directly
  in the console rather than in the Director.
    input - read input from a file
    output - write output to file
    tee    - tee output to file and terminal
    time   - print current time
    version - print current version
    exit    - quit
    quit    - quit
- Implemented new bfile io routines that will permit implementation of
  Windows native APIs for reading/writing files.
- By default always update hard links with the prefix, but do
  not update soft links.  It seems inconsistent, but it is what
  cp does.
- Implement wait command in Console that waits until no jobs are running. 
- Fixed the < code in Include/Excludes, which forgot to skip over the <.
- Do NOT attempt to chmod() a soft link as it will change the file behind
  the link!
- OOPS! Lutz ran into a problem. In attempting to prevent string overflows,
  I used bstrncpy() on a variable that was malloc'ed thus truncating 
  Volume names!  Arrrggggg!
- Lots of documentation of new features.
- Rework Volume name scanning in console, made much more logical
  and corrected a bug (confusion in calling sequence).
- Cleaned up a few error messages in cats adding more info.
- Add the IP address to error messages due to bad connects to servers.
- Implement default File output in config files so user can start
  saving right away without a tape drive.
- Protect inet_ntoa() with mutex in case it is not thread safe.
- Eliminate termcap from use in gnome console.
- Remove unused SD maximum volume files and maximum volume jobs. They
  are implemented in the Director.
- Make the default for Incremental and Differential saves to compare
  against both st_mtime and st_ctime rather than just st_mtime.
  This includes files moved or copied.
  Thanks to Matthias Wamser for bringing this fix to our attention.
- On Win32 clients, make a pass through the include/exclude patterns
  and change any back slashes to foward slashes. Prevents creating
  unusable directory names containing both conventions.
- Move ls -l output on restored files to M_RESTORED class.
- Make gnome-console compile correctly on RH8.0
- label barcodes now works.
- Implemented "update slots".
- Tweak btape "test" to always print suggestion for re-read last block.
- Implemented "Cleaning Prefix"
- Update alter_mysql_tables.in
- More work on barcodes.
- Zap VolHdr in SD when attempting to label a tape that is not
  there.  Prevents false tape names the next label command.
- First cut of bar code reading is implemented. It doesn't do
  anything but return the list to the Director.
- Implemented relabel command that relabels "Purged" Volumes.
- Check exit code in RunBeforeJob and err the job if it is non-zero.
- Remove old testsuite -- too complicated
- Print length when Authorization fails because of bad length.
- Fix problem of NumVols in Pool getting reset on startup.
- Implement full listing of DB records by listing them vertically
  instead of horizontally.
- Make changes to mysql scripts as suggested by Lutz Kittler
- Fixed code to write Uname to Client record.
- Fixed a problem (in btape) where Pmsg() was not printing.
- Moved re-read last block test to last in "test" command of btape.
- Lots of new documentation.
- Fixed newvol.c to handle retrying 10 times if the volume name already exists.
- Removed int_least16_t from sha1.h because it does not exist on some systems.
- Release job in SD if canceled and waiting on a mount (better cancellation).
- Prompt for Client in restore if not specified.
- Print "Selection is empty!" if no selection list found.
- Add new spec files and bacula.desktop from Scott.
- Update client every time a job is run.
- Add verbose option to daemons for printing more user error info.
- Test if console works with readline 4.3 (yes, it is OK).
- Release new depkgs
- Release new winbacula.
- Add Uname info to Client DB record.
- Improve error messages and make them more consistent when a non-existent
  Device is requested.
- Separated the -mwindows option so that only Windows programs
  are built with it (bacula-fd, ...) the tty tools such as console
  dbcheck, smtp, testfind, ... are now able to be run in a standard
  Windows DOS box.
- Add | and < options to Exclude the same as in Include.  
- Add typed in input to the text window in the gnome-console.
- Change function that gets the entry text (previously had orphaned buffer).
- Fix multiple Director problem in gnome-console (thanks to Lutz Kittler).
- Thanks to Renato, I was able to test Bacula on a FreeBSD tape drive.
  There are a number of significant differences: 1. reading less than
  the number of bytes in a record returns an I/O error.  2. ioctl(MTEOM)
  looses the file position. 3. Reading two two EOF marks (or ioctl(MTEOM))
  leave you positioned after the second EOF, so you must backspace file
  to be able to append.
- Added BSF at EOM = yes/no to Device resource to allow proper positioning
  at the end of a FreeBSD tape.
- Made btape "test" do much better testing of error conditions (i.e.
  it now ensures that the append went well). It will automatically
  detect problems and apply fixes and then retry the test, if it
  finally succeeds, it clearly says what directives need changing.
- Add Scott Barninger's rpm changes to the build environment
- Add changes to mtx-changer so users can add eject and sleep for
  certain autochangers.
- Implemented FreeBSD chflags (user defined flags).
- Turn restore errors during setting of owner and modes into
  warning messages -- for restoring files as non-root.
- Fix how prefixes are handled in restoring soft links.
- Modified btape "test" command to do only those things that
  Bacula actually does. There is much more explanation, ...
- Update manual.
- Finally had to back up to gcc version 2 from version 3 to avoid version
  3 nightmares.   
- Final cygwin tweaks.
- Move start time to *after* the resource locks are acquired.
- Unable to duplicate Phil's disabling of Bacula with nmap, but
  did make the authentication code a bit more conservative for
  dealing with bad input.
- Added code to the query command to escape all strings input
  before substitution and sending to the SQL engine.
- Escape user entered filenames for restore command.              
- Cleaned up the waiting code a bit -- using broadcast instead
  of signal and counting the waiters.
- Implement new pthreads semaphore code.
- #define new semaphore code rather than workq on USE_SEMAPHORE.
- Lots of improvements to the document to address recent support requests.
- Implement cycle through a set of tapes suggested by Eugeny
  Fisher with the "RecycleOldestVolume=yes" record in the
  Pool resource.  Basically this record causes Bacula to purge
  to oldest tape when no more tapes are found.
- Correct a number of small incorrect interactions between limit
  variables during recycling.
- Corrected a bug in db_create_media_record() where VolMaxFiles and
  VolMaxJobs was not written to the database.
- When the Director starts, the Pool record is updated in the
  database with the current contents of the Pool resource.
- Corrected bnet_connect() to immediately stop (rather than looping
  for the timeout period) if there is a fatal error (socket, or
  hostname to ip).
- Reworked "purge" code to make purging Volumes easier.
- Made "list volumes" list the volumes in all pools -- also
  "list volumes pool=Default" does not produce and error message.
- List Pool record after doing "update pool".
- Remove pid file code from Win32 -- not really necessary.
- Make bnet handle null jcr during cancel rather than crash.
- Add CygwinInstall.bat and CygwinUnInstall.bat
- Add db_lock() around newvol.c code to prevent race condition if multiple
  callers want a new Volume name.
- Lots of cleanup to Win32 code, with additional error messages.
- Make Bacula work on Win95 (test for GetAttributesEx).
- Add better error messages when end of media is reached or volume
  capacity execed.
- Turned off signal catching in readline(), necessary to keep console
  from crashing on ctl-Z with RedHat 8.0.  Thanks to David Craigon for
  testing this.
- Make the 3rd and hopefully final change to the Finclude structures.
  This new version permits multiple sets of options (more code to be
  written) to be applied against the same set of files. Thus one can
  have options that with match of *.gz and a different set of options
  for *.c, ...
- Integrated GNOME Console font resource code supplied by Phil Stracchino.
- Check for job_cancelled() in bnet_connect() code to stop wait loop if
  client not available.
- Fix early end of file scanning conf file in lex.c, which previously
  caused ABORT -- now reports error.  This could happen with an unterminated
  string for example.
- Move Maximum File Size code before write and detect error on writing EOF.
- Additional fix for Solaris 2.6 and a bdb.c fix submitted by Armin Buehler.
- Added detection for Solaris 2.6, which uses older setsockopt() calls.
- Defined sockopt_val_t for setsockopt() calls.
- Added fixes sent by Bevan Anderson that fix multiple connects to FD
  (I zap keys for security, so must put back dummy key).  Also a fix
  to the Internal database that wrote garbage after the filename in
  the database.
- Back out the __SVR4 changes.
- Add automatic configuration of socklen_t
- Attempt to fix problems reported by Lutz with multiple simultaneous
  open file Volumes (experimental code). Serialized acquire. This may
  fix the problem, but more thought and testing is necessary.
- Add table of "supported" autochanger models.
- Add Solaris 2.6 (__SRV4) changes sent by Peter Schmitz.
- Correct tape selection code in SD (|| => &&). Thanks to Chuck Hemker
  for the patch.
- Eliminate FileOptions. Implement new Finclude and Fexclude that
  have file options contained in it.  New structure must be transmitted
  to FD.
- Split Include/Exclude into new inc_conf.c file.
- Cleanup new Include/Exclude and FileOptions.  Structures in Director
  now correct.  Must transmit FileOptions to FD.
- WARNING:  With the adddress and port code in the Catalog you MUST
  remove old address= and dport= records!!!!!
- Added code to remove cancelled jobs from the workq -- needs testing.
- Added first cut AIX from output James MacLean sent me.
- Second cut of FileOptions.
- First cut of parsing FileOptions and Counters.
- Added address:port for MySQL as well as socket for local access.
- Fixed job.c in filed to properly handle excluded files. Apparently
  I changed the daemon protocol but forgot to update the code.
- Enhance testfind to handle include and exclude files
- Fix getdomainname() prototype for Darwin
- Added new -u and -g options for specifying userid and groupid to
  use when running, so that Bacula can reduce its privileges.
- When Bacula was hanging due to an NFS volume being down, I 
  fixed a few places in the File daemon where is should have immediately
  terminated the connection instead of waiting for the Director to do so.
- Added first cut support for Darwin.
- Temporarily comment out the O_NOFOLLOW to avoid possible subtle problems.
- Implemented O_NOFOLLOW in creating files in create_file.c to prevent
  creating a file at the end of a symbolic link.
- Use chown() if lchown() does not exist (e.g. Darwin).
- Always close stdin on startup to avoid having /dev/console attached.
- Change all DATE occurrences to BDATE because it is used by Cygwin headers.
- Add printing of Volume names in SD status output.
- Display all open devices in device chain in SD status output.
- All changing Pool in console run command.
- Thanks to Eric Bollengier for pointing out that the run_program()
  return status was not correctly generated. Now fixed (I hope).
- Corrected crash in Internal Database getting Volumes.
- Flush all daemon messages at the end of every job.
- Fix Install.bat script so that Bacula restarts after reboot on WinNT/2K systems.
- Minor changes in the gnome-console directory.
- Integrated in my old Tcl/Tk code into src/tconsole and moved it up
  from C to C++.  Pretty crude, but it is a beginning.
- Close syslog() %n exploit in message.c
- Edit space before each line in gconsole.
- Added INCEXE structure so we can have FileOptions.
- Added support for multiple simultaneous open file volumes.
- Fix hard linked files so that the one saved is always restored.
- Add * to restore "dir" listing to indicate marked files.
- Add ability to make md5sum and sha1sum in lib directory.
- Work a bit more on the new daemon protocol.
- Use unmask of 022 or more restive
- Create File volumes with 0640 permissions
- Added support for SHA1 signature. Need to modify DB to have type.
- Document SHA1.
- Work a bit on getting proper child status from bpipe calls.
- Added Ludwig's mtx-changer to the examples/devices directory.
- Added a Warning not to use the Internal Database when it is initialized.
- Compiled and tested sha1, and added it to the library.
- Added code to print the "load slot" status after autoloading.