Kern's ToDo List
- 31 May 2003
+ 30 September 2003
Documentation to do: (any release a little bit at a time)
- Document running a test version.
hours of operation.
- Lookup HP cleaning recommendations.
- Lookup HP tape replacement recommendations (see trouble shooting autochanger)
-- Document FInclude ...
-- Document need to add "-u root" to most of MySQL script calls
- (./create_mys... ./make_my...).
-
+- Create a man page for each binary (Debian package requirement).
Testing to do: (painful)
- that ALL console command line options work and are always implemented
- blocksize recognition code.
-- multiple simultaneous Volumes
- Test if rewind at end of tape waits for tape to rewind.
- Test cancel at EOM.
- Test not zeroing Autochanger slot when it is wrong.
-- Test multiple simultaneous Volumes
-
- Figure out how to use ssh or stunnel to protect Bacula communications.
-For 1.31 release:
-- Implement FileSet VolIndex.
-- Sort JobIds entered into recover tree.
-- The bsr for Dan's job has file indexes covering the whole range rather
- than only the range contained on the volume.
- Constrain FileIndex to be within range for Volume.
-- Fix Verify VolumeToCatalog to use BSRs -- it is broken.
-- Should Bacula make an Append tape as Purged when purging?
+For 1.33 Testing/Documentation:
+- bextract is sending everything to the log file ****FIXME****
+- Document new records in Director. SDAddress SDDeviceName, SDPassword.
+ FDPassword, FDAddress, DBAddress, DBPort, DBPassword.
+- Document new Include/Exclude ...
+- Add test of exclusion, test multiple Include {} statements.
+- Add counter variable test.
+- Document ln -sf /usr/lib/libncurses.so /usr/lib/libtermcap.so
+ and install the esound-dev package for compiling Console on SuSE.
+- Add an example of using a FIFO in dirdconf.wml
+- Add an item to the FAQ about running jobs in different timezones.
+
+For 1.32c
+- Add Volume name to "I cannot write on this volume because"
+
+For 1.33
+- Write your PID file and chwon root:wheel before drop.
+- Make sure there is no symlink in a file before creating a
+ file (attack).
+- Look at mktemp or mkstemp(3).
+ mktemp and mkstemp create files with predictable names too. That's
+ not the vulnerability. The vulnerability is in creating files without
+ using the O_EXCL flag, which means "only create this file if it doesn't
+ exist, including if the file is a dangling symlink."
+
+ It is *NOT* enough to do the equivalent of
+
+ if doesn't exist $filename
+ then create $filename
+
+ because between the test and the create another process could have
+ gotten the CPU and created the file. You must use atomic functions
+ (those that don't get interrupted by other processes) and O_EXCL is
+ the only way for this particular example.
+- Keep last 5 or 10 completed jobs and show them in a similar
+ list.
+- Make a Running Jobs: output similar to current Scheduled Jobs:
+- Use ioctl() fsf if it exists. Figure out where we are from
+ the mt_status command. Use slow fsf only if other does
+ not work.
+- Add flag to write only one EOF mark on the tape.
+- Mount a tape that is not right for the job (wrong # files on tape)
+ Bacula asks for another tape, fix problems with first tape and
+ say "mount". All works OK, but status shows:
+ Device /dev/nst0 open but no Bacula volume is mounted.
+ Total Bytes=1,153,820,213 Blocks=17,888 Bytes/block=64,502
+ Positioned at File=9 Block=3,951
+ Full Backup job Rufus.2003-10-26_16.45.31 using Volume "DLT-24Oct03" on device /dev/nst0
+ Files=21,003 Bytes=253,954,408 Bytes/sec=2,919,016
+ FDReadSeqNo=192,134 in_msg=129830 out_msg=5 fd=7
+- Automatically create pools, but instead of looking for what
+ in in Job records, walk through the pool resources.
+- Check and double check tree code, why does it take so long?
+- Upgrade to cygwin 1.5
+- Fix time difference problem between Bacula and Client
+ so that everything is in GMT.
+- Finish implementation of Verify=DiskToCatalog
+- Change console to bconsole.
+- Change smtp to bsmtp.
+- Possibly up network buffers to 65K.
+- Add device name to "Current Volume not acceptable because ..."
+- Make sure that Bacula rechecks the tape after the 20 min wait.
+- Set IO_NOWAIT on Bacula TCP/IP packets.
+- Try doing a raw partition backup and restore by mounting a
+ Windows partition.
+- Report CVS problems to SourceForge.
+- Implement .consolerc for Console
+- Is it really important to make Job name the same to find the
+ Full backup to avoid promoting an Incremental job?
+- Start label, then run job when tape labeled, it should broadcast.
+- Zap illegal characters in job name for mail files (e.g. /).
+- From Lars Köllers:
+ Yes, it would allow to highly automatic the request for new tapes. If a
+ tape is empty, bacula reads the barcodes (native or simulated), and if
+ an unused tape is found, it runs the label command with all the
+ necessary parameters.
+
+ By the way can bacula automatically "move" an empry/purged volume say
+ in the "short" pool to the "long" pool if this pool runs out of volume
+ space?
+- Either restrict the characters in a name, or fix the problem
+ emailing with names containing / (smtp command line breaks).
+- Eliminate orphaned jobs: dbcheck, normal pruning, delete job command.
+ Hm. Well, there are the remaining orphaned job records:
+
+ | 105 | Llioness Save | 0000-00-00 00:00:00 | B | D | 0 | 0 | f |
+ | 110 | Llioness Save | 0000-00-00 00:00:00 | B | I | 0 | 0 | f |
+ | 115 | Llioness Save | 2003-09-10 02:22:03 | B | I | 0 | 0 | A |
+ | 128 | Catalog Save | 2003-09-11 03:53:32 | B | I | 0 | 0 | C |
+ | 131 | Catalog Save | 0000-00-00 00:00:00 | B | I | 0 | 0 | f |
+
+ As you can see, three of the five are failures. I already deleted the
+ one restore and one other failure using the by-client option. Deciding
+ what is an orphaned job is a tricky problem though, I agree. All these
+ records have or had 0 files/ 0 bytes, except for the restore. With no
+ files, of course, I don't know of the job ever actually becomes
+ associated with a Volume.
+
+ (I'm not sure if this is documented anywhere -- what are the meanings of
+ all the possible JobStatus codes?)
+
+ Looking at my database, it appears to me as though all the "orphaned"
+ jobs fit into one of two categories:
+
+ 1) The Job record has a StartTime but no EndTime, and the job is not
+ currently running;
+ or
+ 2) The Job record has an EndTime, indicating that it completed, but
+ it has no associated JobMedia record.
+
+
+ This does suggest an approach. If failed jobs (or jobs that, for some
+ other reason, write no files) are associated with a volume via a
+ JobMedia record, then they should be purged when the associated volume
+ is purged. I see two ways to handle jobs that are NOT associated with a
+ specific volume:
+
+ 1) purge them automatically whenever any volume is manually purged;
+ or
+ 2) add an option to the purge command to manually purge all jobs with
+ no associated volume.
+
+ I think Restore jobs also fall into category 2 above .... so one might
+ want to make that "The Job record has an EndTime,, but no associated
+ JobMedia record, and is not a Restore job."
+- Implement RestoreJobRetention? Maybe better "JobRetention" in a Job,
+ which would take precidence over the Catalog "JobRetention".
+- Implement Label Format in Add and Label console commands.
+- make "btape /tmp" work.
+- Make sure a rescheduled job is properly reported by status.
+- Walk through the Pool records rather than the Job records
+ in dird.c to create/update pools.
+- What to do about "list files job=xxx".
+- Implement scan: for every slot it finds, zero the slot of
+ Volume other volume having that slot.
+- When job rescheduled, status gives is waiting for Client Rufus
+ to connect to Storage File. Dir needs to inform SD that job
+ is rescheduled.
+- Fix get_storage_from_media_type (ua_restore) to use command line
+ storage=
+- Enhance "update slots" to include a "scan" feature
+ scan 1; scan 1-5; scan 1,2,4 ... to update the catalog
+- Allow a slot or range of slots on the label barcodes command.
+- Don't print "Warning: Wrong Volume mounted ..." if mounting second volume.
+- Make Dmsg look at global before calling subroutine.
+- Enable trace output at runtime for Win32
+- Make sure that Volumes are recycled based on "Least recently used"
+ rather than lowest MediaId.
+- Available volumes for autochangers (see patrick@baanboard.com 3 Sep 03
+ and 4 Sep) scan slots.
+- Get MySQL 3.23.58
+- Get and test MySQL 4.0
+- Do a complete audit of all pthreads_mutex, cond, ... to ensure that
+ any that are dynamically initialized are destroyed when no longer used.
+- Write a mini-readline with history and editing.
+- Look at how fuser works and /proc/PID/fd that is how Nic found the
+ file descriptor leak in Bacula.
+- Implement WrapCounters in Counters.
+- Turn on SIGHUP in dird.c and test.
+- Use system dependent calls to get more precise info on tape errors.
+- Add heartbeat from FD to SD if hb interval expires.
+- Suppress read error on blank tape when doing a label.
+- Can we dynamically change FileSets?
+- If pool specified to label command and Label Format is specified,
+ automatically generate the Volume name.
+- Take a careful look a the Basic recycling algorithm. When Bacula
+ chooses, the order should be:
+ - Look for Append
+ - Look for Recycle or Purged
+ - Prune volumes
+ - Look for purged
+ Instead of using lowest media Id, find the least recently used
+ volume.
+
+ When the tape is mounted and Bacula requests the status
+ - Do everything possible to use it.
+
+ Define a "available" status, which is the currently mounted
+ Volume and all volumes that are currently in the autochanger.
+
+- Why can't SQL do the filename sort for restore?
+- Is a pool specification really needed for a restore? Yes, and
+ you may want to exclude archive Pools.
+- Look at libkse (man kse) for FreeBSD threading.
+- Look into Microsoft Volume Shadowcopy Service VSS for backing
+ up system state components (Active Directory, System Volume, ...)
+- Add ExhautiveRestoreSearch
+- Look at the possibility of loading only the necessary
+ data into the restore tree (i.e. do it one directory at a
+ time as the user walks through the tree).
+- Possibly use the hash code if the user selects all for a restore command.
+- Orphaned Dir buffer at parse_conf.c:373 => store_dir
+- Implement some way for the File daemon to contact the Director
+ to start a job or pass its DHCP obtained IP number.
+- Implement multiple Consoles.
+- Add Console usr permissions.
+- Fix "restore all" to bypass building the tree.
+- Fix restore to list errors if Invalid block found, and if # files
+ restored does not match # expected.
+- Prohibit backing up archive device (findlib/find_one.c:128)
+- Implement Release Device in the Job resource to unmount a drive.
+- Implement Acquire Device in the Job resource to mount a drive,
+ be sure this works with admin jobs so that the user can get
+ prompted to insert the correct tape. Possibly some way to say to
+ run the job but don't save the files.
+- Add JobName= to VerifyToCatalog so that all verifies can be done at the end.
+- Implement FileOptions (see end of this document)
+- Make things like list where a file is saved case independent for
+ Windows.
+- Edit the Client/Storage name into authentication failure messages.
+- Implement migrate
+- Implement a PostgreSQL driver.
+- Bacula needs to propagate SD errors.
+ > > cluster-dir: Start Backup JobId 252, Job=REUTERS.2003-08-11_15.04.12
+ > > prod4-sd: REUTERS.2003-08-11_15.04.12 Error: Write error on device
+ /dev/nst0. ERR=Input/output error.
+ > > prod4-sd: REUTERS.2003-08-11_15.04.12 Error: Re-read of last block failed.
+ Last block=5162 Current block=5164.
+ > > prod4-sd: End of medium on Volume "REU007" Bytes=16,303,521,933
+
+- Use autochanger to handle multiple devices.
+- Fix packet too big problem. This is most likely a Windows TCP stack
+ problem.
+- Add SuSE install doc to list.
+- Check and rechedk "Invalid block number"
+- Make bextract release the drive properly between tapes
+ so that an autochanger can be made to work.
+- User wants to NOT backup up certain big files (email files).
+- Maybe remove multiple simultaneous devices code in SD.
+- On Windows with very long path names, it may be impossible to create
+ a file (and thus restore it) because the total length is too long.
+ We must cd into the directory then create the file without the
+ full path name.
+- lstat() is not going to work on Win32 for testing date.
+- Something is not right in last block of fill command.
+- Implement a Recycle command
+- Add FileSet to command line arguments for restore.
+- Allow multiple Storage specifications (or multiple names on
+ a single Storage specification) in the Job record. Thus a job
+ can be backed up to a number of storage devices.
+- Add client name to cram-md5 challenge so Director can immediately
+ verify if it is the correct client.
+- Implement ClientRunBeforeJob and ClientRunAfterJob.
+- Add JobLevel in FD status (but make sure it is defined).
+- Audit all UA commands to ensure that we always prompt where possible.
+- Restrict characters permitted in a Resource name, and don't permit
+ duplicate names.
+- Check Jmsg in bnet, may not work, must dup bsock.
+- Suppress Job Name in Jmsg for console
+- Create Pools that are referenced in a Run statement at startup if possible.
+- Use runbeforejob to unload, then reload a volume previously used,
+ then the next job run gets an error reading the drive.
+- Make bootstrap filename unique.
- Test a second language e.g. french.
- Start working on Base jobs.
- Make "make binary-release" work from any directory.
-- Document c:/working directory better than /working directory.
-- Unsaved Flag in Job record.
-- Base Flag in Job record.
- Implement UnsavedFiles DB record.
-- Use switch() in backup.c and restore.c in FD instead of giant if statement.
- Implement argc/argv for daemon command line scanning using table driven
stuff below.
- Implement table driven single argc/argv scanner to pickup all arguments.
Much like xxx_conf.c scan table.
keyword, handler(store_routine), store_address, code, flags, default.
-- Make | and < work on FD side.
-- Pass prefix_links to FD.
-- Implement a M_SECURITY message class.
- From Phil Stracchino:
It would probably be a per-client option, and would be called
something like, say, "Automatically purge obsoleted jobs". What it
all Differential and Incremental jobs obsoleted by that Full backup.
This would let people minimize the number of tapes they're keeping on
hand without having to master the art of retention times.
-- Prohibit backing up archive device (findlib/find_one.c:128)
-- Make Restore report an error if FD or SD term codes are not OK.
-- Add JobLevel in FD status (but make sure it is defined).
-- Make Pool resource handle Counter resources.
-- Restrict characters permitted in a Resource name, and don't permit
- duplicate names.
- Implement new serialize subroutines
send(socket, "string", &Vol, "uint32", &i, NULL)
-- Audit all UA commands to ensure that we always prompt where possible.
-
-
-After 1.31:
+- Scratch Pool where the volumes can be re-assigned to any Pool.
+- Implement a M_SECURITY message class.
+- Implement forward spacing block/file: position_device(bsr) --
+ just before read_block_from_device();
+- When doing a Backup send all attributes back to the Director, who
+ would then figure out what files have been deleted.
- Currently in mount.c:236 the SD simply creates a Volume. It should have
explicit permission to do so. It should also mark the tape in error
if there is an error.
- Make sure all restore counters are working correctly in the FD.
- SD Bytes Read is wrong.
-- Configure mtx-changer to have correct path to mtx.
- Look at ALL higher level routines that call block.c to be sure
they don't expect something in errmsg.
- Investigate doing RAW backup of Win32 partition.
-- Add JobName= to VerifyToCatalog so that all verifies can be done at the end.
- Add thread specific data to hold the jcr -- send error messages from
low level routines by accessing it and using Jmsg().
- Cancel waiting for Client connect in SD if FD goes away.
-- Testing Tibs job erred and hung director on Storage resource. This was
- because there were a whole pile of jobs hanging around in the SD
- waiting for a connection from the FD that was never coming.
-- Possibly update all client records at startup.
- Add Progress command that periodically reports the progress of
a job or all jobs.
-- Implement "Reschedule OnError=yes interval=nnn times=xxx"
- One block was orphaned in the SD probably after cancel.
-- Add all command line arguments to "update", e.g. slot=nn volStatus=append, ...
- Examine Bare Metal restore problem (a FD crash exists somewhere ...).
- Implement single pane restore (much like the Gftp panes).
- Implement Automatic Mount even in operator wait.
- Implement create "FileSet"?
-- Implement Release Device in the Job resource to unmount a drive.
-- Implement Acquire Device in the Job resource to mount a drive,
- be sure this works with admin jobs so that the user can get
- prompted to insert the correct tape. Possibly some way to say to
- run the job but don't save the files.
-- Implement all command line args on run.
-- Implement command line "restore" args.
-- Implement "restore current select=no"
- Fix watchdog pthread crash on Win32 (this is pthread_kill() Cygwin bug)
- Implement "scratch pool" where tapes are defined and can be
taken by any pool that needs them.
disk only when the tape is full, then when a tape is hung move
it to tape.
- Implement a relocatable bacula.spec
-- Allow multiple Storage specifications (or multiple names on
- a single Storage specification) in the Job record. Thus a job
- can be backed up to a number of storage devices.
- Implement dump/print label to UA
- Add prefixlinks to where or not where absolute links to FD.
- Issue message to mount a new tape before the rewind.
to the user, who would then use "mount" as described above
once he had actually inserted the disk.
-- Make some way so that if a machine is skipped because it is not up
- that Bacula will continue retrying for a specified period of time --
- periodically.
- If tape is marked read-only, then try opening it read-only rather than
failing, and remember that it cannot be written.
- Refine SD waiting output:
- Figure out some way to estimate output size and to avoid splitting
a backup across two Volumes -- this could be useful for writing CDROMs
where you really prefer not to have it split -- not serious.
-- Add RunBeforeJob and RunAfterJob to the Client program.
- Have SD compute MD5 or SHA1 and compare to what FD computes.
- Make VolumeToCatalog calculate an MD5 or SHA1 from the
actual data on the Volume and compare it.
-- Implement FileOptions (see end of this document)
- Implement Bacula plugins -- design API
- Make bcopy read through bad tape records.
-- Fix read_record to handle multiple sessions.
- Program files (i.e. execute a program to read/write files).
Pass read date of last backup, size of file last time.
- Add Signature type to File DB record.
- Check if we can increase Bacula FD priorty in Win2000
- Make sure the MaxVolFiles is fully implemented in SD
- Check if both CatalogFiles and UseCatalog are set to SD.
-- Need return status on read_cb() from read_records(). Need multiple
- records -- one per Job, maybe a JCR or some other structure with
- a block and a record.
- Figure out how to do a bare metal Windows restore
- Possibly add email to Watchdog if drive is unmounted too
long and a job is waiting on the drive.
-- Use read_record.c in SD code.
-- Restore program that errors in SD due to no tape reports
+- Restore program that errs in SD due to no tape, reports
OK incorrectly in output.
- After unmount, if restore job started, ask to mount.
- Convert all %x substitution variables, which are hard to remember
- Compare tape to Client files (attributes, or attributes and data)
- Make all database Ids 64 bit.
- Write an applet for Linux.
-- Add estimate to Console commands
-- Implement new daemon communications protocol.
+- Implement new inter-daemon communications protocol.
- Allow console commands to detach or run in background.
- Fix status delay on storage daemon during rewind.
- Add SD message variables to control operator wait time
- Set flag for uname -a. Add to Volume label.
- Implement throttled work queue.
- Restore files modified after date
-- Restore file modified before date
-- Restore -- do nothing but show what would happen
- SET LD_RUN_PATH=$HOME/mysql/lib/mysql
- Implement Restore FileSet=
- Create a protocol.h and protocol.c where all protocol messages
- Make bcopy copy with a single tape drive.
- Permit changing ownership during restore.
+- From Phil:
+ > My suggestion: Add a feature on the systray menu-icon menu to request
+ > an immediate backup now. This would be useful for laptop users who may
+ > not be on the network when the regular scheduled backup is run.
+ >
+ > My wife's suggestion: Add a setting to the win32 client to allow it to
+ > shut down the machine after backup is complete (after, of course,
+ > displaying a "System will shut down in one minute, click here to cancel"
+ > warning dialog). This would be useful for sites that want user
+ > woorkstations to be shut down overnight to save power.
+ >
+
+- From Terry Manderson <terry@apnic.net>
+ jobset { # new structure
+ name = "monthlyUnixBoxen"
+ type = backup
+ level = full
+ jobs = "wakame;durian;soy;wasabi;miso" #new!
+ schedule = monthly
+ storage = DLT
+ messages = Standard
+ pool = MonthlyPool
+ priority = 10
+ }
+
+ job {
+ name = "wakame"
+ fileset = "genericUnixSet"
+ client = wakame-fd
+ }
+
+ job {
+ name = "durian"
+ fileset = "genericUnixSet"
+ client = durian-fd
+ }
+
+ job {
+ name = "soy"
+ fileset = "UnixDevelBoxSet"
+ client = soy-fd
+ }
+
+
- Autolabel should be specified by DIR instead of SD.
- Storage daemon
- Add media capacity
Item 2: Make the Storage daemon use intermediate file storage to buffer data.
-Deferred -- not necessary yet.
+Deferred -- not necessary yet -- possibly implement with Migration.
What: If data is coming into the SD too fast, buffer it to
disk if the user has configured this option.
Item 5: Implement Label templates
+Done
What: This is a mechanism whereby Bacula can automatically create
a tape label for new tapes according to a detailed specification
Item 9: Add SSL to daemon communications.
-Inprogress as of version 1.31.
What: This provides for secure communications between the daemons.
Proposed Implementation:
To solve this problem, I propose the following:
- - Add a new Director resource type called FileOptions.
+ - Add a new Director resource type called Options.
- - The FileOptions resource will have records for all
+ - The Options resource will have records for all
options that can currently be specified on the Include record
(in a FileSet). Examples below.
- - The FileOptions resource will permit an exclude option as well
+ - The Options resource will permit an exclude option as well
as a number of additional options.
- - The heart of the FileOptions resource is the ability to
- supply any number of ApplyTo records which specify POSIX
- regular expressions. These ApplyTo regular expressions are
+ - The heart of the Options resource is the ability to
+ supply any number of Match records which specify POSIX
+ regular expressions. These Match regular expressions are
applied to the fully qualified filename (path and all). If
- one matches, then the FileOptions will be used.
+ one matches, then the Options will be used.
- - When an ApplyTo specification matches an included file, the
- options specified in the FileOptions resource will override
+ - When an Match specification matches an included file, the
+ options specified in the Options resource will override
the default options specified on the Include record.
- Include records will be modified to permit referencing one or
- more FileOptions resources. The FileOptions will be used
+ more Options resources. The Options will be used
in the order listed on the Include record and the first
one that matches will be applied.
year or so from now).
- The Exclude record will be deprecated as the same functionality
- can be obtained by using an Exclude = yes in the FileOptions.
+ can be obtained by using an Exclude = yes in the Options.
-FileOptions records:
- The following records can appear in the FileOptions resource. An
+Options records:
+ The following records can appear in the Options resource. An
asterisk preceding the name indicates a feature not currently
implemented.
For Restore Jobs:
- replace= (always/ifnewer/ifolder/never) - replace options currently
- implemented in 1.27
+ implemented in 1.31
- *Writer= (filename) - external write (restore) program
Implementation:
Currently options specifying compression, MD5 signatures, recursion,
... of a FileSet are supplied on the Include record. These will now
- all be collected into a FileOptions resource, which will be
- specified on the Include in place of the options. Multiple FileOptions
- may be specified. Since the FileOptions contain regular expressions
+ all be collected into a Options resource, which will be
+ specified in the Include in place of the options. Multiple Options
+ may be specified. Since the Options may contain regular expressions
that are applied to the full filename, this will give the ability
to specify backup options on a file by file basis to whatever level
of detail you wish.
FileSet {
Name = "FullSet"
- Include = FileOptions=Opts {
- /
+ Include {
+ Compression = GZIP;
+ Signature = MD5
+ Match = /*.?*/ # matches all files.
+ File = /
}
}
- FileOptions {
- Name = Opts
- Compression = GZIP
- Signature = MD5
- ApplyTo = /*.?*/
- }
That's a lot more to do the same thing, but it gives the ability to
apply options on a file by file basis. For example, suppose you
want to compress all files but not any file with extensions .gz or .Z.
- You could do so as follows:
+ In that case, you will need to group two sets of options using
+ the Options resource as follows:
+
FileSet {
Name = "FullSet"
- Include = FileOptions=NoCompress FileOptions=Opts {
- /
+ Include {
+ Options {
+ Signature = MD5
+ # Note multiple Matches are ORed
+ Match = /*.gz/ # matches .gz files */
+ Match = /*.Z/ # matches .Z files */
+ }
+ Options {
+ Compression = GZIP
+ Signature = MD5
+ Match = /*.?*/ # matches all files
+ }
+ File = /
}
}
- FileOptions {
- Name = Opts
- Compression = GZIP
- Signature = MD5
- ApplyTo = /*.?*/ # matches all files
- }
- FileOptions {
- Name = NoCompress
- Signature = MD5
- # Note multiple ApplyTos are ORed
- ApplyTo = /*.gz/ # matches .gz files */
- ApplyTo = /*.Z/ # matches .Z files */
- }
- Now, since the NoCompress FileOptions is specified first on the
- Include line, any *.gz or *.Z file will have an MD5 signature computed,
- but will not be compressed. For all other files, the NoCompress will not
- match, so the Opts options will be used which will include GZIP
+ Now, since the no Compression option is specified in the
+ first group of Options, *.gz or *.Z file will have an MD5 signature computed,
+ but will not be compressed. For all other files, the *.gz *.Z will not
+ match, so the second group of options will be used which will include GZIP
compression.
Questions:
- Is it necessary to provide some means of ANDing regular expressions
and negation? (not currently planned)
- e.g. ApplyTo = /*.gz/ && !/big.gz/
+ e.g. Match = /*.gz/ && !/big.gz/
- I see that Networker has a "null" module which, if specified, does not
backup the file, but does make an record of the file in the catalog
Done: (see kernsdone for more)
-- Heartbeat between daemons.
-- Fix Dir heartbeat in restore and verify vol. Be sure to make
- bnet_recv() ignore BNET_HEARTBEAT.
-- Implement HEART_BEAT while SD waiting for tapes.
-- Include RunBeforeJob and RunAfterJob output in the message
- stream.
-- Change M_INFO to M_RESTORED for all restored files.
-- Fix command prompt in gnome-console by checking on Ready.
-- Merge SQLite, MySQL, and Rel spec into a single file.
-- Fix config of "console"
-- Check if cancel works with FD (fixed).
-- Properly configure console and gconsole (currently for source not
- configured for installation).
-- Error labeling tape from console gets Jmsg error because of no Job.
-- Test and implement get_pint and get_yesno.
-- Implement global with DB name and add to btraceback.gdb
-- Remove subsysdir from conf files (used only in autostart scripts).
-- Fix the following:
- rufus-dir: Max configured use duration exceeded. Marking Volume "MatouBackup" as Used.
- rufus-sd: Volume "" previously written, moving to end of data.
- rufus-sd: Matou.2003-05-10_10.39.18 Error: I canot write on this volume because:
- The number of files mismatch! Volume=1 Catalog=0
- rufus-sd: Matou.2003-05-10_10.39.18 Error: askdir.c:155 NULL Volume name. This shouldn't happen!!!
-- Shell character expansion is failing occassionally.
-- Add a section to the doc on Manual cycling of Volumes.
-- Check if Job/File retentions apply to multivolume jobs.
-- Fix missing casette in autoloader during read:
- 14-May-2003 14:41 undef-sd: RestoreFiles.2003-05-14_14.41.00 Warning: acquire.c:106 Volume name mismatch. Wanted TestVolume0005 got TestVolume0010
- 14-May-2003 14:41 undef-sd: 3301 Issuing autochanger "loaded" command.
- 14-May-2003 14:41 undef-sd: 3302 Issuing autochanger "unload" command.
- 14-May-2003 14:42 undef-sd: 3303 Issuing autochanger "load slot 1" command.
- 14-May-2003 14:42 undef-sd: 3304 Autochanger "load slot 1" status is OK.
- 14-May-2003 14:42 undef-sd: RestoreFiles.2003-05-14_14.41.00 Warning: acquire.c:106 Volume name mismatch. Wanted TestVolume0005 got TestVolume0009
- 14-May-2003 14:42 undef-sd: 3301 Issuing autochanger "loaded" command.
- 14-May-2003 14:42 undef-sd: RestoreFiles.2003-05-14_14.41.00 Warning: acquire.c:106 Volume name mismatch. Wanted TestVolume0005 got TestVolume0009
- 14-May-2003 14:42 undef-sd: 3301 Issuing autochanger "loaded" command.
- 14-May-2003 14:42 undef-sd: RestoreFiles.2003-05-14_14.41.00 Warning: acquire.c:106 Volume name mismatch. Wanted TestVolume0005 got TestVolume0009
- 14-May-2003 14:42 undef-sd: 3301 Issuing autochanger "loaded" command.
- 14-May-2003 14:42 undef-sd: RestoreFiles.2003-05-14_14.41.00 Warning: acquire.c:106 Volume name mismatch. Wanted TestVolume0005 got TestVolume0009
- 14-May-2003 14:42 undef-sd: 3301 Issuing autochanger "loaded" command.
- 14-May-2003 14:42 undef-sd: RestoreFiles.2003-05-14_14.41.00 Fatal error: acquire.c:129 Too many errors trying to mount device "/dev/nrsa0".
- 14-May-2003 14:42 undef-dir: Bacula 1.31 (12May03): 14-May-2003 14:42
-- Fix problem reported by Christopher McCurdy <cmccurdy@eecis.udel.edu>
- xeon-fd: Could not stat c:/Documents and Settings/All
- Users/Application Data/Humc:\Documents and Settings\All User98_AIX.kbf:
- ERR=No such file or directory
- Cannot reproduce.
-- The following Re-read last block at EOT failed. ERR=block.c:523 Read zero bytes on device /dev/nrsa0.
- undef-sd: block.c:523 Read zero bytes on device /dev/nrsa0.
- apparently masks the standard EOM message.
-- BSD (probably) does not have strtoll()
-- BSD does not have ioctl() MTEOM
-- BSD defines a number of MT_xxx variables which conflict
- with those defined by Bacula.
-- Make default duration days if no qualifier (e.g. s) is specified.
-- BSDI fix finding gcc version
-- When the FD errs (e.g. disk full) have a more graceful shutdown.
-- Make sure Bacula prunes/purges canceled and failed jobs too and all
- jobs with zero JobFiles.
-- Implement Volume name checking.
-- Document what characters can go into Volume names.
-- Getting the following on all directories on Win32
- 19-May-2003 01:14 tibs-fd: Could not access c:/cygwin/home/kern/rxvt: ERR=Permission denied
-- Cancellation caused JobMedia error:
- babylon5-dir: Last FULL backup time not found. Doing FULL backup.
- babylon5-dir: Start Backup JobId 416, Job=Zocalo_Save.2003-05-19_02.15.06
- babylon5-sd: End of media on Volume VXA-V17-Inc-001 Bytes=31,982,900,672 Blocks=495,781.
- babylon5-sd: Job Zocalo_Save.2003-05-19_02.15.06 waiting. Cannot find any appendable volumes.
- babylon5-sd: Someone woke me up, but I cannot find any appendable volumes
- for Job=Zocalo_Save.2003-05-19_02.15.06.
- babylon5-sd: Zocalo_Save.2003-05-19_02.15.06 Fatal error: Job
- Zocalo_Save.2003-05-19_02.15.06 canceled while waiting for mount on
- Storage Device "Ecrix_VXA-1".
- babylon5-sd: Zocalo_Save.2003-05-19_02.15.06 Fatal error: Cannot fixup device
- error. Job Zocalo_Save.2003-05-19_02.15.06 canceled while waiting for
- mount on Storage Device "Ecrix_VXA-1".
-
- babylon5-dir: Zocalo_Save.2003-05-19_02.15.06 Error: Catalog error creating
- JobMedia record. sql_create.c:125 Create JobMedia failed. Record already
- exists.
- babylon5-sd: Zocalo_Save.2003-05-19_02.15.06 Error: Error creating JobMedia
- record: 1991 Update JobMedia error
-
- babylon5-sd: Zocalo_Save.2003-05-19_02.15.06 Error: askdir.c:158 NULL Volume
- name. This shouldn't happen!!!
- zocalo-fd: Zocalo_Save.2003-05-19_02.15.06 Error: bnet.c:310 Write error
- sending to Storage daemon:babylon5:9103: ERR=Broken pipe
-- Volume names with spaces get jammed into the catalog with 0x1
- i.e. the SD bashes the Volume but they are not unbased by Dir.
- jerom-dir: MonthlySave.2003-05-10_17.12.01 Error: Unable to get Media
- record for Volume Tape^A1: ERR=sql_get.c:788 Media record for Volume
- "Tape^A1" not found.
- jerom-sd: MonthlySave.2003-05-10_17.12.01 Error: Error updating Volume
- Info: 1991 Catalog Request failed: sql_get.c:788 Media record for Volume
- "Tape^A1" not found.
-- ChangeServiceConfig2A does not exist on WinNT (ADVAPI32.DLL).
-- Fix "access not allowed" for backup of files on WinXP.
-- Check for existence of all new Win32 API's. See LoadLibrary in
- winservice.cpp
-- Count errors during restore and print them in the Job report.
-- Bug: fix access problems on files restored on WinXP.
-- Put system type returned by FD into catalog.
-- Finish WIN32_DATA stream code (bextract, check if can handle stream)
-- Make SD keep track of Files, Bytes during restore.
-- If you enter the userid by hand for restore, you get:
- Enter JobId(s), comma separated, to restore: 74
- You have selected the following JobId: 74
- Building directory tree for JobId 74 ...
- 134645140 items inserted into the tree and marked for extraction.
-- Add SDWriteSeqNo to SD, and probably Read on FD side.
-- If bootstrap is non-zero for restore, do not show JobId in the
- OK to run? (yes/mod/no): list.
-- When all cassettes in magazine are used, got:
- 22-May-2003 18:24 undef-sd: 3304 Autochanger "load slot 1" status is OK.
- 22-May-2003 18:24 undef-sd: NightlySave.2003-05-22_14.08.16 Warning: mount.c:245 Director wanted Volume "TestVolume0009".
- Current Volume "TestVolume0005" not acceptable because:
- 1998 Volume "TestVolume0005" not Append or Recycle.
- 22-May-2003 18:24 undef-sd: NightlySave.2003-05-22_14.08.16 Error: Autochanger Volume "TestVolume0009" not found in slot 1.
- Setting slot to zero in catalog.
- 22-May-2003 18:24 undef-sd: Please mount Volume "TestVolume0009" on Storage Device "ARCHIVE 4586" for Job NightlySave.2003-05-22_14
- .08.16
- Use "mount" command to release Job.
- 22-May-2003 19:24 undef-sd: Please mount Volume "TestVolume0009" on Storage Device "ARCHIVE 4586" for Job NightlySave.2003-05-22_14
- .08.16
- Use "mount" command to release Job.
-- Don't zero the Slot when the wrong volume is found -- simply ask
- the operator.
-- Implement MTIOCERRSTAT on FreeBSD to clear tape error conditions.
-- Shell expansion fails for working_directory in SD from time to time.
-- File the Automatically selected: xxx
- to say Automatically selected Pool: xxx
-- Default duration with no qualifier is sec should be 1 day
-- zap sd_auth_key in SD after FD connection.
-- Find a solution for the multiple FileSet problem (when it is changed). Add date?
-- Look at Python for a Bacula scripting language -- www.python.org
-- When Marking a file in Restore that is a hard link, also
- mark the link so that the data will be reloaded.
-- Emergency restore info:
- - Backup Bacula
- - Backup working directory
- - Backup Catalog
-- Why don't we get an error message from Win32 FD when bootstrap
- file cannot be created for restore command?
-- Fix Win2000 error with no messages during startup.
-- Make restore more robust in counting error and not immediately bailing
- out. Also print error message once, but try to continue.
-- Add code to check that blocks are sequential on restore.
-- Remove "rufus" and such references from regress.
-- No READLINE_SRC if found in alternate directory.
-- If ./btape is called without /dev, assume argument is a Storage resource name.
-- Find general solution for sscanf size problems (as well as sprintf. Do at run time?
-- Bytes restored is wrong.
-- The "List last 20 Jobs run" doesnt work correctly in restore.
- It doesnt show the last 20 jobs , but some older ones.
+- Implement new alist in FileSet scanning.
+- bls should continue reading even if it finds Win32 data on the tape.
+ The error should be Warning rather the Error.
+- Add user configurable timeout for connecting to SD.
+- Unsaved Flag in Job record (use JobMissingFiles).
+- Base Flag in Job record.
+- Configure mtx-changer to have correct path to mtx.
+- Add all command line arguments to "update", e.g. slot=nn volStatus=append, ...
+- Make some way so that if a machine is skipped because it is not up
+ that Bacula will continue retrying for a specified period of time --
+ periodically.
+- Implement all command line args on run.
+- Implement command line "restore" args.
+- Implement "restore current select=no"
+- Restore file modified before date
+- Restore -- do nothing but show what would happen
+- Add estimate to Console commands
+- Use read_record.c in SD code.
+- Fix read_record to handle multiple sessions.
+- Tip from Steve Allam
+ mt -f /dev/nst0 defblksize 0
+- Document "status" in the console.
+- Document driving console from shell script.
+- Write JobMedia records with max file size is reached on tape.
+- Handle the case of multiple JobMedia records pending (i.e. the
+ thread is slow and multiple situations requiring a JobMedia
+ record occur).
+- Do performance analysis on the restore tree routines.
+- Fix maximum file size (block.c) to generate JobMedia records.
+- Make the default file size 1GB on the tape.
+- Implement forward spacing between files.
+- Add Machine type (Linux/Windows) to Status report for daemons.
+ Look at src/host.h
+- Use repositioning at the beginning of the tape.
+- Do full check the command line args in update (e.g. VolStatus ...).
+- Specify list of files to restore
+- Implement ClientRunBeforeJob and ClientRunAfterJob.
+- Make | and < work on FD side.
+- Check to see if "blocked" is set during restore.
+- Figure out what is interrupting sql command in console.
+- Make new job print warning User Unmounted Tape.
+- Test recycling and purging (code changed in db_find_next_volume and
+ in recycle.c).
+- Document SDConnectTimeout (in FD).
+- Add restore by filename test.
+- Document restore by files.
+- Make variable expansion work correctly.
+- Implement List Volume Job=xxx or List scheduled volumes or Status Director
+- Copy static programs into install directory.
+- Think about changing Storage resource Device record to be
+ SDDeviceName.
+- Add RunBeforeJob and RunAfterJob to the Client program.
+- Need return status on read_cb() from read_records(). Need multiple
+ records -- one per Job, maybe a JCR or some other structure with
+ a block and a record.
+- LabelFormat on tape volume apparently creates the db record but
+ never actually labels the volume.
+- Recycling a volume when two jobs are using it is going to break. Fixed.
+- Document list nextvol and new format status dir.
+- Client files in Win32 with Unix eol conventions doesn't work.
+- Either fix or document that fill command in btape can be
+ compressed enormously by the hardware - a 36GB tape wrote 750GB!
+- Add multiple character duration qualifiers.
+- Require some modifer.
+- Restrict characters permitted in a Resource name, and don't permit
+ duplicate names.
+- Figure out some way to ignore or get past checksum errors in
+ reading.
+- The SD spooling file gets created even if it is not used.
+- Look at Cleaning tape in ua_label.c for media create/update
+- Add regression testing to the manual
+- End time: in job output of rescheduled job is time of first run.
+- Document list nextvol and status output.
+- Separate Dir heartbeat in FD from the SD heartbeat.
+- Fix sparse file handeling so that it always reads a multiple
+ of 512. Currently, it subtracts 8 bytes (for faddr).
+ Kludged with #ifdef for FreeBSD.
+- Document that Volume pruning can delete last Full backup and
+ hence you will not have a valid backup.
+- Clarify the fact that having the Bacula cygwin1.dll loaded
+ is not the same as having cygwin installed.
+- Document that it is safe to use the drive when the lights stop flashing.
+- Document all the status codes JobLevel, JobType, JobStatus.
+- Add GUI interface to manual
+- Combine the 3 places that search run records for the next
+ job. Use find_job_pool() modified in ua_output.c
+- Test connect timeouts.
+- Fix FreeBSD build with tcp_wrapper -- should not have -lnsl
+- Implement fast block rejection.
+- I want to restore by file to some date.
+---- 1.32b released
+- Figure out a way to move Volumes from one pool to another.
+- Implement a RunAfterFailedJob
+- Limit the number of block checksum/header BB01, ... errors printed.
+- If last Full back is purged and an Incremental or Differential remains,
+ Bacula does not promote the Incremental to a Full.
+- Document verify_disk_to_catalog
+- Document delete job command.
+- Document update volume pool and other command line keywords.
+- Add VerifyJob to "run" summary (yes/mod/no) prompt.
+- For listing, eliminate multiple JobIds in restore Jobs listing.
+- Document to start higher priorty jobs before lower ones.
+- suppress "Do not forget to mount the drive!!!" if error
+- Change error message when closing brace left off ...
+- Implement a move Volume from one pool to another.
+- Implement delete Job.
+- Document need to put LabelFormat in quotes.
+- Implement job in VerifyToCatalog
+- Eliminate ua_retention.c (retentioncmd) if possible.
projects / bacula / bacula / blobdiff