Projects:
Bacula Projects Roadmap
- 29 November 2005
+ 07 December 2005
+ (prioritized by user vote)
Summary:
-Item 1: Implement Migration that moves Jobs from one Pool to another.
-Item 2: Implement extraction of Win32 BackupWrite data.
-Item 3: Implement a Bacula GUI/management tool using Python.
-Item 4: Implement a Python interface to the Bacula catalog.
-Item 5: Implement more Python events in Bacula.
-Item 6: Implement Base jobs.
-Item 7: Add Plug-ins to the FileSet Include statements.
-Item 8: Implement huge exclude list support using hashing.
-Item 9: Implement data encryption (as opposed to comm encryption)
-Item 10: Permit multiple Media Types in an Autochanger
-Item 11: Allow different autochanger definitions for one autochanger.
-Item 12: Implement red/black binary tree routines.
-Item 13: Improve Baculas tape and drive usage and cleaning management.
-Item 14: Merge multiple backups (Synthetic Backup or Consolidation).
-Item 15: Automatic disabling of devices
-Item 16: Directive/mode to backup only file changes, not entire file
-Item 17: Quick release of FD-SD connection after backup.
-Item 18: Add support for FileSets in user directories CACHEDIR.TAG
-Item 19: Implement new {Client}Run{Before|After}Job feature.
-Item 20: Allow FD to initiate a backup
-Item 21: Multiple threads in file daemon for the same job
-Item 22: Archival (removal) of User Files to Tape
-Item 23: Deletion of Disk-Based BaculaVolumes
-Item 24: Accurate restoration of renamed/deleted files from
-Item 25: Implement creation and maintenance of copy pools
+Item 1: Implement data encryption (as opposed to comm encryption)
+Item 2: Implement Migration that moves Jobs from one Pool to another.
+Item 3: Accurate restoration of renamed/deleted files from
+Item 4: Implement a Bacula GUI/management tool using Python.
+Item 5: Implement Base jobs.
+Item 6: Allow FD to initiate a backup
+Item 7: Improve Bacula's tape and drive usage and cleaning management.
+Item 8: Implement creation and maintenance of copy pools
+Item 9: Implement new {Client}Run{Before|After}Job feature.
+Item 10: Merge multiple backups (Synthetic Backup or Consolidation).
+Item 11: Deletion of Disk-Based Bacula Volumes
+Item 12: Directive/mode to backup only file changes, not entire file
+Item 13: Multiple threads in file daemon for the same job
+Item 14: Implement red/black binary tree routines.
+Item 15: Add support for FileSets in user directories CACHEDIR.TAG
+Item 16: Implement extraction of Win32 BackupWrite data.
+Item 17: Implement a Python interface to the Bacula catalog.
+Item 18: Archival (removal) of User Files to Tape
+Item 19: Add Plug-ins to the FileSet Include statements.
+Item 20: Implement more Python events in Bacula.
+Item 21: Quick release of FD-SD connection after backup.
+Item 22: Permit multiple Media Types in an Autochanger
+Item 23: Allow different autochanger definitions for one autochanger.
+Item 24: Automatic disabling of devices
+Item 25: Implement huge exclude list support using hashing.
Below, you will find more information on future projects:
-Item 1: Implement Migration that moves Jobs from one Pool to another.
+Item 1: Implement data encryption (as opposed to comm encryption)
+ Date: 28 October 2005
+ Origin: Sponsored by Landon and 13 contributors to EFF.
+ Status: Landon Fuller is currently implementing this.
+
+ What: Currently the data that is stored on the Volume is not
+ encrypted. For confidentiality, encryption of data at
+ the File daemon level is essential.
+ Data encryption encrypts the data in the File daemon and
+ decrypts the data in the File daemon during a restore.
+
+ Why: Large sites require this.
+
+Item 2: Implement Migration that moves Jobs from one Pool to another.
Origin: Sponsored by Riege Software International GmbH. Contact:
Daniel Holtkamp <holtkamp at riege dot com>
Date: 28 October 2005
Number of Jobs
Number of Volumes
+Item 3: Accurate restoration of renamed/deleted files from
+ Incremental/Differential backups
+ Date: 28 November 2005
+ Origin: Martin Simmons (martin at lispworks dot com)
+ Status:
-Item 2: Implement extraction of Win32 BackupWrite data.
- Origin: Thorsten Engel <thorsten.engel at matrix-computer dot com>
- Date: 28 October 2005
- Status: Assigned to Thorsten. Implemented in current CVS
+ What: When restoring a fileset for a specified date (including "most
+ recent"), Bacula should give you exactly the files and directories
+ that existed at the time of the last backup prior to that date.
- What: This provides the Bacula File daemon with code that
- can pick apart the stream output that Microsoft writes
- for BackupWrite data, and thus the data can be read
- and restored on non-Win32 machines.
+ Currently this only works if the last backup was a Full backup.
+ When the last backup was Incremental/Differential, files and
+ directories that have been renamed or deleted since the last Full
+ backup are not currently restored correctly. Ditto for files with
+ extra/fewer hard links than at the time of the last Full backup.
- Why: BackupWrite data is the portable=no option in Win32
- FileSets, and in previous Baculas, this data could
- only be extracted using a Win32 FD. With this new code,
- the Windows data can be extracted and restored on
- any OS.
+ Why: Incremental/Differential would be much more useful if this worked.
+
+ Notes: Item 14 (Merging of multiple backups into a single one) seems to
+ rely on this working, otherwise the merged backups will not be
+ truly equivalent to a Full backup.
+ Kern: notes shortened. This can be done without the need for
+ inodes. It is essentially the same as the current Verify job,
+ but one additional database record must be written, which does
+ not need any database change.
-Item 3: Implement a Bacula GUI/management tool using Python.
+ Kern: see if we can correct restoration of directories if
+ replace=ifnewer is set. Currently, if the directory does not
+ exist, a "dummy" directory is created, then when all the files
+ are updated, the dummy directory is newer so the real values
+ are not updated.
+
+Item 4: Implement a Bacula GUI/management tool using Python.
Origin: Kern
Date: 28 October 2005
Status:
Notes: This is currently being implemented using Python-GTK by
Lucas Di Pentima <lucas at lunix dot com dot ar>
-
-Item 4: Implement a Python interface to the Bacula catalog.
- Date: 28 October 2005
- Origin: Kern
- Status:
-
- What: Implement an interface for Python scripts to access
- the catalog through Bacula.
-
- Why: This will permit users to customize Bacula through
- Python scripts.
-
-Item 5: Implement more Python events in Bacula.
- Date: 28 October 2005
- Origin:
- Status:
-
- What: Allow Python scripts to be called at more places
- within Bacula and provide additional access to Bacula
- internal variables.
-
- Why: This will permit users to customize Bacula through
- Python scripts.
-
- Notes: Recycle event
- Scratch pool event
- NeedVolume event
- MediaFull event
-
- Also add a way to get a listing of currently running
- jobs (possibly also scheduled jobs).
-
-
-Item 6: Implement Base jobs.
+Item 5: Implement Base jobs.
Date: 28 October 2005
Origin: Kern
Status:
FD a list of files/attribs, and the FD must search the
list and compare it for each file to be saved.
-Item 7: Add Plug-ins to the FileSet Include statements.
- Date: 28 October 2005
- Origin:
- Status: Partially coded in 1.37 -- much more to do.
-
- What: Allow users to specify wild-card and/or regular
- expressions to be matched in both the Include and
- Exclude directives in a FileSet. At the same time,
- allow users to define plug-ins to be called (based on
- regular expression/wild-card matching).
-
- Why: This would give the users the ultimate ability to control
- how files are backed up/restored. A user could write a
- plug-in knows how to backup his Oracle database without
- stopping/starting it, for example.
-
-Item 8: Implement huge exclude list support using hashing.
- Date: 28 October 2005
- Origin: Kern
- Status:
-
- What: Allow users to specify very large exclude list (currently
- more than about 1000 files is too many).
-
- Why: This would give the users the ability to exclude all
- files that are loaded with the OS (e.g. using rpms
- or debs). If the user can restore the base OS from
- CDs, there is no need to backup all those files. A
- complete restore would be to restore the base OS, then
- do a Bacula restore. By excluding the base OS files, the
- backup set will be *much* smaller.
-
-
-Item 9: Implement data encryption (as opposed to comm encryption)
- Date: 28 October 2005
- Origin: Sponsored by Landon and 13 contributors to EFF.
- Status: Landon Fuller is currently implementing this.
-
- What: Currently the data that is stored on the Volume is not
- encrypted. For confidentiality, encryption of data at
- the File daemon level is essential.
- Data encryption encrypts the data in the File daemon and
- decrypts the data in the File daemon during a restore.
-
- Why: Large sites require this.
-
-Item 10: Permit multiple Media Types in an Autochanger
- Origin: Kern
- Status: Now implemented
-
- What: Modify the Storage daemon so that multiple Media Types
- can be specified in an autochanger. This would be somewhat
- of a simplistic implementation in that each drive would
- still be allowed to have only one Media Type. However,
- the Storage daemon will ensure that only a drive with
- the Media Type that matches what the Director specifies
- is chosen.
-
- Why: This will permit user with several different drive types
- to make full use of their autochangers.
-
-Item 11: Allow different autochanger definitions for one autochanger.
- Date: 28 October 2005
- Origin: Kern
- Status:
-
- What: Currently, the autochanger script is locked based on
- the autochanger. That is, if multiple drives are being
- simultaneously used, the Storage daemon ensures that only
- one drive at a time can access the mtx-changer script.
- This change would base the locking on the control device,
- rather than the autochanger. It would then permit two autochanger
- definitions for the same autochanger, but with different
- drives. Logically, the autochanger could then be "partitioned"
- for different jobs, clients, or class of jobs, and if the locking
- is based on the control device (e.g. /dev/sg0) the mtx-changer
- script will be locked appropriately.
-
- Why: This will permit users to partition autochangers for specific
- use. It would also permit implementation of multiple Media
- Types with no changes to the Storage daemon.
-
-Item 12: Implement red/black binary tree routines.
- Date: 28 October 2005
- Origin: Kern
- Status:
+Item 6: Allow FD to initiate a backup
+ Origin: Frank Volf (frank at deze dot org)
+ Date: 17 November 2005
+ Status:
- What: Implement a red/black binary tree class. This could
- then replace the current binary insert/search routines
- used in the restore in memory tree. This could significantly
- speed up the creation of the in memory restore tree.
+ What: Provide some means, possibly by a restricted console that
+ allows a FD to initiate a backup, and that uses the connection
+ established by the FD to the Director for the backup so that
+ a Director that is firewalled can do the backup.
- Why: Performance enhancement.
+ Why: Makes backup of laptops much easier.
-Item 13: Improve Baculas tape and drive usage and cleaning management.
+Item 7: Improve Bacula's tape and drive usage and cleaning management.
Date: 8 November 2005, November 11, 2005
Origin: Adam Thornton <athornton at sinenomine dot net>,
Arno Lehmann <al at its-lehmann dot de>
sub-projects: Measuring Tape and Drive usage, retiring
volumes, and handling drive cleaning and TAPEALERTs.
+Item 8: Implement creation and maintenance of copy pools
+ Date: 27 November 2005
+ Origin: David Boyes (dboyes at sinenomine dot net)
+ Status:
-Item 14: Merge multiple backups (Synthetic Backup or Consolidation).
- Origin: Marc Cousin and Eric Bollengier
- Date: 15 November 2005
- Status: Depends on first implementing project Item 1 (Migration).
+ What: I would like Bacula to have the capability to write copies
+ of backed-up data on multiple physical volumes selected
+ from different pools without transferring the data
+ multiple times, and to accept any of the copy volumes
+ as valid for restore.
- What: A merged backup is a backup made without connecting to the Client.
- It would be a Merge of existing backups into a single backup.
- In effect, it is like a restore but to the backup medium.
+ Why: In many cases, businesses are required to keep offsite
+ copies of backup volumes, or just wish for simple
+ protection against a human operator dropping a storage
+ volume and damaging it. The ability to generate multiple
+ volumes in the course of a single backup job allows
+ customers to simple check out one copy and send it
+ offsite, marking it as out of changer or otherwise
+ unavailable. Currently, the library and magazine
+ management capability in Bacula does not make this process
+ simple.
- For instance, say that last Sunday we made a full backup. Then
- all week long, we created incremental backups, in order to do
- them fast. Now comes Sunday again, and we need another full.
- The merged backup makes it possible to do instead an incremental
- backup (during the night for instance), and then create a merged
- backup during the day, by using the full and incrementals from
- the week. The merged backup will be exactly like a full made
- Sunday night on the tape, but the production interruption on the
- Client will be minimal, as the Client will only have to send
- incrementals.
+ Restores would use the copy of the data on the first
+ available volume, in order of copy pool chain definition.
- In fact, if it's done correctly, you could merge all the
- Incrementals into single Incremental, or all the Incrementals
- and the last Differential into a new Differential, or the Full,
- last differential and all the Incrementals into a new Full
- backup. And there is no need to involve the Client.
+ This is also a major scalability issue -- as the number of
+ clients increases beyond several thousand, and the volume
+ of data increases, transferring the data multiple times to
+ produce additional copies of the backups will become
+ physically impossible due to transfer speed
+ issues. Generating multiple copies at server side will
+ become the only practical option.
- Why: The benefit is that :
- - the Client just does an incremental ;
- - the merged backup on tape is just as a single full backup,
- and can be restored very fast.
+ How: I suspect that this will require adding a multiplexing
+ SD that appears to be a SD to a specific FD, but 1-n FDs
+ to the specific back end SDs managing the primary and copy
+ pools. Storage pools will also need to acquire parameters
+ to define the pools to be used for copies.
- This is also a way of reducing the backup data since the old
- data can then be pruned (or not) from the catalog, possibly
- allowing older volumes to be recycled
+ Notes: I would commit some of my developers' time if we can agree
+ on the design and behavior.
-Item 15: Automatic disabling of devices
- Date: 2005-11-11
- Origin: Peter Eriksson <peter at ifm.liu dot se>
- Status:
+Item 9: Implement new {Client}Run{Before|After}Job feature.
+ Date: 26 September 2005
+ Origin: Phil Stracchino <phil.stracchino at speakeasy dot net>
+ Status:
- What: After a configurable amount of fatal errors with a tape drive
- Bacula should automatically disable further use of a certain
- tape drive. There should also be "disable"/"enable" commands in
- the "bconsole" tool.
+ What: Some time ago, there was a discussion of RunAfterJob and
+ ClientRunAfterJob, and the fact that they do not run after failed
+ jobs. At the time, there was a suggestion to add a
+ RunAfterFailedJob directive (and, presumably, a matching
+ ClientRunAfterFailedJob directive), but to my knowledge these
+ were never implemented.
- Why: On a multi-drive jukebox there is a possibility of tape drives
- going bad during large backups (needing a cleaning tape run,
- tapes getting stuck). It would be advantageous if Bacula would
- automatically disable further use of a problematic tape drive
- after a configurable amount of errors has occurred.
-
- An example: I have a multi-drive jukebox (6 drives, 380+ slots)
- where tapes occasionally get stuck inside the drive. Bacula will
- notice that the "mtx-changer" command will fail and then fail
- any backup jobs trying to use that drive. However, it will still
- keep on trying to run new jobs using that drive and fail -
- forever, and thus failing lots and lots of jobs... Since we have
- many drives Bacula could have just automatically disabled
- further use of that drive and used one of the other ones
- instead.
-
-
-Item 16: Directive/mode to backup only file changes, not entire file
- Date: 11 November 2005
- Origin: Joshua Kugler <joshua dot kugler at uaf dot edu>
- Marek Bajon <mbajon at bimsplus dot com dot pl>
- Status: RFC
-
- What: Currently when a file changes, the entire file will be backed up in
- the next incremental or full backup. To save space on the tapes
- it would be nice to have a mode whereby only the changes to the
- file would be backed up when it is changed.
-
- Why: This would save lots of space when backing up large files such as
- logs, mbox files, Outlook PST files and the like.
-
- Notes: This would require the usage of disk-based volumes as comparing
- files would not be feasible using a tape drive.
-
-Item 17: Quick release of FD-SD connection after backup.
- Origin: Frank Volf (frank at deze dot org)
- Date: 17 November 2005
- Status:
-
- What: In the Bacula implementation a backup is finished after all data
- and attributes are successfully written to storage. When using a
- tape backup it is very annoying that a backup can take a day,
- simply because the current tape (or whatever) is full and the
- administrator has not put a new one in. During that time the
- system cannot be taken off-line, because there is still an open
- session between the storage daemon and the file daemon on the
- client.
-
- Although this is a very good strategy for making "safe backups"
- This can be annoying for e.g. laptops, that must remain
- connected until the backup is completed.
-
- Using a new feature called "migration" it will be possible to
- spool first to harddisk (using a special 'spool' migration
- scheme) and then migrate the backup to tape.
-
- There is still the problem of getting the attributes committed.
- If it takes a very long time to do, with the current code, the
- job has not terminated, and the File daemon is not freed up. The
- Storage daemon should release the File daemon as soon as all the
- file data and all the attributes have been sent to it (the SD).
- Currently the SD waits until everything is on tape and all the
- attributes are transmitted to the Director before signaling
- completion to the FD. I don't think I would have any problem
- changing this. The reason is that even if the FD reports back to
- the Dir that all is OK, the job will not terminate until the SD
- has done the same thing -- so in a way keeping the SD-FD link
- open to the very end is not really very productive ...
-
- Why: Makes backup of laptops much easier.
-
-
-Item 18: Add support for FileSets in user directories CACHEDIR.TAG
- Origin: Norbert Kiesel <nkiesel at tbdnetworks dot com>
- Date: 21 November 2005
- Status:
-
- What: CACHDIR.TAG is a proposal for identifying directories which
- should be ignored for archiving/backup. It works by ignoring
- directory trees which have a file named CACHEDIR.TAG with a
- specific content. See
- http://www.brynosaurus.com/cachedir/spec.html
- for details.
-
- From Peter Eriksson:
- I suggest that if this is implemented (I've also asked for this
- feature some year ago) that it is made compatible with Legato
- Networkers ".nsr" files where you can specify a lot of options on
- how to handle files/directories (including denying further
- parsing of .nsr files lower down into the directory trees). A
- PDF version of the .nsr man page can be viewed at:
-
- http://www.ifm.liu.se/~peter/nsr.pdf
-
- Why: It's a nice alternative to "exclude" patterns for directories
- which don't have regular pathnames. Also, it allows users to
- control backup for themselves. Implementation should be pretty
- simple. GNU tar >= 1.14 or so supports it, too.
-
- Notes: I envision this as an optional feature to a fileset
- specification.
-
-Item 19: Implement new {Client}Run{Before|After}Job feature.
- Date: 26 September 2005
- Origin: Phil Stracchino <phil.stracchino at speakeasy dot net>
- Status:
-
- What: Some time ago, there was a discussion of RunAfterJob and
- ClientRunAfterJob, and the fact that they do not run after failed
- jobs. At the time, there was a suggestion to add a
- RunAfterFailedJob directive (and, presumably, a matching
- ClientRunAfterFailedJob directive), but to my knowledge these
- were never implemented.
-
- An alternate way of approaching the problem has just occurred to
- me. Suppose the RunBeforeJob and RunAfterJob directives were
- expanded in a manner something like this example:
+ An alternate way of approaching the problem has just occurred to
+ me. Suppose the RunBeforeJob and RunAfterJob directives were
+ expanded in a manner something like this example:
RunBeforeJob {
Command = "/opt/bacula/etc/checkhost %c"
line so that the same script can be used both before/after.
David Boyes.
-Item 20: Allow FD to initiate a backup
- Origin: Frank Volf (frank at deze dot org)
- Date: 17 November 2005
- Status:
+Item 10: Merge multiple backups (Synthetic Backup or Consolidation).
+ Origin: Marc Cousin and Eric Bollengier
+ Date: 15 November 2005
+ Status: Depends on first implementing project Item 1 (Migration).
- What: Provide some means, possibly by a restricted console that
- allows a FD to initiate a backup, and that uses the connection
- established by the FD to the Director for the backup so that
- a Director that is firewalled can do the backup.
+ What: A merged backup is a backup made without connecting to the Client.
+ It would be a Merge of existing backups into a single backup.
+ In effect, it is like a restore but to the backup medium.
- Why: Makes backup of laptops much easier.
+ For instance, say that last Sunday we made a full backup. Then
+ all week long, we created incremental backups, in order to do
+ them fast. Now comes Sunday again, and we need another full.
+ The merged backup makes it possible to do instead an incremental
+ backup (during the night for instance), and then create a merged
+ backup during the day, by using the full and incrementals from
+ the week. The merged backup will be exactly like a full made
+ Sunday night on the tape, but the production interruption on the
+ Client will be minimal, as the Client will only have to send
+ incrementals.
+
+ In fact, if it's done correctly, you could merge all the
+ Incrementals into single Incremental, or all the Incrementals
+ and the last Differential into a new Differential, or the Full,
+ last differential and all the Incrementals into a new Full
+ backup. And there is no need to involve the Client.
+
+ Why: The benefit is that :
+ - the Client just does an incremental ;
+ - the merged backup on tape is just as a single full backup,
+ and can be restored very fast.
+
+ This is also a way of reducing the backup data since the old
+ data can then be pruned (or not) from the catalog, possibly
+ allowing older volumes to be recycled
+
+Item 11: Deletion of Disk-Based Bacula Volumes
+ Date: Nov 25, 2005
+ Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
+ by Kern)
+ Status:
+
+ What: Provide a way for Bacula to automatically remove Volumes
+ from the filesystem, or optionally to truncate them.
+ Obviously, the Volume must be pruned prior removal.
+
+ Why: This would allow users more control over their Volumes and
+ prevent disk based volumes from consuming too much space.
+
+ Notes: The following two directives might do the trick:
+
+ Volume Data Retention = <time period>
+ Remove Volume After = <time period>
+
+ The migration project should also remove a Volume that is
+ migrated. This might also work for tape Volumes.
+
+Item 12: Directive/mode to backup only file changes, not entire file
+ Date: 11 November 2005
+ Origin: Joshua Kugler <joshua dot kugler at uaf dot edu>
+ Marek Bajon <mbajon at bimsplus dot com dot pl>
+ Status: RFC
+
+ What: Currently when a file changes, the entire file will be backed up in
+ the next incremental or full backup. To save space on the tapes
+ it would be nice to have a mode whereby only the changes to the
+ file would be backed up when it is changed.
+
+ Why: This would save lots of space when backing up large files such as
+ logs, mbox files, Outlook PST files and the like.
-Item 21: Multiple threads in file daemon for the same job
+ Notes: This would require the usage of disk-based volumes as comparing
+ files would not be feasible using a tape drive.
+
+Item 13: Multiple threads in file daemon for the same job
Date: 27 November 2005
Origin: Ove Risberg (Ove.Risberg at octocode dot com)
Status:
Notes: I am willing to try to implement this but I will probably
need some help and advice. (No problem -- Kern)
-Item 22: Archival (removal) of User Files to Tape
+Item 14: Implement red/black binary tree routines.
+ Date: 28 October 2005
+ Origin: Kern
+ Status:
+
+ What: Implement a red/black binary tree class. This could
+ then replace the current binary insert/search routines
+ used in the restore in memory tree. This could significantly
+ speed up the creation of the in memory restore tree.
+
+ Why: Performance enhancement.
+
+Item 15: Add support for FileSets in user directories CACHEDIR.TAG
+ Origin: Norbert Kiesel <nkiesel at tbdnetworks dot com>
+ Date: 21 November 2005
+ Status:
+
+ What: CACHDIR.TAG is a proposal for identifying directories which
+ should be ignored for archiving/backup. It works by ignoring
+ directory trees which have a file named CACHEDIR.TAG with a
+ specific content. See
+ http://www.brynosaurus.com/cachedir/spec.html
+ for details.
+
+ From Peter Eriksson:
+ I suggest that if this is implemented (I've also asked for this
+ feature some year ago) that it is made compatible with Legato
+ Networkers ".nsr" files where you can specify a lot of options on
+ how to handle files/directories (including denying further
+ parsing of .nsr files lower down into the directory trees). A
+ PDF version of the .nsr man page can be viewed at:
+
+ http://www.ifm.liu.se/~peter/nsr.pdf
+
+ Why: It's a nice alternative to "exclude" patterns for directories
+ which don't have regular pathnames. Also, it allows users to
+ control backup for themselves. Implementation should be pretty
+ simple. GNU tar >= 1.14 or so supports it, too.
+
+ Notes: I envision this as an optional feature to a fileset
+ specification.
+
+
+Item 16: Implement extraction of Win32 BackupWrite data.
+ Origin: Thorsten Engel <thorsten.engel at matrix-computer dot com>
+ Date: 28 October 2005
+ Status: Assigned to Thorsten. Implemented in current CVS
+
+ What: This provides the Bacula File daemon with code that
+ can pick apart the stream output that Microsoft writes
+ for BackupWrite data, and thus the data can be read
+ and restored on non-Win32 machines.
+
+ Why: BackupWrite data is the portable=no option in Win32
+ FileSets, and in previous Baculas, this data could
+ only be extracted using a Win32 FD. With this new code,
+ the Windows data can be extracted and restored on
+ any OS.
+
+
+Item 18: Implement a Python interface to the Bacula catalog.
+ Date: 28 October 2005
+ Origin: Kern
+ Status:
+
+ What: Implement an interface for Python scripts to access
+ the catalog through Bacula.
+
+ Why: This will permit users to customize Bacula through
+ Python scripts.
+
+Item 18: Archival (removal) of User Files to Tape
Date: Nov. 24/2005
access time. Then after another 6 months (or possibly as one
storage pool gets full) data is migrated to Tape.
+Item 19: Add Plug-ins to the FileSet Include statements.
+ Date: 28 October 2005
+ Origin:
+ Status: Partially coded in 1.37 -- much more to do.
-Item 23: Deletion of Disk-Based Bacula Volumes
- Date: Nov 25, 2005
- Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
- by Kern)
- Status:
+ What: Allow users to specify wild-card and/or regular
+ expressions to be matched in both the Include and
+ Exclude directives in a FileSet. At the same time,
+ allow users to define plug-ins to be called (based on
+ regular expression/wild-card matching).
- What: Provide a way for Bacula to automatically remove Volumes
- from the filesystem, or optionally to truncate them.
- Obviously, the Volume must be pruned prior removal.
+ Why: This would give the users the ultimate ability to control
+ how files are backed up/restored. A user could write a
+ plug-in knows how to backup his Oracle database without
+ stopping/starting it, for example.
- Why: This would allow users more control over their Volumes and
- prevent disk based volumes from consuming too much space.
+Item 20: Implement more Python events in Bacula.
+ Date: 28 October 2005
+ Origin:
+ Status:
- Notes: The following two directives might do the trick:
+ What: Allow Python scripts to be called at more places
+ within Bacula and provide additional access to Bacula
+ internal variables.
- Volume Data Retention = <time period>
- Remove Volume After = <time period>
+ Why: This will permit users to customize Bacula through
+ Python scripts.
- The migration project should also remove a Volume that is
- migrated. This might also work for tape Volumes.
+ Notes: Recycle event
+ Scratch pool event
+ NeedVolume event
+ MediaFull event
+
+ Also add a way to get a listing of currently running
+ jobs (possibly also scheduled jobs).
-Item 24: Accurate restoration of renamed/deleted files from
- Incremental/Differential backups
- Date: 28 November 2005
- Origin: Martin Simmons (martin at lispworks dot com)
+Item 21: Quick release of FD-SD connection after backup.
+ Origin: Frank Volf (frank at deze dot org)
+ Date: 17 November 2005
Status:
- What: When restoring a fileset for a specified date (including "most
- recent"), Bacula should give you exactly the files and directories
- that existed at the time of the last backup prior to that date.
+ What: In the Bacula implementation a backup is finished after all data
+ and attributes are successfully written to storage. When using a
+ tape backup it is very annoying that a backup can take a day,
+ simply because the current tape (or whatever) is full and the
+ administrator has not put a new one in. During that time the
+ system cannot be taken off-line, because there is still an open
+ session between the storage daemon and the file daemon on the
+ client.
- Currently this only works if the last backup was a Full backup.
- When the last backup was Incremental/Differential, files and
- directories that have been renamed or deleted since the last Full
- backup are not currently restored correctly. Ditto for files with
- extra/fewer hard links than at the time of the last Full backup.
+ Although this is a very good strategy for making "safe backups"
+ This can be annoying for e.g. laptops, that must remain
+ connected until the backup is completed.
- Why: Incremental/Differential would be much more useful if this worked.
+ Using a new feature called "migration" it will be possible to
+ spool first to harddisk (using a special 'spool' migration
+ scheme) and then migrate the backup to tape.
- Notes: Item 14 (Merging of multiple backups into a single one) seems to
- rely on this working, otherwise the merged backups will not be
- truely equivalent to a Full backup.
+ There is still the problem of getting the attributes committed.
+ If it takes a very long time to do, with the current code, the
+ job has not terminated, and the File daemon is not freed up. The
+ Storage daemon should release the File daemon as soon as all the
+ file data and all the attributes have been sent to it (the SD).
+ Currently the SD waits until everything is on tape and all the
+ attributes are transmitted to the Director before signaling
+ completion to the FD. I don't think I would have any problem
+ changing this. The reason is that even if the FD reports back to
+ the Dir that all is OK, the job will not terminate until the SD
+ has done the same thing -- so in a way keeping the SD-FD link
+ open to the very end is not really very productive ...
- Kern: notes shortened. This can be done without the need for
- inodes. It is essentially the same as the current Verify job,
- but one additional database record must be written, which does
- not need any database change.
+ Why: Makes backup of laptops much easier.
-Item 25: Implement creation and maintenance of copy pools
- Date: 27 November 2005
- Origin: David Boyes (dboyes at sinenomine dot net)
- Status:
+Item 22: Permit multiple Media Types in an Autochanger
+ Origin: Kern
+ Status: Now implemented
- What: I would like Bacula to have the capability to write copies
- of backed-up data on multiple physical volumes selected
- from different pools without transferring the data
- multiple times, and to accept any of the copy volumes
- as valid for restore.
+ What: Modify the Storage daemon so that multiple Media Types
+ can be specified in an autochanger. This would be somewhat
+ of a simplistic implementation in that each drive would
+ still be allowed to have only one Media Type. However,
+ the Storage daemon will ensure that only a drive with
+ the Media Type that matches what the Director specifies
+ is chosen.
- Why: In many cases, businesses are required to keep offsite
- copies of backup volumes, or just wish for simple
- protection against a human operator dropping a storage
- volume and damaging it. The ability to generate multiple
- volumes in the course of a single backup job allows
- customers to simple check out one copy and send it
- offsite, marking it as out of changer or otherwise
- unavailable. Currently, the library and magazine
- management capability in Bacula does not make this process
- simple.
+ Why: This will permit user with several different drive types
+ to make full use of their autochangers.
- Restores would use the copy of the data on the first
- available volume, in order of copy pool chain definition.
+Item 23: Allow different autochanger definitions for one autochanger.
+ Date: 28 October 2005
+ Origin: Kern
+ Status:
- This is also a major scalability issue -- as the number of
- clients increases beyond several thousand, and the volume
- of data increases, transferring the data multiple times to
- produce additional copies of the backups will become
- physically impossible due to transfer speed
- issues. Generating multiple copies at server side will
- become the only practical option.
+ What: Currently, the autochanger script is locked based on
+ the autochanger. That is, if multiple drives are being
+ simultaneously used, the Storage daemon ensures that only
+ one drive at a time can access the mtx-changer script.
+ This change would base the locking on the control device,
+ rather than the autochanger. It would then permit two autochanger
+ definitions for the same autochanger, but with different
+ drives. Logically, the autochanger could then be "partitioned"
+ for different jobs, clients, or class of jobs, and if the locking
+ is based on the control device (e.g. /dev/sg0) the mtx-changer
+ script will be locked appropriately.
- How: I suspect that this will require adding a multiplexing
- SD that appears to be a SD to a specific FD, but 1-n FDs
- to the specific back end SDs managing the primary and copy
- pools. Storage pools will also need to acquire parameters
- to define the pools to be used for copies.
+ Why: This will permit users to partition autochangers for specific
+ use. It would also permit implementation of multiple Media
+ Types with no changes to the Storage daemon.
- Notes: I would commit some of my developers' time if we can agree
- on the design and behavior.
+Item 24: Automatic disabling of devices
+ Date: 2005-11-11
+ Origin: Peter Eriksson <peter at ifm.liu dot se>
+ Status:
+
+ What: After a configurable amount of fatal errors with a tape drive
+ Bacula should automatically disable further use of a certain
+ tape drive. There should also be "disable"/"enable" commands in
+ the "bconsole" tool.
+
+ Why: On a multi-drive jukebox there is a possibility of tape drives
+ going bad during large backups (needing a cleaning tape run,
+ tapes getting stuck). It would be advantageous if Bacula would
+ automatically disable further use of a problematic tape drive
+ after a configurable amount of errors has occurred.
+
+ An example: I have a multi-drive jukebox (6 drives, 380+ slots)
+ where tapes occasionally get stuck inside the drive. Bacula will
+ notice that the "mtx-changer" command will fail and then fail
+ any backup jobs trying to use that drive. However, it will still
+ keep on trying to run new jobs using that drive and fail -
+ forever, and thus failing lots and lots of jobs... Since we have
+ many drives Bacula could have just automatically disabled
+ further use of that drive and used one of the other ones
+ instead.
+
+Item 25: Implement huge exclude list support using hashing.
+ Date: 28 October 2005
+ Origin: Kern
+ Status:
+
+ What: Allow users to specify very large exclude list (currently
+ more than about 1000 files is too many).
+
+ Why: This would give the users the ability to exclude all
+ files that are loaded with the OS (e.g. using rpms
+ or debs). If the user can restore the base OS from
+ CDs, there is no need to backup all those files. A
+ complete restore would be to restore the base OS, then
+ do a Bacula restore. By excluding the base OS files, the
+ backup set will be *much* smaller.
+
+===============================================
+Not in Dec 2005 Vote:
+Item n: Allow skipping execution of Jobs
+ Date: 29 November 2005
+ Origin: Florian Schnabel <florian.schnabel at docufy dot de>
+ Status:
+ What: An easy option to skip a certain job on a certain date.
+ Why: You could then easily skip tape backups on holidays. Especially
+ if you got no autochanger and can only fit one backup on a tape
+ that would be really handy, other jobs could proceed normally
+ and you won't get errors that way.
============= Empty Feature Request form ===========
Item n: One line summary ...