-Item 8: Implement creation and maintenance of copy pools
- Date: 27 November 2005
- Origin: David Boyes (dboyes at sinenomine dot net)
- Status:
-
- What: I would like Bacula to have the capability to write copies
- of backed-up data on multiple physical volumes selected
- from different pools without transferring the data
- multiple times, and to accept any of the copy volumes
- as valid for restore.
-
- Why: In many cases, businesses are required to keep offsite
- copies of backup volumes, or just wish for simple
- protection against a human operator dropping a storage
- volume and damaging it. The ability to generate multiple
- volumes in the course of a single backup job allows
- customers to simple check out one copy and send it
- offsite, marking it as out of changer or otherwise
- unavailable. Currently, the library and magazine
- management capability in Bacula does not make this process
- simple.
-
- Restores would use the copy of the data on the first
- available volume, in order of copy pool chain definition.
-
- This is also a major scalability issue -- as the number of
- clients increases beyond several thousand, and the volume
- of data increases, transferring the data multiple times to
- produce additional copies of the backups will become
- physically impossible due to transfer speed
- issues. Generating multiple copies at server side will
- become the only practical option.
-
- How: I suspect that this will require adding a multiplexing
- SD that appears to be a SD to a specific FD, but 1-n FDs
- to the specific back end SDs managing the primary and copy
- pools. Storage pools will also need to acquire parameters
- to define the pools to be used for copies.
-
- Notes: I would commit some of my developers' time if we can agree
- on the design and behavior.
-
-Item 9: Implement new {Client}Run{Before|After}Job feature.
- Date: 26 September 2005
- Origin: Phil Stracchino
- Status: Done. This has been implemented by Eric Bollengier
-
- What: Some time ago, there was a discussion of RunAfterJob and
- ClientRunAfterJob, and the fact that they do not run after failed
- jobs. At the time, there was a suggestion to add a
- RunAfterFailedJob directive (and, presumably, a matching
- ClientRunAfterFailedJob directive), but to my knowledge these
- were never implemented.
-
- The current implementation doesn't permit to add new feature easily.
-
- An alternate way of approaching the problem has just occurred to
- me. Suppose the RunBeforeJob and RunAfterJob directives were
- expanded in a manner like this example:
-
- RunScript {
- Command = "/opt/bacula/etc/checkhost %c"
- RunsOnClient = No # default
- AbortJobOnError = Yes # default
- RunsWhen = Before
- }
- RunScript {
- Command = c:/bacula/systemstate.bat
- RunsOnClient = yes
- AbortJobOnError = No
- RunsWhen = After
- RunsOnFailure = yes
- }
-
- RunScript {
- Command = c:/bacula/deletestatefile.bat
- Target = rico-fd
- RunsWhen = Always
- }
-
- It's now possible to specify more than 1 command per Job.
- (you can stop your database and your webserver without a script)
-
- ex :
- Job {
- Name = "Client1"
- JobDefs = "DefaultJob"
- Write Bootstrap = "/tmp/bacula/var/bacula/working/Client1.bsr"
- FileSet = "Minimal"
-
- RunBeforeJob = "echo test before ; echo test before2"
- RunBeforeJob = "echo test before (2nd time)"
- RunBeforeJob = "echo test before (3rd time)"
- RunAfterJob = "echo test after"
- ClientRunAfterJob = "echo test after client"
-
- RunScript {
- Command = "echo test RunScript in error"
- Runsonclient = yes
- RunsOnSuccess = no
- RunsOnFailure = yes
- RunsWhen = After # never by default
- }
- RunScript {
- Command = "echo test RunScript on success"
- Runsonclient = yes
- RunsOnSuccess = yes # default
- RunsOnFailure = no # default
- RunsWhen = After
- }
- }
-
- Why: It would be a significant change to the structure of the
- directives, but allows for a lot more flexibility, including
- RunAfter commands that will run regardless of whether the job
- succeeds, or RunBefore tasks that still allow the job to run even
- if that specific RunBefore fails.
-
- Notes: (More notes from Phil, Kern, David and Eric)
- I would prefer to have a single new Resource called
- RunScript.
-
- RunsWhen = After|Before|Always
- RunsAtJobLevels = All|Full|Diff|Inc # not yet implemented
-
- The AbortJobOnError, RunsOnSuccess and RunsOnFailure directives
- could be optional, and possibly RunWhen as well.
-
- AbortJobOnError would be ignored unless RunsWhen was set to Before
- and would default to Yes if omitted.
- If AbortJobOnError was set to No, failure of the script
- would still generate a warning.
-
- RunsOnSuccess would be ignored unless RunsWhen was set to After
- (or RunsBeforeJob set to No), and default to Yes.
-
- RunsOnFailure would be ignored unless RunsWhen was set to After,
- and default to No.
-
- Allow having the before/after status on the script command
- line so that the same script can be used both before/after.
-
-Item 10: Merge multiple backups (Synthetic Backup or Consolidation).
- Origin: Marc Cousin and Eric Bollengier
- Date: 15 November 2005
- Status: Waiting implementation. Depends on first implementing
- project Item 2 (Migration).
-
- What: A merged backup is a backup made without connecting to the Client.
- It would be a Merge of existing backups into a single backup.
- In effect, it is like a restore but to the backup medium.
-
- For instance, say that last Sunday we made a full backup. Then
- all week long, we created incremental backups, in order to do
- them fast. Now comes Sunday again, and we need another full.
- The merged backup makes it possible to do instead an incremental
- backup (during the night for instance), and then create a merged
- backup during the day, by using the full and incrementals from
- the week. The merged backup will be exactly like a full made
- Sunday night on the tape, but the production interruption on the
- Client will be minimal, as the Client will only have to send
- incrementals.
-
- In fact, if it's done correctly, you could merge all the
- Incrementals into single Incremental, or all the Incrementals
- and the last Differential into a new Differential, or the Full,
- last differential and all the Incrementals into a new Full
- backup. And there is no need to involve the Client.
-
- Why: The benefit is that :
- - the Client just does an incremental ;
- - the merged backup on tape is just as a single full backup,
- and can be restored very fast.
-
- This is also a way of reducing the backup data since the old
- data can then be pruned (or not) from the catalog, possibly
- allowing older volumes to be recycled
-
-Item 11: Deletion of Disk-Based Bacula Volumes
- Date: Nov 25, 2005
- Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
- by Kern)
- Status:
-
- What: Provide a way for Bacula to automatically remove Volumes
- from the filesystem, or optionally to truncate them.
- Obviously, the Volume must be pruned prior removal.
-
- Why: This would allow users more control over their Volumes and
- prevent disk based volumes from consuming too much space.
-
- Notes: The following two directives might do the trick:
-
- Volume Data Retention = <time period>
- Remove Volume After = <time period>
-
- The migration project should also remove a Volume that is
- migrated. This might also work for tape Volumes.
-
-Item 12: Directive/mode to backup only file changes, not entire file
- Date: 11 November 2005
- Origin: Joshua Kugler <joshua dot kugler at uaf dot edu>
- Marek Bajon <mbajon at bimsplus dot com dot pl>
- Status:
-
- What: Currently when a file changes, the entire file will be backed up in
- the next incremental or full backup. To save space on the tapes
- it would be nice to have a mode whereby only the changes to the
- file would be backed up when it is changed.
-
- Why: This would save lots of space when backing up large files such as
- logs, mbox files, Outlook PST files and the like.
-
- Notes: This would require the usage of disk-based volumes as comparing
- files would not be feasible using a tape drive.
-
-Item 13: Multiple threads in file daemon for the same job