- What: Provide some means, possibly by a restricted console that
- allows a FD to initiate a backup, and that uses the connection
- established by the FD to the Director for the backup so that
- a Director that is firewalled can do the backup.
-
- Why: Makes backup of laptops much easier.
-
-
-Item 4: Merge multiple backups (Synthetic Backup or Consolidation).
- Origin: Marc Cousin and Eric Bollengier
- Date: 15 November 2005
- Status: Waiting implementation. Depends on first implementing
- project Item 2 (Migration) which is now done.
-
- What: A merged backup is a backup made without connecting to the Client.
- It would be a Merge of existing backups into a single backup.
- In effect, it is like a restore but to the backup medium.
-
- For instance, say that last Sunday we made a full backup. Then
- all week long, we created incremental backups, in order to do
- them fast. Now comes Sunday again, and we need another full.
- The merged backup makes it possible to do instead an incremental
- backup (during the night for instance), and then create a merged
- backup during the day, by using the full and incrementals from
- the week. The merged backup will be exactly like a full made
- Sunday night on the tape, but the production interruption on the
- Client will be minimal, as the Client will only have to send
- incrementals.
-
- In fact, if it's done correctly, you could merge all the
- Incrementals into single Incremental, or all the Incrementals
- and the last Differential into a new Differential, or the Full,
- last differential and all the Incrementals into a new Full
- backup. And there is no need to involve the Client.
-
- Why: The benefit is that :
- - the Client just does an incremental ;
- - the merged backup on tape is just as a single full backup,
- and can be restored very fast.
-
- This is also a way of reducing the backup data since the old
- data can then be pruned (or not) from the catalog, possibly
- allowing older volumes to be recycled
-
-Item 5: Deletion of Disk-Based Bacula Volumes
- Date: Nov 25, 2005
- Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
- by Kern)
- Status:
-
- What: Provide a way for Bacula to automatically remove Volumes
- from the filesystem, or optionally to truncate them.
- Obviously, the Volume must be pruned prior removal.
-
- Why: This would allow users more control over their Volumes and
- prevent disk based volumes from consuming too much space.
-
- Notes: The following two directives might do the trick:
-
- Volume Data Retention = <time period>
- Remove Volume After = <time period>
-
- The migration project should also remove a Volume that is
- migrated. This might also work for tape Volumes.
-
-Item 6: Implement Base jobs.
- Date: 28 October 2005
- Origin: Kern
- Status:
-
- What: A base job is sort of like a Full save except that you
- will want the FileSet to contain only files that are
- unlikely to change in the future (i.e. a snapshot of
- most of your system after installing it). After the
- base job has been run, when you are doing a Full save,
- you specify one or more Base jobs to be used. All
- files that have been backed up in the Base job/jobs but
- not modified will then be excluded from the backup.
- During a restore, the Base jobs will be automatically
- pulled in where necessary.
-
- Why: This is something none of the competition does, as far as
- we know (except perhaps BackupPC, which is a Perl program that
- saves to disk only). It is big win for the user, it
- makes Bacula stand out as offering a unique
- optimization that immediately saves time and money.
- Basically, imagine that you have 100 nearly identical
- Windows or Linux machine containing the OS and user
- files. Now for the OS part, a Base job will be backed
- up once, and rather than making 100 copies of the OS,
- there will be only one. If one or more of the systems
- have some files updated, no problem, they will be
- automatically restored.
-
- Notes: Huge savings in tape usage even for a single machine.
- Will require more resources because the DIR must send
- FD a list of files/attribs, and the FD must search the
- list and compare it for each file to be saved.
-
-Item 7: Implement creation and maintenance of copy pools
- Date: 27 November 2005
- Origin: David Boyes (dboyes at sinenomine dot net)
- Status:
-
- What: I would like Bacula to have the capability to write copies
- of backed-up data on multiple physical volumes selected
- from different pools without transferring the data
- multiple times, and to accept any of the copy volumes
- as valid for restore.
-
- Why: In many cases, businesses are required to keep offsite
- copies of backup volumes, or just wish for simple
- protection against a human operator dropping a storage
- volume and damaging it. The ability to generate multiple
- volumes in the course of a single backup job allows
- customers to simple check out one copy and send it
- offsite, marking it as out of changer or otherwise
- unavailable. Currently, the library and magazine
- management capability in Bacula does not make this process
- simple.
-
- Restores would use the copy of the data on the first
- available volume, in order of copy pool chain definition.
-
- This is also a major scalability issue -- as the number of
- clients increases beyond several thousand, and the volume
- of data increases, transferring the data multiple times to
- produce additional copies of the backups will become
- physically impossible due to transfer speed
- issues. Generating multiple copies at server side will
- become the only practical option.
-
- How: I suspect that this will require adding a multiplexing
- SD that appears to be a SD to a specific FD, but 1-n FDs
- to the specific back end SDs managing the primary and copy
- pools. Storage pools will also need to acquire parameters
- to define the pools to be used for copies.
-
- Notes: I would commit some of my developers' time if we can agree
- on the design and behavior.
-
-Item 8: Directive/mode to backup only file changes, not entire file
- Date: 11 November 2005
- Origin: Joshua Kugler <joshua dot kugler at uaf dot edu>
- Marek Bajon <mbajon at bimsplus dot com dot pl>
- Status:
-
- What: Currently when a file changes, the entire file will be backed up in
- the next incremental or full backup. To save space on the tapes
- it would be nice to have a mode whereby only the changes to the
- file would be backed up when it is changed.
-
- Why: This would save lots of space when backing up large files such as
- logs, mbox files, Outlook PST files and the like.
-
- Notes: This would require the usage of disk-based volumes as comparing
- files would not be feasible using a tape drive.
-
-Item 9: Implement a server-side compression feature
- Date: 18 December 2006
- Origin: Vadim A. Umanski , e-mail umanski@ext.ru
- Status:
- What: The ability to compress backup data on server receiving data
- instead of doing that on client sending data.
- Why: The need is practical. I've got some machines that can send
- data to the network 4 or 5 times faster than compressing
- them (I've measured that). They're using fast enough SCSI/FC
- disk subsystems but rather slow CPUs (ex. UltraSPARC II).
- And the backup server has got a quite fast CPUs (ex. Dual P4
- Xeons) and quite a low load. When you have 20, 50 or 100 GB
- of raw data - running a job 4 to 5 times faster - that
- really matters. On the other hand, the data can be
- compressed 50% or better - so losing twice more space for
- disk backup is not good at all. And the network is all mine
- (I have a dedicated management/provisioning network) and I
- can get as high bandwidth as I need - 100Mbps, 1000Mbps...
- That's why the server-side compression feature is needed!
- Notes:
-
-Item 10: Improve Bacula's tape and drive usage and cleaning management.
- Date: 8 November 2005, November 11, 2005
- Origin: Adam Thornton <athornton at sinenomine dot net>,
- Arno Lehmann <al at its-lehmann dot de>
- Status:
-
- What: Make Bacula manage tape life cycle information, tape reuse
- times and drive cleaning cycles.
-
- Why: All three parts of this project are important when operating
- backups.
- We need to know which tapes need replacement, and we need to
- make sure the drives are cleaned when necessary. While many
- tape libraries and even autoloaders can handle all this
- automatically, support by Bacula can be helpful for smaller
- (older) libraries and single drives. Limiting the number of
- times a tape is used might prevent tape errors when using
- tapes until the drives can't read it any more. Also, checking
- drive status during operation can prevent some failures (as I
- [Arno] had to learn the hard way...)
-
- Notes: First, Bacula could (and even does, to some limited extent)
- record tape and drive usage. For tapes, the number of mounts,
- the amount of data, and the time the tape has actually been
- running could be recorded. Data fields for Read and Write
- time and Number of mounts already exist in the catalog (I'm
- not sure if VolBytes is the sum of all bytes ever written to
- that volume by Bacula). This information can be important
- when determining which media to replace. The ability to mark
- Volumes as "used up" after a given number of write cycles
- should also be implemented so that a tape is never actually
- worn out. For the tape drives known to Bacula, similar
- information is interesting to determine the device status and
- expected life time: Time it's been Reading and Writing, number
- of tape Loads / Unloads / Errors. This information is not yet
- recorded as far as I [Arno] know. A new volume status would
- be necessary for the new state, like "Used up" or "Worn out".
- Volumes with this state could be used for restores, but not
- for writing. These volumes should be migrated first (assuming
- migration is implemented) and, once they are no longer needed,
- could be moved to a Trash pool.
-
- The next step would be to implement a drive cleaning setup.
- Bacula already has knowledge about cleaning tapes. Once it
- has some information about cleaning cycles (measured in drive
- run time, number of tapes used, or calender days, for example)
- it can automatically execute tape cleaning (with an
- autochanger, obviously) or ask for operator assistance loading
- a cleaning tape.
-
- The final step would be to implement TAPEALERT checks not only
- when changing tapes and only sending the information to the
- administrator, but rather checking after each tape error,
- checking on a regular basis (for example after each tape
- file), and also before unloading and after loading a new tape.
- Then, depending on the drives TAPEALERT state and the known
- drive cleaning state Bacula could automatically schedule later
- cleaning, clean immediately, or inform the operator.
-
- Implementing this would perhaps require another catalog change
- and perhaps major changes in SD code and the DIR-SD protocol,
- so I'd only consider this worth implementing if it would
- actually be used or even needed by many people.
-
- Implementation of these projects could happen in three distinct
- sub-projects: Measuring Tape and Drive usage, retiring
- volumes, and handling drive cleaning and TAPEALERTs.
-
-Item 11: Allow skipping execution of Jobs
- Date: 29 November 2005
- Origin: Florian Schnabel <florian.schnabel at docufy dot de>
- Status:
-
- What: An easy option to skip a certain job on a certain date.
- Why: You could then easily skip tape backups on holidays. Especially
- if you got no autochanger and can only fit one backup on a tape
- that would be really handy, other jobs could proceed normally
- and you won't get errors that way.
-
-Item 12: Add a scheduling syntax that permits weekly rotations
- Date: 15 December 2006
- Origin: Gregory Brauer (greg at wildbrain dot com)
- Status:
-
- What: Currently, Bacula only understands how to deal with weeks of the