+ What: I would like Bacula to have the capability to write copies
+ of backed-up data on multiple physical volumes selected
+ from different pools without transferring the data
+ multiple times, and to accept any of the copy volumes
+ as valid for restore.
+
+ Why: In many cases, businesses are required to keep offsite
+ copies of backup volumes, or just wish for simple
+ protection against a human operator dropping a storage
+ volume and damaging it. The ability to generate multiple
+ volumes in the course of a single backup job allows
+ customers to simple check out one copy and send it
+ offsite, marking it as out of changer or otherwise
+ unavailable. Currently, the library and magazine
+ management capability in Bacula does not make this process
+ simple.
+
+ Restores would use the copy of the data on the first
+ available volume, in order of Copy pool chain definition.
+
+ This is also a major scalability issue -- as the number of
+ clients increases beyond several thousand, and the volume
+ of data increases, transferring the data multiple times to
+ produce additional copies of the backups will become
+ physically impossible due to transfer speed
+ issues. Generating multiple copies at server side will
+ become the only practical option.
+
+ How: I suspect that this will require adding a multiplexing
+ SD that appears to be a SD to a specific FD, but 1-n FDs
+ to the specific back end SDs managing the primary and copy
+ pools. Storage pools will also need to acquire parameters
+ to define the pools to be used for copies.
+
+ Notes: I would commit some of my developers' time if we can agree
+ on the design and behavior.
+
+ Notes: Additional notes from David:
+ I think there's two areas where new configuration would be needed.
+
+ 1) Identify a "SD mux" SD (specify it in the config just like a normal
+ SD. The SD configuration would need something like a "Daemon Type =
+ Normal/Mux" keyword to identify it as a multiplexor. (The director code
+ would need modification to add the ability to do the multiple session
+ setup, but the impact of the change would be new code that was invoked
+ only when a SDmux is needed).
+
+ 2) Additional keywords in the Pool definition to identify the need to
+ create copies. Each pool would acquire a Copypool= attribute (may be
+ repeated to generate more than one copy. 3 is about the practical limit,
+ but no point in hardcoding that).
+
+ Example:
+ Pool {
+ Name = Primary
+ Pool Type = Backup
+ Copypool = Copy1
+ Copypool = OffsiteCopy2
+ }
+
+ where Copy1 and OffsiteCopy2 are valid pools.
+
+ In terms of function (shorthand):
+ Backup job X is defined normally, specifying pool Primary as the pool to
+ use. Job gets scheduled, and Bacula starts scheduling resources.
+ Scheduler looks at pool definition for Primary, sees that there are a
+ non-zero number of copypool keywords. The director then connects to an
+ available SDmux, passes it the pool ids for Primary, Copy1, and
+ OffsiteCopy2 and waits. SDmux then goes out and reserves devices and
+ volumes in the normal SDs that serve Primary, Copy1 and OffsiteCopy2.
+ When all are ready, the SDmux signals ready back to the director, and
+ the FD is given the address of the SDmux as the SD to communicate with.
+ Backup proceeds normally, with the SDmux duplicating blocks to each
+ connected normal SD, and returning ready when all defined copies have
+ been written. At EOJ, FD shuts down connection with SDmux, which closes
+ down the normal SD connections and goes back to an idle state.
+ SDmux does not update database; normal SDs do (noting that file is
+ present on each volume it has been written to).
+
+ On restore, director looks for the volume containing the file in pool
+ Primary first, then Copy1, then OffsiteCopy2. If the volume holding the
+ file in pool Primary is missing or busy (being written in another job,
+ etc), or one of the volumes from the copypool list that have the file in
+ question is already mounted and ready for some reason, use it to do the
+ restore, else mount one of the copypool volumes and proceed.
+
+
+Item 9: Scheduling syntax that permits more flexibility and options
+ Date: 15 December 2006
+ Origin: Gregory Brauer (greg at wildbrain dot com) and
+ Florian Schnabel <florian.schnabel at docufy dot de>
+ Status:
+
+ What: Currently, Bacula only understands how to deal with weeks of the
+ month or weeks of the year in schedules. This makes it impossible
+ to do a true weekly rotation of tapes. There will always be a
+ discontinuity that will require disruptive manual intervention at
+ least monthly or yearly because week boundaries never align with
+ month or year boundaries.
+
+ A solution would be to add a new syntax that defines (at least)
+ a start timestamp, and repetition period.
+
+ An easy option to skip a certain job on a certain date.
+
+
+ Why: Rotated backups done at weekly intervals are useful, and Bacula
+ cannot currently do them without extensive hacking.
+
+ You could then easily skip tape backups on holidays. Especially
+ if you got no autochanger and can only fit one backup on a tape
+ that would be really handy, other jobs could proceed normally
+ and you won't get errors that way.
+
+
+ Notes: Here is an example syntax showing a 3-week rotation where full
+ Backups would be performed every week on Saturday, and an
+ incremental would be performed every week on Tuesday. Each
+ set of tapes could be removed from the loader for the following
+ two cycles before coming back and being reused on the third
+ week. Since the execution times are determined by intervals
+ from a given point in time, there will never be any issues with
+ having to adjust to any sort of arbitrary time boundary. In
+ the example provided, I even define the starting schedule
+ as crossing both a year and a month boundary, but the run times
+ would be based on the "Repeat" value and would therefore happen
+ weekly as desired.
+
+
+ Schedule {
+ Name = "Week 1 Rotation"
+ #Saturday. Would run Dec 30, Jan 20, Feb 10, etc.
+ Run {
+ Options {
+ Type = Full
+ Start = 2006-12-30 01:00
+ Repeat = 3w
+ }
+ }
+ #Tuesday. Would run Jan 2, Jan 23, Feb 13, etc.
+ Run {
+ Options {
+ Type = Incremental
+ Start = 2007-01-02 01:00
+ Repeat = 3w
+ }
+ }
+ }
+
+ Schedule {
+ Name = "Week 2 Rotation"
+ #Saturday. Would run Jan 6, Jan 27, Feb 17, etc.
+ Run {
+ Options {
+ Type = Full
+ Start = 2007-01-06 01:00
+ Repeat = 3w
+ }
+ }
+ #Tuesday. Would run Jan 9, Jan 30, Feb 20, etc.
+ Run {
+ Options {
+ Type = Incremental
+ Start = 2007-01-09 01:00
+ Repeat = 3w
+ }
+ }
+ }
+
+ Schedule {
+ Name = "Week 3 Rotation"
+ #Saturday. Would run Jan 13, Feb 3, Feb 24, etc.
+ Run {
+ Options {
+ Type = Full
+ Start = 2007-01-13 01:00
+ Repeat = 3w
+ }
+ }
+ #Tuesday. Would run Jan 16, Feb 6, Feb 27, etc.
+ Run {
+ Options {
+ Type = Incremental
+ Start = 2007-01-16 01:00
+ Repeat = 3w
+ }
+ }
+ }
+
+ Notes: Kern: I have merged the previously separate project of skipping
+ jobs (via Schedule syntax) into this.
+
+
+Item 10: Message mailing based on backup types
+ Origin: Evan Kaufman <evan.kaufman@gmail.com>
+ Date: January 6, 2006
+ Status:
+
+ What: In the "Messages" resource definitions, allowing messages
+ to be mailed based on the type (backup, restore, etc.) and level
+ (full, differential, etc) of job that created the originating
+ message(s).
+
+ Why: It would, for example, allow someone's boss to be emailed
+ automatically only when a Full Backup job runs, so he can
+ retrieve the tapes for offsite storage, even if the IT dept.
+ doesn't (or can't) explicitly notify him. At the same time, his
+ mailbox wouldnt be filled by notifications of Verifies, Restores,
+ or Incremental/Differential Backups (which would likely be kept
+ onsite).
+
+ Notes: One way this could be done is through additional message types, for example:
+
+ Messages {
+ # email the boss only on full system backups
+ Mail = boss@mycompany.com = full, !incremental, !differential, !restore,
+ !verify, !admin
+ # email us only when something breaks
+ MailOnError = itdept@mycompany.com = all
+ }
+
+ Notes: Kern: This should be rather trivial to implement.
+
+
+Item 11: Cause daemons to use a specific IP address to source communications
+ Origin: Bill Moran <wmoran@collaborativefusion.com>
+ Date: 18 Dec 2006
+ Status:
+ What: Cause Bacula daemons (dir, fd, sd) to always use the ip address
+ specified in the [DIR|DF|SD]Addr directive as the source IP
+ for initiating communication.
+ Why: On complex networks, as well as extremely secure networks, it's
+ not unusual to have multiple possible routes through the network.
+ Often, each of these routes is secured by different policies
+ (effectively, firewalls allow or deny different traffic depending
+ on the source address)
+ Unfortunately, it can sometimes be difficult or impossible to
+ represent this in a system routing table, as the result is
+ excessive subnetting that quickly exhausts available IP space.
+ The best available workaround is to provide multiple IPs to
+ a single machine that are all on the same subnet. In order
+ for this to work properly, applications must support the ability
+ to bind outgoing connections to a specified address, otherwise
+ the operating system will always choose the first IP that
+ matches the required route.
+ Notes: Many other programs support this. For example, the following
+ can be configured in BIND:
+ query-source address 10.0.0.1;
+ transfer-source 10.0.0.2;
+ Which means queries from this server will always come from
+ 10.0.0.1 and zone transfers will always originate from
+ 10.0.0.2.
+
+
+Item 12: Add Plug-ins to the FileSet Include statements.
+ Date: 28 October 2005
+ Origin: Kern
+ Status: Partially coded in 1.37 -- much more to do.
+
+ What: Allow users to specify wild-card and/or regular
+ expressions to be matched in both the Include and
+ Exclude directives in a FileSet. At the same time,
+ allow users to define plug-ins to be called (based on
+ regular expression/wild-card matching).
+
+ Why: This would give the users the ultimate ability to control
+ how files are backed up/restored. A user could write a
+ plug-in knows how to backup his Oracle database without
+ stopping/starting it, for example.
+
+
+Item 13: Restore only file attributes (permissions, ACL, owner, group...)
+ Origin: Eric Bollengier
+ Date: 30/12/2006
+ Status: Implemented by Eric, see project-restore-attributes-only.patch
+
+ What: The goal of this project is to be able to restore only rights
+ and attributes of files without crushing them.
+
+ Why: Who have never had to repair a chmod -R 777, or a wild update
+ of recursive right under Windows? At this time, you must have
+ enough space to restore data, dump attributes (easy with acl,
+ more complex with unix/windows rights) and apply them to your
+ broken tree. With this options, it will be very easy to compare
+ right or ACL over the time.
+
+ Notes: If the file is here, we skip restore and we change rights.
+ If the file isn't here, we can create an empty one and apply
+ rights or do nothing.
+
+ This will not work with win32 stream, because it seems that we
+ can't split the WriteBackup stream to get only ACL and ownerchip.
+
+Item 14: Add an override in Schedule for Pools based on backup types
+Date: 19 Jan 2005
+Origin: Chad Slater <chad.slater@clickfox.com>
+Status:
+
+ What: Adding a FullStorage=BigTapeLibrary in the Schedule resource
+ would help those of us who use different storage devices for different
+ backup levels cope with the "auto-upgrade" of a backup.
+
+ Why: Assume I add several new devices to be backed up, i.e. several
+ hosts with 1TB RAID. To avoid tape switching hassles, incrementals are
+ stored in a disk set on a 2TB RAID. If you add these devices in the
+ middle of the month, the incrementals are upgraded to "full" backups,
+ but they try to use the same storage device as requested in the
+ incremental job, filling up the RAID holding the differentials. If we
+ could override the Storage parameter for full and/or differential
+ backups, then the Full job would use the proper Storage device, which
+ has more capacity (i.e. a 8TB tape library.
+
+
+Item 15: Implement more Python events and functions
+ Date: 28 October 2005
+ Origin: Kern
+ Status:
+
+ What: Allow Python scripts to be called at more places
+ within Bacula and provide additional access to Bacula
+ internal variables.
+
+ Implement an interface for Python scripts to access the
+ catalog through Bacula.
+
+ Why: This will permit users to customize Bacula through
+ Python scripts.
+
+ Notes: Recycle event
+ Scratch pool event
+ NeedVolume event
+ MediaFull event
+
+ Also add a way to get a listing of currently running
+ jobs (possibly also scheduled jobs).
+
+
+ to start the appropriate job.