-Item 11: Cause daemons to use a specific IP address to source communications
- Origin: Bill Moran <wmoran@collaborativefusion.com>
- Date: 18 Dec 2006
- Status:
- What: Cause Bacula daemons (dir, fd, sd) to always use the ip address
- specified in the [DIR|DF|SD]Addr directive as the source IP
- for initiating communication.
- Why: On complex networks, as well as extremely secure networks, it's
- not unusual to have multiple possible routes through the network.
- Often, each of these routes is secured by different policies
- (effectively, firewalls allow or deny different traffic depending
- on the source address)
- Unfortunately, it can sometimes be difficult or impossible to
- represent this in a system routing table, as the result is
- excessive subnetting that quickly exhausts available IP space.
- The best available workaround is to provide multiple IPs to
- a single machine that are all on the same subnet. In order
- for this to work properly, applications must support the ability
- to bind outgoing connections to a specified address, otherwise
- the operating system will always choose the first IP that
- matches the required route.
- Notes: Many other programs support this. For example, the following
- can be configured in BIND:
- query-source address 10.0.0.1;
- transfer-source 10.0.0.2;
- Which means queries from this server will always come from
- 10.0.0.1 and zone transfers will always originate from
- 10.0.0.2.
-
-
-Item 14: Add an override in Schedule for Pools based on backup types
-Date: 19 Jan 2005
-Origin: Chad Slater <chad.slater@clickfox.com>
-Status:
-
- What: Adding a FullStorage=BigTapeLibrary in the Schedule resource
- would help those of us who use different storage devices for different
- backup levels cope with the "auto-upgrade" of a backup.
-
- Why: Assume I add several new devices to be backed up, i.e. several
- hosts with 1TB RAID. To avoid tape switching hassles, incrementals are
- stored in a disk set on a 2TB RAID. If you add these devices in the
- middle of the month, the incrementals are upgraded to "full" backups,
- but they try to use the same storage device as requested in the
- incremental job, filling up the RAID holding the differentials. If we
- could override the Storage parameter for full and/or differential
- backups, then the Full job would use the proper Storage device, which
- has more capacity (i.e. a 8TB tape library.
-
-
-Item 15: Implement more Python events and functions
- Date: 28 October 2005
- Origin: Kern
- Status: Project abandoned in favor of plugins.
-
- What: Allow Python scripts to be called at more places
- within Bacula and provide additional access to Bacula
- internal variables.
-
- Implement an interface for Python scripts to access the
- catalog through Bacula.
-
- Why: This will permit users to customize Bacula through
- Python scripts.
-
- Notes: Recycle event
- Scratch pool event
- NeedVolume event
- MediaFull event
-
- Also add a way to get a listing of currently running
- jobs (possibly also scheduled jobs).
-
-
- to start the appropriate job.
-
-
-Item 16: Allow inclusion/exclusion of files in a fileset by creation/mod times
- Origin: Evan Kaufman <evan.kaufman@gmail.com>
- Date: January 11, 2006
- Status:
-
- What: In the vein of the Wild and Regex directives in a Fileset's
- Options, it would be helpful to allow a user to include or exclude
- files and directories by creation or modification times.
-
- You could factor the Exclude=yes|no option in much the same way it
- affects the Wild and Regex directives. For example, you could exclude
- all files modified before a certain date:
-
- Options {
- Exclude = yes
- Modified Before = ####
- }
-
- Or you could exclude all files created/modified since a certain date:
-
- Options {
- Exclude = yes
- Created Modified Since = ####
- }
-
- The format of the time/date could be done several ways, say the number
- of seconds since the epoch:
- 1137008553 = Jan 11 2006, 1:42:33PM # result of `date +%s`
-
- Or a human readable date in a cryptic form:
- 20060111134233 = Jan 11 2006, 1:42:33PM # YYYYMMDDhhmmss
-
- Why: I imagine a feature like this could have many uses. It would
- allow a user to do a full backup while excluding the base operating
- system files, so if I installed a Linux snapshot from a CD yesterday,
- I'll *exclude* all files modified *before* today. If I need to
- recover the system, I use the CD I already have, plus the tape backup.
- Or if, say, a Windows client is hit by a particularly corrosive
- virus, and I need to *exclude* any files created/modified *since* the
- time of infection.
-
- Notes: Of course, this feature would work in concert with other
- in/exclude rules, and wouldnt override them (or each other).
-
- Notes: The directives I'd imagine would be along the lines of
- "[Created] [Modified] [Before|Since] = <date>".
- So one could compare against 'ctime' and/or 'mtime', but ONLY 'before'
- or 'since'.
-
-
-Item 17: Automatic promotion of backup levels based on backup size
- Date: 19 January 2006
- Origin: Adam Thornton <athornton@sinenomine.net>
- Status:
-
- What: Amanda has a feature whereby it estimates the space that a
- differential, incremental, and full backup would take. If the
- difference in space required between the scheduled level and the next
- level up is beneath some user-defined critical threshold, the backup
- level is bumped to the next type. Doing this minimizes the number of
- volumes necessary during a restore, with a fairly minimal cost in
- backup media space.
-
- Why: I know at least one (quite sophisticated and smart) user
- for whom the absence of this feature is a deal-breaker in terms of
- using Bacula; if we had it it would eliminate the one cool thing
- Amanda can do and we can't (at least, the one cool thing I know of).
-
-
-Item 19: Automatic disabling of devices
- Date: 2005-11-11
- Origin: Peter Eriksson <peter at ifm.liu dot se>
- Status:
-
- What: After a configurable amount of fatal errors with a tape drive
- Bacula should automatically disable further use of a certain
- tape drive. There should also be "disable"/"enable" commands in
- the "bconsole" tool.
-
- Why: On a multi-drive jukebox there is a possibility of tape drives
- going bad during large backups (needing a cleaning tape run,
- tapes getting stuck). It would be advantageous if Bacula would
- automatically disable further use of a problematic tape drive
- after a configurable amount of errors has occurred.
-
- An example: I have a multi-drive jukebox (6 drives, 380+ slots)
- where tapes occasionally get stuck inside the drive. Bacula will
- notice that the "mtx-changer" command will fail and then fail
- any backup jobs trying to use that drive. However, it will still
- keep on trying to run new jobs using that drive and fail -
- forever, and thus failing lots and lots of jobs... Since we have
- many drives Bacula could have just automatically disabled
- further use of that drive and used one of the other ones
- instead.
-
-Item 20: An option to operate on all pools with update vol parameters
- Origin: Dmitriy Pinchukov <absh@bossdev.kiev.ua>
- Date: 16 August 2006
- Status: Patch made by Nigel Stepp
-
- What: When I do update -> Volume parameters -> All Volumes
- from Pool, then I have to select pools one by one. I'd like
- console to have an option like "0: All Pools" in the list of
- defined pools.
-
- Why: I have many pools and therefore unhappy with manually
- updating each of them using update -> Volume parameters -> All
- Volumes from Pool -> pool #.
-
-
-Item 21: Include timestamp of job launch in "stat clients" output
- Origin: Mark Bergman <mark.bergman@uphs.upenn.edu>
- Date: Tue Aug 22 17:13:39 EDT 2006
- Status:
-
- What: The "stat clients" command doesn't include any detail on when
- the active backup jobs were launched.
-
- Why: Including the timestamp would make it much easier to decide whether
- a job is running properly.
-
- Notes: It may be helpful to have the output from "stat clients" formatted
- more like that from "stat dir" (and other commands), in a column
- format. The per-client information that's currently shown (level,
- client name, JobId, Volume, pool, device, Files, etc.) is good, but
- somewhat hard to parse (both programmatically and visually),
- particularly when there are many active clients.
-
-
-
-Item 22: Implement Storage daemon compression
- Date: 18 December 2006
- Origin: Vadim A. Umanski , e-mail umanski@ext.ru
- Status:
- What: The ability to compress backup data on the SD receiving data
- instead of doing that on client sending data.
- Why: The need is practical. I've got some machines that can send
- data to the network 4 or 5 times faster than compressing
- them (I've measured that). They're using fast enough SCSI/FC
- disk subsystems but rather slow CPUs (ex. UltraSPARC II).
- And the backup server has got a quite fast CPUs (ex. Dual P4
- Xeons) and quite a low load. When you have 20, 50 or 100 GB
- of raw data - running a job 4 to 5 times faster - that
- really matters. On the other hand, the data can be
- compressed 50% or better - so losing twice more space for
- disk backup is not good at all. And the network is all mine
- (I have a dedicated management/provisioning network) and I
- can get as high bandwidth as I need - 100Mbps, 1000Mbps...
- That's why the server-side compression feature is needed!
- Notes:
-
-Item 23: Improve Bacula's tape and drive usage and cleaning management
- Date: 8 November 2005, November 11, 2005
- Origin: Adam Thornton <athornton at sinenomine dot net>,
- Arno Lehmann <al at its-lehmann dot de>
- Status:
-
- What: Make Bacula manage tape life cycle information, tape reuse
- times and drive cleaning cycles.
-
- Why: All three parts of this project are important when operating
- backups.
- We need to know which tapes need replacement, and we need to
- make sure the drives are cleaned when necessary. While many
- tape libraries and even autoloaders can handle all this
- automatically, support by Bacula can be helpful for smaller
- (older) libraries and single drives. Limiting the number of
- times a tape is used might prevent tape errors when using
- tapes until the drives can't read it any more. Also, checking
- drive status during operation can prevent some failures (as I
- [Arno] had to learn the hard way...)
-
- Notes: First, Bacula could (and even does, to some limited extent)
- record tape and drive usage. For tapes, the number of mounts,
- the amount of data, and the time the tape has actually been
- running could be recorded. Data fields for Read and Write
- time and Number of mounts already exist in the catalog (I'm
- not sure if VolBytes is the sum of all bytes ever written to
- that volume by Bacula). This information can be important
- when determining which media to replace. The ability to mark
- Volumes as "used up" after a given number of write cycles
- should also be implemented so that a tape is never actually
- worn out. For the tape drives known to Bacula, similar
- information is interesting to determine the device status and
- expected life time: Time it's been Reading and Writing, number
- of tape Loads / Unloads / Errors. This information is not yet
- recorded as far as I [Arno] know. A new volume status would
- be necessary for the new state, like "Used up" or "Worn out".
- Volumes with this state could be used for restores, but not
- for writing. These volumes should be migrated first (assuming
- migration is implemented) and, once they are no longer needed,
- could be moved to a Trash pool.
-
- The next step would be to implement a drive cleaning setup.
- Bacula already has knowledge about cleaning tapes. Once it
- has some information about cleaning cycles (measured in drive
- run time, number of tapes used, or calender days, for example)
- it can automatically execute tape cleaning (with an
- autochanger, obviously) or ask for operator assistance loading
- a cleaning tape.
-
- The final step would be to implement TAPEALERT checks not only
- when changing tapes and only sending the information to the
- administrator, but rather checking after each tape error,
- checking on a regular basis (for example after each tape
- file), and also before unloading and after loading a new tape.
- Then, depending on the drives TAPEALERT state and the known
- drive cleaning state Bacula could automatically schedule later
- cleaning, clean immediately, or inform the operator.
-
- Implementing this would perhaps require another catalog change
- and perhaps major changes in SD code and the DIR-SD protocol,
- so I'd only consider this worth implementing if it would
- actually be used or even needed by many people.
-
- Implementation of these projects could happen in three distinct
- sub-projects: Measuring Tape and Drive usage, retiring
- volumes, and handling drive cleaning and TAPEALERTs.
-
-Item 24: Multiple threads in file daemon for the same job
- Date: 27 November 2005
- Origin: Ove Risberg (Ove.Risberg at octocode dot com)
- Status:
-
- What: I want the file daemon to start multiple threads for a backup
- job so the fastest possible backup can be made.
-
- The file daemon could parse the FileSet information and start
- one thread for each File entry located on a separate
- filesystem.
-
- A confiuration option in the job section should be used to
- enable or disable this feature. The confgutration option could
- specify the maximum number of threads in the file daemon.
-
- If the theads could spool the data to separate spool files
- the restore process will not be much slower.
-
- Why: Multiple concurrent backups of a large fileserver with many
- disks and controllers will be much faster.
-
-Item 25: Archival (removal) of User Files to Tape
- Date: Nov. 24/2005
- Origin: Ray Pengelly [ray at biomed dot queensu dot ca
- Status:
-
- What: The ability to archive data to storage based on certain parameters
- such as age, size, or location. Once the data has been written to
- storage and logged it is then pruned from the originating
- filesystem. Note! We are talking about user's files and not
- Bacula Volumes.
-
- Why: This would allow fully automatic storage management which becomes
- useful for large datastores. It would also allow for auto-staging
- from one media type to another.
-
- Example 1) Medical imaging needs to store large amounts of data.
- They decide to keep data on their servers for 6 months and then put
- it away for long term storage. The server then finds all files
- older than 6 months writes them to tape. The files are then removed
- from the server.
-
- Example 2) All data that hasn't been accessed in 2 months could be
- moved from high-cost, fibre-channel disk storage to a low-cost
- large-capacity SATA disk storage pool which doesn't have as quick of
- access time. Then after another 6 months (or possibly as one
- storage pool gets full) data is migrated to Tape.
-
-
-========= New Items since the last vote =================
-
-Item 26: Add a new directive to bacula-dir.conf which permits inclusion of all subconfiguration files in a given directory
-Date: 18 October 2008
-Origin: Database, Lda. Maputo, Mozambique
-Contact:Cameron Smith / cameron.ord@database.co.mz
-Status: New request
-
-What: A directive something like "IncludeConf = /etc/bacula/subconfs" Every
- time Bacula Director restarts or reloads, it will walk the given
- directory (non-recursively) and include the contents of any files
- therein, as though they were appended to bacula-dir.conf
-
-Why: Permits simplified and safer configuration for larger installations with
- many client PCs. Currently, through judicious use of JobDefs and
- similar directives, it is possible to reduce the client-specific part of
- a configuration to a minimum. The client-specific directives can be
- prepared according to a standard template and dropped into a known
- directory. However it is still necessary to add a line to the "master"
- (bacula-dir.conf) referencing each new file. This exposes the master to
- unnecessary risk of accidental mistakes and makes automation of adding
- new client-confs, more difficult (it is easier to automate dropping a
- file into a dir, than rewriting an existing file). Ken has previously
- made a convincing argument for NOT including Bacula's core configuration
- in an RDBMS, but I believe that the present request is a reasonable
- extension to the current "flat-file-based" configuration philosophy.
-
-Notes: There is NO need for any special syntax to these files. They should
- contain standard directives which are simply "inlined" to the parent
- file as already happens when you explicitly reference an external file.
-
-Notes: (kes) this can already be done with scripting
- From: John Jorgensen <jorgnsn@lcd.uregina.ca>
- The bacula-dir.conf at our site contains these lines:
-
- #
- # Include subfiles associated with configuration of clients.
- # They define the bulk of the Clients, Jobs, and FileSets.
- #
- @|"sh -c 'for f in /etc/bacula/clientdefs/*.conf ; do echo @${f} ; done'"
-
- and when we get a new client, we just put its configuration into
- a new file called something like:
-
- /etc/bacula/clientdefs/clientname.conf
-
-
- Item n: List inChanger flag when doing restore.
- Origin: Jesper Krogh<jesper@krogh.cc>
- Date: 17 oct. 2008
- Status:
-
- What: When doing a restore the restore selection dialog ends by telling stuff
- like this:
- The job will require the following
- Volume(s) Storage(s) SD Device(s)
- ===========================================================================
- 000741L3 LTO-4 LTO3
- 000866L3 LTO-4 LTO3
- 000765L3 LTO-4 LTO3
- 000764L3 LTO-4 LTO3
- 000756L3 LTO-4 LTO3
- 001759L3 LTO-4 LTO3
- 001763L3 LTO-4 LTO3
- 001762L3 LTO-4 LTO3
- 001767L3 LTO-4 LTO3
-
- When having an autochanger, it would be really nice with an inChanger
- column so the operator knew if this restore job would stop waiting for
- operator intervention. This is done just by selecting the inChanger flag
- from the catalog and printing it in a seperate column.
-
-
- Why: This would help getting large restores through minimizing the
- time spent waiting for operator to drop by and change tapes in the library.
-
- Notes: [Kern] I think it would also be good to have the Slot as well,
- or some indication that Bacula thinks the volume is in the autochanger
- because it depends on both the InChanger flag and the Slot being
- valid.
-
-
-Item 1: Implement an interface between Bacula and Amazon's S3.
- Date: 25 August 2008
- Origin: Soren Hansen <soren@ubuntu.com>
- Status: Not started.
- What: Enable the storage daemon to store backup data on Amazon's
- S3 service.
-
- Why: Amazon's S3 is a cheap way to store data off-site. Current
- ways to integrate Bacula and S3 involve storing all the data
- locally and syncing them to S3, and manually fetching them
- again when they're needed. This is very cumbersome.
-
-
-Item 1: enable/disable compression depending on storage device (disk/tape)
- Origin: Ralf Gross ralf-lists@ralfgross.de
- Date: 2008-01-11
- Status: Initial Request
-
- What: Add a new option to the storage resource of the director. Depending
- on this option, compression will be enabled/disabled for a device.
-
- Why: If different devices (disks/tapes) are used for full/diff/incr
- backups, software compression will be enabled for all backups
- because of the FileSet compression option. For backup to tapes
- wich are able to do hardware compression this is not desired.
-
-
- Notes:
- http://news.gmane.org/gmane.comp.sysutils.backup.bacula.devel/cutoff=11124
- It must be clear to the user, that the FileSet compression option
- must still be enabled use compression for a backup job at all.
- Thus a name for the new option in the director must be
- well-defined.
-
- Notes: KES I think the Storage definition should probably override what
- is in the Job definition or vice-versa, but in any case, it must
- be well defined.
-
-
-Item 1: Backup and Restore of Windows Encrypted Files through raw encryption
- functions
-