Projects:
Bacula Projects Roadmap
- Status updated 14 Jun 2009
+ Status updated 25 February 2010
Summary:
* => item complete
- Item 1: Ability to restart failed jobs
-*Item 2: 'restore' menu: enter a JobId, automatically select dependents
- Item 3: Scheduling syntax that permits more flexibility and options
- Item 4: Data encryption on storage daemon
- Item 5: Deletion of disk Volumes when pruned
- Item 6: Implement Base jobs
- Item 7: Add ability to Verify any specified Job.
- Item 8: Improve Bacula's tape and drive usage and cleaning management
- Item 9: Allow FD to initiate a backup
- Item 10: Restore from volumes on multiple storage daemons
- Item 11: Implement Storage daemon compression
- Item 12: Reduction of communications bandwidth for a backup
- Item 13: Ability to reconnect a disconnected comm line
- Item 14: Start spooling even when waiting on tape
- Item 15: Enable/disable compression depending on storage device (disk/tape)
- Item 16: Include all conf files in specified directory
- Item 17: Multiple threads in file daemon for the same job
- Item 18: Possibilty to schedule Jobs on last Friday of the month
- Item 19: Include timestamp of job launch in "stat clients" output
- Item 20: Cause daemons to use a specific IP address to source communications
- Item 21: Message mailing based on backup types
- Item 22: Ability to import/export Bacula database entities
- Item 23: "Maximum Concurrent Jobs" for drives when used with changer device
- Item 24: Implementation of running Job speed limit.
- Item 25: Add an override in Schedule for Pools based on backup types
- Item 26: Automatic promotion of backup levels based on backup size
- Item 27: Allow inclusion/exclusion of files in a fileset by creation/mod times
- Item 28: Archival (removal) of User Files to Tape
- Item 29: An option to operate on all pools with update vol parameters
- Item 30: Automatic disabling of devices
-*Item 31: List InChanger flag when doing restore.
- Item 32: Ability to defer Batch Insert to a later time
- Item 33: Add MaxVolumeSize/MaxVolumeBytes statement to Storage resource
- Item 34: Enable persistent naming/number of SQL queries
- Item 35: Port bat to Win32
- Item 36: Bacula Dir, FD and SD to support proxies
- Item 37: Add Minumum Spool Size directive
- Item 38: Backup and Restore of Windows Encrypted Files using Win raw encryption
- Item 39: Implement an interface between Bacula and Amazon's S3.
- Item 40: Convert Bacula existing tray monitor on Windows to a stand alone program
+Item 1: Ability to restart failed jobs
+Item 2: Scheduling syntax that permits more flexibility and options
+Item 3: Data encryption on storage daemon
+Item 4: Add ability to Verify any specified Job.
+Item 5: Improve Bacula's tape and drive usage and cleaning management
+Item 6: Allow FD to initiate a backup
+Item 7: Implement Storage daemon compression
+Item 8: Reduction of communications bandwidth for a backup
+Item 9: Ability to reconnect a disconnected comm line
+Item 10: Start spooling even when waiting on tape
+Item 11: Include all conf files in specified directory
+Item 12: Multiple threads in file daemon for the same job
+Item 13: Possibilty to schedule Jobs on last Friday of the month
+Item 14: Include timestamp of job launch in "stat clients" output
+Item 15: Message mailing based on backup types
+Item 16: Ability to import/export Bacula database entities
+Item 17: Implementation of running Job speed limit.
+Item 18: Add an override in Schedule for Pools based on backup types
+Item 19: Automatic promotion of backup levels based on backup size
+Item 20: Allow FileSet inclusion/exclusion by creation/mod times
+Item 21: Archival (removal) of User Files to Tape
+Item 22: An option to operate on all pools with update vol parameters
+Item 23: Automatic disabling of devices
+Item 24: Ability to defer Batch Insert to a later time
+Item 25: Add MaxVolumeSize/MaxVolumeBytes to Storage resource
+Item 26: Enable persistent naming/number of SQL queries
+Item 27: Bacula Dir, FD and SD to support proxies
+Item 28: Add Minumum Spool Size directive
+Item 29: Handle Windows Encrypted Files using Win raw encryption
+Item 30: Implement a Storage device like Amazon's S3.
+Item 31: Convert tray monitor on Windows to a stand alone program
+Item 32: Relabel disk volume after recycling
+Item 33: Command that releases all drives in an autochanger
+Item 34: Run bscan on a remote storage daemon from within bconsole.
+Item 35: Implement a Migration job type that will create a reverse
+Item 36: Job migration between different SDs
+Item 37: Concurrent spooling and despooling withini a single job.
+Item 39: Extend the verify code to make it possible to verify
+Item 40: Separate "Storage" and "Device" in the bacula-dir.conf
+Item 41: Least recently used device selection for tape drives in autochanger.
+
Item 1: Ability to restart failed jobs
Date: 26 April 2009
volume of data or files stored on Volume before enabling.
-Item 2: 'restore' menu: enter a JobId, automatically select dependents
-Origin: Graham Keeling (graham@equiinet.com)
-Date: 13 March 2009
-Status: Done in 3.0.2
-
-What: Add to the bconsole 'restore' menu the ability to select a job
- by JobId, and have bacula automatically select all the
- dependent jobs.
-
- Why: Currently, you either have to...
- a) laboriously type in a date that is greater than the date of the backup that
- you want and is less than the subsequent backup (bacula then figures out the
- dependent jobs), or
- b) manually figure out all the JobIds that you want and laboriously type them
- all in.
- It would be extremely useful (in a programmatical sense, as well as for humans)
- to be able to just give it a single JobId and let bacula do the hard work (work
- that it already knows how to do).
-
- Notes (Kern): I think this should either be modified to have Bacula print
- a list of dates that the user can choose from as is done in bwx-console and
- bat or the name of this command must be carefully chosen so that the user
- clearly understands that the JobId is being used to specify what Job and the
- date to which he wishes the restore to happen.
-
-
-Item 3: Scheduling syntax that permits more flexibility and options
+Item 2: Scheduling syntax that permits more flexibility and options
Date: 15 December 2006
Origin: Gregory Brauer (greg at wildbrain dot com) and
Florian Schnabel <florian.schnabel at docufy dot de>
jobs (via Schedule syntax) into this.
-Item 4: Data encryption on storage daemon
+Item 3: Data encryption on storage daemon
Origin: Tobias Barth <tobias.barth at web-arts.com>
Date: 04 February 2009
Status: new
- What: The storage demon should be able to do the data encryption that can currently be done by the file daemon.
+ What: The storage demon should be able to do the data encryption that can
+ currently be done by the file daemon.
- Why: This would have 2 advantages: 1) one could encrypt the data of unencrypted tapes by doing a migration job, and 2) the storage daemon would be the only machine that would have to keep the encryption keys.
+ Why: This would have 2 advantages:
+ 1) one could encrypt the data of unencrypted tapes by doing a
+ migration job
+ 2) the storage daemon would be the only machine that would have
+ to keep the encryption keys.
Notes from Landon:
As an addendum to the feature request, here are some crypto
http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg28860.html
-Item 5: Deletion of disk Volumes when pruned
- Date: Nov 25, 2005
- Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
- by Kern)
- Status:
-
- What: Provide a way for Bacula to automatically remove Volumes
- from the filesystem, or optionally to truncate them.
- Obviously, the Volume must be pruned prior removal.
-
- Why: This would allow users more control over their Volumes and
- prevent disk based volumes from consuming too much space.
-
- Notes: The following two directives might do the trick:
-
- Volume Data Retention = <time period>
- Remove Volume After = <time period>
-
- The migration project should also remove a Volume that is
- migrated. This might also work for tape Volumes.
-
- Notes: (Kern). The data fields to control this have been added
- to the new 3.0.0 database table structure.
-
-
-Item 6: Implement Base jobs
- Date: 28 October 2005
- Origin: Kern
- Status:
-
- What: A base job is sort of like a Full save except that you
- will want the FileSet to contain only files that are
- unlikely to change in the future (i.e. a snapshot of
- most of your system after installing it). After the
- base job has been run, when you are doing a Full save,
- you specify one or more Base jobs to be used. All
- files that have been backed up in the Base job/jobs but
- not modified will then be excluded from the backup.
- During a restore, the Base jobs will be automatically
- pulled in where necessary.
-
- Why: This is something none of the competition does, as far as
- we know (except perhaps BackupPC, which is a Perl program that
- saves to disk only). It is big win for the user, it
- makes Bacula stand out as offering a unique
- optimization that immediately saves time and money.
- Basically, imagine that you have 100 nearly identical
- Windows or Linux machine containing the OS and user
- files. Now for the OS part, a Base job will be backed
- up once, and rather than making 100 copies of the OS,
- there will be only one. If one or more of the systems
- have some files updated, no problem, they will be
- automatically restored.
-
- Notes: Huge savings in tape usage even for a single machine.
- Will require more resources because the DIR must send
- FD a list of files/attribs, and the FD must search the
- list and compare it for each file to be saved.
-
-
-Item 7: Add ability to Verify any specified Job.
+Item 4: Add ability to Verify any specified Job.
Date: 17 January 2008
Origin: portrix.net Hamburg, Germany.
Contact: Christian Sabelmann
Jobs whose file information are still in the catalog.
-Item 8: Improve Bacula's tape and drive usage and cleaning management
+Item 5: Improve Bacula's tape and drive usage and cleaning management
Date: 8 November 2005, November 11, 2005
Origin: Adam Thornton <athornton at sinenomine dot net>,
Arno Lehmann <al at its-lehmann dot de>
volumes, and handling drive cleaning and TAPEALERTs.
-Item 9: Allow FD to initiate a backup
+Item 6: Allow FD to initiate a backup
Origin: Frank Volf (frank at deze dot org)
Date: 17 November 2005
Status:
-What: Provide some means, possibly by a restricted console that
+What: Provide some means, possibly by a restricted console that
allows a FD to initiate a backup, and that uses the connection
established by the FD to the Director for the backup so that
a Director that is firewalled can do the backup.
-Why: Makes backup of laptops much easier.
-
-
-Item 10: Restore from volumes on multiple storage daemons
-Origin: Graham Keeling (graham@equiinet.com)
-Date: 12 March 2009
-Status: Proposing
-
-What: The ability to restore from volumes held by multiple storage daemons
- would be very useful.
-
-Why: It is useful to be able to backup to any number of different storage
- daemons. For example, your first storage daemon may run out of space, so you
- switch to your second and carry on. Bacula will currently let you do this.
- However, once you come to restore, bacula cannot cope when volumes on different
- storage daemons are required.
-
- Notes: The director knows that more than one storage daemon is needed, as
- bconsole outputs something like the following table.
-
- The job will require the following
- Volume(s) Storage(s) SD Device(s)
- ===========================================================================
-
- backup-0001 Disk 1 Disk 1.0
- backup-0002 Disk 2 Disk 2.0
-
- However, the bootstrap file that it creates gets sent to the first storage
- daemon only, which then stalls for a long time, 'waiting for a mount request'
- for the volume that it doesn't have.
- The bootstrap file contains no knowledge of the storage daemon.
- Under the current design:
-
- The director connects to the storage daemon, and gets an sd_auth_key.
- The director then connects to the file daemon, and gives it the
- sd_auth_key with the 'jobcmd'.
- (restoring of files happens)
- The director does a 'wait_for_storage_daemon_termination()'.
- The director waits for the file daemon to indicate the end of the job.
-
- With my idea:
-
- The director connects to the file daemon.
- Then, for each storage daemon in the .bsr file... {
- The director connects to the storage daemon, and gets an sd_auth_key.
- The director then connects to the file daemon, and gives it the
- sd_auth_key with the 'storaddr' command.
- (restoring of files happens)
- The director does a 'wait_for_storage_daemon_termination()'.
- The director waits for the file daemon to indicate the end of the
- work on this storage.
- }
- The director tells the file daemon that there are no more storages to contact.
- The director waits for the file daemon to indicate the end of the job.
-
- As you can see, each restore between the file daemon and storage daemon is
- handled in the same way that it is currently handled, using the same method
- for authentication, except that the sd_auth_key is moved from the 'jobcmd' to
- the 'storaddr' command - where it logically belongs.
-
-
-Item 11: Implement Storage daemon compression
+Why: Makes backup of laptops much easier.
+Notes: - The FD already has code for the monitor interface
+ - It could be nice to have a .job command that lists authorized
+ jobs.
+ - Commands need to be restricted on the Director side
+ (for example by re-using the runscript flag)
+ - The Client resource can be used to authorize the connection
+ - In a first time, the client can't modify job parameters
+ - We need a way to run a status command to follow job progression
+
+ This project consists of the following points
+ 1. Modify the FD to have a "mini-console" interface that
+ permits it to connect to the Director and start a
+ backup job of itself.
+ 2. The list of jobs that can be started by the FD are
+ defined in the Director (possibly via a restricted
+ console).
+ 3. Modify the existing tray monitor code in the Win32 FD
+ so that it is a separate program from the FD.
+ 4. The tray monitor program should be extended to permit
+ initiating a backup.
+ 5. No new Director directives should be added without
+ prior consultation with the Bacula developers.
+ 6. The comm line used by the FD to connect to the Director
+ should be re-used by the Director to do the backup.
+ This feature is partially implemented in the Director.
+ 7. The FD may have a new directive that allows it to start
+ a backup when the FD starts.
+ 8. The console interface to the FD should be extended to
+ permit a properly authorized console to initiate a
+ backup via the FD.
+
+
+Item 7: Implement Storage daemon compression
Date: 18 December 2006
Origin: Vadim A. Umanski , e-mail umanski@ext.ru
Status:
Notes:
-Item 12: Reduction of communications bandwidth for a backup
+Item 8: Reduction of communications bandwidth for a backup
Date: 14 October 2008
Origin: Robin O'Leary (Equiinet)
Status:
backup that will speed up subsequent backups.
-Item 13: Ability to reconnect a disconnected comm line
+Item 9: Ability to reconnect a disconnected comm line
Date: 26 April 2009
Origin: Kern/Eric
Status:
Notes: *Very* complicated from a design point of view because of authenication.
-Item 14: Start spooling even when waiting on tape
+Item 10: Start spooling even when waiting on tape
Origin: Tobias Barth <tobias.barth@web-arts.com>
Date: 25 April 2008
Status:
implemented.
-Item 15: Enable/disable compression depending on storage device (disk/tape)
- Origin: Ralf Gross ralf-lists@ralfgross.de
- Date: 2008-01-11
- Status: Initial Request
-
- What: Add a new option to the storage resource of the director. Depending
- on this option, compression will be enabled/disabled for a device.
-
- Why: If different devices (disks/tapes) are used for full/diff/incr
- backups, software compression will be enabled for all backups
- because of the FileSet compression option. For backup to tapes
- wich are able to do hardware compression this is not desired.
-
-
- Notes:
- http://news.gmane.org/gmane.comp.sysutils.backup.bacula.devel/cutoff=11124
- It must be clear to the user, that the FileSet compression option
- must still be enabled use compression for a backup job at all.
- Thus a name for the new option in the director must be
- well-defined.
-
- Notes: KES I think the Storage definition should probably override what
- is in the Job definition or vice-versa, but in any case, it must
- be well defined.
-
-
-Item 16: Include all conf files in specified directory
+Item 11: Include all conf files in specified directory
Date: 18 October 2008
Origin: Database, Lda. Maputo, Mozambique
Contact:Cameron Smith / cameron.ord@database.co.mz
/etc/bacula/clientdefs/clientname.conf
-Item 17: Multiple threads in file daemon for the same job
+Item 12: Multiple threads in file daemon for the same job
Date: 27 November 2005
Origin: Ove Risberg (Ove.Risberg at octocode dot com)
Status:
Why: Multiple concurrent backups of a large fileserver with many
disks and controllers will be much faster.
+ Notes: (KES) This is not necessary and could be accomplished
+ by having two jobs. In addition, the current VSS code
+ is single thread.
+
-Item 18: Possibilty to schedule Jobs on last Friday of the month
+Item 13: Possibilty to schedule Jobs on last Friday of the month
Origin: Carsten Menke <bootsy52 at gmx dot net>
Date: 02 March 2008
Status:
would expand to this {first|last} {Month|Week|Day|Mo-Fri} of the
{Year|Month|Week} you would be able to run really flexible jobs.
- To got a certain Job run on the last Friday of the Month for example one could
- then write
+ To got a certain Job run on the last Friday of the Month for example
+ one could then write
Run = pool=Monthly last Fri of the Month at 23:50
Run = pool=Monthly last Day of the Month at 23:50
-Item 19: Include timestamp of job launch in "stat clients" output
+Item 14: Include timestamp of job launch in "stat clients" output
Origin: Mark Bergman <mark.bergman@uphs.upenn.edu>
Date: Tue Aug 22 17:13:39 EDT 2006
Status:
particularly when there are many active clients.
-Item 20: Cause daemons to use a specific IP address to source communications
- Origin: Bill Moran <wmoran@collaborativefusion.com>
- Date: 18 Dec 2006
- Status: Done
- What: Cause Bacula daemons (dir, fd, sd) to always use the ip address
- specified in the [DIR|DF|SD]Addr directive as the source IP
- for initiating communication.
- Why: On complex networks, as well as extremely secure networks, it's
- not unusual to have multiple possible routes through the network.
- Often, each of these routes is secured by different policies
- (effectively, firewalls allow or deny different traffic depending
- on the source address)
- Unfortunately, it can sometimes be difficult or impossible to
- represent this in a system routing table, as the result is
- excessive subnetting that quickly exhausts available IP space.
- The best available workaround is to provide multiple IPs to
- a single machine that are all on the same subnet. In order
- for this to work properly, applications must support the ability
- to bind outgoing connections to a specified address, otherwise
- the operating system will always choose the first IP that
- matches the required route.
- Notes: Many other programs support this. For example, the following
- can be configured in BIND:
- query-source address 10.0.0.1;
- transfer-source 10.0.0.2;
- Which means queries from this server will always come from
- 10.0.0.1 and zone transfers will always originate from
- 10.0.0.2.
-
-
-Item 21: Message mailing based on backup types
+Item 15: Message mailing based on backup types
Origin: Evan Kaufman <evan.kaufman@gmail.com>
Date: January 6, 2006
Status:
or Incremental/Differential Backups (which would likely be kept
onsite).
- Notes: One way this could be done is through additional message types, for example:
+ Notes: One way this could be done is through additional message types, for
+ example:
Messages {
# email the boss only on full system backups
Notes: Kern: This should be rather trivial to implement.
-Item 22: Ability to import/export Bacula database entities
+Item 16: Ability to import/export Bacula database entities
Date: 26 April 2009
Origin: Eric
Status:
other criteria.
-Item 23: "Maximum Concurrent Jobs" for drives when used with changer device
- Origin: Ralf Gross ralf-lists <at> ralfgross.de
- Date: 2008-12-12
- Status: Initial Request
-
- What: respect the "Maximum Concurrent Jobs" directive in the _drives_
- Storage section in addition to the changer section
-
- Why: I have a 3 drive changer where I want to be able to let 3 concurrent
- jobs run in parallel. But only one job per drive at the same time.
- Right now I don't see how I could limit the number of concurrent jobs
- per drive in this situation.
-
- Notes: Using different priorities for these jobs lead to problems that other
- jobs are blocked. On the user list I got the advice to use the "Prefer Mounted
- Volumes" directive, but Kern advised against using "Prefer Mounted
- Volumes" in an other thread:
- http://article.gmane.org/gmane.comp.sysutils.backup.bacula.devel/11876/
-
- In addition I'm not sure if this would be the same as respecting the
- drive's "Maximum Concurrent Jobs" setting.
-
- Example:
-
- Storage {
- Name = Neo4100
- Address = ....
- SDPort = 9103
- Password = "wiped"
- Device = Neo4100
- Media Type = LTO4
- Autochanger = yes
- Maximum Concurrent Jobs = 3
- }
-
- Storage {
- Name = Neo4100-LTO4-D1
- Address = ....
- SDPort = 9103
- Password = "wiped"
- Device = ULTRIUM-TD4-D1
- Media Type = LTO4
- Maximum Concurrent Jobs = 1
- }
-
- [2 more drives]
-
- The "Maximum Concurrent Jobs = 1" directive in the drive's section is ignored.
-
-
-Item 24: Implementation of running Job speed limit.
+Item 17: Implementation of running Job speed limit.
Origin: Alex F, alexxzell at yahoo dot com
Date: 29 January 2009
especially where there is little available.
-Item 25: Add an override in Schedule for Pools based on backup types
+Item 18: Add an override in Schedule for Pools based on backup types
Date: 19 Jan 2005
Origin: Chad Slater <chad.slater@clickfox.com>
Status:
has more capacity (i.e. a 8TB tape library.
-Item 26: Automatic promotion of backup levels based on backup size
+Item 19: Automatic promotion of backup levels based on backup size
Date: 19 January 2006
Origin: Adam Thornton <athornton@sinenomine.net>
Status:
of).
-Item 27: Allow inclusion/exclusion of files in a fileset by creation/mod times
+Item 20: Allow FileSet inclusion/exclusion by creation/mod times
Origin: Evan Kaufman <evan.kaufman@gmail.com>
Date: January 11, 2006
Status:
or 'since'.
-Item 28: Archival (removal) of User Files to Tape
+Item 21: Archival (removal) of User Files to Tape
Date: Nov. 24/2005
Origin: Ray Pengelly [ray at biomed dot queensu dot ca
Status:
storage pool gets full) data is migrated to Tape.
-Item 29: An option to operate on all pools with update vol parameters
+Item 22: An option to operate on all pools with update vol parameters
Origin: Dmitriy Pinchukov <absh@bossdev.kiev.ua>
Date: 16 August 2006
Status: Patch made by Nigel Stepp
Volumes from Pool -> pool #.
-Item 30: Automatic disabling of devices
+Item 23: Automatic disabling of devices
Date: 2005-11-11
Origin: Peter Eriksson <peter at ifm.liu dot se>
Status:
instead.
-Item 31: List InChanger flag when doing restore.
- Origin: Jesper Krogh<jesper@krogh.cc>
- Date: 17 Oct 2008
- Status: Done in version 3.0.2
-
- What: When doing a restore the restore selection dialog ends by telling stuff
- like this:
- The job will require the following
- Volume(s) Storage(s) SD Device(s)
- ===========================================================================
- 000741L3 LTO-4 LTO3
- 000866L3 LTO-4 LTO3
- 000765L3 LTO-4 LTO3
- 000764L3 LTO-4 LTO3
- 000756L3 LTO-4 LTO3
- 001759L3 LTO-4 LTO3
- 001763L3 LTO-4 LTO3
- 001762L3 LTO-4 LTO3
- 001767L3 LTO-4 LTO3
-
- When having an autochanger, it would be really nice with an inChanger
- column so the operator knew if this restore job would stop waiting for
- operator intervention. This is done just by selecting the inChanger flag
- from the catalog and printing it in a seperate column.
-
-
- Why: This would help getting large restores through minimizing the
- time spent waiting for operator to drop by and change tapes in the library.
-
- Notes: [Kern] I think it would also be good to have the Slot as well,
- or some indication that Bacula thinks the volume is in the autochanger
- because it depends on both the InChanger flag and the Slot being
- valid.
-
-
-Item 32: Ability to defer Batch Insert to a later time
+Item 24: Ability to defer Batch Insert to a later time
Date: 26 April 2009
Origin: Eric
Status:
format (i.e. dependent on the import/export entities project).
-Item 33: Add MaxVolumeSize/MaxVolumeBytes statement to Storage resource
+Item 25: Add MaxVolumeSize/MaxVolumeBytes to Storage resource
Origin: Bastian Friedrich <bastian.friedrich@collax.com>
Date: 2008-07-09
Status: -
quite well.
-Item 34: Enable persistent naming/number of SQL queries
+Item 26: Enable persistent naming/number of SQL queries
Date: 24 Jan, 2007
Origin: Mark Bergman
Status:
than by number.
-Item 35: Port bat to Win32
- Date: 26 April 2009
- Origin: Kern/Eric
- Status:
-
- What: Make bat run on Win32/64.
-
- Why: To have GUI on Windows
-
- Notes:
-
-
-Item 36: Bacula Dir, FD and SD to support proxies
+Item 27: Bacula Dir, FD and SD to support proxies
Origin: Karl Grindley @ MIT Lincoln Laboratory <kgrindley at ll dot mit dot edu>
Date: 25 March 2009
Status: proposed
One could also possibly use stunnel, netcat, etc.
-Item 37: Add Minumum Spool Size directive
+Item 28: Add Minumum Spool Size directive
Date: 20 March 2008
Origin: Frank Sweetser <fs@wpi.edu>
gigabytes) it can easily produce multi-megabyte report emails!
-Item 38: Backup and Restore of Windows Encrypted Files using Win raw encryption
+Item 29: Handle Windows Encrypted Files using Win raw encryption
Origin: Michael Mohr, SAG Mohr.External@infineon.com
Date: 22 February 2008
Origin: Alex Ehrlich (Alex.Ehrlich-at-mail.ee)
encrypted-file-related callback functions.
-Item 39: Implement an interface between Bacula and Storage clould like Amazon's S3.
+Item 30: Implement a Storage device like Amazon's S3.
Date: 25 August 2008
Origin: Soren Hansen <soren@ubuntu.com>
Status: Not started.
if bacula want to recycle a volume, it will start by downloading the
file to truncate it few seconds later, if we can avoid that...
-Item 40: Convert Bacula existing tray monitor on Windows to a stand alone program
+Item 31: Convert tray monitor on Windows to a stand alone program
Date: 26 April 2009
Origin: Kern/Eric
Status:
a console connection).
-
-========= End items voted on May 2009 ==================
-
-========= New items after last vote ====================
-
-Item 1: Relabel disk volume after recycling
+Item 32: Relabel disk volume after recycling
Origin: Pasi Kärkkäinen <pasik@iki.fi>
Date: 07 May 2009.
Status: Not implemented yet, no code written.
- What: The ability to relabel the disk volume (and thus rename the file on the disk)
- after it has been recycled. Useful when you have a single job per disk volume,
- and you use a custom Label format, for example:
- Label Format = "${Client}-${Level}-${NumVols:p/4/0/r}-${Year}_${Month}_${Day}-${Hour}_${Minute}"
+ What: The ability to relabel the disk volume (and thus rename the file on the
+ disk) after it has been recycled. Useful when you have a single job
+ per disk volume, and you use a custom Label format, for example:
+ Label Format =
+ "${Client}-${Level}-${NumVols:p/4/0/r}-${Year}_${Month}_${Day}-${Hour}_${Minute}"
- Why: Disk volumes in Bacula get the label/filename when they are used for the first time.
- If you use recycling and custom label format like above, the disk
- volume name doesn't match the contents after it has been recycled.
- This feature makes it possible to keep the label/filename in sync
- with the content and thus makes it easy to check/monitor the backups
- from the shell and/or normal file management tools, because the filenames
- of the disk volumes match the content.
+ Why: Disk volumes in Bacula get the label/filename when they are used for the
+ first time. If you use recycling and custom label format like above,
+ the disk volume name doesn't match the contents after it has been
+ recycled. This feature makes it possible to keep the label/filename
+ in sync with the content and thus makes it easy to check/monitor the
+ backups from the shell and/or normal file management tools, because
+ the filenames of the disk volumes match the content.
Notes: The configuration option could be "Relabel after Recycling = Yes".
+Item 33: Command that releases all drives in an autochanger
+ Origin: Blake Dunlap (blake@nxs.net)
+ Date: 10/07/2009
+ Status: Request
+
+ What: It would be nice if there was a release command that
+ would release all drives in an autochanger instead of having to
+ do each one in turn.
+
+ Why: It can take some time for a release to occur, and the
+ commands must be given for each drive in turn, which can quicky
+ scale if there are several drives in the library. (Having to
+ watch the console, to give each command can waste a good bit of
+ time when you start getting into the 16 drive range when the
+ tapes can take up to 3 minutes to eject each)
+
+ Notes: Due to the way some autochangers/libraries work, you
+ cannot assume that new tapes inserted will go into slots that are
+ not currently believed to be in use by bacula (the tape from that
+ slot is in a drive). This would make any changes in
+ configuration quicker/easier, as all drives need to be released
+ before any modifications to slots.
+
+Item 34: Run bscan on a remote storage daemon from within bconsole.
+ Date: 07 October 2009
+ Origin: Graham Keeling <graham@equiinet.com>
+ Status: Proposing
+
+ What: The ability to be able to run bscan on a remote storage daemon from
+ within bconsole in order to populate your catalog.
+
+ Why: Currently, it seems you have to:
+ a) log in to a console on the remote machine
+ b) figure out where the storage daemon config file is
+ c) figure out the storage device from the config file
+ d) figure out the catalog IP address
+ e) figure out the catalog port
+ f) open the port on the catalog firewall
+ g) configure the catalog database to accept connections from the
+ remote host
+ h) build a 'bscan' command from (b)-(e) above and run it
+ It would be much nicer to be able to type something like this into
+ bconsole:
+ *bscan storage=<storage> device=<device> volume=<volume>
+ or something like:
+ *bscan storage=<storage> all
+ It seems to me that the scan could also do a better job than the
+ external bscan program currently does. It would possibly be able to
+ deduce some extra details, such as the catalog StorageId for the
+ volumes.
+
+ Notes: (Kern). If you need to do a bscan, you have done something wrong,
+ so this functionality should not need to be integrated into the
+ the Storage daemon. However, I am not opposed to someone implementing
+ this feature providing that all the code is in a shared object (or dll)
+ and does not add significantly to the size of the Storage daemon. In
+ addition, the code should be written in a way such that the same source
+ code is used in both the bscan program and the Storage daemon to avoid
+ adding a lot of new code that must be maintained by the project.
+
+Item 35: Implement a Migration job type that will create a reverse
+ incremental (or decremental) backup from two existing full backups.
+ Date: 05 October 2009
+ Origin: Griffith College Dublin. Some sponsorship available.
+ Contact: Gavin McCullagh <gavin.mccullagh@gcd.ie>
+ Status:
+
+ What: The ability to take two full backup jobs and derive a reverse
+ incremental backup from them. The older full backup data may then
+ be discarded.
+
+ Why: Long-term backups based on keeping full backups can be expensive in
+ media. In many cases (eg a NAS), as the client accumulates files
+ over months and years, the same file will be duplicated unchanged,
+ across many media and datasets. Eg, Less than 10% (and
+ shrinking) of our monthly full mail server backup is new files,
+ the other 90% is also in the previous full backup.
+ Regularly converting the oldest full backup into a reverse
+ incremental backup allows the admin to keep access to old backup
+ jobs, but remove all of the duplicated files, freeing up media.
+
+ Notes: This feature was previously discussed on the bacula-devel list
+ here: http://www.mail-archive.com/bacula-devel@lists.sourceforge.net/msg04962.html
+
+Item 36: Job migration between different SDs
+Origin: Mariusz Czulada <manieq AT wp DOT eu>
+Date: 07 May 2007
+Status: NEW
+
+What: Allow to specify in migration job devices on Storage Daemon other then
+ the one used for migrated jobs (possibly on different/distant host)
+
+Why: Sometimes we have more then one system which requires backup
+ implementation. Often, these systems are functionally unrelated and
+ placed in different locations. Having a big backup device (a tape
+ library) in each location is not cost-effective. It would be much
+ better to have one powerful enough tape library which could handle
+ backups from all systems, assuming relatively fast and reliable WAN
+ connections. In such architecture backups are done in service windows
+ on local bacula servers, then migrated to central storage off the peak
+ hours.
+
+Notes: If migration to different SD is working, migration to the same SD, as
+ now, could be done the same way (i mean 'localhost') to unify the
+ whole process
+
+Item 37: Concurrent spooling and despooling withini a single job.
+Date: 17 nov 2009
+Origin: Jesper Krogh <jesper@krogh.cc>
+Status: NEW
+What: When a job has spooling enabled and the spool area size is
+ less than the total volumes size the storage daemon will:
+ 1) Spool to spool-area
+ 2) Despool to tape
+ 3) Go to 1 if more data to be backed up.
+
+ Typical disks will serve data with a speed of 100MB/s when
+ dealing with large files, network it typical capable of doing 115MB/s
+ (GbitE). Tape drives will despool with 50-90MB/s (LTO3) 70-120MB/s
+ (LTO4) depending on compression and data.
+
+ As bacula currently works it'll hold back data from the client until
+ de-spooling is done, now matter if the spool area can handle another
+ block of data. Say given a FileSet of 4TB and a spool-area of 100GB and
+ a Maximum Job Spool Size set to 50GB then above sequence could be
+ changed to allow to spool to the other 50GB while despooling the first
+ 50GB and not holding back the client while doing it. As above numbers
+ show, depending on tape-drive and disk-arrays this potentially leads to
+ a cut of the backup-time of 50% for the individual jobs.
+
+ Real-world example, backing up 112.6GB (large files) to LTO4 tapes
+ (despools with ~75MB/s, data is gzipped on the remote filesystem.
+ Maximum Job Spool Size = 8GB
+
+ Current:
+ Size: 112.6GB
+ Elapsed time (total time): 46m 15s => 2775s
+ Despooling time: 25m 41s => 1541s (55%)
+ Spooling time: 20m 34s => 1234s (45%)
+ Reported speed: 40.58MB/s
+ Spooling speed: 112.6GB/1234s => 91.25MB/s
+ Despooling speed: 112.6GB/1541s => 73.07MB/s
+
+ So disk + net can "keep up" with the LTO4 drive (in this test)
+
+ Prosed change would effectively make the backup run in the "despooling
+ time" 1541s giving a reduction to 55% of the total run time.
+
+ In the situation where the individual job cannot keep up with LTO-drive
+ spooling enables efficient multiplexing of multiple concurrent jobs onto
+ the same drive.
+
+Why: When dealing with larger volumes the general utillization of the
+ network/disk is important to maximize in order to be able to run a full
+ backup over a weekend. Current work-around is to split the FileSet in
+ smaller FileSet and Jobs but that leads to more configuration mangement
+ and is harder to review for completeness. Subsequently it makes restores
+ more complex.
+
+Item 39: Extend the verify code to make it possible to verify
+ older jobs, not only the last one that has finished
+ Date: 10 April 2009
+ Origin: Ralf Gross (Ralf-Lists <at> ralfgross.de)
+ Status: not implemented or documented
+
+ What: At the moment a VolumeToCatalog job compares only the
+ last job with the data in the catalog. It's not possible
+ to compare the data (md5sums) of an older volume with the
+ data in the catalog.
+
+ Why: If a verify job fails, one has to immediately check the
+ source of the problem, fix it and rerun the verify job.
+ This has to happen before the next backup of the
+ verified backup job starts.
+ More important: It's not possible to check jobs that are
+ kept for a long time (archiv). If a jobid could be
+ specified for a verify job, older backups/tapes could be
+ checked on a regular base.
+
+ Notes: verify documentation:
+ VolumeToCatalog: This level causes Bacula to read the file
+ attribute data written to the Volume from the last Job [...]
+
+ Verify Job = <Job-Resource-Name> If you run a verify job
+ without this directive, the last job run will be compared
+ with the catalog, which means that you must immediately
+ follow a backup by a verify command. If you specify a Verify
+ Job Bacula will find the last job with that name that ran [...]
+
+ example bconsole verify dialog:
+
+ Run Verify job
+ JobName: VerifyServerXXX
+ Level: VolumeToCatalog
+ Client: ServerXXX-fd
+ FileSet: ServerXXX-Vol1
+ Pool: Full (From Job resource)
+ Storage: Neo4100 (From Pool resource)
+ Verify Job: ServerXXX-Vol1
+ Verify List:
+ When: 2009-04-20 09:03:04
+ Priority: 10
+ OK to run? (yes/mod/no): m
+ Parameters to modify:
+ 1: Level
+ 2: Storage
+ 3: Job
+ 4: FileSet
+ 5: Client
+ 6: When
+ 7: Priority
+ 8: Pool
+ 9: Verify Job
+
+
+
+Item 40: Separate "Storage" and "Device" in the bacula-dir.conf
+ Date: 29 April 2009
+ Origin: "James Harper" <james.harper@bendigoit.com.au>
+ Status: not implemented or documented
+
+ What: Separate "Storage" and "Device" in the bacula-dir.conf
+ The resulting config would looks something like:
+
+ Storage {
+ Name = name_of_server
+ Address = hostname/IP address
+ SDPort = 9103
+ Password = shh_its_a_secret
+ Maximum Concurrent Jobs = 7
+ }
+
+ Device {
+ Name = name_of_device
+ Storage = name_of_server
+ Device = name_of_device_on_sd
+ Media Type = media_type
+ Maximum Concurrent Jobs = 1
+ }
+
+ Maximum Concurrent Jobs would be specified with a server and a device
+ maximum, which would both be honoured by the director. Almost everything
+ that mentions a 'Storage' would need to be changed to 'Device', although
+ perhaps a 'Storage' would just be a synonym for 'Device' for backwards
+ compatibility...
+
+ Why: If you have multiple Storage definitions pointing to different
+ Devices in the same Storage daemon, the "status storage" command
+ prompts for each different device, but they all give the same
+ information.
+
+ Notes:
+
+Item 41: Least recently used device selection for tape drives in autochanger.
+Date: 12 October 2009
+Origin: Thomas Carter <tcarter@memc.com>
+Status: Proposal
+
+What: A better tape drive selection algorithm for multi-drive
+ autochangers. The AUTOCHANGER class contains an array list of tape
+ devices. When a tape drive is needed, this list is always searched in
+ order. This causes lower number drives (specifically drive 0) to do a
+ majority of the work with higher numbered drives possibly never being
+ used. When a drive in an autochanger is reserved for use, its entry should
+ be moved to the end of the list; this would give a rough LRU drive
+ selection.
+
+Why: The current implementation places a majority of use and wear on drive
+ 0 of a multi-drive autochanger.
+
+Notes:
+
+========= New items after last vote ====================
+
========= Add new items above this line =================
========== Items put on hold by Kern ============================
+
+
+========== Items completed in version 5.0.0 ====================
+*Item 2: 'restore' menu: enter a JobId, automatically select dependents
+*Item 5: Deletion of disk Volumes when pruned (partial -- truncate when pruned)
+*Item 6: Implement Base jobs
+*Item 10: Restore from volumes on multiple storage daemons
+*Item 15: Enable/disable compression depending on storage device (disk/tape)
+*Item 20: Cause daemons to use a specific IP address to source communications
+*Item 23: "Maximum Concurrent Jobs" for drives when used with changer device
+*Item 31: List InChanger flag when doing restore.
+*Item 35: Port bat to Win32
projects / bacula / bacula / blobdiff