X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=bacula%2Fprojects;h=385ee5106e0844ebc7b502966b1db0c397aec490;hb=5821457bbef2394cbf205e2a527917d11c1b6e8b;hp=544fc660cd124453fc3c9b39574a867f41e548b8;hpb=49879d44e0c6fdf9f2bc2773b767a73decb2cafd;p=bacula%2Fbacula

diff --git a/bacula/projects b/bacula/projects
index 544fc660cd..385ee5106e 100644
--- a/bacula/projects
+++ b/bacula/projects
@@ -1,113 +1,208 @@
                 
 Projects:
                      Bacula Projects Roadmap 
-                       24 November 2005
-
-Below, you will find more information on future projects:
-
-Item 1:   Implement a Migration job type that will move the job
-          data from one device to another.
-  Origin: Sponsored by Riege Sofware International GmbH. Contact:
-          Daniel Holtkamp <holtkamp at riege dot com>
-  Date:   28 October 2005
-  Status: Partially coded in 1.37 -- much more to do. Assigned to
-          Kern.
-
-  What:   The ability to copy, move, or archive data that is on a
-          device to another device is very important. 
-
-  Why:    An ISP might want to backup to disk, but after 30 days
-          migrate the data to tape backup and delete it from
-          disk.  Bacula should be able to handle this
-          automatically.  It needs to know what was put where,
-          and when, and what to migrate -- it is a bit like
-          retention periods.  Doing so would allow space to be
-          freed up for current backups while maintaining older
-          data on tape drives.
-
-  Notes:   Riege Software have asked for the following migration
-           triggers:
-           Age of Job
-           Highwater mark (stopped by Lowwater mark?)
-                            
-  Notes:  Migration could be additionally triggered by:
-           Number of Jobs
-           Number of Volumes
-
-
-Item 2:   Implement extraction of Win32 BackupWrite data.
-  Origin: Thorsten Engel <thorsten.engel at matrix-computer dot com>
-  Date:   28 October 2005
-  Status: Assigned to Thorsten. Implemented in current CVS
-
-  What:   This provides the Bacula File daemon with code that
-          can pick apart the stream output that Microsoft writes
-          for BackupWrite data, and thus the data can be read
-          and restored on non-Win32 machines.
-
-  Why:    BackupWrite data is the portable=no option in Win32
-          FileSets, and in previous Baculas, this data could
-          only be extracted using a Win32 FD. With this new code,
-          the Windows data can be extracted and restored on
-          any OS.
-
-
-Item 3:   Implement a Bacula GUI/management tool using Python
-          and Qt.
+                    Status updated 26 April 2009
+
+Summary:
+Item  1: Allow FD to initiate a backup
+Item  2: Ability to restart failed jobs
+Item  3: Port bat to Win32
+Item  4: Convert Bacula existing tray monitor on Windows to a stand alone program
+Item  5: Ability to import/export Bacula database entities
+Item  6: Ability to defer Batch Insert to a later time
+Item  7: List InChanger flag when doing restore.
+Item  8: Deletion of disk Volumes when pruned
+Item  9: Implement Base jobs
+Item 10: Scheduling syntax that permits more flexibility and options
+Item 11: Reduction of communications bandwidth for a backup
+Item 12: Bacula Dir, FD and SD to support proxies
+Item 13: Message mailing based on backup types
+Item 14: Ability to reconnect a disconnected comm line
+Item 15: Include timestamp of job launch in "stat clients" output
+Item 16: Add an override in Schedule for Pools based on backup types
+Item 17: Automatic promotion of backup levels based on backup size
+Item 18: Allow inclusion/exclusion of files in a fileset by creation/mod times
+Item 19: Archival (removal) of User Files to Tape
+Item 20: Include all conf files in specified directory
+Item 21: Implement an interface between Bacula and Amazon's S3.
+Item 22: Enable/disable compression depending on storage device (disk/tape)
+Item 23: Backup and Restore of Windows Encrypted Files using Win raw encryption
+Item 24: Data encryption on storage daemon
+Item 25: "Maximum Concurrent Jobs" for drives when used with changer device
+Item 26: Add MaxVolumeSize/MaxVolumeBytes statement to Storage resource
+Item 27: Start spooling even when waiting on tape
+Item 28: Enable persistent naming/number of SQL queries
+Item 29: Implementation of running Job speed limit.
+Item 30: Restore from volumes on multiple storage daemons
+Item 31: 'restore' menu: enter a JobId, automatically select dependents
+Item 32: Possibilty to schedule Jobs on last Friday of the month
+Item 33: Add Minumum Spool Size directive
+Item 34: Cause daemons to use a specific IP address to source communications
+Item 35: Add ability to Verify any specified Job.
+Item 36: Automatic disabling of devices
+Item 37: An option to operate on all pools with update vol parameters
+Item 38: Implement Storage daemon compression
+Item 39: Improve Bacula's tape and drive usage and cleaning management
+Item 40: Multiple threads in file daemon for the same job
+
+
+Item  1: Allow FD to initiate a backup
+Origin:  Frank Volf (frank at deze dot org)
+Date:    17 November 2005
+Status: 
+
+What: Provide some means, possibly by a restricted console that
+       allows a FD to initiate a backup, and that uses the connection
+       established by the FD to the Director for the backup so that
+       a Director that is firewalled can do the backup.
+Why:  Makes backup of laptops much easier.
+
+Item  2: Ability to restart failed jobs
+   Date: 26 April 2009
+ Origin: Kern/Eric
+ Status: 
+
+  What:  Often jobs fail because of a communications line drop or max run time,
+          cancel, or some other non-critical problem.  Currrently any data
+          saved is lost.  This implementation should modify the Storage daemon
+          so that it saves all the files that it knows are completely backed
+          up to the Volume
+
+          The jobs should then be marked as incomplete and a subsequent
+          Incremental Accurate backup will then take into account all the
+          previously saved job.
+
+  Why:   Avoids backuping data already saved.
+
+  Notes: Requires Accurate to restart correctly.  Must completed have a minimum
+          volume of data or files stored on Volume before enabling.
+
+Item  3: Port bat to Win32
+   Date: 26 April 2009
+ Origin: Kern/Eric
+ Status: 
+
+  What:  Make bat run on Win32/64.
+
+  Why:   To have GUI on Windows
+
+  Notes: 
+
+Item  4: Convert Bacula existing tray monitor on Windows to a stand alone program
+   Date: 26 April 2009
+ Origin: Kern/Eric
+ Status: 
+
+  What:  Separate Win32 tray monitor to be a separate program.
+
+  Why:   Vista does not allow SYSTEM services to interact with the 
+          desktop, so the current tray monitor does not work on Vista
+          machines.  
+
+  Notes: Requires communicating with the FD via the network (simulate
+          a console connection).
+
+
+
+Item  5: Ability to import/export Bacula database entities
+   Date: 26 April 2009
+ Origin: Eric
+ Status: 
+
+  What:  Create a Bacula ASCII SQL database independent format that permits
+          importing and exporting database catalog Job entities.
+
+  Why:   For achival, database clustering, tranfer to other databases
+          of any SQL engine.
+
+  Notes: Job selection should be by Job, time, Volume, Client, Pool and possibly
+          other criteria.
 
-  Origin: Kern
-  Date:   28 October 2005
-  Status: 
 
-  What:   Implement a Bacula console, and management tools
-          using Python and Qt.
-
-  Why:    Don't we already have a wxWidgets GUI?  Yes, but
-          it is written in C++ and changes to the user interface
-          must be hand tailored using C++ code. By developing
-          the user interface using Qt designer, the interface
-          can be very easily updated and most of the new Python       
-          code will be automatically created.  The user interface
-          changes become very simple, and only the new features
-          must be implement.  In addition, the code will be in
-          Python, which will give many more users easy (or easier)
-          access to making additions or modifications.
-
-Item 4:   Implement a Python interface to the Bacula catalog.
-  Date:   28 October 2005
-  Origin: Kern
-  Status: 
+Item  6: Ability to defer Batch Insert to a later time
+   Date: 26 April 2009
+ Origin: Eric
+ Status: 
 
-  What:   Implement an interface for Python scripts to access
-          the catalog through Bacula.
+  What:  Instead of doing a Job Batch Insert at the end of the Job
+          which might create resource contention with lots of Job,
+          defer the insert to a later time.
 
-  Why:    This will permit users to customize Bacula through
-          Python scripts.
+  Why:   Permits to focus on getting the data on the Volume and
+          putting the metadata into the Catalog outside the backup
+          window.
 
-Item 5:   Implement more Python events in Bacula.
-  Date:   28 October 2005
-  Origin: 
-  Status: 
+  Notes: Will use the proposed Bacula ASCII database import/export
+          format (i.e. dependent on the import/export entities project).
+
+
+Item  7: List InChanger flag when doing restore.
+ Origin: Jesper Krogh<jesper@krogh.cc>
+   Date: 17 Oct 2008
+ Status:
+
+   What: When doing a restore the restore selection dialog ends by telling stuff 
+      like this:
+  The job will require the following
+   Volume(s)                 Storage(s)                SD Device(s)
+   ===========================================================================
+    000741L3                  LTO-4                     LTO3 
+    000866L3                  LTO-4                     LTO3 
+    000765L3                  LTO-4                     LTO3 
+    000764L3                  LTO-4                     LTO3 
+    000756L3                  LTO-4                     LTO3 
+    001759L3                  LTO-4                     LTO3 
+    001763L3                  LTO-4                     LTO3 
+    001762L3                  LTO-4                     LTO3 
+    001767L3                  LTO-4                     LTO3 
+
+   When having an autochanger, it would be really nice with an inChanger 
+   column so the operator knew if this restore job would stop waiting for 
+   operator intervention. This is done just by selecting the inChanger flag 
+   from the catalog and printing it in a seperate column.
+
+
+   Why:   This would help getting large restores through minimizing the 
+      time spent waiting for operator to drop by and change tapes in the library.
+
+  Notes: [Kern] I think it would also be good to have the Slot as well,
+      or some indication that Bacula thinks the volume is in the autochanger
+      because it depends on both the InChanger flag and the Slot being
+      valid.
 
-  What:   Allow Python scripts to be called at more places 
-          within Bacula and provide additional access to Bacula
-          internal variables.
 
-  Why:    This will permit users to customize Bacula through
-          Python scripts.
+Item  8: Deletion of disk Volumes when pruned
+  Date:  Nov 25, 2005
+  Origin: Ross Boylan <RossBoylan at stanfordalumni dot org> (edited
+          by Kern)
+  Status:        
 
-  Notes:  Recycle event
-          Scratch pool event
-          NeedVolume event
-          MediaFull event
+   What: Provide a way for Bacula to automatically remove Volumes
+          from the filesystem, or optionally to truncate them.
+          Obviously, the Volume must be pruned prior removal.
 
+  Why:   This would allow users more control over their Volumes and
+          prevent disk based volumes from consuming too much space.
 
-Item 6:   Implement Base jobs.
-  Date:   28 October 2005
+  Notes: The following two directives might do the trick:
+
+          Volume Data Retention = <time period>
+          Remove Volume After = <time period>
+
+          The migration project should also remove a Volume that is
+          migrated. This might also work for tape Volumes.
+
+  Notes: (Kern). The data fields to control this have been added
+          to the new 3.0.0 database table structure.
+
+
+
+Item  9: Implement Base jobs 
+  Date:  28 October 2005
   Origin: Kern
-  Status: 
+  Status:
   
-  What:   A base job is sort of like a Full save except that you 
+  What:  A base job is sort of like a Full save except that you 
           will want the FileSet to contain only files that are
           unlikely to change in the future (i.e.  a snapshot of
           most of your system after installing it).  After the
@@ -118,8 +213,8 @@ Item 6:   Implement Base jobs.
           During a restore, the Base jobs will be automatically
           pulled in where necessary.
 
-  Why:    This is something none of the competition does, as far as
-          we know (except perhpas BackupPC, which is a Perl program that
+  Why:   This is something none of the competition does, as far as
+          we know (except perhaps BackupPC, which is a Perl program that
           saves to disk only).  It is big win for the user, it
           makes Bacula stand out as offering a unique
           optimization that immediately saves time and money.
@@ -131,211 +226,904 @@ Item 6:   Implement Base jobs.
           have some files updated, no problem, they will be
           automatically restored.
 
-  Notes:  Huge savings in tape usage even for a single machine.
+  Notes: Huge savings in tape usage even for a single machine.
           Will require more resources because the DIR must send
           FD a list of files/attribs, and the FD must search the
           list and compare it for each file to be saved.
 
-Item 7:   Add Plug-ins to the FileSet Include statements.
-  Date:   28 October 2005
-  Origin:
-  Status: Partially coded in 1.37 -- much more to do.
 
-  What:   Allow users to specify wild-card and/or regular
-          expressions to be matched in both the Include and
-          Exclude directives in a FileSet.  At the same time,
-          allow users to define plug-ins to be called (based on
-          regular expression/wild-card matching).
+Item 10: Scheduling syntax that permits more flexibility and options
+   Date: 15 December 2006
+  Origin: Gregory Brauer (greg at wildbrain dot com) and
+          Florian Schnabel <florian.schnabel at docufy dot de>
+  Status:
 
-  Why:    This would give the users the ultimate ability to control
-          how files are backed up/restored.  A user could write a
-          plug-in knows how to backup his Oracle database without
-          stopping/starting it, for example.
+   What: Currently, Bacula only understands how to deal with weeks of the
+          month or weeks of the year in schedules.  This makes it impossible
+          to do a true weekly rotation of tapes.  There will always be a
+          discontinuity that will require disruptive manual intervention at
+          least monthly or yearly because week boundaries never align with
+          month or year boundaries.
+
+          A solution would be to add a new syntax that defines (at least)
+          a start timestamp, and repetition period.
+
+          An easy option to skip a certain job  on a certain date.
+   
+
+     Why: Rotated backups done at weekly intervals are useful, and Bacula
+          cannot currently do them without extensive hacking.
+
+          You could then easily skip tape backups on holidays.  Especially
+          if you got no autochanger and can only fit one backup on a tape
+          that would be really handy, other jobs could proceed normally
+          and you won't get errors that way.
+
+
+   Notes: Here is an example syntax showing a 3-week rotation where full
+          Backups would be performed every week on Saturday, and an
+          incremental would be performed every week on Tuesday.  Each
+          set of tapes could be removed from the loader for the following
+          two cycles before coming back and being reused on the third
+          week.  Since the execution times are determined by intervals
+          from a given point in time, there will never be any issues with
+          having to adjust to any sort of arbitrary time boundary.  In
+          the example provided, I even define the starting schedule
+          as crossing both a year and a month boundary, but the run times
+          would be based on the "Repeat" value and would therefore happen
+          weekly as desired.
+
+
+          Schedule {
+              Name = "Week 1 Rotation"
+              #Saturday.  Would run Dec 30, Jan 20, Feb 10, etc.
+              Run {
+                  Options {
+                      Type   = Full
+                      Start  = 2006-12-30 01:00
+                      Repeat = 3w
+                  }
+              }
+              #Tuesday.  Would run Jan 2, Jan 23, Feb 13, etc.
+              Run {
+                  Options {
+                      Type   = Incremental
+                      Start  = 2007-01-02 01:00
+                      Repeat = 3w
+                  }
+              }
+          }
 
-Item 8:   Implement huge exclude list support using hashing.
-  Date:   28 October 2005
-  Origin: Kern
-  Status: 
+          Schedule {
+              Name = "Week 2 Rotation"
+              #Saturday.  Would run Jan 6, Jan 27, Feb 17, etc.
+              Run {
+                  Options {
+                      Type   = Full
+                      Start  = 2007-01-06 01:00
+                      Repeat = 3w
+                  }
+              }
+              #Tuesday.  Would run Jan 9, Jan 30, Feb 20, etc.
+              Run {
+                  Options {
+                      Type   = Incremental
+                      Start  = 2007-01-09 01:00
+                      Repeat = 3w
+                  }
+              }
+          }
+
+          Schedule {
+              Name = "Week 3 Rotation"
+              #Saturday.  Would run Jan 13, Feb 3, Feb 24, etc.
+              Run {
+                  Options {
+                      Type   = Full
+                      Start  = 2007-01-13 01:00
+                      Repeat = 3w
+                  }
+              }
+              #Tuesday.  Would run Jan 16, Feb 6, Feb 27, etc.
+              Run {
+                  Options {
+                      Type   = Incremental
+                      Start  = 2007-01-16 01:00
+                      Repeat = 3w
+                  }
+              }
+          }
 
-  What:   Allow users to specify very large exclude list (currently
-          more than about 1000 files is too many).
-
-  Why:    This would give the users the ability to exclude all
-          files that are loaded with the OS (e.g. using rpms
-          or debs). If the user can restore the base OS from
-          CDs, there is no need to backup all those files. A
-          complete restore would be to restore the base OS, then
-          do a Bacula restore. By excluding the base OS files, the
-          backup set will be *much* smaller.
-
-
-Item  9:  Implement data encryption (as opposed to communications
-          encryption)
-  Date:   28 October 2005
-  Origin: Sponsored by Landon and 13 contributors to EFF.
-  Status: Landon Fuller is currently implementing this.
-                  
-  What:   Currently the data that is stored on the Volume is not
-          encrypted. For confidentiality, encryption of data at
-          the File daemon level is essential. 
-          Data encryption encrypts the data in the File daemon and
-          decrypts the data in the File daemon during a restore.
-
-  Why:    Large sites require this.
-
-Item 10:  Permit multiple Media Types in an Autochanger
-  Origin: 
+   Notes: Kern: I have merged the previously separate project of skipping 
+          jobs (via Schedule syntax) into this.
+
+Item 11: Reduction of communications bandwidth for a backup
+   Date: 14 October 2008
+ Origin: Robin O'Leary (Equiinet)
+ Status: 
+
+  What:  Using rdiff techniques, Bacula could significantly reduce
+          the network data transfer volume to do a backup.
+
+  Why:   Faster backup across the Internet
+
+  Notes: This requires retaining certain data on the client during a Full
+          backup that will speed up subsequent backups.
+          
+
+Item 12: Bacula Dir, FD and SD to support proxies
+Origin: Karl Grindley @ MIT Lincoln Laboratory <kgrindley at ll dot mit dot edu>
+Date:  25 March 2009
+Status: proposed
+
+What:  Support alternate methods for nailing up a TCP session such
+        as SOCKS5, SOCKS4 and HTTP (CONNECT) proxies.  Such a feature
+        would allow tunneling of bacula traffic in and out of proxied
+        networks.
+
+Why:   Currently, bacula is architected to only function on a flat network, with
+        no barriers or limitations.  Due to the large configuration states of
+        any network and the infinite configuration where file daemons and
+        storage daemons may sit in relation to one another, bacula often is
+        not usable on a network where filtered or air-gaped networks exist.
+        While often solutions such as ACL modifications to firewalls or port
+        redirection via SNAT or DNAT will solve the issue, often however,
+        these solutions are not adequate or not allowed by hard policy.
+
+        In an air-gapped network with only a highly locked down proxy services
+        are provided (SOCKS4/5 and/or HTTP and/or SSH outbound) ACLs or
+        iptable rules will not work.
+
+Notes: Director resource tunneling: This configuration option to utilize a
+        proxy to connect to a client should be specified in the client
+        resource Client resource tunneling: should be configured in the client
+        resource in the director config file?  Or configured on the bacula-fd
+        configuration file on the fd host itself?  If the ladder, this would
+        allow only certain clients to use a proxy, where others do not when
+        establishing the TCP connection to the storage server. 
+
+        Also worth noting, there are other 3rd party, light weight apps that
+        could be utilized to bootstrap this.  Instead of sockifing bacula
+        itself, use an external program to broker proxy authentication, and
+        connection to the remote host.  OpenSSH does this by using the
+        "ProxyCommand" syntax in the client configuration and uses stdin and
+        stdout to the command.  Connect.c is a very popular one.
+        (http://bent.latency.net/bent/darcs/goto-san-connect-1.85/src/connect.html).
+        One could also possibly use stunnel, netcat, etc.
+
+
+Item 13: Message mailing based on backup types
+ Origin: Evan Kaufman <evan.kaufman@gmail.com>
+   Date: January 6, 2006
+ Status:
+
+   What: In the "Messages" resource definitions, allowing messages
+          to be mailed based on the type (backup, restore, etc.) and level
+          (full, differential, etc) of job that created the originating
+          message(s).
+
+ Why:    It would, for example, allow someone's boss to be emailed
+          automatically only when a Full Backup job runs, so he can
+          retrieve the tapes for offsite storage, even if the IT dept.
+          doesn't (or can't) explicitly notify him.  At the same time, his
+          mailbox wouldnt be filled by notifications of Verifies, Restores,
+          or Incremental/Differential Backups (which would likely be kept
+          onsite).
+
+ Notes:  One way this could be done is through additional message types, for example:
+
+   Messages {
+     # email the boss only on full system backups
+     Mail = boss@mycompany.com = full, !incremental, !differential, !restore, 
+            !verify, !admin
+     # email us only when something breaks
+     MailOnError = itdept@mycompany.com = all
+   }
+
+   Notes: Kern: This should be rather trivial to implement.
+
+Item 14: Ability to reconnect a disconnected comm line
+  Date:  26 April 2009
+  Origin: Kern/Eric
   Status: 
 
-  What:   Modify the Storage daemon so that multiple Media Types
-          can be specified in an autochanger. This would be somewhat
-          of a simplistic implementation in that each drive would
-          still be allowed to have only one Media Type.  However,
-          the Storage daemon will ensure that only a drive with
-          the Media Type that matches what the Director specifies
-          is chosen.
+  What:  Often jobs fail because of a communications line drop. In that 
+          case, Bacula should be able to reconnect to the other daemon and
+          resume the job.
 
-  Why:    This will permit user with several different drive types
-          to make full use of their autochangers.
+  Why:   Avoids backuping data already saved.
 
-Item 11:  Allow two different autochanger definitions that refer
-          to the same autochanger.
-  Date:   28 October 2005
-  Origin: Kern
-  Status: 
+  Notes: *Very* complicated from a design point of view because of authenication.
 
-  What:   Currently, the autochanger script is locked based on
-          the autochanger. That is, if multiple drives are being
-          simultaneously used, the Storage daemon ensures that only
-          one drive at a time can access the mtx-changer script.
-          This change would base the locking on the control device,
-          rather than the autochanger. It would then permit two autochanger
-          definitions for the same autochanger, but with different 
-          drives. Logically, the autochanger could then be "partitioned"
-          for different jobs, clients, or class of jobs, and if the locking
-          is based on the control device (e.g. /dev/sg0) the mtx-changer
-          script will be locked appropriately.
-
-  Why:    This will permit users to partition autochangers for specific
-          use. It would also permit implementation of multiple Media
-          Types with no changes to the Storage daemon.
-
-Item 12:  Implement red/black binary tree routines.
-  Date:   28 October 2005
-  Origin: Kern
+
+Item 15: Include timestamp of job launch in "stat clients" output
+  Origin: Mark Bergman <mark.bergman@uphs.upenn.edu>
+  Date:  Tue Aug 22 17:13:39 EDT 2006
+  Status:
+
+  What:  The "stat clients" command doesn't include any detail on when
+          the active backup jobs were launched.
+
+  Why:   Including the timestamp would make it much easier to decide whether
+          a job is running properly. 
+
+  Notes: It may be helpful to have the output from "stat clients" formatted 
+          more like that from "stat dir" (and other commands), in a column
+          format. The per-client information that's currently shown (level,
+          client name, JobId, Volume, pool, device, Files, etc.) is good, but
+          somewhat hard to parse (both programmatically and visually), 
+          particularly when there are many active clients.
+
+
+
+
+Item 16: Add an override in Schedule for Pools based on backup types
+Date:    19 Jan 2005
+Origin:  Chad Slater <chad.slater@clickfox.com>
+Status: 
+                                                
+  What:  Adding a FullStorage=BigTapeLibrary in the Schedule resource
+          would help those of us who use different storage devices for different
+          backup levels cope with the "auto-upgrade" of a backup.
+
+  Why:   Assume I add several new devices to be backed up, i.e. several
+          hosts with 1TB RAID.  To avoid tape switching hassles, incrementals are
+          stored in a disk set on a 2TB RAID.  If you add these devices in the
+          middle of the month, the incrementals are upgraded to "full" backups,
+          but they try to use the same storage device as requested in the
+          incremental job, filling up the RAID holding the differentials.  If we
+          could override the Storage parameter for full and/or differential
+          backups, then the Full job would use the proper Storage device, which
+          has more capacity (i.e. a 8TB tape library.
+
+Item 17: Automatic promotion of backup levels based on backup size
+   Date: 19 January 2006
+  Origin: Adam Thornton <athornton@sinenomine.net>
   Status: 
 
-  What:   Implement a red/black binary tree class. This could 
-          then replace the current binary insert/search routines
-          used in the restore in memory tree.  This could significantly
-          speed up the creation of the in memory restore tree.
+    What: Other backup programs have a feature whereby it estimates the space
+          that a differential, incremental, and full backup would take.  If
+          the difference in space required between the scheduled level and the
+          next level up is beneath some user-defined critical threshold, the
+          backup level is bumped to the next type.  Doing this minimizes the
+          number of volumes necessary during a restore, with a fairly minimal
+          cost in backup media space.
+
+    Why: I know at least one (quite sophisticated and smart) user for whom the
+          absence of this feature is a deal-breaker in terms of using Bacula;
+          if we had it it would eliminate the one cool thing other backup
+          programs can do and we can't (at least, the one cool thing I know
+          of).
+
+Item 18: Allow inclusion/exclusion of files in a fileset by creation/mod times
+  Origin: Evan Kaufman <evan.kaufman@gmail.com>
+  Date:  January 11, 2006
+  Status:
+
+  What:  In the vein of the Wild and Regex directives in a Fileset's
+          Options, it would be helpful to allow a user to include or exclude
+          files and directories by creation or modification times.
+
+          You could factor the Exclude=yes|no option in much the same way it
+          affects the Wild and Regex directives.  For example, you could exclude
+          all files modified before a certain date:
+
+   Options {
+     Exclude = yes
+     Modified Before = ####
+   }
+
+           Or you could exclude all files created/modified since a certain date:
+
+   Options {
+      Exclude = yes
+     Created Modified Since = ####
+   }
 
-  Why:    Performance enhancement.
+           The format of the time/date could be done several ways, say the number
+           of seconds since the epoch:
+           1137008553 = Jan 11 2006, 1:42:33PM   # result of `date +%s`
 
-Item 13:  Let Bacula log tape usage and handle drive cleaning cycles.
-  Date:   November 11, 2005
-  Origin: Arno Lehmann <al at its-lehmann dot de>
+           Or a human readable date in a cryptic form:
+           20060111134233 = Jan 11 2006, 1:42:33PM   # YYYYMMDDhhmmss
+
+  Why:   I imagine a feature like this could have many uses. It would
+          allow a user to do a full backup while excluding the base operating
+          system files, so if I installed a Linux snapshot from a CD yesterday,
+          I'll *exclude* all files modified *before* today.  If I need to
+          recover the system, I use the CD I already have, plus the tape backup.
+          Or if, say, a Windows client is hit by a particularly corrosive
+          virus, and I need to *exclude* any files created/modified *since* the
+          time of infection.
+
+  Notes: Of course, this feature would work in concert with other
+          in/exclude rules, and wouldnt override them (or each other).
+
+  Notes: The directives I'd imagine would be along the lines of
+          "[Created] [Modified] [Before|Since] = <date>".
+          So one could compare against 'ctime' and/or 'mtime', but ONLY 'before'
+           or 'since'.
+
+
+
+Item 19: Archival (removal) of User Files to Tape
+  Date:  Nov. 24/2005 
+  Origin: Ray Pengelly [ray at biomed dot queensu dot ca
+  Status: 
+
+  What:  The ability to archive data to storage based on certain parameters
+          such as age, size, or location.  Once the data has been written to
+          storage and logged it is then pruned from the originating
+          filesystem. Note! We are talking about user's files and not
+          Bacula Volumes.
+
+  Why:   This would allow fully automatic storage management which becomes
+          useful for large datastores.  It would also allow for auto-staging
+          from one media type to another.
+
+          Example 1) Medical imaging needs to store large amounts of data.
+          They decide to keep data on their servers for 6 months and then put
+          it away for long term storage.  The server then finds all files
+          older than 6 months writes them to tape.  The files are then removed
+          from the server.
+
+          Example 2) All data that hasn't been accessed in 2 months could be
+          moved from high-cost, fibre-channel disk storage to a low-cost
+          large-capacity SATA disk storage pool which doesn't have as quick of
+          access time.  Then after another 6 months (or possibly as one
+          storage pool gets full) data is migrated to Tape.
+
+
+Item 20: Include all conf files in specified directory
+Date:  18 October 2008
+Origin: Database, Lda. Maputo, Mozambique
+Contact:Cameron Smith / cameron.ord@database.co.mz 
+Status: New request
+
+What: A directive something like "IncludeConf = /etc/bacula/subconfs" Every
+      time Bacula Director restarts or reloads, it will walk the given
+      directory (non-recursively) and include the contents of any files
+      therein, as though they were appended to bacula-dir.conf
+
+Why: Permits simplified and safer configuration for larger installations with
+      many client PCs.  Currently, through judicious use of JobDefs and
+      similar directives, it is possible to reduce the client-specific part of
+      a configuration to a minimum.  The client-specific directives can be
+      prepared according to a standard template and dropped into a known
+      directory.  However it is still necessary to add a line to the "master"
+      (bacula-dir.conf) referencing each new file.  This exposes the master to
+      unnecessary risk of accidental mistakes and makes automation of adding
+      new client-confs, more difficult (it is easier to automate dropping a
+      file into a dir, than rewriting an existing file).  Ken has previously
+      made a convincing argument for NOT including Bacula's core configuration
+      in an RDBMS, but I believe that the present request is a reasonable
+      extension to the current "flat-file-based" configuration philosophy.
+ 
+Notes: There is NO need for any special syntax to these files.  They should
+       contain standard directives which are simply "inlined" to the parent
+       file as already happens when you explicitly reference an external file.
+
+Notes: (kes) this can already be done with scripting
+     From: John Jorgensen <jorgnsn@lcd.uregina.ca>
+     The bacula-dir.conf at our site contains these lines:
+
+   #
+   # Include subfiles associated with configuration of clients.
+   # They define the bulk of the Clients, Jobs, and FileSets.
+   #
+   @|"sh -c 'for f in /etc/bacula/clientdefs/*.conf ; do echo @${f} ; done'"
+
+    and when we get a new client, we just put its configuration into
+    a new file called something like:
+
+    /etc/bacula/clientdefs/clientname.conf
+
+
+
+
+Item 21: Implement an interface between Bacula and Amazon's S3.
+  Date:  25 August 2008
+  Origin: Soren Hansen <soren@ubuntu.com>
+  Status: Not started.
+  What:  Enable the storage daemon to store backup data on Amazon's
+          S3 service.
+
+  Why:   Amazon's S3 is a cheap way to store data off-site. Current
+          ways to integrate Bacula and S3 involve storing all the data
+          locally and syncing them to S3, and manually fetching them
+          again when they're needed. This is very cumbersome.
+
+
+Item 22: Enable/disable compression depending on storage device (disk/tape)
+  Origin: Ralf Gross ralf-lists@ralfgross.de
+  Date:  2008-01-11
+  Status: Initial Request
+
+  What: Add a new option to the storage resource of the director.  Depending
+          on this option, compression will be enabled/disabled for a device.
+
+  Why: If different devices (disks/tapes) are used for full/diff/incr
+          backups, software compression will be enabled for all backups
+          because of the FileSet compression option.  For backup to tapes
+          wich are able to do hardware compression this is not desired.
+          
+
+  Notes:
+         http://news.gmane.org/gmane.comp.sysutils.backup.bacula.devel/cutoff=11124
+         It must be clear to the user, that the FileSet compression option
+         must still be enabled use compression for a backup job at all.
+         Thus a name for the new option in the director must be
+         well-defined.
+
+  Notes: KES I think the Storage definition should probably override what
+         is in the Job definition or vice-versa, but in any case, it must
+         be well defined.
+
+
+Item 31: Backup and Restore of Windows Encrypted Files using Win raw encryption
+  Origin: Michael Mohr, SAG  Mohr.External@infineon.com
+  Date:  22 February 2008
+  Origin: Alex Ehrlich (Alex.Ehrlich-at-mail.ee)
+  Date:  05 August 2008
   Status:
 
-  What:   Make Bacula manage tape life cycle information and drive 
-          cleaning cycles.
+  What: Make it possible to backup and restore Encypted Files from and to
+          Windows systems without the need to decrypt it by using the raw
+          encryption functions API (see:
+          http://msdn2.microsoft.com/en-us/library/aa363783.aspx)
+          that is provided for that reason by Microsoft.
+          If a file ist encrypted could be examined by evaluating the 
+          FILE_ATTRIBUTE_ENCRYTED flag of the GetFileAttributes
+          function.
+          For each file backed up or restored by FD on Windows, check if
+          the file is encrypted; if so then use OpenEncryptedFileRaw,
+          ReadEncryptedFileRaw, WriteEncryptedFileRaw,
+          CloseEncryptedFileRaw instead of BackupRead and BackupWrite
+          API calls.
+
+  Why:   Without the usage of this interface the fd-daemon running
+          under the system account can't read encypted Files because
+          the key needed for the decrytion is missed by them. As a result 
+          actually encrypted files are not backed up
+          by bacula and also no error is shown while missing these files.
+
+   Notes: Using xxxEncryptedFileRaw API would allow to backup and
+           restore EFS-encrypted files without decrypting their data.
+           Note that such files cannot be restored "portably" (at least,
+           easily) but they would be restoreable to a different (or
+           reinstalled) Win32 machine; the restore would require setup
+           of a EFS recovery agent in advance, of course, and this shall
+           be clearly reflected in the documentation, but this is the
+           normal Windows SysAdmin's business.
+           When "portable" backup is requested the EFS-encrypted files
+           shall be clearly reported as errors.
+           See MSDN on the "Backup and Restore of Encrypted Files" topic:
+           http://msdn.microsoft.com/en-us/library/aa363783.aspx
+           Maybe the EFS support requires a new flag in the database for
+           each file, too?
+           Unfortunately, the implementation is not as straightforward as
+           1-to-1 replacement of BackupRead with ReadEncryptedFileRaw,
+           requiring some FD code rewrite to work with
+           encrypted-file-related callback functions.
+
+Item 24: Data encryption on storage daemon
+  Origin: Tobias Barth <tobias.barth at web-arts.com>
+  Date:  04 February 2009
+  Status: new
+
+  What:  The storage demon should be able to do the data encryption that can currently be done by the file daemon.
+
+  Why:   This would have 2 advantages: 1) one could encrypt the data of unencrypted tapes by doing a migration job, and 2) the storage daemon would be the only machine that would have to keep the encryption keys.
+
+  Notes from Landon:
+          As an addendum to the feature request, here are some crypto  
+          implementation details I wrote up regarding SD-encryption back in Jan  
+          2008:
+          http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg28860.html
+
+
+
+Item 25: "Maximum Concurrent Jobs" for drives when used with changer device
+  Origin: Ralf Gross ralf-lists <at> ralfgross.de
+  Date:  2008-12-12
+  Status: Initial Request
+
+  What:  respect the "Maximum Concurrent Jobs" directive in the _drives_
+          Storage section in addition to the changer section
+
+  Why:   I have a 3 drive changer where I want to be able to let 3 concurrent
+          jobs run in parallel. But only one job per drive at the same time.
+          Right now I don't see how I could limit the number of concurrent jobs
+          per drive in this situation.
+
+  Notes: Using different priorities for these jobs lead to problems that other
+          jobs are blocked. On the user list I got the advice to use the "Prefer Mounted
+          Volumes" directive, but Kern advised against using "Prefer Mounted
+          Volumes" in an other thread:
+          http://article.gmane.org/gmane.comp.sysutils.backup.bacula.devel/11876/
+
+          In addition I'm not sure if this would be the same as respecting the
+          drive's "Maximum Concurrent Jobs" setting.
+
+          Example:
+
+          Storage {
+            Name = Neo4100
+            Address = ....
+            SDPort = 9103
+            Password = "wiped"
+            Device = Neo4100
+            Media Type = LTO4
+            Autochanger = yes
+            Maximum Concurrent Jobs = 3
+          }
+          
+          Storage {
+            Name = Neo4100-LTO4-D1
+            Address = ....
+            SDPort = 9103
+            Password = "wiped"
+            Device = ULTRIUM-TD4-D1
+            Media Type = LTO4
+            Maximum Concurrent Jobs = 1
+          }
+          
+          [2 more drives] 
+
+          The "Maximum Concurrent Jobs = 1" directive in the drive's section is ignored.
+
+Item 26: Add MaxVolumeSize/MaxVolumeBytes statement to Storage resource
+   Origin: Bastian Friedrich <bastian.friedrich@collax.com>
+   Date:  2008-07-09
+   Status: -
+
+   What:  SD has a "Maximum Volume Size" statement, which is deprecated and
+           superseded by the Pool resource statement "Maximum Volume Bytes".
+           It would be good if either statement could be used in Storage
+           resources.
+
+   Why:   Pools do not have to be restricted to a single storage type/device;
+           thus, it may be impossible to define Maximum Volume Bytes in the
+           Pool resource.  The old MaxVolSize statement is deprecated, as it
+           is SD side only.  I am using the same pool for different devices.
+
+   Notes: State of idea currently unknown.  Storage resources in the dir
+           config currently translate to very slim catalog entries; these
+           entries would require extensions to implement what is described
+           here.  Quite possibly, numerous other statements that are currently
+           available in Pool resources could be used in Storage resources too
+           quite well.
+
+Item 27: Start spooling even when waiting on tape
+  Origin: Tobias Barth <tobias.barth@web-arts.com>
+  Date:  25 April 2008
+  Status:
 
-  Why:    Both parts of this project are important when operating backups.
-          We need to know which tapes need replacement, and we need to
-          make sure the drives are cleaned when necessary.  While many
-          tape libraries and even autoloaders can handle all this
-          automatically, support by Bacula can be helpful for smaller
-          (older) libraries and single drives.  Also, checking drive
-          status during operation can prevent some failures (as I had to
-          learn the hard way...)
+  What: If a job can be spooled to disk before writing it to tape, it should
+          be spooled immediately.  Currently, bacula waits until the correct
+          tape is inserted into the drive.
+
+  Why:   It could save hours.  When bacula waits on the operator who must insert
+          the correct tape (e.g.  a new tape or a tape from another media
+          pool), bacula could already prepare the spooled data in the spooling
+          directory and immediately start despooling when the tape was
+          inserted by the operator.
+         
+          2nd step: Use 2 or more spooling directories.  When one directory is
+          currently despooling, the next (on different disk drives) could
+          already be spooling the next data.
+
+  Notes: I am using bacula 2.2.8, which has none of those features
+         implemented.
+
+Item 28: Enable persistent naming/number of SQL queries
+  Date:  24 Jan, 2007 
+  Origin: Mark Bergman 
+  Status: 
 
-  Notes:  First, Bacula could (and even does, to some limited extent)
-          record tape and drive usage.  For tapes, the number of mounts,
-          the amount of data, and the time the tape has actually been
-          running could be recorded.  Data fields for Read and Write time
-          and Nmber of mounts already exist in the catalog (I'm not sure
-          if VolBytes is the sum of all bytes ever written to that volume
-          by Bacula).  This information can be important when determining
-          which media to replace.  For the tape drives known to Bacula,
-          similar information is interesting to determine the device
-          status and expected life time: Time it's been Reading and
-          Writing, number of tape Loads / Unloads / Errors.  This
-          information is not yet recorded as far as I know.
-
-          The next step would be implementing drive cleaning setup.
-          Bacula already has knowledge about cleaning tapes.  Once it has
-          some information about cleaning cycles (measured in drive run
-          time, number of tapes used, or calender days, for example) it
-          can automatically execute tape cleaning (with an autochanger,
-          obviously) or ask for operator assistence loading a cleaning
-          tape.
-
-          The next step would be to implement TAPEALERT checks not only
-          when changing tapes and only sending he information to the
-          administrator, but rather checking after each tape error,
-          checking on a regular basis (for example after each tape file),
-          and also before unloading and after loading a new tape.  Then,
-          depending on the drives TAPEALERT state and the know drive
-          cleaning state Bacula could automatically schedule later
-          cleaning, clean immediately, or inform the operator.
+  What: 
+        Change the parsing of the query.sql file and the query command so that
+        queries are named/numbered by a fixed value, not their order in the
+        file.
+
+
+  Why:  
+        One of the real strengths of bacula is the ability to query the
+        database, and the fact that complex queries can be saved and
+        referenced from a file is very powerful. However, the choice
+        of query (both for interactive use, and by scripting input
+        to the bconsole command) is completely dependent on the order
+        within the query.sql file. The descriptve labels are helpful for
+        interactive use, but users become used to calling a particular
+        query "by number", or may use scripts to execute queries. This
+        presents a problem if the number or order of queries in the file
+        changes.
+
+        If the query.sql file used the numeric tags as a real value (rather
+        than a comment), then users could have a higher confidence that they
+        are executing the intended query, that their local changes wouldn't
+        conflict with future bacula upgrades.
+
+        For scripting, it's very important that the intended query is
+        what's actually executed. The current method of parsing the
+        query.sql file discourages scripting because the addition or
+        deletion of queries within the file will require corresponding
+        changes to scripts. It may not be obvious to users that deleting
+        query "17" in the query.sql file will require changing all
+        references to higher numbered queries. Similarly, when new
+        bacula distributions change the number of "official" queries,
+        user-developed queries cannot simply be appended to the file
+        without also changing any references to those queries in scripts
+        or procedural documentation, etc.
+
+        In addition, using fixed numbers for queries would encourage more
+        user-initiated development of queries, by supporting conventions
+        such as:
+
+                queries numbered 1-50 are supported/developed/distributed by
+                        with official bacula releases
+                        
+                queries numbered 100-200 are community contributed, and are
+                related to media management
+
+                queries numbered 201-300 are community contributed, and are
+                related to checksums, finding duplicated files across
+                different backups, etc.
+
+                queries numbered 301-400 are community contributed, and are
+                related to backup statistics (average file size, size per
+                client per backup level, time for all clients by backup level,
+                storage capacity by media type, etc.)
+
+                queries numbered 500-999 are locally created
+
+  Notes:
+        Alternatively, queries could be called by keyword (tag), rather
+        than by number.
+
+Item 29: Implementation of running Job speed limit.
+Origin: Alex F, alexxzell at yahoo dot com
+Date: 29 January 2009
+
+What: I noticed the need for an integrated bandwidth limiter for
+      running jobs.  It would be very useful just to specify another
+      field in bacula-dir.conf, like speed = how much speed you wish
+      for that specific job to run at
+
+Why: Because of a couple of reasons.  First, it's very hard to implement a
+     traffic shaping utility and also make it reliable.  Second, it is very
+     uncomfortable to have to implement these apps to, let's say 50 clients
+     (including desktops, servers).  This would also be unreliable because you
+     have to make sure that the apps are properly working when needed; users
+     could also disable them (accidentally or not).  It would be very useful
+     to provide Bacula this ability.  All information would be centralized,
+     you would not have to go to 50 different clients in 10 different
+     locations for configuration; eliminating 3rd party additions help in
+     establishing efficiency.  Would also avoid bandwidth congestion,
+     especially where there is little available.
+
+Item 30: Restore from volumes on multiple storage daemons
+Origin: Graham Keeling (graham@equiinet.com)
+Date:  12 March 2009
+Status: Proposing
+
+What:  The ability to restore from volumes held by multiple storage daemons
+        would be very useful.
+
+Why:   It is useful to be able to backup to any number of different storage
+        daemons. For example, your first storage daemon may run out of space, so you
+        switch to your second and carry on. Bacula will currently let you do this.
+        However, once you come to restore, bacula cannot cope when volumes on different
+        storage daemons are required.
+
+        Notes: The director knows that more than one storage daemon is needed, as
+        bconsole outputs something like the following table.
+
+        The job will require the following
+           Volume(s)                 Storage(s)                SD Device(s)
+        ===========================================================================
+           
+           backup-0001               Disk 1                    Disk 1.0                 
+           backup-0002               Disk 2                    Disk 2.0             
+
+        However, the bootstrap file that it creates gets sent to the first storage
+        daemon only, which then stalls for a long time, 'waiting for a mount request'
+        for the volume that it doesn't have.
+        The bootstrap file contains no knowledge of the storage daemon.
+        Under the current design:
+
+                The director connects to the storage daemon, and gets an sd_auth_key.
+                The director then connects to the file daemon, and gives it the
+                        sd_auth_key with the 'jobcmd'.
+                (restoring of files happens)
+                The director does a 'wait_for_storage_daemon_termination()'.
+                The director waits for the file daemon to indicate the end of the job.
+
+        With my idea:
+
+        The director connects to the file daemon.
+        Then, for each storage daemon in the .bsr file...  {
+                The director connects to the storage daemon, and gets an sd_auth_key.
+                The director then connects to the file daemon, and gives it the
+                        sd_auth_key with the 'storaddr' command.
+                (restoring of files happens)
+                The director does a 'wait_for_storage_daemon_termination()'.
+                The director waits for the file daemon to indicate the end of the
+                        work on this storage.
+        }
+        The director tells the file daemon that there are no more storages to contact.
+        The director waits for the file daemon to indicate the end of the job.
+
+        As you can see, each restore between the file daemon and storage daemon is
+        handled in the same way that it is currently handled, using the same method
+        for authentication, except that the sd_auth_key is moved from the 'jobcmd' to
+        the 'storaddr' command - where it logically belongs.
+
+
+Item 31: 'restore' menu: enter a JobId, automatically select dependents 
+Origin: Graham Keeling (graham@equiinet.com)
+Date:  13 March 2009
+
+Status: Proposing
+
+What:  Add to the bconsole 'restore' menu the ability to select a job 
+        by JobId, and have bacula automatically select all the dependent jobs.
+
+        Why: Currently, you either have to...
+        a) laboriously type in a date that is greater than the date of the backup that
+        you want and is less than the subsequent backup (bacula then figures out the
+        dependent jobs), or
+        b) manually figure out all the JobIds that you want and laboriously type them
+        all in.
+        It would be extremely useful (in a programmatical sense, as well as for humans)
+        to be able to just give it a single JobId and let bacula do the hard work (work
+        that it already knows how to do).
+
+        Notes (Kern): I think this should either be modified to have Bacula print
+        a list of dates that the user can choose from as is done in bwx-console and
+        bat or the name of this command must be carefully chosen so that the user
+        clearly understands that the JobId is being used to specify what Job and the
+        date to which he wishes the restore to happen.
+
+
+
+Item 32: Possibilty to schedule Jobs on last Friday of the month
+Origin: Carsten Menke <bootsy52 at gmx dot net>
+Date:   02 March 2008
+Status:
+
+   What: Currently if you want to run your monthly Backups on the last
+           Friday of each month this is only possible with workarounds (e.g
+           scripting) (As some months got 4 Fridays and some got 5 Fridays)
+           The same is true if you plan to run your yearly Backups on the
+           last Friday of the year.  It would be nice to have the ability to
+           use the builtin scheduler for this.
+
+   Why:   In many companies the last working day of the week is Friday (or 
+           Saturday), so to get the most data of the month onto the monthly
+           tape, the employees are advised to insert the tape for the
+           monthly backups on the last friday of the month.
+
+   Notes: To give this a complete functionality it would be nice if the
+           "first" and "last" Keywords could be implemented in the
+           scheduler, so it is also possible to run monthy backups at the
+           first friday of the month and many things more.  So if the syntax
+           would expand to this {first|last} {Month|Week|Day|Mo-Fri} of the
+           {Year|Month|Week} you would be able to run really flexible jobs.
+
+           To got a certain Job run on the last Friday of the Month for example one could 
+           then write
+
+              Run = pool=Monthly last Fri of the Month at 23:50
+
+              ## Yearly Backup
+
+              Run = pool=Yearly last Fri of the Year at 23:50
+
+              ## Certain Jobs the last Week of a Month
+
+              Run = pool=LastWeek last Week of the Month at 23:50
+
+              ## Monthly Backup on the last day of the month
+
+              Run = pool=Monthly last Day of the Month at 23:50
+
+Item 33: Add Minumum Spool Size directive
+Date: 20 March 2008
+Origin: Frank Sweetser <fs@wpi.edu>
+
+   What: Add a new SD directive, "minimum spool size" (or similar).  This
+         directive would specify a minimum level of free space available for
+         spooling.  If the unused spool space is less than this level, any
+         new spooling requests would be blocked as if the "maximum spool
+         size" threshold had bee reached.  Already spooling jobs would be
+         unaffected by this directive.
+
+   Why: I've been bitten by this scenario a couple of times:
+
+        Assume a maximum spool size of 100M. Two concurrent jobs, A and B,
+        are both running.  Due to timing quirks and previously running jobs,
+        job A has used 99.9M of space in the spool directory.  While A is
+        busy despooling to disk, B is happily using the remaining 0.1M of
+        spool space.  This ends up in a spool/despool sequence every 0.1M of
+        data.  In addition to fragmenting the data on the volume far more
+        than was necessary, in larger data sets (ie, tens or hundreds of
+        gigabytes) it can easily produce multi-megabyte report emails!
+
+
+Item 34: Cause daemons to use a specific IP address to source communications
+ Origin: Bill Moran <wmoran@collaborativefusion.com>
+ Date:   18 Dec 2006
+ Status:
+ What:   Cause Bacula daemons (dir, fd, sd) to always use the ip address
+          specified in the [DIR|DF|SD]Addr directive as the source IP
+          for initiating communication.
+ Why:    On complex networks, as well as extremely secure networks, it's
+          not unusual to have multiple possible routes through the network.
+          Often, each of these routes is secured by different policies
+          (effectively, firewalls allow or deny different traffic depending
+          on the source address)
+          Unfortunately, it can sometimes be difficult or impossible to
+          represent this in a system routing table, as the result is
+          excessive subnetting that quickly exhausts available IP space.
+          The best available workaround is to provide multiple IPs to
+          a single machine that are all on the same subnet.  In order
+          for this to work properly, applications must support the ability
+          to bind outgoing connections to a specified address, otherwise
+          the operating system will always choose the first IP that
+          matches the required route.
+ Notes:  Many other programs support this.  For example, the following
+          can be configured in BIND:
+          query-source address 10.0.0.1;
+          transfer-source 10.0.0.2;
+          Which means queries from this server will always come from
+          10.0.0.1 and zone transfers will always originate from
+          10.0.0.2.
+
+
+
+Item 35: Add ability to Verify any specified Job.
+Date: 17 January 2008
+Origin: portrix.net Hamburg, Germany.
+Contact: Christian Sabelmann
+Status: 70% of the required Code is part of the Verify function since v. 2.x
+
+   What:
+   The ability to tell Bacula which Job should verify instead of 
+   automatically verify just the last one.
+
+   Why: 
+   It is sad that such a powerfull feature like Verify Jobs
+   (VolumeToCatalog) is restricted to be used only with the last backup Job
+   of a client.  Actual users who have to do daily Backups are forced to
+   also do daily Verify Jobs in order to take advantage of this useful
+   feature.  This Daily Verify after Backup conduct is not always desired
+   and Verify Jobs have to be sometimes scheduled.  (Not necessarily
+   scheduled in Bacula).  With this feature Admins can verify Jobs once a
+   Week or less per month, selecting the Jobs they want to verify.  This
+   feature is also not to difficult to implement taking in account older bug
+   reports about this feature and the selection of the Job to be verified.
+          
+   Notes: For the verify Job, the user could select the Job to be verified 
+   from a List of the latest Jobs of a client. It would also be possible to 
+   verify a certain volume.  All of these would naturaly apply only for 
+   Jobs whose file information are still in the catalog.
+
+Item 36: Automatic disabling of devices
+   Date: 2005-11-11
+  Origin: Peter Eriksson <peter at ifm.liu dot se>
+  Status:
 
-          Implementing this would perhaps require another catalog change
-          and perhaps major changes in SD code and the DIR-SD protocoll,
-          so I'd only consider this worth implementing if it would
-          actually be used or even needed by many people. 
-
-Item 14:  Merging of multiple backups into a single one. (Also called Synthetic
-          Backup or Consolidation).
-
-  Origin: Marc Cousin and Eric Bollengier 
-  Date:   15 November 2005
-  Status: Depends on first implementing project Item 1 (Migration).
-
-  What:   A merged backup is a backup made without connecting to the Client.
-          It would be a Merge of existing backups into a single backup.
-          In effect, it is like a restore but to the backup medium.
-
-          For instance, say that last sunday we made a full backup.  Then
-          all week long, we created incremental backups, in order to do
-          them fast.  Now comes sunday again, and we need another full.
-          The merged backup makes it possible to do instead an incremental
-          backup (during the night for instance), and then create a merged
-          backup during the day, by using the full and incrementals from
-          the week.  The merged backup will be exactly like a full made
-          sunday night on the tape, but the production interruption on the
-          Client will be minimal, as the Client will only have to send
-          incrementals.
-
-          In fact, if it's done correctly, you could merge all the
-          Incrementals into single Incremental, or all the Incrementals
-          and the last Differential into a new Differential, or the Full,
-          last differential and all the Incrementals into a new Full
-          backup.  And there is no need to involve the Client.
-
-  Why:    The benefit is that :
-          - the Client just does an incremental ;
-          - the merged backup on tape is just as a single full backup,
-            and can be restored very fast.
-
-          This is also a way of reducing the backup data since the old
-          data can then be pruned (or not) from the catalog, possibly
-          allowing older volumes to be recycled
-
-Item 15:  Automatic disabling of devices
-   Date:   2005-11-11
-   Origin: Peter Eriksson <peter at ifm.liu dot se>
-   Status:
-
-   What:  After a configurable amount of fatal errors with a tape drive
+   What: After a configurable amount of fatal errors with a tape drive
           Bacula should automatically disable further use of a certain
           tape drive. There should also be "disable"/"enable" commands in
           the "bconsole" tool.
 
-   Why:   On a multi-drive jukebox there is a possibility of tape drives
+   Why:  On a multi-drive jukebox there is a possibility of tape drives
           going bad during large backups (needing a cleaning tape run,
           tapes getting stuck). It would be advantageous if Bacula would
           automatically disable further use of a problematic tape drive
-          after a configurable amount of errors has occured.
+          after a configurable amount of errors has occurred.
 
           An example: I have a multi-drive jukebox (6 drives, 380+ slots)
           where tapes occasionally get stuck inside the drive. Bacula will
@@ -347,223 +1135,171 @@ Item 15:  Automatic disabling of devices
           further use of that drive and used one of the other ones
           instead.
 
+Item 37: An option to operate on all pools with update vol parameters
+  Origin: Dmitriy Pinchukov <absh@bossdev.kiev.ua>
+   Date: 16 August 2006
+  Status: Patch made by  Nigel Stepp
 
-Item 16:  Directive/mode to backup only file changes, not entire file
-  Date:   11 November 2005
-  Origin: Joshua Kugler <joshua dot kugler at uaf dot edu>
-          Marek Bajon <mbajon at bimsplus dot com dot pl>
-  Status: RFC
+   What: When I do update -> Volume parameters -> All Volumes
+          from Pool, then I have to select pools one by one.  I'd like
+          console to have an option like "0: All Pools" in the list of
+          defined pools.
 
-  What:   Currently when a file changes, the entire file will be backed up in
-          the next incremental or full backup.  To save space on the tapes
-          it would be nice to have a mode whereby only the changes to the
-          file would be backed up when it is changed.
+   Why:  I have many pools and therefore unhappy with manually
+          updating each of them using update -> Volume parameters -> All
+          Volumes from Pool -> pool #.
 
-  Why:    This would save lots of space when backing up large files such as 
-          logs, mbox files, Outlook PST files and the like.
+Item 38: Implement Storage daemon compression
+  Date:  18 December 2006
+  Origin: Vadim A. Umanski , e-mail umanski@ext.ru
+  Status:
+  What:  The ability to compress backup data on the SD receiving data
+          instead of doing that on client sending data.
+  Why:   The need is practical. I've got some machines that can send
+          data to the network 4 or 5 times faster than compressing
+          them (I've measured that). They're using fast enough SCSI/FC
+          disk subsystems but rather slow CPUs (ex. UltraSPARC II).
+          And the backup server has got a quite fast CPUs (ex. Dual P4
+          Xeons) and quite a low load. When you have 20, 50 or 100 GB
+          of raw data - running a job 4 to 5 times faster - that
+          really matters. On the other hand, the data can be
+          compressed 50% or better - so losing twice more space for
+          disk backup is not good at all. And the network is all mine
+          (I have a dedicated management/provisioning network) and I
+          can get as high bandwidth as I need - 100Mbps, 1000Mbps...
+          That's why the server-side compression feature is needed!
+  Notes:
+
+Item 39: Improve Bacula's tape and drive usage and cleaning management 
+  Date:  8 November 2005, November 11, 2005
+  Origin: Adam Thornton <athornton at sinenomine dot net>,
+          Arno Lehmann <al at its-lehmann dot de>
+  Status:
 
-  Notes:  This would require the usage of disk-based volumes as comparing 
-          files would not be feasible using a tape drive.
+  What:  Make Bacula manage tape life cycle information, tape reuse
+          times and drive cleaning cycles.
 
-Item 17:  Quick release of FD-SD connection
-  Origin: Frank Volf (frank at deze dot org)
-  Date:   17 november 2005
-  Status:
+  Why:   All three parts of this project are important when operating
+          backups.
+          We need to know which tapes need replacement, and we need to
+          make sure the drives are cleaned when necessary.  While many
+          tape libraries and even autoloaders can handle all this
+          automatically, support by Bacula can be helpful for smaller
+          (older) libraries and single drives.  Limiting the number of
+          times a tape is used might prevent tape errors when using
+          tapes until the drives can't read it any more.  Also, checking
+          drive status during operation can prevent some failures (as I
+          [Arno] had to learn the hard way...)
 
-   What:  In the bacula implementation a backup is finished after all data
-          and attributes are succesfully written to storage.  When using a
-          tape backup it is very annoying that a backup can take a day,
-          simply because the current tape (or whatever) is full and the
-          administrator has not put a new one in.  During that time the
-          system cannot be taken off-line, because there is still an open
-          session between the storage daemon and the file daemon on the
-          client.
-
-          Although this is a very good strategey for making "safe backups"
-          This can be annoying for e.g.  laptops, that must remain
-          connected until the bacukp is completed.
-
-          Using a new feature called "migration" it will be possible to
-          spool first to harddisk (using a special 'spool' migration
-          scheme) and then migrate the backup to tape.
-
-          There is still the problem of getting the attributes committed.
-          If it takes a very long time to do, with the current code, the
-          job has not terminated, and the File daemon is not freed up.  The
-          Storage daemon should release the File daemon as soon as all the
-          file data and all the attributes have been sent to it (the SD).
-          Currently the SD waits until everything is on tape and all the
-          attributes are transmitted to the Director before signalling
-          completion to the FD. I don't think I would have any problem
-          changing this.  The reason is that even if the FD reports back to
-          the Dir that all is OK, the job will not terminate until the SD
-          has done the same thing -- so in a way keeping the SD-FD link
-          open to the very end is not really very productive ...
-
-   Why:   Makes backup of laptops much easier.
-
-
-Item 18:  Add support for CACHEDIR.TAG
-  Origin: Norbert Kiesel <nkiesel at tbdnetworks dot com>
-  Date:   21 November 2005
-  Status:
+  Notes: First, Bacula could (and even does, to some limited extent)
+          record tape and drive usage.  For tapes, the number of mounts,
+          the amount of data, and the time the tape has actually been
+          running could be recorded.  Data fields for Read and Write
+          time and Number of mounts already exist in the catalog (I'm
+          not sure if VolBytes is the sum of all bytes ever written to
+          that volume by Bacula).  This information can be important
+          when determining which media to replace.  The ability to mark
+          Volumes as "used up" after a given number of write cycles
+          should also be implemented so that a tape is never actually
+          worn out.  For the tape drives known to Bacula, similar
+          information is interesting to determine the device status and
+          expected life time: Time it's been Reading and Writing, number
+          of tape Loads / Unloads / Errors.  This information is not yet
+          recorded as far as I [Arno] know.  A new volume status would
+          be necessary for the new state, like "Used up" or "Worn out".
+          Volumes with this state could be used for restores, but not
+          for writing. These volumes should be migrated first (assuming
+          migration is implemented) and, once they are no longer needed,
+          could be moved to a Trash pool.
+
+          The next step would be to implement a drive cleaning setup.
+          Bacula already has knowledge about cleaning tapes.  Once it
+          has some information about cleaning cycles (measured in drive
+          run time, number of tapes used, or calender days, for example)
+          it can automatically execute tape cleaning (with an
+          autochanger, obviously) or ask for operator assistance loading
+          a cleaning tape.
+
+          The final step would be to implement TAPEALERT checks not only
+          when changing tapes and only sending the information to the
+          administrator, but rather checking after each tape error,
+          checking on a regular basis (for example after each tape
+          file), and also before unloading and after loading a new tape.
+          Then, depending on the drives TAPEALERT state and the known
+          drive cleaning state Bacula could automatically schedule later
+          cleaning, clean immediately, or inform the operator.
 
-  What:   CACHDIR.TAG is a proposal for identifying directories which
-          should be ignored for archiving/backup.  It works by ignoring
-          directory trees which have a file named CACHEDIR.TAG with a
-          specific content.  See
-          http://www.brynosaurus.com/cachedir/spec.html
-          for details.
-
-          From Peter Eriksson:
-          I suggest that if this is implemented (I've also asked for this
-          feature some year ago) that it is made compatible with Legato
-          Networkers ".nsr" files where you can specify a lot of options on
-          how to handle files/directories (including denying further
-          parsing of .nsr files lower down into the directory trees).  A
-          PDF version of the .nsr man page can be viewed at:
-
-          http://www.ifm.liu.se/~peter/nsr.pdf
-
-  Why:    It's a nice alternative to "exclude" patterns for directories
-          which don't have regular pathnames.  Also, it allows users to
-          control backup for themself.  Implementation should be pretty
-          simple.  GNU tar >= 1.14 or so supports it, too.
-
-  Notes:  I envision this as an optional feature to a fileset
-          specification.
-
-Item 19:  Implement new {Client}Run{Before|After}Job feature.
-  Date:   26 September 2005
-  Origin: Phil Stracchino <phil.stracchino at speakeasy dot net>
-  Status: 
+          Implementing this would perhaps require another catalog change
+          and perhaps major changes in SD code and the DIR-SD protocol,
+          so I'd only consider this worth implementing if it would
+          actually be used or even needed by many people.
 
-  What:   Some time ago, there was a discussion of RunAfterJob and
-          ClientRunAfterJob, and the fact that they do not run after failed
-          jobs.  At the time, there was a suggestion to add a
-          RunAfterFailedJob directive (and, presumably, a matching
-          ClientRunAfterFailedJob directive), but to my knowledge these
-          were never implemented.
-
-          An alternate way of approaching the problem has just occurred to
-          me.  Suppose the RunBeforeJob and RunAfterJob directives were
-          expanded in a manner something like this example:
-
-          RunBeforeJob {
-              Command = "/opt/bacula/etc/checkhost %c"
-              RunsOnClient = No
-              RunsAtJobLevels = All       # All, Full, Diff, Inc
-              AbortJobOnError = Yes
-          }
-          RunBeforeJob {
-              Command = c:/bacula/systemstate.bat
-              RunsOnClient = yes
-              RunsAtJobLevels = All       # All, Full, Diff, Inc
-              AbortJobOnError = No
-          }
+          Implementation of these projects could happen in three distinct
+          sub-projects: Measuring Tape and Drive usage, retiring
+          volumes, and handling drive cleaning and TAPEALERTs.
 
-          RunAfterJob {
-              Command = c:/bacula/deletestatefile.bat
-              RunsOnClient = Yes
-              RunsAtJobLevels = All       # All, Full, Diff, Inc
-              RunsOnSuccess = Yes
-              RunsOnFailure = Yes
-          }
-          RunAfterJob {
-              Command = c:/bacula/somethingelse.bat
-              RunsOnClient = Yes
-              RunsAtJobLevels = All
-              RunsOnSuccess = No
-              RunsOnFailure = Yes
-          }
-          RunAfterJob {
-              Command = "/opt/bacula/etc/checkhost -v %c"
-              RunsOnClient = No
-              RunsAtJobLevels = All
-              RunsOnSuccess = No
-              RunsOnFailure = Yes
-          }
+Item 40: Multiple threads in file daemon for the same job
+  Date:  27 November 2005
+  Origin: Ove Risberg (Ove.Risberg at octocode dot com)
+  Status:
 
+  What:  I want the file daemon to start multiple threads for a backup
+          job so the fastest possible backup can be made.
 
-  Why:    It would be a significant change to the structure of the
-          directives, but allows for a lot more flexibility, including
-          RunAfter commands that will run regardless of whether the job
-          succeeds, or RunBefore tasks that still allow the job to run even
-          if that specific RunBefore fails.
+          The file daemon could parse the FileSet information and start
+          one thread for each File entry located on a separate
+          filesystem.
 
-  Notes:  By Kern: I would prefer to have a single new Resource called
-          RunScript. More notes from Phil:
+          A confiuration option in the job section should be used to
+          enable or disable this feature. The confgutration option could
+          specify the maximum number of threads in the file daemon.
 
-            RunBeforeJob = yes|no
-            RunAfterJob = yes|no
-            RunsAtJobLevels = All|Full|Diff|Inc
+          If the theads could spool the data to separate spool files
+          the restore process will not be much slower.
 
-          The AbortJobOnError, RunsOnSuccess and RunsOnFailure directives
-          could be optional, and possibly RunsWhen as well.
+  Why:   Multiple concurrent backups of a large fileserver with many
+          disks and controllers will be much faster.
 
-          AbortJobOnError would be ignored unless RunsWhen was set to Before
-          (or RunsBefore Job set to Yes), and would default to Yes if
-          omitted.  If AbortJobOnError was set to No, failure of the script
-          would still generate a warning.
+========= End items voted on May 2009 ==================
 
-          RunsOnSuccess would be ignored unless RunsWhen was set to After
-          (or RunsBeforeJob set to No), and default to Yes.
+========= New items after last vote ====================
 
-          RunsOnFailure would be ignored unless RunsWhen was set to After,
-          and default to No.
+Item 1:   Relabel disk volume after recycling
+  Origin: Pasi Kärkkäinen <pasik@iki.fi>
+  Date:   07 May 2009.
+  Status: Not implemented yet, no code written.
 
-          Allow having the before/after status on the script command
-          line so that the same script can be used both before/after.
-          David Boyes.
+  What:   The ability to relabel the disk volume (and thus rename the file on the disk) 
+          after it has been recycled. Useful when you have a single job per disk volume, 
+          and you use a custom Label format, for example: 
+          Label Format = "${Client}-${Level}-${NumVols:p/4/0/r}-${Year}_${Month}_${Day}-${Hour}_${Minute}"
 
-Item 20:  "Maximum Rewrite (Recycle) Times" for a tape
-    Date: 8 November 2005
-  Origin: Adam Thornton <athornton at sinenomine dot net>
-  Status: 
+  Why:    Disk volumes in Bacula get the label/filename when they are used for the first time.
+          If you use recycling and custom label format like above, the disk
+          volume name doesn't match the contents after it has been recycled.
+          This feature makes it possible to keep the label/filename in sync
+          with the content and thus makes it easy to check/monitor the backups
+          from the shell and/or normal file management tools, because the filenames 
+          of the disk volumes match the content.
 
-    What: The ability to use a Volume for at most N re-write times, marking  
-          it unavailable after that.
-
-     Why: I was working with a customer this morning who mentioned that it
-          would be useful to automatically age out tapes that had been
-          rewritten enough times that the media lifespan was becoming
-          questionable (his old backup system supported saying "rewrite this
-          tape 50 times at most." Neither Maximum Volume Jobs nor Volume Use
-          Duration quite does this, because you can have multiple jobs per
-          volume, and it's not entirely guaranteed that time and number of
-          write cycles map linearly (although they probably do in actual
-          usage), and also because "Used" volumes can be recycled, and the
-          status we want here is something like "full and now unwriteable."
-          I haven't looked but I suspect this would require a change in the
-          database format, to keep track of how many cycles a volume has
-          been through, and maybe an additional Status type.
-
-
-Item 21:  Allow FD to initiate a backup
-  Origin: Frank Volf (frank at deze dot org)
-  Date:   17 november 2005
-  Status:
+  Notes:  The configuration option could be "Relabel after Recycling = Yes".
 
-   What:  Provide some means, possibly by a restricted console that
-          allows a FD to initiate a backup, and that uses the connection
-          established by the FD to the Director for the backup so that
-          a Director that is firewalled can do the backup.
 
-   Why:   Makes backup of laptops much easier.
+========= Add new items above this line =================
 
 
-============= Empty RFC form ===========
-Item n:   One line summary ...
-  Date:   Date submitted 
+============= Empty Feature Request form ===========
+Item  n: One line summary ...
+  Date:  Date submitted 
   Origin: Name and email of originator.
   Status: 
 
-  What:   More detailed explanation ...
+  What:  More detailed explanation ...
 
-  Why:    Why it is important ...
+  Why:   Why it is important ...
 
-  Notes:  Additional notes or features (omit if not used)
-============== End RFC form ==============
+  Notes: Additional notes or features (omit if not used)
+============== End Feature Request form ==============
 
 
-Items completed for release 1.38.0 -- see kernsdone
+========== Items put on hold by Kern ============================