X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=bacula%2Fprojects;h=3da15a6a3275083664935f2379979e008531a21c;hb=6f6aaa5a1ab969a158abb2c6698a440585ae94c4;hp=312a3b65019db9aa23d49ea3699c5c41c8a0e9fe;hpb=5de97d6f0ee8002de6c0c1a907b86a2c2f42f9bc;p=bacula%2Fbacula diff --git a/bacula/projects b/bacula/projects index 312a3b6501..3da15a6a32 100644 --- a/bacula/projects +++ b/bacula/projects @@ -1,8 +1,8 @@ Projects: Bacula Projects Roadmap - 07 December 2005 - (prioritized by user vote) + Prioritized by user vote 07 December 2005 + Status updated 30 July 2006 Summary: Item 1: Implement data encryption (as opposed to comm encryption) @@ -37,7 +37,7 @@ Below, you will find more information on future projects: Item 1: Implement data encryption (as opposed to comm encryption) Date: 28 October 2005 Origin: Sponsored by Landon and 13 contributors to EFF. - Status: Landon Fuller is currently implementing this. + Status: Done: Landon Fuller has implemented this in 1.39.x. What: Currently the data that is stored on the Volume is not encrypted. For confidentiality, encryption of data at @@ -51,7 +51,7 @@ Item 2: Implement Migration that moves Jobs from one Pool to another. Origin: Sponsored by Riege Software International GmbH. Contact: Daniel Holtkamp Date: 28 October 2005 - Status: Partially coded in 1.37 -- much more to do. Assigned to + Status: 90% complete: Working in 1.39, more to do. Assigned to Kern. What: The ability to copy, move, or archive data that is on a @@ -102,10 +102,16 @@ Item 3: Accurate restoration of renamed/deleted files from but one additional database record must be written, which does not need any database change. + Kern: see if we can correct restoration of directories if + replace=ifnewer is set. Currently, if the directory does not + exist, a "dummy" directory is created, then when all the files + are updated, the dummy directory is newer so the real values + are not updated. + Item 4: Implement a Bacula GUI/management tool using Python. Origin: Kern Date: 28 October 2005 - Status: + Status: Lucus is working on this for Python GTK+. What: Implement a Bacula console, and management tools using Python and Qt or GTK. @@ -282,8 +288,8 @@ Item 8: Implement creation and maintenance of copy pools Item 9: Implement new {Client}Run{Before|After}Job feature. Date: 26 September 2005 - Origin: Phil Stracchino - Status: + Origin: Phil Stracchino + Status: Done. This has been implemented by Eric Bollengier What: Some time ago, there was a discussion of RunAfterJob and ClientRunAfterJob, and the fact that they do not run after failed @@ -292,45 +298,63 @@ Item 9: Implement new {Client}Run{Before|After}Job feature. ClientRunAfterFailedJob directive), but to my knowledge these were never implemented. + The current implementation doesn't permit to add new feature easily. + An alternate way of approaching the problem has just occurred to me. Suppose the RunBeforeJob and RunAfterJob directives were - expanded in a manner something like this example: + expanded in a manner like this example: - RunBeforeJob { + RunScript { Command = "/opt/bacula/etc/checkhost %c" - RunsOnClient = No - RunsAtJobLevels = All # All, Full, Diff, Inc - AbortJobOnError = Yes + RunsOnClient = No # default + AbortJobOnError = Yes # default + RunsWhen = Before } - RunBeforeJob { + RunScript { Command = c:/bacula/systemstate.bat RunsOnClient = yes - RunsAtJobLevels = All # All, Full, Diff, Inc AbortJobOnError = No + RunsWhen = After + RunsOnFailure = yes } - RunAfterJob { + RunScript { Command = c:/bacula/deletestatefile.bat - RunsOnClient = Yes - RunsAtJobLevels = All # All, Full, Diff, Inc - RunsOnSuccess = Yes - RunsOnFailure = Yes - } - RunAfterJob { - Command = c:/bacula/somethingelse.bat - RunsOnClient = Yes - RunsAtJobLevels = All - RunsOnSuccess = No - RunsOnFailure = Yes - } - RunAfterJob { - Command = "/opt/bacula/etc/checkhost -v %c" - RunsOnClient = No - RunsAtJobLevels = All - RunsOnSuccess = No - RunsOnFailure = Yes + Target = rico-fd + RunsWhen = Always } + It's now possible to specify more than 1 command per Job. + (you can stop your database and your webserver without a script) + + ex : + Job { + Name = "Client1" + JobDefs = "DefaultJob" + Write Bootstrap = "/tmp/bacula/var/bacula/working/Client1.bsr" + FileSet = "Minimal" + + RunBeforeJob = "echo test before ; echo test before2" + RunBeforeJob = "echo test before (2nd time)" + RunBeforeJob = "echo test before (3rd time)" + RunAfterJob = "echo test after" + ClientRunAfterJob = "echo test after client" + + RunScript { + Command = "echo test RunScript in error" + Runsonclient = yes + RunsOnSuccess = no + RunsOnFailure = yes + RunsWhen = After # never by default + } + RunScript { + Command = "echo test RunScript on success" + Runsonclient = yes + RunsOnSuccess = yes # default + RunsOnFailure = no # default + RunsWhen = After + } + } Why: It would be a significant change to the structure of the directives, but allows for a lot more flexibility, including @@ -338,19 +362,19 @@ Item 9: Implement new {Client}Run{Before|After}Job feature. succeeds, or RunBefore tasks that still allow the job to run even if that specific RunBefore fails. - Notes: By Kern: I would prefer to have a single new Resource called - RunScript. More notes from Phil: + Notes: (More notes from Phil, Kern, David and Eric) + I would prefer to have a single new Resource called + RunScript. - RunBeforeJob = yes|no - RunAfterJob = yes|no - RunsAtJobLevels = All|Full|Diff|Inc + RunsWhen = After|Before|Always + RunsAtJobLevels = All|Full|Diff|Inc # not yet implemented The AbortJobOnError, RunsOnSuccess and RunsOnFailure directives - could be optional, and possibly RunsWhen as well. + could be optional, and possibly RunWhen as well. AbortJobOnError would be ignored unless RunsWhen was set to Before - (or RunsBefore Job set to Yes), and would default to Yes if - omitted. If AbortJobOnError was set to No, failure of the script + and would default to Yes if omitted. + If AbortJobOnError was set to No, failure of the script would still generate a warning. RunsOnSuccess would be ignored unless RunsWhen was set to After @@ -361,12 +385,12 @@ Item 9: Implement new {Client}Run{Before|After}Job feature. Allow having the before/after status on the script command line so that the same script can be used both before/after. - David Boyes. Item 10: Merge multiple backups (Synthetic Backup or Consolidation). Origin: Marc Cousin and Eric Bollengier Date: 15 November 2005 - Status: Depends on first implementing project Item 1 (Migration). + Status: Waiting implementation. Depends on first implementing + project Item 2 (Migration). What: A merged backup is a backup made without connecting to the Client. It would be a Merge of existing backups into a single backup. @@ -423,7 +447,7 @@ Item 12: Directive/mode to backup only file changes, not entire file Date: 11 November 2005 Origin: Joshua Kugler Marek Bajon - Status: RFC + Status: What: Currently when a file changes, the entire file will be backed up in the next incremental or full backup. To save space on the tapes @@ -464,7 +488,8 @@ Item 13: Multiple threads in file daemon for the same job Item 14: Implement red/black binary tree routines. Date: 28 October 2005 Origin: Kern - Status: + Status: Class code is complete. Code needs to be integrated into + restore tree code. What: Implement a red/black binary tree class. This could then replace the current binary insert/search routines @@ -476,7 +501,8 @@ Item 14: Implement red/black binary tree routines. Item 15: Add support for FileSets in user directories CACHEDIR.TAG Origin: Norbert Kiesel Date: 21 November 2005 - Status: + Status: (I think this is better done using a Python event that I + will implement in version 1.39.x). What: CACHDIR.TAG is a proposal for identifying directories which should be ignored for archiving/backup. It works by ignoring @@ -507,7 +533,7 @@ Item 15: Add support for FileSets in user directories CACHEDIR.TAG Item 16: Implement extraction of Win32 BackupWrite data. Origin: Thorsten Engel Date: 28 October 2005 - Status: Assigned to Thorsten. Implemented in current CVS + Status: Done. Assigned to Thorsten. Implemented in current CVS What: This provides the Bacula File daemon with code that can pick apart the stream output that Microsoft writes @@ -637,7 +663,7 @@ Item 21: Quick release of FD-SD connection after backup. Item 22: Permit multiple Media Types in an Autochanger Origin: Kern - Status: Now implemented + Status: Done. Implemented in 1.38.9 (I think). What: Modify the Storage daemon so that multiple Media Types can be specified in an autochanger. This would be somewhat @@ -713,8 +739,25 @@ Item 25: Implement huge exclude list support using hashing. do a Bacula restore. By excluding the base OS files, the backup set will be *much* smaller. + +============= Empty Feature Request form =========== +Item n: One line summary ... + Date: Date submitted + Origin: Name and email of originator. + Status: + + What: More detailed explanation ... + + Why: Why it is important ... + + Notes: Additional notes or features (omit if not used) +============== End Feature Request form ============== + + +=============================================== +Feature requests submitted after cutoff for December 2005 vote + and not yet discussed. =============================================== -Not in Dec 2005 Vote: Item n: Allow skipping execution of Jobs Date: 29 November 2005 Origin: Florian Schnabel @@ -726,15 +769,387 @@ Item n: Allow skipping execution of Jobs that would be really handy, other jobs could proceed normally and you won't get errors that way. -============= Empty Feature Request form =========== -Item n: One line summary ... - Date: Date submitted - Origin: Name and email of originator. - Status: +=================================================== + +Item n: archive data + + Origin: calvin streeting calvin at absentdream dot com + Date: 15/5/2006 + + What: The abilty to archive to media (dvd/cd) in a uncompressd format + for dead filing (archiving not backing up) + + Why: At my works when jobs are finished and moved off of the main file + servers (raid based systems) onto a simple linux file server (ide based + system) so users can find old information without contacting the IT + dept. + + So this data dosn't realy change it only gets added to, + But it also needs backing up. At the moment it takes + about 8 hours to back up our servers (working data) so + rather than add more time to existing backups i am trying + to implement a system where we backup the acrhive data to + cd/dvd these disks would only need to be appended to + (burn only new/changed files to new disks for off site + storage). basialy understand the differnce between + achive data and live data. + + Notes: scan the data and email me when it needs burning divide + into predifind chunks keep a recored of what is on what + disk make me a label (simple php->mysql=>pdf stuff) i + could do this bit ability to save data uncompresed so + it can be read in any other system (future proof data) + save the catalog with the disk as some kind of menu + system + +Item : Tray monitor window cleanups + Origin: Alan Brown ajb2 at mssl dot ucl dot ac dot uk + Date: 24 July 2006 + Status: + What: Resizeable and scrollable windows in the tray monitor. - What: More detailed explanation ... + Why: With multiple clients, or with many jobs running, the displayed + window often ends up larger than the available screen, making + the trailing items difficult to read. - Why: Why it is important ... + Notes: - Notes: Additional notes or features (omit if not used) -============== End Feature Request form ============== + Item : Clustered file-daemons + Origin: Alan Brown ajb2 at mssl dot ucl dot ac dot uk + Date: 24 July 2006 + Status: + What: A "virtual" filedaemon, which is actually a cluster of real ones. + + Why: In the case of clustered filesystems (SAN setups, GFS, or OCFS2, etc) + multiple machines may have access to the same set of filesystems + + For performance reasons, one may wish to initate backups from + several of these machines simultaneously, instead of just using + one backup source for the common clustered filesystem. + + For obvious reasons, normally backups of $A-FD/$PATH and + B-FD/$PATH are treated as different backup sets. In this case + they are the same communal set. + + Likewise when restoring, it would be easier to just specify + one of the cluster machines and let bacula decide which to use. + + This can be faked to some extent using DNS round robin entries + and a virtual IP address, however it means "status client" will + always give bogus answers. Additionally there is no way of + spreading the load evenly among the servers. + + What is required is something similar to the storage daemon + autochanger directives, so that Bacula can keep track of + operating backups/restores and direct new jobs to a "free" + client. + + Notes: + +Item : Tray monitor window cleanups + Origin: Alan Brown ajb2 at mssl dot ucl dot ac dot uk + Date: 24 July 2006 + Status: + What: Resizeable and scrollable windows in the tray monitor. + + Why: With multiple clients, or with many jobs running, the displayed + window often ends up larger than the available screen, making + the trailing items difficult to read. + + Notes: + +Item: Commercial database support + Origin: Russell Howe + Date: 26 July 2006 + Status: + + What: It would be nice for the database backend to support more + databases. I'm thinking of SQL Server at the moment, but I guess Oracle, + DB2, MaxDB, etc are all candidates. SQL Server would presumably be + implemented using FreeTDS or maybe an ODBC library? + + Why: We only really have one database server, which is MS SQL Server + 2000. Maintaining a second one for the backup software (we grew out of + SQLite, which I liked, but which didn't work so well with our database + size). We don't really have a machine with the resources to run + postgres, and would rather only maintain a single DBMS. We're stuck with + SQL Server because pretty much all the company's custom applications + (written by consultants) are locked into SQL Server 2000. I can imagine + this scenario is fairly common, and it would be nice to use the existing + properly specced database server for storing Bacula's catalog, rather + than having to run a second DBMS. + + +Item n: Split documentation + Origin: Maxx + Date: 27th July 2006 + Status: + + What: Split documentation in several books + + Why: Bacula manual has now more than 600 pages, and looking for + implementation details is getting complicated. I think + it would be good to split the single volume in two or + maybe three parts: + + 1) Introduction, requirements and tutorial, typically + are useful only until first installation time + + 2) Basic installation and configuration, with all the + gory details about the directives supported 3) + Advanced Bacula: testing, troubleshooting, GUI and + ancillary programs, security managements, scripting, + etc. + + Notes: + +Item n: Include an option to operate on all pools when doing + update vol parameters + + Origin: Dmitriy Pinchukov + Date: 16 August 2006 + Status: + + What: When I do update -> Volume parameters -> All Volumes + from Pool, then I have to select pools one by one. I'd like + console to have an option like "0: All Pools" in the list of + defined pools. + + Why: I have many pools and therefore unhappy with manually + updating each of them using update -> Volume parameters -> All + Volumes from Pool -> pool #. + +Item n: Automatic promotion of backup levels + Date: 19 January 2006 + Origin: Adam Thornton + Status: Blue sky + + What: Amanda has a feature whereby it estimates the space that a + differential, incremental, and full backup would take. If the + difference in space required between the scheduled level and the next + level up is beneath some user-defined critical threshold, the backup + level is bumped to the next type. Doing this minimizes the number of + volumes necessary during a restore, with a fairly minimal cost in + backup media space. + + Why: I know at least one (quite sophisticated and smart) user + for whom the absence of this feature is a deal-breaker in terms of + using Bacula; if we had it it would eliminate the one cool thing + Amanda can do and we can't (at least, the one cool thing I know of). + + + + +Item n+1: Incorporation of XACML2/SAML2 parsing + Date: 19 January 2006 + Origin: Adam Thornton + Status: Blue sky + + What: XACML is "eXtensible Access Control Markup Language" and + "SAML is the "Security Assertion Markup Language"--an XML standard + for making statements about identity and authorization. Having these + would give us a framework to approach ACLs in a generic manner, and + in a way flexible enough to support the four major sorts of ACLs I + see as a concern to Bacula at this point, as well as (probably) to + deal with new sorts of ACLs that may appear in the future. + + Why: Bacula is beginning to need to back up systems with ACLs + that do not map cleanly onto traditional Unix permissions. I see + four sets of ACLs--in general, mutually incompatible with one + another--that we're going to need to deal with. These are: NTFS + ACLs, POSIX ACLs, NFSv4 ACLS, and AFS ACLS. (Some may question the + relevance of AFS; AFS is one of Sine Nomine's core consulting + businesses, and having a reputable file-level backup and restore + technology for it (as Tivoli is probably going to drop AFS support + soon since IBM no longer supports AFS) would be of huge benefit to + our customers; we'd most likely create the AFS support at Sine Nomine + for inclusion into the Bacula (and perhaps some changes to the + OpenAFS volserver) core code.) + + Now, obviously, Bacula already handles NTFS just fine. However, I + think there's a lot of value in implementing a generic ACL model, so + that it's easy to support whatever particular instances of ACLs come + down the pike: POSIX ACLS (think SELinux) and NFSv4 are the obvious + things arriving in the Linux world in a big way in the near future. + XACML, although overcomplicated for our needs, provides this + framework, and we should be able to leverage other people's + implementations to minimize the amount of work *we* have to do to get + a generic ACL framework. Basically, the costs of implementation are + high, but they're largely both external to Bacula and already sunk. + +Item 1: Add an over-ride in the Schedule configuration to use a + different pool for different backup types. + +Date: 19 Jan 2005 +Origin: Chad Slater +Status: + + What: Adding a FullStorage=BigTapeLibrary in the Schedule resource + would help those of us who use different storage devices for different + backup levels cope with the "auto-upgrade" of a backup. + + Why: Assume I add several new device to be backed up, i.e. several + hosts with 1TB RAID. To avoid tape switching hassles, incrementals are + stored in a disk set on a 2TB RAID. If you add these devices in the + middle of the month, the incrementals are upgraded to "full" backups, + but they try to use the same storage device as requested in the + incremental job, filling up the RAID holding the differentials. If we + could override the Storage parameter for full and/or differential + backups, then the Full job would use the proper Storage device, which + has more capacity (i.e. a 8TB tape library. + + +Item: Implement multiple numeric backup levels as supported by dump +Date: 3 April 2006 +Origin: Daniel Rich +Status: +What: Dump allows specification of backup levels numerically instead of just + "full", "incr", and "diff". In this system, at any given level, all + files are backed up that were were modified since the last backup of a + higher level (with 0 being the highest and 9 being the lowest). A + level 0 is therefore equivalent to a full, level 9 an incremental, and + the levels 1 through 8 are varying levels of differentials. For + bacula's sake, these could be represented as "full", "incr", and + "diff1", "diff2", etc. + +Why: Support of multiple backup levels would provide for more advanced backup + rotation schemes such as "Towers of Hanoi". This would allow better + flexibility in performing backups, and can lead to shorter recover + times. + +Notes: Legato Networker supports a similar system with full, incr, and 1-9 as + levels. + +Kern notes: I think this would add very little functionality, but a *lot* of + additional overhead to Bacula. + +Item 1: include JobID in spool file name + Origin: Mark Bergman + Date: Tue Aug 22 17:13:39 EDT 2006 + Status: + + What: Change the name of the spool file to include the JobID + + Why: JobIDs are the common key used to refer to jobs, yet the + spoolfile name doesn't include that information. The date/time + stamp is useful (and should be retained). + + + +Item 2: include timestamp of job launch in "stat clients" output + Origin: Mark Bergman + Date: Tue Aug 22 17:13:39 EDT 2006 + Status: + + What: The "stat clients" command doesn't include any detail on when + the active backup jobs were launched. + + Why: Including the timestamp would make it much easier to decide whether + a job is running properly. + + Notes: It may be helpful to have the output from "stat clients" formatted + more like that from "stat dir" (and other commands), in a column + format. The per-client information that's currently shown (level, + client name, JobId, Volume, pool, device, Files, etc.) is good, but + somewhat hard to parse (both programmatically and visually), + particularly when there are many active clients. + +Item 1: Filesystemwatch triggered backup. + Date: 31 August 2006 + Origin: Jesper Krogh + Status: Unimplemented, depends probably on "client initiated backups" + + What: With inotify and similar filesystem triggeret notification + systems is it possible to have the file-daemon to monitor + filesystem changes and initiate backup. + + Why: There are 2 situations where this is nice to have. + 1) It is possible to get a much finer-grained backup than + the fixed schedules used now.. A file created and deleted + a few hours later, can automatically be caught. + + 2) The introduced load on the system will probably be + distributed more even on the system. + + Notes: This can be combined with configration that specifies + something like: "at most every 15 minutes or when changes + consumed XX MB". + +Item n: Message mailing based on backup types +Origin: Evan Kaufman + Date: January 6, 2006 +Status: + + What: In the "Messages" resource definitions, allowing messages + to be mailed based on the type (backup, restore, etc.) and level + (full, differential, etc) of job that created the originating + message(s). + +Why: It would, for example, allow someone's boss to be emailed + automatically only when a Full Backup job runs, so he can + retrieve the tapes for offsite storage, even if the IT dept. + doesn't (or can't) explicitly notify him. At the same time, his + mailbox wouldnt be filled by notifications of Verifies, Restores, + or Incremental/Differential Backups (which would likely be kept + onsite). + +Notes: + One way this could be done is through additional message types, for example: + + Messages { + # email the boss only on full system backups + Mail = boss@mycompany.com = full, !incremental, !differential, !restore, + !verify, !admin + # email us only when something breaks + MailOnError = itdept@mycompany.com = all + } + + +Item n: Allow inclusion/exclusion of files in a fileset by creation/mod times + Origin: Evan Kaufman + Date: January 11, 2006 + Status: + + What: In the vein of the Wild and Regex directives in a Fileset's + Options, it would be helpful to allow a user to include or exclude + files and directories by creation or modification times. + + You could factor the Exclude=yes|no option in much the same way it + affects the Wild and Regex directives. For example, you could exclude + all files modified before a certain date: + + Options { + Exclude = yes + Modified Before = #### + } + + Or you could exclude all files created/modified since a certain date: + + Options { + Exclude = yes + Created Modified Since = #### + } + + The format of the time/date could be done several ways, say the number + of seconds since the epoch: + 1137008553 = Jan 11 2006, 1:42:33PM # result of `date +%s` + + Or a human readable date in a cryptic form: + 20060111134233 = Jan 11 2006, 1:42:33PM # YYYYMMDDhhmmss + + Why: I imagine a feature like this could have many uses. It would + allow a user to do a full backup while excluding the base operating + system files, so if I installed a Linux snapshot from a CD yesterday, + I'll *exclude* all files modified *before* today. If I need to + recover the system, I use the CD I already have, plus the tape backup. + Or if, say, a Windows client is hit by a particularly corrosive + virus, and I need to *exclude* any files created/modified *since* the + time of infection. + + Notes: Of course, this feature would work in concert with other + in/exclude rules, and wouldnt override them (or each other). + + Notes: The directives I'd imagine would be along the lines of + "[Created] [Modified] [Before|Since] = ". + So one could compare against 'ctime' and/or 'mtime', but ONLY 'before' + or 'since'.