X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=bacula%2Fprojects;h=94c6047c4e3948a1d347b6b936f06286bfd0c5b4;hb=2fe63270075b56f945034210985242ba26ea24c7;hp=cc79273c48323f172b6691d8a42aa5d9a8ac18c5;hpb=96cf382bdff2d9b83a46bddc8367f0b8706d1b5f;p=bacula%2Fbacula diff --git a/bacula/projects b/bacula/projects index cc79273c48..94c6047c4e 100644 --- a/bacula/projects +++ b/bacula/projects @@ -1,226 +1,1091 @@ Projects: Bacula Projects Roadmap - 22 February 2004 + Status updated 15 December 2006 -Item 1: Implement Base jobs. - What: A base job is sort of like a Full save except that you - will want the FileSet to contain only files that are unlikely - to change in the future (i.e. a snapshot of most of your - system after installing it). After the base job has been run, - when you are doing a Full save, you specify one or more - Base jobs to be used. All files that have been backed up in - the Base job/jobs but not modified will then be excluded from - the backup. During a restore, the Base jobs will be - automatically pulled in where necessary. - - Why: This is something none of the competition does, as far as we -know - (except BackupPC, which is a Perl program that saves to disk - only). It is big win for the user, it makes Bacula stand out - as offering a unique optimization that immediately saves time - and money. Basically, imagine that you have 100 nearly -identical - Windows or Linux machine containing the OS and user files. - Now for the OS part, a Base job will be backed up once, and - rather than making 100 copies of the OS, there will be only - one. If one or more of the systems have some files updated, - no problem, they will be automatically restored. - - Notes: Huge savings in tape usage even for a single machine. Will - require more resources because the DIR must send FD a list of - files/attribs, and the FD must search the list and compare it - for each file to be saved. - - -Item 2: Job Data Spooling. - - What: Make the Storage daemon use intermediate file storage to -buffer - the data to disk before writing it to the tape. +Summary: +Item 1: Accurate restoration of renamed/deleted files +Item 2: Implement a Bacula GUI/management tool. +Item 3: Implement Base jobs. +Item 4: Implement from-client and to-client on restore command line. +Item 5: Implement creation and maintenance of copy pools +Item 6: Merge multiple backups (Synthetic Backup or Consolidation). +Item 8: Deletion of Disk-Based Bacula Volumes +Item 9: Implement a Python interface to the Bacula catalog. +Item 10: Archival (removal) of User Files to Tape +Item 11: Add Plug-ins to the FileSet Include statements. +Item 12: Implement more Python events in Bacula. +Item 13: Quick release of FD-SD connection after backup. +Item 14: Implement huge exclude list support using hashing. +Item 15: Allow skipping execution of Jobs +Item 16: Tray monitor window cleanups +Item 17: Split documentation +Item 18: Automatic promotion of backup levels +Item 19: Add an override in Schedule for Pools based on backup types. +Item 20: An option to operate on all pools with update vol parameters +Item 21: Include JobID in spool file name +Item 22: Include timestamp of job launch in "stat clients" output +Item 23: Message mailing based on backup types +Item 24: Allow inclusion/exclusion of files in a fileset by creation/mod times +Item 25: Add a scheduling syntax that permits weekly rotations +Item 26: Improve Bacula's tape and drive usage and cleaning management. +Item 27: Implement support for stacking arbitrary stream filters, sinks. +Item 28: Allow FD to initiate a backup +Item 29: Directive/mode to backup only file changes, not entire file +Item 30: Automatic disabling of devices +Item 31: Incorporation of XACML2/SAML2 parsing +Item 32: Clustered file-daemons +Item 33: Commercial database support +Item 34: Archive data +Item 35: Filesystem watch triggered backup. +Item 36: Implement multiple numeric backup levels as supported by dump + + +Below, you will find more information on future projects: + +Item 1: Accurate restoration of renamed/deleted files + Date: 28 November 2005 + Origin: Martin Simmons (martin at lispworks dot com) + Status: Robert Nelson will implement this + + What: When restoring a fileset for a specified date (including "most + recent"), Bacula should give you exactly the files and directories + that existed at the time of the last backup prior to that date. - Why: This would be a nice project and is the most requested -feature. - Even though you may finish a client job quicker by spooling to - disk, you still have to eventually get it onto tape. If - intermediate disk buffering allows us to improve write - bandwidth to tape, it may make sense. In addition, you can - run multiple simultaneous jobs all spool to disk, then the - data can be written one job at a time to the tape at full - tape speed. This keeps the tape running smoothly and prevents - blocks from different simultaneous jobs from being intermixed - on the tape, which is very inefficient for restores. + Currently this only works if the last backup was a Full backup. + When the last backup was Incremental/Differential, files and + directories that have been renamed or deleted since the last Full + backup are not currently restored correctly. Ditto for files with + extra/fewer hard links than at the time of the last Full backup. - Notes: Need multiple spool directories. Should possibly be able - to spool by Job type, ... Possibly need high and low spool - data levels. + Why: Incremental/Differential would be much more useful if this worked. + Notes: Merging of multiple backups into a single one seems to + rely on this working, otherwise the merged backups will not be + truly equivalent to a Full backup. -Item 3: GUI for interactive restore -Item 4: GUI for interactive backup + Kern: notes shortened. This can be done without the need for + inodes. It is essentially the same as the current Verify job, + but one additional database record must be written, which does + not need any database change. - What: The current interactive restore is implemented with a tty - interface. It would be much nicer to be able to "see" the - list of files backed up in typical GUI tree format. - The same mechanism could also be used for creating - ad-hoc backup FileSets (item 8). + Kern: see if we can correct restoration of directories if + replace=ifnewer is set. Currently, if the directory does not + exist, a "dummy" directory is created, then when all the files + are updated, the dummy directory is newer so the real values + are not updated. - Why: Ease of use -- especially for the end user. +Item 2: Implement a Bacula GUI/management tool. + Origin: Kern + Date: 28 October 2005 + Status: - Notes: Rather than implementing in Gtk, we probably should go -directly - for a Browser implementation, even if doing so meant the - capability wouldn't be available until much later. Not only - is there the question of Windows sites, most - Solaris/HP/IRIX, etc, shops can't currently run Gtk programs - without installing lots of stuff admins are very wary about. - Real sysadmins will always use the command line anyway, and - the user who's doing an interactive restore or backup of his - own files will in most cases be on a Windows machine running - Exploder. + What: Implement a Bacula console, and management tools + probably using Qt3 and C++. + + Why: Don't we already have a wxWidgets GUI? Yes, but + it is written in C++ and changes to the user interface + must be hand tailored using C++ code. By developing + the user interface using Qt designer, the interface + can be very easily updated and most of the new Python + code will be automatically created. The user interface + changes become very simple, and only the new features + must be implement. In addition, the code will be in + Python, which will give many more users easy (or easier) + access to making additions or modifications. + + Notes: There is a partial Python-GTK implementation + Lucas Di Pentima but + it is no longer being developed. + + +Item 3: Implement Base jobs. + Date: 28 October 2005 + Origin: Kern + Status: + + What: A base job is sort of like a Full save except that you + will want the FileSet to contain only files that are + unlikely to change in the future (i.e. a snapshot of + most of your system after installing it). After the + base job has been run, when you are doing a Full save, + you specify one or more Base jobs to be used. All + files that have been backed up in the Base job/jobs but + not modified will then be excluded from the backup. + During a restore, the Base jobs will be automatically + pulled in where necessary. + Why: This is something none of the competition does, as far as + we know (except perhaps BackupPC, which is a Perl program that + saves to disk only). It is big win for the user, it + makes Bacula stand out as offering a unique + optimization that immediately saves time and money. + Basically, imagine that you have 100 nearly identical + Windows or Linux machine containing the OS and user + files. Now for the OS part, a Base job will be backed + up once, and rather than making 100 copies of the OS, + there will be only one. If one or more of the systems + have some files updated, no problem, they will be + automatically restored. -Item 5: Implement a Migration job type that will move the job - data from one device to another. + Notes: Huge savings in tape usage even for a single machine. + Will require more resources because the DIR must send + FD a list of files/attribs, and the FD must search the + list and compare it for each file to be saved. - What: The ability to copy, move, or archive data that is on a - device to another device is very important. +Item 4: Implement from-client and to-client on restore command line. + Date: 11 December 2006 + Origin: Discussion on Bacula-users entitled 'Scripted restores to + different clients', December 2006 + Status: New feature request + + What: While using bconsole interactively, you can specify the client + that a backup job is to be restored for, and then you can + specify later a different client to send the restored files + back to. However, using the 'restore' command with all options + on the command line, this cannot be done, due to the ambiguous + 'client' parameter. Additionally, this parameter means different + things depending on if it's specified on the command line or + afterwards, in the Modify Job screens. + + Why: This feature would enable restore jobs to be more completely + automated, for example by a web or GUI front-end. + + Notes: client can also be implied by specifying the jobid on the command + line - Why: An ISP might want to backup to disk, but after 30 days - migrate the data to tape backup and delete it from disk. - Bacula should be able to handle this automatically. It needs -to - know what was put where, and when, and what to migrate -- it - is a bit like retention periods. Doing so would allow space to - be freed up for current backups while maintaining older data -on - tape drives. +Item 5: Implement creation and maintenance of copy pools + Date: 27 November 2005 + Origin: David Boyes (dboyes at sinenomine dot net) + Status: - Notes: Migration could be triggered by: - Number of Jobs - Number of Volumes - Age of Jobs - Highwater size (keep total size) - Lowwater mark + What: I would like Bacula to have the capability to write copies + of backed-up data on multiple physical volumes selected + from different pools without transferring the data + multiple times, and to accept any of the copy volumes + as valid for restore. + Why: In many cases, businesses are required to keep offsite + copies of backup volumes, or just wish for simple + protection against a human operator dropping a storage + volume and damaging it. The ability to generate multiple + volumes in the course of a single backup job allows + customers to simple check out one copy and send it + offsite, marking it as out of changer or otherwise + unavailable. Currently, the library and magazine + management capability in Bacula does not make this process + simple. -Item 6: Embedded Perl Scripting (precursor to 7). + Restores would use the copy of the data on the first + available volume, in order of copy pool chain definition. - What: On a configuration parameter, embed the Perl language in - Bacula. + This is also a major scalability issue -- as the number of + clients increases beyond several thousand, and the volume + of data increases, transferring the data multiple times to + produce additional copies of the backups will become + physically impossible due to transfer speed + issues. Generating multiple copies at server side will + become the only practical option. - Why: The embedded Perl scripting can be called to implement - Events such as "Volume Name needed", "End of Tape", - "Tape at x% of rated capacity", "Job started", - "Job Ended", "Job error", ... + How: I suspect that this will require adding a multiplexing + SD that appears to be a SD to a specific FD, but 1-n FDs + to the specific back end SDs managing the primary and copy + pools. Storage pools will also need to acquire parameters + to define the pools to be used for copies. - Notes: This needs Events. + Notes: I would commit some of my developers' time if we can agree + on the design and behavior. +Item 6: Merge multiple backups (Synthetic Backup or Consolidation). + Origin: Marc Cousin and Eric Bollengier + Date: 15 November 2005 + Status: Waiting implementation. Depends on first implementing + project Item 2 (Migration) which is now done. -Item 7: Implement Events (requires 6). + What: A merged backup is a backup made without connecting to the Client. + It would be a Merge of existing backups into a single backup. + In effect, it is like a restore but to the backup medium. - What: When a particular user defined Event occurs, call the - embedded Perl interpreter. + For instance, say that last Sunday we made a full backup. Then + all week long, we created incremental backups, in order to do + them fast. Now comes Sunday again, and we need another full. + The merged backup makes it possible to do instead an incremental + backup (during the night for instance), and then create a merged + backup during the day, by using the full and incrementals from + the week. The merged backup will be exactly like a full made + Sunday night on the tape, but the production interruption on the + Client will be minimal, as the Client will only have to send + incrementals. - Why: This will provide the ultimate in user customization for - Bacula. Almost anything imaginable can be done if Events - are called at the appropriate place. + In fact, if it's done correctly, you could merge all the + Incrementals into single Incremental, or all the Incrementals + and the last Differential into a new Differential, or the Full, + last differential and all the Incrementals into a new Full + backup. And there is no need to involve the Client. - Notes: There is a certain amount of work to be done on how - the user defines or "registers" events. + Why: The benefit is that : + - the Client just does an incremental ; + - the merged backup on tape is just as a single full backup, + and can be restored very fast. + This is also a way of reducing the backup data since the old + data can then be pruned (or not) from the catalog, possibly + allowing older volumes to be recycled -Item 8: Multiple Storage Devices for a Single Job +Item 8: Deletion of Disk-Based Bacula Volumes + Date: Nov 25, 2005 + Origin: Ross Boylan (edited + by Kern) + Status: - What: Allow any Job to use more than one Storage device. + What: Provide a way for Bacula to automatically remove Volumes + from the filesystem, or optionally to truncate them. + Obviously, the Volume must be pruned prior removal. - Why: With two devices, for example, the second device could - have the next backup tape pre-mounted reducing operator - intervention in the middle of the night. + Why: This would allow users more control over their Volumes and + prevent disk based volumes from consuming too much space. + Notes: The following two directives might do the trick: -Item 9: Backup a Single Job Simultaneously to Multiple Storage - Devices + Volume Data Retention =