3 Bacula Projects Roadmap
6 Below, you will find more information on future projects:
8 Item 1: Implement a Migration job type that will move the job
9 data from one device to another.
10 Origin: Sponsored by Riege Sofware International GmbH. Contact:
11 Daniel Holtkamp <holtkamp at riege dot com>
13 Status: Partially coded in 1.37 -- much more to do. Assigned to
16 What: The ability to copy, move, or archive data that is on a
17 device to another device is very important.
19 Why: An ISP might want to backup to disk, but after 30 days
20 migrate the data to tape backup and delete it from
21 disk. Bacula should be able to handle this
22 automatically. It needs to know what was put where,
23 and when, and what to migrate -- it is a bit like
24 retention periods. Doing so would allow space to be
25 freed up for current backups while maintaining older
28 Notes: Migration could be triggered by:
32 Highwater size (keep total size)
35 Item 2: Implement extraction of Win32 BackupWrite data.
36 Origin: Thorsten Engel <thorsten.engel at matrix-computer dot com>
38 Status: Assigned to Thorsten. Implemented in current CVS
40 What: This provides the Bacula File daemon with code that
41 can pick apart the stream output that Microsoft writes
42 for BackupWrite data, and thus the data can be read
43 and restored on non-Win32 machines.
45 Why: BackupWrite data is the portable=no option in Win32
46 FileSets, and in previous Baculas, this data could
47 only be extracted using a Win32 FD. With this new code,
48 the Windows data can be extracted and restored on
52 Item 3: Implement a Bacula GUI/management tool using Python
59 What: Implement a Bacula console, and management tools
62 Why: Don't we already have a wxWidgets GUI? Yes, but
63 it is written in C++ and changes to the user interface
64 must be hand tailored using C++ code. By developing
65 the user interface using Qt designer, the interface
66 can be very easily updated and most of the new Python
67 code will be automatically created. The user interface
68 changes become very simple, and only the new features
69 must be implement. In addition, the code will be in
70 Python, which will give many more users easy (or easier)
71 access to making additions or modifications.
73 Item 4: Implement a Python interface to the Bacula catalog.
78 What: Implement an interface for Python scripts to access
79 the catalog through Bacula.
81 Why: This will permit users to customize Bacula through
84 Item 5: Implement more Python events in Bacula.
89 What: Allow Python scripts to be called at more places
90 within Bacula and provide additional access to Bacula
93 Why: This will permit users to customize Bacula through
101 Item 6: Implement Base jobs.
102 Date: 28 October 2005
106 What: A base job is sort of like a Full save except that you
107 will want the FileSet to contain only files that are
108 unlikely to change in the future (i.e. a snapshot of
109 most of your system after installing it). After the
110 base job has been run, when you are doing a Full save,
111 you specify one or more Base jobs to be used. All
112 files that have been backed up in the Base job/jobs but
113 not modified will then be excluded from the backup.
114 During a restore, the Base jobs will be automatically
115 pulled in where necessary.
117 Why: This is something none of the competition does, as far as
118 we know (except perhpas BackupPC, which is a Perl program that
119 saves to disk only). It is big win for the user, it
120 makes Bacula stand out as offering a unique
121 optimization that immediately saves time and money.
122 Basically, imagine that you have 100 nearly identical
123 Windows or Linux machine containing the OS and user
124 files. Now for the OS part, a Base job will be backed
125 up once, and rather than making 100 copies of the OS,
126 there will be only one. If one or more of the systems
127 have some files updated, no problem, they will be
128 automatically restored.
130 Notes: Huge savings in tape usage even for a single machine.
131 Will require more resources because the DIR must send
132 FD a list of files/attribs, and the FD must search the
133 list and compare it for each file to be saved.
135 Item 7: Add Plug-ins to the FileSet Include statements.
136 Date: 28 October 2005
138 Status: Partially coded in 1.37 -- much more to do.
140 What: Allow users to specify wild-card and/or regular
141 expressions to be matched in both the Include and
142 Exclude directives in a FileSet. At the same time,
143 allow users to define plug-ins to be called (based on
144 regular expression/wild-card matching).
146 Why: This would give the users the ultimate ability to control
147 how files are backed up/restored. A user could write a
148 plug-in knows how to backup his Oracle database without
149 stopping/starting it, for example.
151 Item 8: Implement huge exclude list support using hashing.
152 Date: 28 October 2005
156 What: Allow users to specify very large exclude list (currently
157 more than about 1000 files is too many).
159 Why: This would give the users the ability to exclude all
160 files that are loaded with the OS (e.g. using rpms
161 or debs). If the user can restore the base OS from
162 CDs, there is no need to backup all those files. A
163 complete restore would be to restore the base OS, then
164 do a Bacula restore. By excluding the base OS files, the
165 backup set will be *much* smaller.
168 Item 9: Implement data encryption (as opposed to communications
170 Date: 28 October 2005
171 Origin: Sponsored by Landon and 13 contributors to EFF.
172 Status: Landon Fuller is currently implementing this.
174 What: Currently the data that is stored on the Volume is not
175 encrypted. For confidentiality, encryption of data at
176 the File daemon level is essential.
177 Data encryption encrypts the data in the File daemon and
178 decrypts the data in the File daemon during a restore.
180 Why: Large sites require this.
182 Item 10: Permit multiple Media Types in an Autochanger
186 What: Modify the Storage daemon so that multiple Media Types
187 can be specified in an autochanger. This would be somewhat
188 of a simplistic implementation in that each drive would
189 still be allowed to have only one Media Type. However,
190 the Storage daemon will ensure that only a drive with
191 the Media Type that matches what the Director specifies
194 Why: This will permit user with several different drive types
195 to make full use of their autochangers.
197 Item 11: Allow two different autochanger definitions that refer
198 to the same autochanger.
199 Date: 28 October 2005
203 What: Currently, the autochanger script is locked based on
204 the autochanger. That is, if multiple drives are being
205 simultaneously used, the Storage daemon ensures that only
206 one drive at a time can access the mtx-changer script.
207 This change would base the locking on the control device,
208 rather than the autochanger. It would then permit two autochanger
209 definitions for the same autochanger, but with different
210 drives. Logically, the autochanger could then be "partitioned"
211 for different jobs, clients, or class of jobs, and if the locking
212 is based on the control device (e.g. /dev/sg0) the mtx-changer
213 script will be locked appropriately.
215 Why: This will permit users to partition autochangers for specific
216 use. It would also permit implementation of multiple Media
217 Types with no changes to the Storage daemon.
219 Item 12: Implement red/black binary tree routines.
220 Date: 28 October 2005
224 What: Implement a red/black binary tree class. This could
225 then replace the current binary insert/search routines
226 used in the restore in memory tree. This could significantly
227 speed up the creation of the in memory restore tree.
229 Why: Performance enhancement.
232 Item 13: Merging of multiple backups into a single one. (Also called Synthetic
233 Backup or Consolidation).
235 Origin: Marc Cousin and Eric Bollengier
236 Date: 15 November 2005
237 Status: Depends on first implementing project Item 1 (Migration).
239 What: A merged backup is a backup made without connecting to the Client.
240 It would be a Merge of existing backups into a single backup. In
241 effect, it is like a restore but to the backup medium.
243 For instance, say that last sunday we made a full backup. Then all
244 week long, we created incremental backups, in order to do them
245 fast. Now comes sunday again, and we need another full. The merged
246 backup makes it possible to do instead an incremental backup (during
247 the night for instance), and then create a merged backup during the
248 day, by using the full and incrementals from the week. The merged
249 backup will be exactly like a full made sunday night on the tape, but
250 the production interruption on the Client will be minimal, as the
251 Client will only have to send incrementals.
253 In fact, if it's done correctly, you could merge all the Incrementals
254 into single Incremental, or all the Incrementals and the last
255 Differential into a new Differential, or the Full, last differential
256 and all the Incrementals into a new Full backup. And there is no
257 need to involve the Client.
259 Why: The benefit is that :
260 - the Client just does an incremental ;
261 - the merged backup on tape is just as a single full backup,
262 and can be restored very fast.
264 This is also a way of reducing the backup data since the old data can
265 then be pruned (or not) from the catalog, possibly allowing older
266 volumes to be recycled
270 ============= Empty RFC form ===========
271 Item n: One line summary ...
273 Origin: Name and email of originator.
276 What: More detailed explanation ...
278 Why: Why it is important ...
280 Notes: Additional notes or features (omit if not used)
281 ============== End RFC form ==============
284 Items completed for release 1.38.0:
285 #4 Embedded Python Scripting (implemented in all Daemons)
286 #5 Events that call a Python program (Implemented in all
287 daemons, but more cleanup work to be done).
288 #6 Select one from among Multiple Storage Devices for Job.
289 This is already implemented in 1.37.
290 #7 Single Job Writing to Multiple Storage Devices. This is
291 currently implemented with a Clone feature.
292 #- Full multiple drive Autochanger support (done in 1.37)
293 #- Built in support for communications encryption (TLS)
294 done by Landon Fuller.
295 # Support for Unicode characters
296 (via UTF-8) on Win32 machines thanks to Thorsten Engel.
297 Item 8: Break the one-to-one Relationship between a Job and a
298 Specific Storage Device (or Devices if #10 is implemented).
300 Completed items from last year's list:
301 Item 1: Multiple simultaneous Jobs. (done)
302 Item 3: Write the bscan program -- also write a bcopy program (done).
303 Item 5: Implement Label templates (done).
304 Item 6: Write a regression script (done)
305 Item 9: Add SSL to daemon communications (done by Landon Fuller)
306 Item 10: Define definitive tape format (done)
307 Item 3: GUI for interactive restore. Partially Implemented in 1.34
308 Note, there is now a complete Webmin plugin, a partial
309 GNOME console, and an excellent wx-console GUI.
310 Item 4: GUI for interactive backup
311 Item 2: Job Data Spooling.
312 Done: Regular expression matching.
313 Item 10: New daemon communication protocol (this has been dropped).