X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=bacula%2Fkernstodo;h=3218418e8399deebdd25771fadc5002e8b18fe11;hb=09343cb4421dbd76b82b9b289016f06fe6e620cf;hp=ab8881e22b7c4041148e03d22f7512a2d990f9fd;hpb=444d14e5e29b052b8710a318bcae9573692b632c;p=bacula%2Fbacula diff --git a/bacula/kernstodo b/bacula/kernstodo index ab8881e22b..3218418e83 100644 --- a/bacula/kernstodo +++ b/bacula/kernstodo @@ -1,99 +1,131 @@ Kern's ToDo List - 12 June 2005 + 13 August 2005 Major development: Project Developer ======= ========= -TLS Landon Fuller -Unicode in Win32 Thorsten Engel (done) -VSS Thorsten Engel (in beta testing) Version 1.37 Kern (see below) ======================================================== -1.37 Major Projects: -#3 Migration (Move, Copy, Archive Jobs) - (probably not this version) -#7 Single Job Writing to Multiple Storage Devices - (probably not this version) +Final items for 1.37 before release: +1. Fix bugs +- Look at fixing restore status stats in SD. +- Check "update slots=7 scan storage=DLT drive=0" with + non-bacula tape in the drive. -## Integrate web-bacula into a new Bacula project with - bimagemgr. -## Create a new GUI chapter explaining all the GUI programs. +- --without-openssl breaks at least on Solaris. +- Figure out how to package gui, and rescue programs. +- Test TLS. +- Arno had to do -- to get update slots=x to work + UPDATE Media SET InChanger=0,Slot=0 WHERE InChanger>0 AND Slot>0; (MySQL) -Autochangers: -- 7. Implement new Console commands to allow offlining/reserving drives, - and possibly manipulating the autochanger (much asked for). -- Make "update slots" when pointing to Autochanger, remove - all Volumes from other drives. "update slots all-drives"? +- Add recycle event. +- Add scratch pool event. +- Implement NeedVolume event +- Add Win32 FileSet definition somewhere + Document: -- Document that Bootstrap files can be written with cataloging - turned off. -- Pruning with Admin job. -- Add better documentation on how restores can be done -- OS linux 2.4 - 1) ADIC, DLT, FastStor 4000, 7*20GB - 2) Sun, DDS, (Suns name unknown - Archive Python DDS drive), 1.2GB - 3) Wangtek, QIC, 6525ES, 525MB (fixed block size 1k, block size etc. - driver dependent - aic7xxx works, ncr53c8xx with problems) - 4) HP, DDS-2, C1553A, 6*4GB -- Doc the following - to activate, check or disable the hardware compression feature on my - exb-8900 i use the exabyte "MammothTool" you can get it here: - http://www.exabyte.com/support/online/downloads/index.cfm - There is a solaris version of this tool. With option -C 0 or 1 you can - disable or activate compression. Start this tool without any options for - a small reference. -- Linux Sony LIB-D81, AIT-3 library works. -- Document PostgreSQL performance problems bug 131. -- Document testing -- Document that ChangerDevice is used for Alert command. -- Document new CDROM directory. -- Document Heartbeat Interval in the dealing with firewalls section. +- datadir for po files. +- AM_GNU_GETTEXT finds the library if you specify + --with-libintl-prefix +- Document cleaning up the spool files: + db, pid, state, bsr, mail, conmsg, spool - Document the multiple-drive-changer.txt script. - -For 1.37: -- Implement "PreferMountedVolumes = yes|no" in Job resource. -=== rate design - jcr->last_rate - jcr->last_runtime - MA = (last_MA * 3 + rate) / 4 - rate = (bytes - last_bytes) / (runtime - last_runtime) -- Despool attributes simultaneously with data in a separate - thread, rejoined at end of data spooling. -- Implement Files/Bytes,... stats for restore job. -- Implement Total Bytes Written, ... for restore job. -- Add setting Volume State via Python. -- Max Vols limit in Pool off by one? -- Make bootstrap file handle multiple MediaTypes (SD) -- Test restoring into a user restricted directory on Win32 -- see - bug report. -- --without-openssl breaks at least on Solaris. +- Pruning with Admin job. +- Restore of all files for a Job or set of jobs even if the file + records have been removed from the catalog. +- Does WildFile match against full name? Doc. + +For 1.39: +- Look at simplifying File exclusions. +- Fix store_yesno to be store_bitmask. +- New directive "Delete purged Volumes" +- new pool XXX with ScratchPoolId = MyScratchPool's PoolId and + let it fill itself, and RecyclePoolId = XXX's PoolId so I can + see if it become stable and I just have to supervise + MyScratchPool +- If I want to remove this pool, I set RecyclePoolId = MyScratchPool's + PoolId, and when it is empty remove it. +- Figure out how to recycle Scratch volumes back to the Scratch + Pool. +- Add Volume=SCRTCH +- Allow Check Labels to be used with Bacula labels. +- "Resuming" a failed backup (lost line for example) by using the + failed backup as a sort of "base" job. +- Look at NDMP +- Email to the user when the tape is about to need changing x + days before it needs changing. +- Command to show next tape that will be used for a job even + if the job is not scheduled. +--- create_file.c.orig Fri Jul 8 12:13:05 2005 ++++ create_file.c Fri Jul 8 12:13:07 2005 +@@ -195,6 +195,8 @@ + attr->ofname, be.strerror()); + return CF_ERROR; + } ++ } else if(S_ISSOCK(attr->statp.st_mode)) { ++ Dmsg1(200, "Skipping socket: %s\n", attr->ofname); + } else { + Dmsg1(200, "Restore node: %s\n", attr->ofname); + if (mknod(attr->ofname, attr->statp.st_mode, attr->statp.st_rdev) != 0 && errno != EEXIST) { + +- Fix bpipe.c so that it does not modify results pointer. + ***FIXME*** calling sequence should be changed. +1.xx Major Projects: +#3 Migration (Move, Copy, Archive Jobs) +#7 Single Job Writing to Multiple Storage Devices +- Reserve blocks other restore jobs when first cannot connect + to SD. +- Add true/false to conf same as yes/no +- For Windows disaster recovery see http://unattended.sf.net/ +- regardless of the retention period, Bacula will not prune the + last Full, Diff, or Inc File data until a month after the + retention period for the last Full backup that was done. +- update volume=xxx --- add status=Full +- Remove old spool files on startup. +- Exclude SD spool/working directory. +- Refuse to prune last valid Full backup. Same goes for Catalog. - Python: - Make a callback when Rerun failed levels is called. - Give Python program access to Scheduled jobs. + - Add setting Volume State via Python. - Python script to save with Python, not save, save with Bacula. - Python script to do backup. - What events? - Change the Priority, Client, Storage, JobStatus (error) at the start of a job. - - Make sure that Python has access to Client address/port so that - it can check if Clients are alive. - +- Why is SpoolDirectory = /home/bacula/spool; not reported + as an error when writing a DVD? +- Make bootstrap file handle multiple MediaTypes (SD) - Remove all old Device resource code in Dir and code to pass it back in SD -- better, rework it to pass back device statistics. - Check locking of resources -- be sure to lock devices where previously resources were locked. -- Add global lock on all devices when creating a device structure. +- The last part is left in the spool dir. -Maybe in 1.37: -- Add start/end date editing in messages (%t %T, %e?) ... -- Add ClientDefs similar to JobDefs. -- Print more info when bextract -p accepts a bad block. + +- In restore don't compare byte count on a raw device -- directory + entry does not contain bytes. - To mark files as deleted, run essentially a Verify to disk, and when a file is found missing (MarkId != JobId), then create a new File record with FileIndex == -1. This could be done by the FD at the same time as the backup. +=== rate design + jcr->last_rate + jcr->last_runtime + MA = (last_MA * 3 + rate) / 4 + rate = (bytes - last_bytes) / (runtime - last_runtime) +- Max Vols limit in Pool off by one? +- Implement Files/Bytes,... stats for restore job. +- Implement Total Bytes Written, ... for restore job. +- Despool attributes simultaneously with data in a separate + thread, rejoined at end of data spooling. +- 7. Implement new Console commands to allow offlining/reserving drives, + and possibly manipulating the autochanger (much asked for). +- Add start/end date editing in messages (%t %T, %e?) ... +- Add ClientDefs similar to JobDefs. +- Print more info when bextract -p accepts a bad block. - Fix FD JobType to be set before RunBeforeJob in FD. - Look at adding full Volume and Pool information to a Volume label so that bscan can get *all* the info. @@ -127,6 +159,61 @@ Maybe in 1.37: - Bug: if a job is manually scheduled to run later, it does not appear in any status report and cannot be cancelled. +==== Keeping track of deleted files ==== + My "trick" for keeping track of deletions is the following. + Assuming the user turns on this option, after all the files + have been backed up, but before the job has terminated, the + FD will make a pass through all the files and send their + names to the DIR (*exactly* the same as what a Verify job + currently does). This will probably be done at the same + time the files are being sent to the SD avoiding a second + pass. The DIR will then compare that to what is stored in + the catalog. Any files in the catalog but not in what the + FD sent will receive a catalog File entry that indicates + that at that point in time the file was deleted. + + During a restore, any file initially picked up by some + backup (Full, ...) then subsequently having a File entry + marked "delete" will be removed from the tree, so will not + be restored. If a file with the same name is later OK it + will be inserted in the tree -- this already happens. All + will be consistent except for possible changes during the + running of the FD. + + Since I'm on the subject, some of you may be wondering what + the utility of the in memory tree is if you are going to + restore everything (at least it comes up from time to time + on the list). Well, it is still *very* useful because it + allows only the last item found for a particular filename + (full path) to be entered into the tree, and thus if a file + is backed up 10 times, only the last copy will be restored. + I recently (last Friday) restored a complete directory, and + the Full and all the Differential and Incremental backups + spanned 3 Volumes. The first Volume was not even mounted + because all the files had been updated and hence backed up + since the Full backup was made. In this case, the tree + saved me a *lot* of time. + + Make sure this information is stored on the tape too so + that it can be restored directly from the tape. + + Comments from Martin Simmons (I think they are all covered): + Ok, that should cover the basics. There are few issues though: + + - Restore will depend on the catalog. I think it is better to include the + extra data in the backup as well, so it can be seen by bscan and bextract. + + - I'm not sure if it will preserve multiple hard links to the same inode. Or + maybe adding or removing links will cause the data to be dumped again? + + - I'm not sure if it will handle renamed directories. Possibly it will work + by dumping the whole tree under a renamed directory? + + - It remains to be seen how the backup performance of the DIR's will be + affected when comparing the catalog for a large filesystem. + +===== + Regression tests: - Add Pool/Storage override regression test. - Add delete JobId to regression. @@ -1287,3 +1374,71 @@ Block Position: 0 in an Autochanger. - Upgrade to MySQL 4.1.12 See: http://dev.mysql.com/doc/mysql/en/Server_SQL_mode.html +- Add # Job Level date to bsr file +- Implement "PreferMountedVolumes = yes|no" in Job resource. +## Integrate web-bacula into a new Bacula project with + bimagemgr. +- Cleaning tapes should have Status "Cleaning" rather than append. +- Make sure that Python has access to Client address/port so that + it can check if Clients are alive. +- Review all items in "restore". +- Fix PostgreSQL GROUP BY problems in restore. +- Fix PostgreSQL sql problems in bugs. +- After rename + 04-Jul 13:01 MainSD: Rufus.2005-07-04_01.05.02 Warning: Director wanted Volume + "DLT-13Feb04". + Current Volume "DLT-04Jul05" not acceptable because: + 1997 Volume "DLT-13Feb04" not in catalog. + 04-Jul 13:01 MainSD: Please mount Volume "DLT-04Jul05" on Storage Device + "HP DLT 80" (/dev/nst0) for Job Rufus.2005-07-04_01.05.02 +## Create a new GUI chapter explaining all the GUI programs. +- Make "update slots" when pointing to Autochanger, remove + all Volumes from other drives. "update slots all-drives"? + No, this is done by modifying mtx-changer to list what is + in the drives. +- Finish TLS implementation. +- Port limiting -m in iptables to prevent DoS attacks + could cause broken pipes on Bacula. +6. Build and test the Volume Shadow Copy (VSS) for Win32. +- Allow cancel of unknown Job +- State not saved when closing Win32 FD by icon +- bsr-opt-test fails. bsr deleted. Fix. +- Move Python daemon variables from Job to Bacula object. + WorkingDir, ConfigFile +- Document that Bootstrap files can be written with cataloging + turned off. +- Document details of ANSI/IBM labels +- OS linux 2.4 + 1) ADIC, DLT, FastStor 4000, 7*20GB +- Linux Sony LIB-D81, AIT-3 library works. +- Doc the following + to activate, check or disable the hardware compression feature on my + exb-8900 i use the exabyte "MammothTool" you can get it here: + http://www.exabyte.com/support/online/downloads/index.cfm + There is a solaris version of this tool. With option -C 0 or 1 you can + disable or activate compression. Start this tool without any options for + a small reference. +- Document Heartbeat Interval in the dealing with firewalls section. +- Document new CDROM directory. +- On Win32 working directory must have drive letter ???? +- On Win32 working directory must be writable by SYSTEM to + do restores. +- Document that ChangerDevice is used for Alert command. +- Add better documentation on how restores can be done +8. Take one more try at making DVD writing work (no go) +7. Write a bacula-web document +- Why isn't the DEVICE structure defined when doing + a reservation? +- Multi-drive changer seems to only use drive 0 + Multiple drives don't seem to be opened. +- My database is growing +- Call GetLastError() in the berrno constructor rather + than delaying until strerror. +- Tape xxx in drive 0, requested in drive 1 +- The mount command does not work with drives other than 0. +- A mount should cause the SD to re-examine what Slot is + loaded. +- The SD locks on to the first available drive then + wants a Volume that is released but in another drive -- + chaos. +- Run the regression scripts on Solaris and FreeBSD