X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=bacula%2Fkernstodo;h=0da65c17df58c13ab8e59dc9d210fcba7914b531;hb=90e04dd4f4c1bcad214b4ac5dd4ffbc55bbe0763;hp=0c3e7b5dedf4b574e9140e4d1869858443942433;hpb=72ddc7a22f11f9841cc61f6f7c2d0e84602ff72e;p=bacula%2Fbacula diff --git a/bacula/kernstodo b/bacula/kernstodo index 0c3e7b5ded..0da65c17df 100644 --- a/bacula/kernstodo +++ b/bacula/kernstodo @@ -1,5 +1,5 @@ Kern's ToDo List - 22 February 2006 + 12 November 2006 Major development: Project Developer @@ -19,18 +19,150 @@ Document: workaround: specify absolute working and pid directory in bacula-fd.conf (e.g. c:\bacula\working instead of \bacula\working). +- Document techniques for restoring large numbers of files. +- Document setting my.cnf to big file usage. +- Add example of proper index output to doc. show index from File; +- Correct the Include syntax in the m4.xxx files in examples/conf +- Document JobStatus and Termination codes. +- Fix the error with the "DVI file can't be opened" while + building the French PDF. +- Document more DVD stuff +- Doc + { "JobErrors", "i"}, + { "JobFiles", "i"}, + { "SDJobFiles", "i"}, + { "SDErrors", "i"}, + { "FDJobStatus","s"}, + { "SDJobStatus","s"}, +- Document all the little details of setting up certificates for + the Bacula data encryption code. +- Document more precisely how to use master keys -- especially + for disaster recovery. + Priority: +- Look at moving the Storage directive from the Job to the + Pool in the default conf files. +- Make sure the new level=Full syntax is used in all + example conf files (especially in the manual). +- Fix prog copyright (SD) all other files. +- Migration Volume span bug +- Rescue release +- Bug reports +- Test FIFO backup/restore -- make regression +- Doc items +- Add encryption regression tests +- Test Volume compatibility between machine architectures +- Encryption documentation +- Wrong jobbytes with query 12 (todo) +- bacula-1.38.2-ssl.patch +- Bare-metal recovery Windows (todo) +- Document need for UTF-8 format + + For 1.39: -- Fix re-read of last block to check if job has actually written - a block, and check if block was written by a different job - (i.e. multiple simultaneous jobs writing). -- JobStatus and Termination codes. +- Implement Python event for backing up/restoring a file. +- Change dbcheck to tell users to use native tools for fixing + broken databases, and to ensure they have the proper indexes. +- add udev rules for Bacula devices. +- If a job terminates, the DIR connection can close before the + Volume info is updated, leaving the File count wrong. +- Look at why SIGPIPE during connection can cause seg fault in + writing the daemon message, when Dir dropped to bacula:bacula +- Look at zlib 32 => 64 problems. +- Try turning on disk seek code. +- Possibly turn on St. Bernard code. +- Fix bextract to restore ACLs, or better yet, use common routines. +- Do we migrate appendable Volumes? +- Remove queue.c code. - Some users claim that they must do two prune commands to get a Volume marked as purged. - Print warning message if LANG environment variable does not specify UTF-8. +- New dot commands from Arno. + .show device=xxx lists information from one storage device, including + devices (I'm not even sure that information exists in the DIR...) + .move eject device=xxx mostly the same as 'unmount xxx' but perhaps with + better machine-readable output like "Ok" or "Error busy" + .move eject device=xxx toslot=yyy the same as above, but with a new + target slot. The catalog should be updated accordingly. + .move transfer device=xxx fromslot=yyy toslot=zzz + +Low priority: +- Check to see if jcr->stime is lost during rescheduling of + jobs in jobq.c +- Fix re-read of last block to check if job has actually written + a block, and check if block was written by a different job + (i.e. multiple simultaneous jobs writing). +- Figure out how to configure query.sql. Suggestion to use m4: + == changequote.m4 === + changequote(`[',`]')dnl + ==== query.sql.in === + :List next 20 volumes to expire + SELECT + Pool.Name AS PoolName, + Media.VolumeName, + Media.VolStatus, + Media.MediaType, + ifdef([MySQL], + [ FROM_UNIXTIME(UNIX_TIMESTAMP(Media.LastWritten) Media.VolRetention) AS Expire, ])dnl + ifdef([PostgreSQL], + [ media.lastwritten + interval '1 second' * media.volretention as expire, ])dnl + Media.LastWritten + FROM Pool + LEFT JOIN Media + ON Media.PoolId=Pool.PoolId + WHERE Media.LastWritten>0 + ORDER BY Expire + LIMIT 20; + ==== + Command: m4 -DmySQL changequote.m4 query.sql.in >query.sql + + The problem is that it requires m4, which is not present on all machines + at ./configure time. +- Get Perl replacement for bregex.c +- Given all the problems with FIFOs, I think the solution is to do something a + little different, though I will look at the code and see if there is not some + simple solution (i.e. some bug that was introduced). What might be a better + solution would be to use a FIFO as a sort of "key" to tell Bacula to read and + write data to a program rather than the FIFO. For example, suppose you + create a FIFO named: + + /home/kern/my-fifo + + Then, I could imagine if you backup and restore this file with a direct + reference as is currently done for fifos, instead, during backup Bacula will + execute: + + /home/kern/my-fifo.backup + + and read the data that my-fifo.backup writes to stdout. For restore, Bacula + will execute: + + /home/kern/my-fifo.restore + + and send the data backed up to stdout. These programs can either be an + executable or a shell script and they need only read/write to stdin/stdout. + + I think this would give a lot of flexibility to the user without making any + significant changes to Bacula. + + +==== SQL +# get null file +select FilenameId from Filename where Name=''; +# Get list of all directories referenced in a Backup. +select Path.Path from Path,File where File.JobId=nnn and + File.FilenameId=(FilenameId-from-above) and File.PathId=Path.PathId + order by Path.Path ASC; + +- Look into using Dart for testing + http://public.kitware.com/Dart/HTML/Index.shtml + +- Look into replacing autotools with cmake + http://www.cmake.org/HTML/Index.html + === Migration from David === What I'd like to see: @@ -205,6 +337,9 @@ minutes). 3905 Device "LTO-Drive1" (/dev/nst0) open but no Bacula volume is mounted. If this is not a blank tape, try unmounting and remounting the Volume. +- Add VolumeState (enable, disable, archive) +- Add VolumeLock to prevent all but lock holder (SD) from updating + the Volume data (with the exception of VolumeState). - The btape fill command does not seem to use the Autochanger - Make Windows installer default to system disk drive. - Look at using ioctl(FIOBMAP, ...) on Linux, and @@ -324,10 +459,6 @@ minutes). - In restore don't compare byte count on a raw device -- directory entry does not contain bytes. -- To mark files as deleted, run essentially a Verify to disk, and - when a file is found missing (MarkId != JobId), then create - a new File record with FileIndex == -1. This could be done - by the FD at the same time as the backup. === rate design jcr->last_rate jcr->last_runtime @@ -376,7 +507,12 @@ minutes). - Bug: if a job is manually scheduled to run later, it does not appear in any status report and cannot be cancelled. -==== Keeping track of deleted files ==== +==== Keeping track of deleted/new files ==== +- To mark files as deleted, run essentially a Verify to disk, and + when a file is found missing (MarkId != JobId), then create + a new File record with FileIndex == -1. This could be done + by the FD at the same time as the backup. + My "trick" for keeping track of deletions is the following. Assuming the user turns on this option, after all the files have been backed up, but before the job has terminated, the @@ -387,7 +523,14 @@ minutes). pass. The DIR will then compare that to what is stored in the catalog. Any files in the catalog but not in what the FD sent will receive a catalog File entry that indicates - that at that point in time the file was deleted. + that at that point in time the file was deleted. This + either transmitted to the FD or simultaneously computed in + the FD, so that the FD can put a record on the tape that + indicates that the file has been deleted at this point. + A delete file entry could potentially be one with a FileIndex + of 0 or perhaps -1 (need to check if FileIndex is used for + some other thing as many of the Bacula fields are "overloaded" + in the SD). During a restore, any file initially picked up by some backup (Full, ...) then subsequently having a File entry @@ -414,6 +557,12 @@ minutes). Make sure this information is stored on the tape too so that it can be restored directly from the tape. + All the code (with the exception of formally generating and + saving the delete file entries) already exists in the Verify + Catalog command. It explicitly recognizes added/deleted files since + the last InitCatalog. It is more or less a "simple" matter of + taking that code and adapting it slightly to work for backups. + Comments from Martin Simmons (I think they are all covered): Ok, that should cover the basics. There are few issues though: @@ -1535,3 +1684,52 @@ Block Position: 0 accept time qualifiers. - Does ClientRunAfterJob fail the job on a bad return code? - Make hardlink code at line 240 of find_one.c use binary search. +- Add ACL error messages in src/filed/acl.c. +- Make authentication failures single threaded. +- Make Dir and SD authentication errors single threaded. +- Fix catreq.c digestbuf at line 411 in src/dird/catreq.c +- Make base64.c (bin_to_base64) take a buffer length + argument to avoid overruns. + and verify that other buffers cannot overrun. +- Implement VolumeState as discussed with Arno. +- Add LocationId to update volume +- Add LocationLog + LogId + Date + User text + MediaId + LocationId + NewState??? +- Add Comment to Media record +- Fix auth compatibility with 1.38 +- Update dbcheck to include Log table +- Update llist to include new fields. +- Make unmount unload autochanger. Make mount load slot. +- Fix bscan to report the JobType when restoring a job. +- Fix wx-console scanning problem with commas in names. +- Add manpages to the list of directories for make install. Notify + Scott +- Add bconsole option to use stdin/out instead of conio. +- Fix ClientRunBefore/AfterJob compatibility. +- Ensure that connection to daemon failure always indicates what + daemon it was trying to connect to. +- Freespace on DVD requested over and over even with no intervening + writes. +- .update volume [enabled|disabled|*see below] + > However, I could easily imagine an option to "update slots" that says + > "enable=yes|no" that would automatically enable or disable all the Volumes + > found in the autochanger. This will permit the user to optionally mark all + > the Volumes in the magazine disabled prior to taking them offsite, and mark + > them all enabled when bringing them back on site. Coupled with the options + > to the slots keyword, you can apply the enable/disable to any or all volumes. +- Restricted consoles start in the Default catalog even if it + is not permitted. +- When reading through parts on the DVD, the DVD is mounted and + unmounted for each part. +- Make sure that the restore options don't permit "seeing" other + Client's job data. +- Restore of a raw drive should not try to check the volume size. +- Lock tape drive door when open() +- Make release unload any autochanger. +- Arno's reservation deadlock. +- Eric's SD patch