Updates for debugging disk seek

[bacula/bacula] / bacula / kernstodo

diff --git a/bacula/kernstodo b/bacula/kernstodo
index 948d44122d8f0cbb11011ef6701940ebe7d52691..0da65c17df58c13ab8e59dc9d210fcba7914b531 100644 (file)
--- a/bacula/kernstodo
+++ b/bacula/kernstodo
@@ -1,5 +1,5 @@
                     Kern's ToDo List
                     Kern's ToDo List

-                     22 August 2006
+                     12 November 2006

 
 Major development:      
 Project                     Developer
 
 Major development:      
 Project                     Developer


@@ -21,59 +21,66 @@ Document:
   \bacula\working).
 - Document techniques for restoring large numbers of files.
 - Document setting my.cnf to big file usage.
   \bacula\working).
 - Document techniques for restoring large numbers of files.
 - Document setting my.cnf to big file usage.

-- Add example of proper index output to doc.
-   show index from File;
+- Add example of proper index output to doc. show index from File;

 - Correct the Include syntax in the m4.xxx files in examples/conf
 - Document JobStatus and Termination codes.
 - Fix the error with the "DVI file can't be opened" while
   building the French PDF.
 - Correct the Include syntax in the m4.xxx files in examples/conf
 - Document JobStatus and Termination codes.
 - Fix the error with the "DVI file can't be opened" while
   building the French PDF.

-- Document more DVD stuff -- particularly that recycling doesn't work,
-  and all the other things too.
+- Document more DVD stuff
+- Doc
+   { "JobErrors",  "i"},
+   { "JobFiles",   "i"},
+   { "SDJobFiles", "i"},
+   { "SDErrors",   "i"},
+   { "FDJobStatus","s"},
+   { "SDJobStatus","s"},
+- Document all the little details of setting up certificates for
+  the Bacula data encryption code.
+- Document more precisely how to use master keys -- especially
+  for disaster recovery.
+ 

 
 Priority:
 
 Priority:

+- Look at moving the Storage directive from the Job to the
+  Pool in the default conf files.
+- Make sure the new level=Full syntax is used in all 
+  example conf files (especially in the manual).
+- Fix prog copyright (SD) all other files.
+- Migration Volume span bug
+- Rescue release
+- Bug reports
+- Test FIFO backup/restore -- make regression
+- Doc items
+- Add encryption regression tests
+- Test Volume compatibility between machine architectures
+- Encryption documentation
+- Wrong jobbytes with query 12 (todo)
+- bacula-1.38.2-ssl.patch
+- Bare-metal recovery Windows (todo)
+- Document need for UTF-8 format
+
+

 
 For 1.39:
 
 For 1.39:

-- Restricted consoles start in the Default catalog even if it
-  is not permitted.
-- Make sure that the restore options don't permit "seeing" other
-  Client's job data.

 - Implement Python event for backing up/restoring a file.
 - Implement Python event for backing up/restoring a file.

-- Fix wx-console scanning problem with commas in names.

 - Change dbcheck to tell users to use native tools for fixing
   broken databases, and to ensure they have the proper indexes.
 - add udev rules for Bacula devices.
 - Change dbcheck to tell users to use native tools for fixing
   broken databases, and to ensure they have the proper indexes.
 - add udev rules for Bacula devices.

-- Add manpages to the list of directories for make install. Notify
-  Scott

 - If a job terminates, the DIR connection can close before the
   Volume info is updated, leaving the File count wrong.
 - Look at why SIGPIPE during connection can cause seg fault in
   writing the daemon message, when Dir dropped to bacula:bacula
 - Look at zlib 32 => 64 problems.
 - If a job terminates, the DIR connection can close before the
   Volume info is updated, leaving the File count wrong.
 - Look at why SIGPIPE during connection can cause seg fault in
   writing the daemon message, when Dir dropped to bacula:bacula
 - Look at zlib 32 => 64 problems.

-- Ensure that connection to daemon failure always indicates what
-  daemon it was trying to connect to.

 - Try turning on disk seek code.
 - Possibly turn on St. Bernard code.
 - Try turning on disk seek code.
 - Possibly turn on St. Bernard code.

-- Fix bextract to restore ACLs, or better yet, use common
-  routines.
+- Fix bextract to restore ACLs, or better yet, use common routines.

 - Do we migrate appendable Volumes?
 - Remove queue.c code.
 - Do we migrate appendable Volumes?
 - Remove queue.c code.

-- Add bconsole option to use stdin/out instead of conio.
-- Fix ClientRunBefore/AfterJob compatibility.
-- Fix re-read of last block to check if job has actually written
-  a block, and check if block was written by a different job
-  (i.e. multiple simultaneous jobs writing).

 - Some users claim that they must do two prune commands to get a
   Volume marked as purged.
 - Print warning message if LANG environment variable does not specify
   UTF-8.
 - New dot commands from Arno.
 - Some users claim that they must do two prune commands to get a
   Volume marked as purged.
 - Print warning message if LANG environment variable does not specify
   UTF-8.
 - New dot commands from Arno.

-  .update volume [enabled|disabled|*see below]
-    > However, I could easily imagine an option to "update slots" that says 
-    > "enable=yes|no" that would automatically enable or disable all the Volumes 
-    > found in the autochanger.  This will permit the user to optionally mark all 
-    > the Volumes in the magazine disabled prior to taking them offsite, and mark 
-    > them all enabled when bringing them back on site.   Coupled with the options 
-    > to the slots keyword, you can apply the enable/disable to any or all volumes.

   .show device=xxx lists information from one storage device, including 
      devices (I'm not even sure that information exists in the DIR...)
   .move eject device=xxx mostly the same as 'unmount xxx' but perhaps with 
   .show device=xxx lists information from one storage device, including 
      devices (I'm not even sure that information exists in the DIR...)
   .move eject device=xxx mostly the same as 'unmount xxx' but perhaps with 


@@ -83,6 +90,37 @@ For 1.39:
   .move transfer device=xxx fromslot=yyy toslot=zzz
 
 Low priority:
   .move transfer device=xxx fromslot=yyy toslot=zzz
 
 Low priority:

+- Check to see if jcr->stime is lost during rescheduling of
+  jobs in jobq.c
+- Fix re-read of last block to check if job has actually written
+  a block, and check if block was written by a different job
+  (i.e. multiple simultaneous jobs writing).
+- Figure out how to configure query.sql.  Suggestion to use m4:
+    == changequote.m4 ===
+    changequote(`[',`]')dnl
+    ==== query.sql.in ===
+    :List next 20 volumes to expire
+    SELECT
+        Pool.Name AS PoolName,
+        Media.VolumeName,
+        Media.VolStatus,
+        Media.MediaType,
+    ifdef([MySQL],
+    [ FROM_UNIXTIME(UNIX_TIMESTAMP(Media.LastWritten) Media.VolRetention) AS Expire, ])dnl
+    ifdef([PostgreSQL],
+    [ media.lastwritten + interval '1 second' * media.volretention as expire, ])dnl
+      Media.LastWritten
+      FROM Pool
+      LEFT JOIN Media
+      ON Media.PoolId=Pool.PoolId
+      WHERE Media.LastWritten>0
+      ORDER BY Expire
+      LIMIT 20;
+    ====
+    Command: m4 -DmySQL changequote.m4 query.sql.in >query.sql
+
+  The problem is that it requires m4, which is not present on all machines
+  at ./configure time.

 - Get Perl replacement for bregex.c
 - Given all the problems with FIFOs, I think the solution is to do something a
   little different, though I will look at the code and see if there is not some
 - Get Perl replacement for bregex.c
 - Given all the problems with FIFOs, I think the solution is to do something a
   little different, though I will look at the code and see if there is not some


@@ -421,10 +459,6 @@ minutes).
 
 - In restore don't compare byte count on a raw device -- directory
   entry does not contain bytes.
 
 - In restore don't compare byte count on a raw device -- directory
   entry does not contain bytes.

-- To mark files as deleted, run essentially a Verify to disk, and
-  when a file is found missing (MarkId != JobId), then create
-  a new File record with FileIndex == -1. This could be done
-  by the FD at the same time as the backup.

 === rate design
   jcr->last_rate
   jcr->last_runtime
 === rate design
   jcr->last_rate
   jcr->last_runtime


@@ -473,7 +507,12 @@ minutes).
 - Bug: if a job is manually scheduled to run later, it does not appear
   in any status report and cannot be cancelled.
 
 - Bug: if a job is manually scheduled to run later, it does not appear
   in any status report and cannot be cancelled.
 

-==== Keeping track of deleted files ====
+==== Keeping track of deleted/new files ====
+- To mark files as deleted, run essentially a Verify to disk, and
+  when a file is found missing (MarkId != JobId), then create
+  a new File record with FileIndex == -1. This could be done
+  by the FD at the same time as the backup.
+

      My "trick" for keeping track of deletions is the following.
      Assuming the user turns on this option, after all the files
      have been backed up, but before the job has terminated, the
      My "trick" for keeping track of deletions is the following.
      Assuming the user turns on this option, after all the files
      have been backed up, but before the job has terminated, the


@@ -484,7 +523,14 @@ minutes).
      pass.  The DIR will then compare that to what is stored in
      the catalog.  Any files in the catalog but not in what the
      FD sent will receive a catalog File entry that indicates
      pass.  The DIR will then compare that to what is stored in
      the catalog.  Any files in the catalog but not in what the
      FD sent will receive a catalog File entry that indicates

-     that at that point in time the file was deleted.
+     that at that point in time the file was deleted. This
+     either transmitted to the FD or simultaneously computed in
+     the FD, so that the FD can put a record on the tape that
+     indicates that the file has been deleted at this point.
+     A delete file entry could potentially be one with a FileIndex
+     of 0 or perhaps -1 (need to check if FileIndex is used for
+     some other thing as many of the Bacula fields are "overloaded"
+     in the SD).

 
      During a restore, any file initially picked up by some
      backup (Full, ...) then subsequently having a File entry
 
      During a restore, any file initially picked up by some
      backup (Full, ...) then subsequently having a File entry


@@ -511,6 +557,12 @@ minutes).
      Make sure this information is stored on the tape too so
      that it can be restored directly from the tape.
 
      Make sure this information is stored on the tape too so
      that it can be restored directly from the tape.
 

+     All the code (with the exception of formally generating and
+     saving the delete file entries) already exists in the Verify
+     Catalog command.  It explicitly recognizes added/deleted files since
+     the last InitCatalog.  It is more or less a "simple" matter of
+     taking that code and adapting it slightly to work for backups.
+

   Comments from Martin Simmons (I think they are all covered):
   Ok, that should cover the basics.  There are few issues though:
 
   Comments from Martin Simmons (I think they are all covered):
   Ok, that should cover the basics.  There are few issues though:
 


@@ -1654,3 +1706,30 @@ Block Position: 0
 - Update llist to include new fields.
 - Make unmount unload autochanger.  Make mount load slot.
 - Fix bscan to report the JobType when restoring a job.
 - Update llist to include new fields.
 - Make unmount unload autochanger.  Make mount load slot.
 - Fix bscan to report the JobType when restoring a job.

+- Fix wx-console scanning problem with commas in names.
+- Add manpages to the list of directories for make install. Notify
+  Scott
+- Add bconsole option to use stdin/out instead of conio.
+- Fix ClientRunBefore/AfterJob compatibility.
+- Ensure that connection to daemon failure always indicates what
+  daemon it was trying to connect to.
+- Freespace on DVD requested over and over even with no intervening
+  writes.
+- .update volume [enabled|disabled|*see below]
+    > However, I could easily imagine an option to "update slots" that says 
+    > "enable=yes|no" that would automatically enable or disable all the Volumes 
+    > found in the autochanger.  This will permit the user to optionally mark all 
+    > the Volumes in the magazine disabled prior to taking them offsite, and mark 
+    > them all enabled when bringing them back on site.   Coupled with the options 
+    > to the slots keyword, you can apply the enable/disable to any or all volumes.
+- Restricted consoles start in the Default catalog even if it
+  is not permitted.
+- When reading through parts on the DVD, the DVD is mounted and
+  unmounted for each part.
+- Make sure that the restore options don't permit "seeing" other
+  Client's job data.
+- Restore of a raw drive should not try to check the volume size.
+- Lock tape drive door when open()
+- Make release unload any autochanger.
+- Arno's reservation deadlock.
+- Eric's SD patch