kes Added the following to provide solutions to the plugin/Accurate

     problem -- bug #1236 Cannot restore incremental backups with
     the Exchange plugin.
     - New Bacula read-only variable bVarAccurate -- returns accurate flag
     - New Bacula write-only variable bVarFileSeen -- marks a file as seen
     - New plugin entry point -- checkFile that is called at the end
       of an Accurate job and allows the plugin to mark a file as seen.

git-svn-id: https://bacula.svn.sourceforge.net/svnroot/bacula/trunk@8595 91ce42f0-d328-0410-95d8-f526ca767f89

diff --git a/bacula/release-3-status.txt b/bacula/release-3-status.txt
new file mode 100644 (file)
index 0000000..5a12ad6
--- /dev/null
+++ b/bacula/release-3-status.txt
@@ -0,0 +1,130 @@
+                   Release 3.0 Status
+
+New features implemented:
+- Accurate Backup
+- Copy Jobs
+- ACL code overhaul
+- Solaris ZFS/NFSv4 ACL support
+- Extended Attribute support
+- Shared Objects
+- Virtual Backup
+- Catalog Format Enhancements
+  - Support for more than 4 Billion file entries
+  - New stats tables
+- Duplicate Job Control
+- TLS Authentication
+- bextract non-portable Win32 data
+- State File updated at Job Termination
+- Bacula Plugins
+  - FD bpipe plugin
+  - FD Exchange plugin
+- libdbi DB driver
+- Console Enhancements
+  - status slots
+  - list joblog
+  - command separator
+- Bootstrap regex
+- Restore seek optimization
+- Virtual Tape Emulation
+- Bat Enhancements
+- RunScript Enhancements
+- Status enhancements
+- FD connect timeout default to 3 minutes
+- ftruncate implemented for NFS Volumes
+- FD Version compatibily flag
+- Statistics Enhancements
+- Faster Attribute despooling
+- Document split into 6 manuals
+- GPLv2 license now compatible with OpenSSL
+- Win64 File daemon
+- Postmortum dump improvements in SD
+- Deadlock mutex detection in SD
+ New Directives:
+- Max Full Interval
+- Max Diff Interval
+- Honor No Dump Flag
+- Exclude Dirs Containing
+- Recycle Pool
+- Max Run Sched Time
+- Max Wait Time
+- Full/Diff/Incr Max Run Time
+- Scratch Pool
+- Max Console Connections
+- Spool Size (in Job resource)
+- Allow Mixed Priorities
+- Allow Duplicate Jobs
+- Allow Higher Duplicates
+- Cancel Queued Duplicates
+- Cancel Running Duplicates
+- TLS Authenticate
+- Console (in RunScript)
+
+Project Items Completed for version 3.0.0:
+Item  1:  Accurate restoration of renamed/deleted files
+Item  3:  Merge multiple backups (Synthetic Backup or Consolidation)
+Item  4:  Implement Catalog directive for Pool resource in Director
+Item  5:  Add an item to the restore option where you can select a Pool
+Item  8:  Implement Copy pools
+Item 12:  Add Plug-ins to the FileSet Include statements.
+Item 13:  Restore only file attributes (permissions, ACL, owner, group...)
+Item 18:  Better control over Job execution
+Item 26:  Store and restore extended attributes, especially selinux file contexts
+Item 27:  make changing "spooldata=yes|no" possible for
+Item 28:  Implement an option to modify the last written date for volumes
+Item  n:  Split documentation into several books
+
+New features to be documented:
+- Update src/plugins/README
+- command separator in console (!$%&'()*+,-/:;<>?[]^`{|}~)
+- examples/database/dbcheck.sql
+- dbport in dbcheck
+- filepattern (restore with regex in bsr)
+- Allow Mixed Priorities  Kjetil Torgrim Homme <kjetilho@linpro.no> 30Jun08
+- Eliminated license problems with OpenSSL.
+- The exchange plugin truncates the exchange logs as soon as it has
+  sent the data to the sd.  If the sd is spooling and something
+  goes wrong and the data doesn't make it to the backup medium then
+  the logs are lost, which could be a problem under some
+  restore scenarios.
+- Regex filtered bsrs Kjetil Torgrim Homme <kjetilho@linpro.no> 15Aug08
+- Added FD bVarAccurate and bVarFileSeen variables
+- Added plugin entrypoint checkFile
+
+
+Code to be completed before 3.0.0 is released:
+2. Fix Exchange plugin bugs 1234,1235,1236,1237
+   *1234 fixed.
+4. Finish documentation (some noted above)
+6. Make plugins work with Accurate -- needs two new API calls
+   - mark file not deleted
+   - is file deleted (at end of job)
+   - Pass accurate flag to Plugin
+7. Proper installation of bat files
+8. Turn off plugin options code until fully implemented.
+9. Review whether or not plugins co-exist with encryption,
+   compression, ...
+
+Bat help files:
+/usr/share/doc/bat/help (default)
+Typical /usr/share/doc/xxx
+  changelog.Debian.gz
+  changelog.gz
+  CHANGES.txt.gz
+  copyright
+  CREDITS.txt
+  examples
+  FAQ.txt
+  help
+  README.Debian
+  README.txt
+
+Features that will not be completed:
+- Plugin Options
+- Dir plugin implementation somewhat minimal
+- No example Dir plugin
+- SD plugin implementation somewhat minimal
+- No example SD plugin
+- Install manual should be split into Install a
+  separate Configuration manual.
+- Not all new features are tested or tested well
+  enough

diff --git a/bacula/src/console/console.c b/bacula/src/console/console.c
index c1f0d2c128602db4abd8dcc6a2706d1446732ff1..6098e92ad4f00288ca9acf16a7b9a81875652827 100644 (file)
--- a/bacula/src/console/console.c
+++ b/bacula/src/console/console.c
@@ -1,7 +1,7 @@
 /*
    Bacula® - The Network Backup Solution
 
-   Copyright (C) 2000-2008 Free Software Foundation Europe e.V.
+   Copyright (C) 2000-2009 Free Software Foundation Europe e.V.
 
    The main author of Bacula is Kern Sibbald, with contributions from
    many others, a complete list can be found in the file AUTHORS.

diff --git a/bacula/src/console/console_conf.c b/bacula/src/console/console_conf.c
index ec40512c3fa1a0bc2116d727564a6ea93712f15b..d7c33a3098831122a959b32b2b723fbd8ae04ccf 100644 (file)
--- a/bacula/src/console/console_conf.c
+++ b/bacula/src/console/console_conf.c
@@ -1,7 +1,7 @@
 /*
    Bacula® - The Network Backup Solution
 
-   Copyright (C) 2000-2007 Free Software Foundation Europe e.V.
+   Copyright (C) 2000-2009 Free Software Foundation Europe e.V.
 
    The main author of Bacula is Kern Sibbald, with contributions from
    many others, a complete list can be found in the file AUTHORS.

diff --git a/bacula/src/filed/accurate.c b/bacula/src/filed/accurate.c
index bb98acadd67526164c39c001e24a6478fb448e73..b3435898547fcf9fdbaaa23adffb416be5c0f35f 100644 (file)
--- a/bacula/src/filed/accurate.c
+++ b/bacula/src/filed/accurate.c
@@ -1,7 +1,7 @@
 /*
    Bacula® - The Network Backup Solution
 
-   Copyright (C) 2000-2008 Free Software Foundation Europe e.V.
+   Copyright (C) 2000-2009 Free Software Foundation Europe e.V.
 
    The main author of Bacula is Kern Sibbald, with contributions from
    many others, a complete list can be found in the file AUTHORS.
@@ -43,6 +43,19 @@ typedef struct PrivateCurFile {
    bool seen;
 } CurFile;
 
+bool accurate_mark_file_as_seen(JCR *jcr, char *fname)
+{
+   if (!jcr->accurate || !jcr->file_list) {
+      return false;
+   }
+   /* TODO: just use elt->seen = 1 */
+   CurFile *temp = (CurFile *)jcr->file_list->lookup(fname);
+   if (temp) {
+      temp->seen = 1;              /* records are in memory */
+   }
+   return true;
+}
+
 static bool accurate_mark_file_as_seen(JCR *jcr, CurFile *elt)
 {
    /* TODO: just use elt->seen = 1 */
@@ -98,14 +111,15 @@ bool accurate_send_deleted_list(JCR *jcr)
    ff_pkt->type = FT_DELETED;
 
    foreach_htable(elt, jcr->file_list) {
-      if (!elt->seen) { /* already seen */
-         Dmsg2(dbglvl, "deleted fname=%s seen=%i\n", elt->fname, elt->seen);
-         ff_pkt->fname = elt->fname;
-         ff_pkt->statp.st_mtime = elt->mtime;
-         ff_pkt->statp.st_ctime = elt->ctime;
-         encode_and_send_attributes(jcr, ff_pkt, stream);
+      if (elt->seen || plugin_check_file(jcr, elt->fname)) {
+         continue;
       }
-//      free(elt->fname);
+      Dmsg2(dbglvl, "deleted fname=%s seen=%i\n", elt->fname, elt->seen);
+      ff_pkt->fname = elt->fname;
+      ff_pkt->statp.st_mtime = elt->mtime;
+      ff_pkt->statp.st_ctime = elt->ctime;
+      encode_and_send_attributes(jcr, ff_pkt, stream);
+//    free(elt->fname);
    }
 
    term_find_files(ff_pkt);

diff --git a/bacula/src/filed/fd_plugins.c b/bacula/src/filed/fd_plugins.c
index 3b54d81cd49ad76546e08612abbf29e292868e41..6e547c6dca60917e61ac04f4b6e770049594b6ed 100644 (file)
--- a/bacula/src/filed/fd_plugins.c
+++ b/bacula/src/filed/fd_plugins.c
@@ -150,6 +150,45 @@ void generate_plugin_event(JCR *jcr, bEventType eventType, void *value)
    return;
 }
 
+/*
+ * Check if file was seen for accurate
+ */
+bool plugin_check_file(JCR *jcr, char *fname)
+{
+   Plugin *plugin;
+   bool ok = false;
+   int i = 0;
+
+   if (!plugin_list || !jcr || !jcr->plugin_ctx_list) {
+      return false;                      /* Return if no plugins loaded */
+   }
+
+   bpContext *plugin_ctx_list = (bpContext *)jcr->plugin_ctx_list;
+
+   Dmsg2(dbglvl, "plugin_ctx=%p JobId=%d\n", jcr->plugin_ctx_list, jcr->JobId);
+
+   /* Pass event to every plugin */
+   foreach_alist(plugin, plugin_list) {
+      jcr->plugin_ctx = &plugin_ctx_list[i++];
+      jcr->plugin = plugin;
+      if (is_plugin_disabled(jcr)) {
+         continue;
+      }
+      if (plug_func(plugin)->checkFile == NULL) {
+         continue;
+      }
+      ok = plug_func(plugin)->checkFile(jcr->plugin_ctx, fname);
+      if (ok) {
+         break;
+      }
+   }
+
+   jcr->plugin = NULL;
+   jcr->plugin_ctx = NULL;
+   return ok;
+}
+
+
 /*   
  * Sequence of calls for a backup:
  * 1. plugin_save() here is called with ff_pkt
@@ -732,12 +771,12 @@ static boffset_t my_plugin_blseek(BFILE *bfd, boffset_t offset, int whence)
 static bRC baculaGetValue(bpContext *ctx, bVariable var, void *value)
 {
    JCR *jcr;
-   jcr = ((bacula_ctx *)ctx->bContext)->jcr;
-// Dmsg1(dbglvl, "bacula: baculaGetValue var=%d\n", var);
    if (!value || !ctx) {
       return bRC_Error;
    }
    jcr = ((bacula_ctx *)ctx->bContext)->jcr;
+// Dmsg1(dbglvl, "bacula: baculaGetValue var=%d\n", var);
+   jcr = ((bacula_ctx *)ctx->bContext)->jcr;
    if (!jcr) {
       return bRC_Error;
    }
@@ -752,22 +791,61 @@ static bRC baculaGetValue(bpContext *ctx, bVariable var, void *value)
       Dmsg1(dbglvl, "Bacula: return my_name=%s\n", my_name);
       break;
    case bVarLevel:
+      *((int *)value) = jcr->get_JobLevel();
+      Dmsg1(dbglvl, "Bacula: return bVarJobLevel=%d\n", jcr->get_JobLevel());
+      break;
    case bVarType:
+      *((int *)value) = jcr->get_JobType();
+      Dmsg1(dbglvl, "Bacula: return bVarJobType=%d\n", jcr->get_JobType());
+      break;
    case bVarClient:
+      *((char **)value) = jcr->client_name;
+      Dmsg1(dbglvl, "Bacula: return Client_name=%s\n", jcr->client_name);
+      break;
    case bVarJobName:
+      *((char **)value) = jcr->Job;
+      Dmsg1(dbglvl, "Bacula: return Job name=%s\n", jcr->Job);
+      break;
    case bVarJobStatus:
+      *((int *)value) = jcr->JobStatus;
+      Dmsg1(dbglvl, "Bacula: return bVarJobStatus=%d\n", jcr->JobStatus);
+      break;
    case bVarSinceTime:
+      *((int *)value) = (int)jcr->mtime;
+      Dmsg1(dbglvl, "Bacula: return since=%d\n", (int)jcr->mtime);
       break;
+   case bVarAccurate:
+      *((int *)value) = (int)jcr->accurate;
+      Dmsg1(dbglvl, "Bacula: return accurate=%d\n", (int)jcr->accurate);
+      break;
+   case bVarFileSeen:
+      break;                 /* a write only variable, ignore read request */
    }
    return bRC_OK;
 }
 
 static bRC baculaSetValue(bpContext *ctx, bVariable var, void *value)
 {
+   JCR *jcr;
    if (!value || !ctx) {
       return bRC_Error;
    }
-   Dmsg1(dbglvl, "bacula: baculaSetValue var=%d\n", var);
+   jcr = ((bacula_ctx *)ctx->bContext)->jcr;
+// Dmsg1(dbglvl, "bacula: baculaGetValue var=%d\n", var);
+   jcr = ((bacula_ctx *)ctx->bContext)->jcr;
+   if (!jcr) {
+      return bRC_Error;
+   }
+// Dmsg1(dbglvl, "Bacula: jcr=%p\n", jcr); 
+   switch (var) {
+   case bVarFileSeen:
+      if (!accurate_mark_file_as_seen(jcr, (char *)value)) {
+         return bRC_Error;
+      } 
+      break;
+   default:
+      break;
+   }
    return bRC_OK;
 }
 

diff --git a/bacula/src/filed/fd_plugins.h b/bacula/src/filed/fd_plugins.h
index 42dc47d34b711c9aa16504204348035bf55e199c..e5b0097e443a8a9a1e280bdbfe90dd01389e9eab 100644 (file)
--- a/bacula/src/filed/fd_plugins.h
+++ b/bacula/src/filed/fd_plugins.h
@@ -144,7 +144,9 @@ typedef enum {
   bVarClient    = 5,
   bVarJobName   = 6,
   bVarJobStatus = 7,
-  bVarSinceTime = 8
+  bVarSinceTime = 8,
+  bVarAccurate  = 9,
+  bVarFileSeen  = 10
 } bVariable;
 
 /* Events that are passed to plugin */
@@ -186,6 +188,7 @@ bool plugin_name_stream(JCR *jcr, char *name);
 int plugin_create_file(JCR *jcr, ATTR *attr, BFILE *bfd, int replace);
 bool plugin_set_attributes(JCR *jcr, ATTR *attr, BFILE *ofd);
 int plugin_save(JCR *jcr, FF_PKT *ff_pkt, bool top_level);
+bool plugin_check_file(JCR *jcr, char *fname);
 #endif
 
 #ifdef __cplusplus
@@ -227,7 +230,7 @@ typedef enum {
 
 
 #define FD_PLUGIN_MAGIC     "*FDPluginData*" 
-#define FD_PLUGIN_INTERFACE_VERSION  2
+#define FD_PLUGIN_INTERFACE_VERSION  3
 
 typedef struct s_pluginInfo {
    uint32_t size;
@@ -259,6 +262,7 @@ typedef struct s_pluginFuncs {
    bRC (*pluginIO)(bpContext *ctx, struct io_pkt *io);
    bRC (*createFile)(bpContext *ctx, struct restore_pkt *rp);
    bRC (*setFileAttributes)(bpContext *ctx, struct restore_pkt *rp);
+   bool (*checkFile)(bpContext *ctx, char *fname);
 } pFuncs;
 
 #define plug_func(plugin) ((pFuncs *)(plugin->pfuncs))

diff --git a/bacula/src/filed/protos.h b/bacula/src/filed/protos.h
index 428a9ddf7c1bcc22edd73dadb7c5f883fb1ae83b..72300f18c680a131f6dc4c6de6384c405d392a74 100644 (file)
--- a/bacula/src/filed/protos.h
+++ b/bacula/src/filed/protos.h
@@ -1,7 +1,7 @@
 /*
    Bacula® - The Network Backup Solution
 
-   Copyright (C) 2000-2008 Free Software Foundation Europe e.V.
+   Copyright (C) 2000-2009 Free Software Foundation Europe e.V.
 
    The main author of Bacula is Kern Sibbald, with contributions from
    many others, a complete list can be found in the file AUTHORS.
@@ -53,6 +53,7 @@ bool parse_acl_stream(JCR *jcr, int stream);
 /* from accurate.c */
 bool accurate_send_deleted_list(JCR *jcr);
 bool accurate_check_file(JCR *jcr, FF_PKT *ff_pkt);
+bool accurate_mark_file_as_seen(JCR *jcr, char *fname);
 
 /* from backup.c */
 bool encode_and_send_attributes(JCR *jcr, FF_PKT *ff_pkt, int &data_stream);

diff --git a/bacula/src/filed/xattr.c b/bacula/src/filed/xattr.c
index 2b01bff2d88624139d160e251a627a635bfc9cc6..88e33a5a76bc1f1e7a29e2722370e00ede29be5e 100644 (file)
--- a/bacula/src/filed/xattr.c
+++ b/bacula/src/filed/xattr.c
@@ -40,7 +40,7 @@
  *
  *   Written by Marco van Wieringen, November MMVIII
  *
- *   Version $Id: xattr.c 7879 2008-10-23 10:12:36Z kerns $
+ *   Version $Id$
  */
 
 #include "bacula.h"

diff --git a/bacula/src/lib/plugins.h b/bacula/src/lib/plugins.h
index 9934a4eb10c2d2451e8024865f29eda2081fb658..ead012e3460f97440460905a1b015c47b3f023b1 100644 (file)
--- a/bacula/src/lib/plugins.h
+++ b/bacula/src/lib/plugins.h
@@ -1,7 +1,7 @@
 /*
    Bacula® - The Network Backup Solution
 
-   Copyright (C) 2007-2008 Free Software Foundation Europe e.V.
+   Copyright (C) 2007-2009 Free Software Foundation Europe e.V.
 
    The main author of Bacula is Kern Sibbald, with contributions from
    many others, a complete list can be found in the file AUTHORS.

diff --git a/bacula/technotes-2.5 b/bacula/technotes-2.5
index c081cce26c2ed1f794115dc62d54e9a6aee34963..773fe455ef21c9db7ee937172000465bee8e5b1e 100644 (file)
--- a/bacula/technotes-2.5
+++ b/bacula/technotes-2.5
@@ -1,53 +1,18 @@
               Technical notes on version 2.5
 
           
-Projecs features implemented:
-Items Completed for version 3.0.0:
-Item  1:  Accurate restoration of renamed/deleted files
-Item  3:  Merge multiple backups (Synthetic Backup or Consolidation)
-Item  4:  Implement Catalog directive for Pool resource in Director
-Item  5:  Add an item to the restore option where you can select a Pool
-Item  8:  Implement Copy pools
-Item 12:  Add Plug-ins to the FileSet Include statements.
-Item 13:  Restore only file attributes (permissions, ACL, owner, group...)
-Item 18:  Better control over Job execution
-Item 26:  Store and restore extended attributes, especially selinux file contexts
-Item 27:  make changing "spooldata=yes|no" possible for
-Item 28:  Implement an option to modify the last written date for volumes
-Item  n:  Split documentation into several books
-
-New techical changes:
-libtool
-plugins
-bpipe fd plugin
-Exchange plugin
-
-
-New features to be documented:
-- Update src/plugins/README
-- command separator in console (!$%&'()*+,-/:;<>?[]^`{|}~)
-- examples/database/dbcheck.sql
-- enhancement to wait command: wait mount  ...
-- dbport in dbcheck
-- Allow Mixed Priorities  Kjetil Torgrim Homme <kjetilho@linpro.no> 30Jun08
-- Eliminated license problems with OpenSSL.
-- The exchange plugin truncates the exchange logs as soon as it has
-  sent the data to the sd.  If the sd is spooling and something
-  goes wrong and the data doesn't make it to the backup medium then
-  the logs are lost, which could be a problem under some
-  restore scenarios.
-- Regex filtered bsrs Kjetil Torgrim Homme <kjetilho@linpro.no> 15Aug08
-
-Code to be completed before 3.0.0 is released:
-*1. Fix bug #1221 problem with NODUMP flag
-2. Fix Exchange plugin bugs 1234,1235,1236,1237
-   *1234 fixed.
-3. Resolve bug #1227
-4. Finish documentation (some noted above)
-*5. Diff 2.4.4 to current SVN and check differences
-
-
 General:
+25Mar09
+kes  Added the following to provide solutions to the plugin/Accurate
+     problem -- bug #1236 Cannot restore incremental backups with 
+     the Exchange plugin.
+     - New Bacula read-only variable bVarAccurate -- returns accurate flag
+     - New Bacula write-only variable bVarFileSeen -- marks a file as seen
+     - New plugin entry point -- checkFile that is called at the end
+       of an Accurate job and allows the plugin to mark a file as seen.
+24Mar09
+kes  Temporarly turn off comm timers because it causes bat to seg fault.
+     Must research making SIGUSR2 work with bat and Qt.
 24Mar09
 ebl  Use MTIMEONLY fileset option in accurate check
 23Mar09