From: Kern Sibbald Date: Fri, 1 Dec 2006 20:20:58 +0000 (+0000) Subject: kes Update ReleaseNotes X-Git-Tag: Release-2.0.0~188 X-Git-Url: https://git.sur5r.net/?a=commitdiff_plain;h=14b69bb1c569a70feb2650a0a9d19b55ae720f1c;p=bacula%2Fbacula kes Update ReleaseNotes kes Purge Job records from the catalog for Migration. This was not properly done previously. This fixes bug #718. git-svn-id: https://bacula.svn.sourceforge.net/svnroot/bacula/trunk@3729 91ce42f0-d328-0410-95d8-f526ca767f89 --- diff --git a/bacula/ReleaseNotes b/bacula/ReleaseNotes index 5caab67da3..39c60e62d5 100644 --- a/bacula/ReleaseNotes +++ b/bacula/ReleaseNotes @@ -114,8 +114,68 @@ Version 1.39.28 contains mainly bug fixes to 1.39.26. Please After one of these has been done the upgrade should work fine. +Major new Features in 1.40.0: +- The basic features of Migration are now complete, and + are documented in a new Migration chapter in the manual + http://www.bacula.org/dev-manual/Migration.html +- There is an Encryption chapter in the doc. + http://www.bacula.org/dev-manual/Bacula_Data_Encryption.html +- Additional support for removable devices. See the + Requires Mount, Mount Point, Mount Command, and Unmount + Commands in the Storage daemon configuration chapter: + http://www.bacula.org/dev-manual/Storage_Daemon_Configuratio.html + Also see the Edit Codes for Mount and Unmount Directives in the + same chapter. +- Switch the Win32 build from using Microsoft C++ to using mingw32 + cross-compiling. The initial work was done by Howard Thomson, + then tweaked by me. Robert Nelson then totally reworked the + cross-compiling code so that it not only cross-compiles, but + also compiles on Visual Studio, and at the same time, he added + all the current Unix features to the FD, such as selection on + drives, encryption support, building *all* the tools, ... +- The Director and Storage daemon have now been ported to Win32. + This code is working but should still be considered beta (i.e. + test carefully before putting into production). +- Data encryption done in the Client is now supported due to code + submitted by Landon Fuller. +- Bacula restore and bextract can now extract non-portable Win32 data to + any client (including Unix/Linux clients). Of course, in doing so, + the Microsoft specific permissions and ACLs will be lost. Thanks + to Thorsten Engel for this code. +- The 260 character limitation for Win32 paths name lengths is now + eliminated thanks to Thorsten Engel. +- Eric Bollengier wrote new RunScript directives that includes + the old RunBefore/AfterJob and ClientRunBefore/AfterJob features + plus a *lot* more, allowing you to control just about every aspect + of running scripts. See the manual for detailed documentation. + http://www.bacula.org/dev-manual/Configuring_Director.html#5227 +- SunOS ACLs should now work thanks to a patch from David Duchscher. +- Apply days keyword patch from Alexander.Bergolth at wu-wien.ac.at + If this patch is applied, the number of days can be specified with + "list nextvol days=xx" + or + "status dir days=xx" + This can be used to preview the next scheduled job (and the + next tape to be used) on Fridays if there are no scheduled jobs during + the weekend. +- Apply patch for enhancing wait from Eric Bollengier. One can now: + wait (wait for all jobs to stop) + wait jobid=nn + wait jobuid=unique id + wait job=job-name +- Volumes can now be set to Enable, Disable, or Archive. If they + are not enabled, Volumes will not be mounted. + Implement update volume enable=(on|off|true|false|archived|0|1|2) +- Add Catalog message destination in Messages resource that puts the + job report in the Log database table. + New Features in 1.40.0: +- Add enable/disable job=. This command prevents + the specified job from being scheduled. Even when disabled, + the job can be manually started from the console. +- The database Id records should be 32/64 bit independent now. 64 bits + can be enabled by changing one define, but this has never been tested. - Relative path specifications (i.e. ../xxx) are now permitted in the restore cd command. - When running multiple simultaneous jobs, most jobs that use spooling @@ -125,9 +185,9 @@ New Features in 1.40.0: conf files containing Window cr/lf and Mac cr line termination characters are now accepted thanks to Robert Nelson. - Windows tray status windows are scrollable and resizable. -- Win32 external script excution is much more flexible -- handles +- Win32 external script execution is much more flexible -- handles spaces in names better, ... -- Lots of DVD fixes -- I think they now work! +- Lots of DVD fixes -- writing DVDs is now reported to work. - Fix opening of database in a restricted console to respect any Catalog ACL. - Permit multiple console/director resources in bconsole.conf. @@ -137,28 +197,17 @@ New Features in 1.40.0: - Apply patch supplied in bug #656 to pass priority field in the run dialog to the Director in gnome console. - Add support of encrypted data stream to bscan from Eric. - display data_len instead of data content (may be binary) + display data_len instead of data content (may be binary). - Add Enabled=xxx on update slots command. -- Add host:port to connect failure messages to FD and SD from Dir +- Add host:port to connect failure messages to FD and SD from Dir/ - Add WhereACL to console ACL list. If nothing is specified, only the default is permitted for restore. Otherwise, *all* allows any path, or you can specify permitted paths. This should allow control over where users can restore files. This is untested. -- Install man pages with 'make install' +- Install man pages with 'make install'. - Add Media.Enabled flag to client backups for dotcmds.c - Enforce Media.Enabled=1 for a current restore to work - Require restore case 3 to have sqlquery permission to work. -- The basic features of Migration are now complete. -- There is a Migration chapter in the doc. - http://www.bacula.org/dev-manual/Migration.html -- There is an Encryption chapter in the doc. - http://www.bacula.org/dev-manual/Bacula_Data_Encryption.html -- Additional support for removable devices. See the - Requires Mount, Mount Point, Mount Command, and Unmount - Commands in the Storage daemon configuration chapter: - http://www.bacula.org/dev-manual/Storage_Daemon_Configuratio.html - Also see the Edit Codes for Mount and Unmount Directives in the - same chapter. - Add -n option to bconsole to turn off conio. - The bytes field in the terminated jobs part of the status command now reports in KB, MB, ... units. @@ -172,42 +221,9 @@ New Features in 1.40.0: do a rewind on Solaris when no tape is in the drive (Solaris does not have the detailed errno found on Linux). Added Solaris specific code. Note, this may apply to other OSes as well. -- Switch the Win32 build from using Microsoft C++ to using mingw32 - cross-compiling. The initial work was done by Howard Thomson, - then tweaked by me. Robert Nelson then totally reworked the - cross-compiling code so that it not only cross-compiles, but - also compiles on Visual Studio, and at the same time, he added - all the current Unix features to the FD, such as selection on - drives, encryption support, building *all* the tools, ... - Finally, he also ported the Director and the Storage daemon to - Win32. -- Data encryption done in the Client is now supported due to code - submitted by Landon Fuller. -- Bacula restore and bextract can now extract non-portable Win32 data to - any client (including Unix/Linux clients). Of course, in doing so, - the Microsoft specific permissions and ACLs will be lost. Thanks - to Thorsten Engel for this code. -- The 260 character limitation for Win32 paths name lengths is now - eliminated thanks to Thorsten Engel. -- Eric Bollengier wrote new RunScript directives that includes - the old RunBefore/AfterJob and ClientRunBefore/AfterJob features - plus a *lot* more, allowing you to control just about every aspect - of running scripts. See the manual for detailed documentation. - http://www.bacula.org/dev-manual/Configuring_Director.html#5227 - The examples directory has a new bacula_mail_summary.sh file that creates a single email summary of any number of jobs. Submitted by Andrew J. Millar. -- SunOS ACLs should now work thanks to a patch from David Duchscher. -- The database Id records should be 32/64 bit independent now. 64 bits - can be enabled by changing one define, but this has never been tested. -- Apply days keyword patch from Alexander.Bergolth at wu-wien.ac.at - If this patch is applied, the number of days can be specified with - "list nextvol days=xx" - or - "status dir days=xx" - My use case is to be able to preview the next scheduled job (and the - next tape to be used) on Fridays if there are no scheduled jobs during - the weekend. This patch was probably back ported to 1.38.x - Add nagios plugin to the examples directory. Submitted by Christian Masopust. - Modify most restore error messages to be queued so that they @@ -217,9 +233,6 @@ New Features in 1.40.0: correct detection of holes in block devices and FIFOs. Bug # 506. - Added a report.pl program to the examples directory from Jonas Bjorklund. -- Add enable/disable job=. This command prevents - the specified job from being scheduled. Even when disabled, - the job can be manually started from the console. - Add two new queries to query.sql provided by Arno. One list volumes known to the Storage device, and the other lists volumes possibly needing replacement (error, ...). @@ -242,11 +255,6 @@ New Features in 1.40.0: accept time qualifiers. - Implement jobuid to replace old usage of job in keywords as suggested by Eric Bollengier. -- Apply patch for enhancing wait from Eric Bollengier. One can now: - wait (wait for all jobs to stop) - wait jobid=nn - wait jobuid=unique id - wait job=job-name - Implement write variables for Python to set Priority (anytime), and Job Level, only during JobInit event. - Use the keyword ujobid to mean the unique job id; job or jobname @@ -312,8 +320,8 @@ New Features in 1.40.0: and forcing a 6 second wait. - If using GCC (actually g++) add the following compiler flags -fno-strict-aliasing -fno-exceptions -fno-rtti -- Turn on new bsnprintf() code. This could cause print output - to look different. The reason for this code is to eliminate the +- Turn on new bsnprintf() code. + The reason for this code is to eliminate the security problems associated with using the system libraries print routines. - Implement job report that indicates where Storage and Pool @@ -323,26 +331,22 @@ New Features in 1.40.0: for restore. - Add new VOLMGMT message class. No messages are yet sent with this class. -- Add Catalog message destination in Messages resource that puts the - job report in the Log database table. - Spend a lot of time integrating mkcdrec with the rescue disk. - Add spooling/despooling info in status output of SD. - Add Comment field to llist of a volume. - Allow true/false in some yes/no questions. -- Implement update volume enable=(on|off|true|false|archived|0|1|2) Additional Features Added for Win32: - -Added DriveType directive to the Director's Include Option FileSet -resource. Allowed values are: fixed, removable, cdrom, and remote. There -is only an implementation for Windows because it is the only platform that -has the concept of drives. +- Added DriveType directive to the Director's Include Option FileSet + resource. Allowed values are: fixed, removable, cdrom, and remote. There + is only an implementation for Windows because it is the only platform that + has the concept of drives. -Adds EnhancedWild directive to the Director's Include Option FileSet -resource. Allowed values are: yes and no. +- Adds EnhancedWild directive to the Director's Include Option FileSet + resource. Allowed values are: yes and no. -When EnhancedWild is enabled then the processing of the -Wild, WildDir and WildFile is changed in the following ways. + When EnhancedWild is enabled then the processing of the + Wild, WildDir and WildFile is changed in the following ways. Patterns conform to Posix \ is not a special character in character classification [] @@ -352,23 +356,18 @@ Wild, WildDir and WildFile is changed in the following ways. fnmatch option FNM_FILE_NAME is specified * doesn't match a / so it won't match multiple directory levels in a path -Relative WildFile patterns (ones without a leading /) match -against the filename portion. This in combination with the -FNM_FILE_NAME fnmatch() flag makes directives such as WildFile = -abc*.def work as expected. +- Relative WildFile patterns (ones without a leading /) match + against the filename portion. This in combination with the + FNM_FILE_NAME fnmatch() flag makes directives such as WildFile = + abc*.def work as expected. -Adds support for the shell's feature of brace expansion. +- Adds support for the shell's feature of brace expansion. -Here is an example where braces allow 24 lines to be expressed in 5. + Here is an example where braces allow 24 lines to be expressed in 5. -# Exclude directories full of lots and lots of useless little files -WildDir = "[A-Z]:/{Documents and -Settings,{WINNT,Windows}/Profiles}/*/Cookies" -WildDir = "[A-Z]:/{Documents and -Settings,{WINNT,Windows}/Profiles}/*/Recent" -WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local -Settings,LOCALS~1}/History" -WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local -Settings,LOCALS~1}/Temp" -WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local -Settings,LOCALS~1}/Temporary Internet Files" + # Exclude directories full of lots and lots of useless little files + WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Cookies" + WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/Recent" + WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/History" + WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temp" + WildDir = "[A-Z]:/{Documents and Settings,{WINNT,Windows}/Profiles}/*/{Local Settings,LOCALS~1}/Temporary Internet Files" diff --git a/bacula/kernstodo b/bacula/kernstodo index e97927a303..381b412024 100644 --- a/bacula/kernstodo +++ b/bacula/kernstodo @@ -92,6 +92,18 @@ For 1.39: .move transfer device=xxx fromslot=yyy toslot=zzz Low priority: +- It appears to me that you have run into some sort of race + condition where two threads want to use the same Volume and they + were both given access. Normally that is no problem. However, + one thread wanted the particular Volume in drive 0, but it was + loaded into drive 1 so it decided to unload it from drive 1 and + then loaded it into drive 0, while the second thread went on + thinking that the Volume could be used in drive 1 not realizing + that in between time, it was loaded in drive 0. + I'll look at the code to see if there is some way we can avoid + this kind of problem. Probably the best solution is to make the + first thread simply start using the Volume in drive 1 rather than + transferring it to drive 0. - After pruning, check to see if the Volume retention period has expired. - Check to see if jcr->stime is lost during rescheduling of diff --git a/bacula/src/dird/migrate.c b/bacula/src/dird/migrate.c index 8d2dd6500e..36ac5de4f8 100644 --- a/bacula/src/dird/migrate.c +++ b/bacula/src/dird/migrate.c @@ -347,7 +347,7 @@ bool do_migration(JCR *jcr) migration_cleanup(jcr, jcr->JobStatus); if (mig_jcr) { UAContext *ua = new_ua_context(jcr); - purge_files_from_job(ua, jcr->previous_jr.JobId); + purge_job_records_from_catalog(ua, jcr->previous_jr.JobId); free_ua_context(ua); } return true; diff --git a/bacula/src/dird/protos.h b/bacula/src/dird/protos.h index 25dbfa3db4..717cc861c8 100644 --- a/bacula/src/dird/protos.h +++ b/bacula/src/dird/protos.h @@ -255,6 +255,7 @@ void purge_files_from_volume(UAContext *ua, MEDIA_DBR *mr ); int purge_jobs_from_volume(UAContext *ua, MEDIA_DBR *mr); void purge_files_from_job(UAContext *ua, JobId_t JobId); void purge_job_from_catalog(UAContext *ua, JobId_t JobId); +void purge_job_records_from_catalog(UAContext *ua, JobId_t JobId); /* ua_run.c */ diff --git a/bacula/src/dird/ua_purge.c b/bacula/src/dird/ua_purge.c index ed4e9103fc..1b7e9661af 100644 --- a/bacula/src/dird/ua_purge.c +++ b/bacula/src/dird/ua_purge.c @@ -419,11 +419,29 @@ void purge_job_from_catalog(UAContext *ua, JobId_t JobId) POOL_MEM query(PM_MESSAGE); char ed1[50]; + /* Remove job records */ + purge_job_records_from_catalog(ua, JobId); + + /* Now remove the Job record itself */ edit_int64(JobId, ed1); Mmsg(query, "DELETE FROM Job WHERE JobId=%s", ed1); db_sql_query(ua->db, query.c_str(), NULL, (void *)NULL); Dmsg1(050, "Delete Job sql=%s\n", query.c_str()); +} + +/* + * This removes all the records associated with a Job from + * the catalog (i.e. prunes it) without removing the Job + * record itself. + */ +void purge_job_records_from_catalog(UAContext *ua, JobId_t JobId) +{ + POOL_MEM query(PM_MESSAGE); + char ed1[50]; + + purge_files_from_job(ua, JobId); + edit_int64(JobId, ed1); Mmsg(query, "DELETE FROM JobMedia WHERE JobId=%s", ed1); db_sql_query(ua->db, query.c_str(), NULL, (void *)NULL); Dmsg1(050, "Delete JobMedia sql=%s\n", query.c_str()); @@ -434,6 +452,10 @@ void purge_job_from_catalog(UAContext *ua, JobId_t JobId) } + +/* + * Remove File records for a particular Job. + */ void purge_files_from_job(UAContext *ua, JobId_t JobId) { POOL_MEM query(PM_MESSAGE); @@ -450,6 +472,7 @@ void purge_files_from_job(UAContext *ua, JobId_t JobId) * could grow very large. */ Mmsg(query, upd_Purged, ed1); + db_sql_query(ua->db, query.c_str(), NULL, (void *)NULL); } void purge_files_from_volume(UAContext *ua, MEDIA_DBR *mr ) diff --git a/bacula/src/version.h b/bacula/src/version.h index cb36b2c6de..172bb9748a 100644 --- a/bacula/src/version.h +++ b/bacula/src/version.h @@ -4,8 +4,8 @@ #undef VERSION #define VERSION "1.39.29" -#define BDATE "28 November 2006" -#define LSMDATE "28Nov06" +#define BDATE "01 December 2006" +#define LSMDATE "01Dec06" #define PROG_COPYRIGHT "Copyright (C) %d-2006 Free Software Foundation Europe e.V.\n" #define BYEAR "2006" /* year for copyright messages in progs */ diff --git a/bacula/technotes-1.39 b/bacula/technotes-1.39 index db27dc3e1f..eedab76861 100644 --- a/bacula/technotes-1.39 +++ b/bacula/technotes-1.39 @@ -1,6 +1,10 @@ Technical notes on version 1.39 General: +01Dec06 +kes Update ReleaseNotes +kes Purge Job records from the catalog for Migration. This was + not properly done previously. This fixes bug #718. 28Nov06 kes Initialize msg_type to M_INFO in migration. Pointed out by Robert Nelson.