From: Kern Sibbald Date: Sun, 14 Sep 2008 17:06:49 +0000 (+0000) Subject: Update new features chapter X-Git-Tag: Release-3.0.0~982 X-Git-Url: https://git.sur5r.net/?a=commitdiff_plain;h=b0e5b1b24eb1ad6004883ea728885ea7b0c47042;p=bacula%2Fdocs Update new features chapter --- diff --git a/docs/manuals/en/concepts/concepts.kilepr b/docs/manuals/en/concepts/concepts.kilepr index ba04ea15..8d234ade 100644 --- a/docs/manuals/en/concepts/concepts.kilepr +++ b/docs/manuals/en/concepts/concepts.kilepr @@ -161,10 +161,10 @@ order=-1 [item:newfeatures.tex] archive=true -column=21 +column=0 encoding= highlight=LaTeX -line=279 +line=439 open=true order=0 diff --git a/docs/manuals/en/concepts/newfeatures.tex b/docs/manuals/en/concepts/newfeatures.tex index abd32512..dbc907c2 100644 --- a/docs/manuals/en/concepts/newfeatures.tex +++ b/docs/manuals/en/concepts/newfeatures.tex @@ -8,6 +8,30 @@ This chapter presents the new features added to the development 2.5.x versions to be released as Bacula version 3.0.0 near the end of 2008. +\section{Accurate} +\index[general]{Accurate Backup} +As with most other backup programs, Bacula decides what files to backup +for Incremental and Differental backup by comparing the change (st\_ctime) +and modification (st\_mtime) times of the file to the time the last backup completed. +If one of those two times is later than the last backup time, then the file +will be backed up. This does not, however, permit tracking what files have +been deleted and will miss any file with an old time that may have been +restored or moved on the client filesystem. + +If the {\bf Accurate = \lt{}yes|no\gt{}} directive is enabled (default no) in the +Job resource, the job will be run as an Accurate Job. For a {\bf Full} +backup, there is no difference, but for {\bf Differential} and {\bf Incremental} +backups, the Director will send a list of all previous files backed up, and the +File daemon will use that list to determine if any new files have been added or +or moved and if any files have been deleted. This allows Bacula to make an accurate +backup of your system to that point in time so that if you do a restore, it +will restore your system exactly. One note of caution about using Accurate backup is that +it requires more resources (CPU and memory) on both the Director and +the Client machine to create the list of previous files backed up, to send that +list to the File daemon, and do comparisons on the File daemon between every file +and the list. + + \section{Copy Jobs} \index[general]{Copy Jobs} A new {\bf Copy} job type has been implemented. It is essentially @@ -56,9 +80,9 @@ Job { Pool { Name = Default Pool Type = Backup - Recycle = yes # Bacula can automatically recycle Volumes - AutoPrune = yes # Prune expired volumes - Volume Retention = 365d # one year + Recycle = yes # Automatically recycle Volumes + AutoPrune = yes # Prune expired volumes + Volume Retention = 365d # one year NextPool = Full Storage = File } @@ -66,9 +90,9 @@ Pool { Pool { Name = Full Pool Type = Backup - Recycle = yes # Bacula can automatically recycle Volumes - AutoPrune = yes # Prune expired volumes - Volume Retention = 365d # one year + Recycle = yes # Automatically recycle Volumes + AutoPrune = yes # Prune expired volumes + Volume Retention = 365d # one year Storage = DiskChanger } @@ -85,7 +109,7 @@ Storage { # Definition of DDS Virtual tape disk storage device Storage { Name = DiskChanger - Address = localhost # N.B. Use a fully qualified name here + Address = localhost # N.B. Use a fully qualified name here Password = "yyy" Device = DiskChanger Media Type = DiskChangerMedia @@ -119,6 +143,42 @@ run job=MyBackup level=VirtualFull And it would produce a new Full backup without using the client, and the output would be written to the {\bf Full} Pool which uses the Diskchanger Storage. +\section{Duplicate Job Control} +\index[general]{Duplicate Jobs} +The new version of Bacula provides four new directives that +give additional control over what Bacula does if duplicate jobs +are started. A duplicate job in the sense we use it here means +a second or subsequent job with the same name starts. This +happens most frequently when the first job runs longer than expected because no +tapes are available. + +The four directives each take as an argument a {\bf yes} or {\bf no} value and +are specified in the Job resource. + +They are: + +\begin{description} +\item [Allow Duplicate Jobs = \lt{}yes|no\gt{}] + If this directive is enabled duplicate jobs will be run. If + the directive is set to {\bf no} (default) then only one job of a given name + may run at one time, and the action that Bacula takes to ensure only + one job runs is determined by the other directives (see below). + +\item [Allow Higher Duplicates = \lt{}yes|no\gt{}] + If this directive is set to {\bf yes} (default) the job with a higher + priority (lower priority number) will be permitted to run. If the + priorities of the two jobs are the same, the outcome is determined by + other directives (see below). + +\item [Cancel Queued Duplicates = \lt{}yes|no\gt{}] + If this directive is set to {\bf yes} (default) any job that is + already queued to run but not yet running will be canceled. + +\item [Cancel Running Duplicates = \lt{}yes|no\gt{}] + If this directive is set to {\bf yes} any job that is already running + will be canceled. The default is {\bf no}. +\end{description} + \section{TLS Authentication} \index[general]{TLS Authentication} In Bacula version 2.5.x and later, in addition to the normal Bacula @@ -194,43 +254,6 @@ obey this flag. The new directive is: The default value is {\bf no}. -\section{Duplicate Job Control} -\index[general]{Duplicate Jobs} -The new version of Bacula provides four new directives that -give additional control over what Bacula does if duplicate jobs -are started. A duplicate job in the sense we use it here means -a second or subsequent job with the same name starts. This -happens most frequently when the first job runs longer than expected because no -tapes are available. - -The four directives each take as an argument a yes or no value and -are specified in the Job resource. - -They are: - -\begin{description} -\item [Allow Duplicate Jobs = \lt{}yes|no\gt{}] - If this directive is enabled duplicate jobs will be run. If - the directive is set to {\bf no} (default) then only one job of a given name - may run at one time, and the action that Bacula takes to ensure only - one job runs is determined by the other directives (see below). - -\item [Allow Higher Duplicates = \lt{}yes|no\gt{}] - If this directive is set to {\bf yes} (default) the job with a higher - priority (lower priority number) will be permitted to run. If the - priorities of the two jobs are the same, the outcome is determined by - other directives (see below). - -\item [Cancel Queued Duplicates = \lt{}yes|no\gt{}] - If this directive is set to {\bf yes} (default) any job that is - already queued to run but not yet running will be canceled. - -\item [Cancel Running Duplicates = \lt{}yes|no\gt{}] - If this directive is set to {\bf yes} any job that is already running - will be canceled. The default is {\bf no}. -\end{description} - - \section{Ignore Dir} \index[general]{IgnoreDir} The {\bf Ignore Dir = \lt{}filename\gt{}} is a new directive that can be added to the Include @@ -240,17 +263,17 @@ the whole directory will be ignored (not backed up). For example: \begin{verbatim} - # List of files to be backed up - FileSet { - Name = "MyFileSet" - Include { - Options { - signature = MD5 - } - File = /home - IgnoreDir = .excludeme - } - } + # List of files to be backed up + FileSet { + Name = "MyFileSet" + Include { + Options { + signature = MD5 + } + File = /home + IgnoreDir = .excludeme + } + } \end{verbatim} But in /home, there may be hundreds of directories of users and some @@ -273,28 +296,6 @@ then Bacula will not backup the two directories named: -\section{Accurate} -\index[general]{Accurate Backup} -As with most other backup programs, Bacula decides what files to backup -for Incremental and Differental backup by comparing the change (st\_ctime) -and modification (st\_mtime) times of the file to the time the last backup completed. -If one of those two times is later than the last backup time, then the file -will be backed up. This does not, however, permit tracking what files have -been deleted and will miss any file with an old time that may have been -restored or moved on the client filesystem. - -If the {\bf Accurate = \lt{}yes|no\gt{}} directive is enabled (default no) in the -Job resource, the job will be run as an Accurate Job. For a {\bf Full} -backup, there is no difference, but for {\bf Differential} and {\bf Incremental} -backups, the Director will send a list of all previous files backed up, and the -File daemon will use that list to determine if any new files have been added or -or moved and if any files have been deleted. This allows Bacula to make an accurate -backup of your system to that point in time so that if you do a restore, it -will restore your system exactly. The downside of using Accurate backup is that -it requires significantly more resources (CPU and memory) on both the Director and -the Client machine to create the list of previous files backed up, to send that -list to the File daemon, and do comparisons on the File daemon between every file -and the list. \section{Bacula Plugins} \index[general]{Plugin} @@ -305,8 +306,136 @@ a single plugin named {\bf bpipe} that allows an external program to get control to backup and restore a file. Plugins are also planned (partially implemented) in the Director and the -Storage daemon. Also we plan (at some point) to port (partially implemented) -the plugin code to Win32 machines. +Storage daemon. The code is also implemented to work on Win32 machines, +but it has not yet been tested. + +\subsection{Plugin Directory} +Each daemon (DIR, FD, SD) has a new {\bf Plugin Directory} directive that may +be added to the daemon definition resource. The directory takes a quoted +string argument, which is the name of the directory in which the daemon can +find the Bacula plugins. If this directive is not specified, Bacula will not +load any plugins. Since each plugin has a distinctive name, all the daemons +can share the same plugin directory. + + + +\subsection{Plugin Options} +The {\bf Plugin Options} directive takes a quoted string +arguement (after the equal sign) and may be specified in the +Job resource. The options specified will be passed to the plugin +when it is run. The value defined in the Job resource can be modified +by the user when he runs a Job via the {\bf bconsole} command line +prompts. + +Note: this directive may be specified, but it is not yet passed to +the plugin (i.e. not fully implemented). + +\subsection{Plugin Options ACL} +The {\bf Plugin Options ACL} directive may be specified in the +Director's Console resource. It functions as all the other ACL commands +do by permitting users running restricted consoles to specify a +{\bf Plugin Options} that overrides the one specified in the Job +definition. Without this directive restricted consoles may not modify +the Plugin Options. + +\subsection{Plugin} +The {\bf Plugin} directive is specified in the Include section of +a FileSet resource where you put your {\bf File = xxx} directives. +For example: + +\begin{verbatim} + FileSet { + Name = "MyFileSet" + Include { + Options { + signature = MD5 + } + File = /home + Plugin = "bpipe:..." + } + } +\end{verbatim} + +In the above example, when the File daemon is processing the directives +in the Include section, it will first backup all the files in {\bf /home} +then it will load the plugin named {\bf bpipe} (actually bpipe-dir.so) from +the Plugin Directory. The syntax and semantics of the Plugin directive +require the first part of the string up to the colon (:) to be the name +of the plugin. Everything after the first colon is ignored by the File daemon but +is passed to the plugin. Thus the plugin writer may define the meaning of the +rest of the string as he wishes. + +Please see the next section for information about the {\bf bpipe} Bacula +plugin. + +\section{The bpipe Plugin} +The {\bf bpipe} plugin is provided in the directory src/plugins/fd/bpipe-fd.c of +the Bacula source distribution. When the plugin is compiled and linking into +the resulting dynamic shared object (DSO), it will have the name {\bf bpipe-fd.so}. + +The purpose of the plugin is to provide an interface to any system program for +backup and restore. As specified above the {\bf bpipe} plugin is specified in +the Include section of your Job's FileSet resource. The full syntax of the +plugin directive as interpreted by the {\bf bpipe} plugin (each plugin is free +to specify the sytax as it wishes) is: + +\begin{verbatim} + Plugin = ":::" +\end{verbatim} + +where +\begin{description} +\item {\bf field1} is the name of the plugin with the trailing {\bf -fd.so} +stripped off, so in this case, we would put {\bf bpipe} in this field. + +\item {\bf field2} specifies the namespace, which for {\bf bpipe} is the +pseudo path and filename under which the backup will be saved. This pseudo +path and filename will be seen by the user in the restore file tree. +For example, if the value is {\bf /MYSQL/regress.sql}, the data +backed up by the plugin will be put under that "pseudo" path and filename. +You must be careful to choose a naming convention that is unique to avoid +a conflict with a path and filename that actually exists on your system. + +\item {\bf field3} for the {\bf bpipe} plugin +specifies the "reader" program that is called by the plugin during +backup to read the data. {\bf bpipe} will call this program by doing a +{\bf popen} on it. + +\item {\bf field4} for the {\bf bpipe} plugin +specifies the "writer" program that is called by the plugin during +restore to write the data back to the filesystem. +\end{description} + +Putting it all together, the full plugin directive line might look +like the following: + +\begin{verbatim} +Plugin = "bpipe:/MYSQL/regress.sql:mysqldump -f + --opt --databases bacula:mysql" +\end{verbatim} + +The directive has been split into two lines, but within the {\bf bacula-dir.conf} file +would be written on a single line. + +This causes the File daemon to call the {\bf bpipe} plugin, which will write +its data into the "pseudo" file {\bf /MYSQL/regress.sql} by calling the +program {\bf mysqldump -f --opt --database bacula} to read the data during +backup. The mysqldump command outputs all the data for the database named +{\bf bacula}, which will be read by the plugin and stored in the backup. +During restore, the data that was backed up will be sent to the program +specified in the last field, which in this case is {\bf mysql}. When +{\bf mysql} is called, it will read the data sent to it by the plugn +then write it back to the same database from which it came ({\bf bacula} +in this case). + +The {\bf bpipe} plugin is a generic pipe program, that simply transmits +the data from a specified program to Bacula for backup, and then from Bacula to +a specified program for restore. + +By using different command lines to {\bf bpipe}, +you can backup any kind of data (ASCII or binary) depending +on the program called. + \section{Display Autochanger Content} @@ -383,34 +512,68 @@ Storage daemon job spooling and despooling activity. The default connect timeout to the File daemon has been set to 3 minutes. Previously it was 30 minutes. -\item [Statistics Enhancements] +\item [ftruncate for NFS Volumes] +If you write to a Volume mounted by NFS (say on a local file server), +in previous Bacula versions, when the Volume was recycled, it was not +properly truncated because NFS does not implement ftruncate (file +truncate). This is now corrected in the new version because we have +written code (actually a kind user) that deletes and recreates the Volume, +thus accomplishing the same thing as a truncate. + +\item [Support for Ubuntu] +The new version of Bacula now recognizes the Ubuntu (and Kubuntu) +version of Linux, and thus now provides correct autostart routines. +Since Ubuntu officially supports Bacula, you can also obtain any +recent release of Bacula from the Ubuntu repositories. + + +\item [FD Version] +The File daemon to Director protocol now includes a version +number, which will help us in future versions automatically determine +if a File daemon is not compatible. + +\item [Max Run Sched Time] + +\item [Full Max Wait Time] -If you (or you boss) want to have statistics on your backups, you could use -some SQL stuffs on the Job table to report how many: +\item [Incremental Max Wait Time] + +\item [Differential Max Wait Time] + +\item [Full Max Run Time] + +\item [Differential Max Run Time] + +\item [Incremental Max Run Time] + + +\item [Statistics Enhancements] +If you (or your boss) want to have statistics on your backups, you could use +a few SQL queries on the Job table to report how many: \begin{itemize} \item jobs have run \item jobs have been successful -\item files have been backuped +\item files have been backed up \item ... \end{itemize} -Theses statistics are accurate only if your job retention is greater than -your statistic period. Ie, if jobs are purged from the catalog, you won't be +However, these statistics are accurate only if your job retention is greater than +your statistics period. Ie, if jobs are purged from the catalog, you won't be able to use them. -Now, you can use the \textbf{update stats [days=num]} console to fill the +Now, you can use the \textbf{update stats [days=num]} console command to fill the JobStat table with new Job records. -The \textbt{Statistics Retention = \lt{}time\gt{}} director directive defines +The \textbf{Statistics Retention = \lt{}time\gt{}} director directive defines the length of time that Bacula will keep statistics job records in the Catalog database after the Job End time. (In \texttt{JobStat} table) When this time period expires, and if user runs \texttt{prune stats} command, Bacula will prune (remove) Job records that are older than the specified period. -Theses statistics records aren't use for restore purpose, but mainly for +These statistics records aren't used for restore purpose, but mainly for capacity planning, billings, etc. -You can use this setup in your \textbf{BackupCatalog} job to maintain +You can use the following Job resource in your nightly \textbf{BackupCatalog} job to maintain statistics. \begin{verbatim} Job { @@ -426,9 +589,7 @@ Job { \end{verbatim} \item [Spooling Enhancements] - A new job directive permits to specify the spool size per job. This is used -in advance job tunning. {\bf SpoolSize={\it bytes}} - +in advanced job tunning. {\bf SpoolSize={\it bytes}} \end{description}