X-Git-Url: https://git.sur5r.net/?a=blobdiff_plain;f=docs%2Fmanuals%2Fen%2Fmain%2Fnewfeatures.tex;h=77e6068f89da0be55142e601590eee78f2106e32;hb=4df9e46d36a6c4040908609af712dd718f663f83;hp=6810d6bc912284de01094d4a5878e8ece6d05beb;hpb=51b624a2f8c8a565c353695c2e70f4fbc755ebc0;p=bacula%2Fdocs diff --git a/docs/manuals/en/main/newfeatures.tex b/docs/manuals/en/main/newfeatures.tex index 6810d6bc..77e6068f 100644 --- a/docs/manuals/en/main/newfeatures.tex +++ b/docs/manuals/en/main/newfeatures.tex @@ -1,7 +1,970 @@ +\chapter{New Features in 9.0.0} +Note: The first beta versions are released as version 7.9.0, and the first +production release will be 9.0.0. + +\subsection{Maximum Virtual Full Interval Option} +Two new director directives have been added: + +\begin{verbatim} + Max Virtual Full Interval +and + Virtual Full Backup Pool +\end{verbatim} + +The {\bf Max Virtual Full Interval} directive should behave similar to the +{\bf Max Full Interval}, but for Virtual Full jobs. If Bacula sees that +there has not been a Full backup in Max Virtual Full Interval time then it +will upgrade the job to Virtual Full. If you have both {\bf Max Full +Interval} and {\bf Max Virtual Full Interval} set then Max Full Interval +should take precedence. + +The {\bf Virtual Full Backup Pool} directive allows one to change the pool +as well. You probably want to use these two directives in +conjunction with each other but that may depend on the specifics of one's +setup. If you set the {\bf Max Full Interval} without setting {\bf Max +Virtual Full Interval} then Bacula will use whatever the "default" pool is +set to which is the same behavior as with the Max Full Interval. + +\subsection{Progressive Virtual Full} + +In Bacula version 9.0.0, we have added a new Directive named {\bf Backups To Keep} that +permits you to implement Progressive Virtual Fulls within Bacula. Sometimes +this feature is known as Incremental Forever with Consolidation. + +\smallskip + +\begin{figure}[htbp] + \centering + \includegraphics[width=.8\linewidth]{pvf-slidingbackups} + \caption{Backup Sequence Slides Forward One Day, Each Day} + \label{fig:slidingbackups} +\end{figure} + +To implement the Progressive Virtual Full feature, simply add the +{\bf Backups To Keep} directive to your Virtual Full backup Job resource. +The value specified on the directive indicates the number of backup jobs +that should not be merged into the Virtual Full (i.e. the number of backup +jobs that should remain after the Virtual Full has completed. The default +is zero, which reverts to a standard Virtual Full than consolidates all the +backup jobs that it finds. + +\subsubsection{Backups To Keep Directive} +The new {\bf BackupsToKeep} directive is specified in the Job Resource and +has the form: + +\begin{verbatim} + Backups To Keep = 30 +\end{verbatim} + +where the value (30 in the above figure and example) is the number of +backups to retain. When this directive is present during a Virtual Full +(it is ignored for other Job types), it will look for the most recent Full +backup that has more subsequent backups than the value specified. In the +above example the Job will simply terminate unless there is a Full back +followed by at least 31 backups of either level Differential or +Incremental. + +\smallskip +Assuming that the last Full backup is followed by 32 Incremental backups, a +Virtual Full will be run that consolidates the Full with the first two +Incrementals that were run after the Full. The result is that you will end +up with a Full followed by 30 Incremental backups. The Job Resource +in {\bf bacula-dir.conf} to accomplish this would be: + +\begin{verbatim} + Job { + Name = "VFull" + Type = Backup + Level = VirtualFull + Client = "my-fd" + File Set = "FullSet" + Accurate = Yes + Backups To Keep = 10 + } +\end{verbatim} + +\subsubsection{Delete Consolidated Jobs} +The new directive {\bf Delete Consolidated Jobs} expects a {\bf yes} +or {\bf no} value that if set to {\bf yes} will cause any old Job that is +consolidated during a Virtual Full to be deleted. In the example above +we saw that a Full plus one other job (either an Incremental or +Differential) were consolidated into a new Full backup. The original Full +plus the other Job consolidated will be deleted. The default value is +{\bf no}. + +\subsubsection{Virtual Full Compatibility} +Virtual Full as well as Progressive Virtual Full works with any +standard backup Job. + +\smallskip +However, it should be noted that Virtual Full jobs are not compatible with +any plugins that you may be using. + +\subsection{TapeAlert Enhancements} +There are some significant enhancements to the TapeAlert feature of Bacula. +Several directives are used slightly differently, which unfortunately +causes a compatibility problem with the old TapeAlert implementation. +Consequently, if you are already using TapeAlert, you must modify your +{\bf bacula-sd.conf} in order for Tape Alerts to work. See below +for the details ... + +\subsubsection{What is New} +First, you must define a \textbf{Alert Command} directive in the Device +resource that calls the new \textbf{tapealert} script that is installed in +the scripts directory (normally: /opt/bacula/scripts). It is defined as +follows: + +\begin{verbatim} +Device { + Name = ... + Archive Device = /dev/nst0 + Alert Command = "/opt/bacula/scripts/tapealert %l" + Control Device = /dev/sg1 # must be SCSI ctl for /dev/nst0 + ... +} +\end{verbatim} + +In addition the \textbf{Control Device} directive in the Storage Daemon's +conf file must be specified in each Device resource to permit Bacula to +detect tape alerts on a specific devices (normally only tape devices). + +Once the above mentioned two directives (Alert Command and Control Device) +are in place in each of your Device resources, Bacula will check for tape +alerts at two points: + +\begin{itemize} +\item After the Drive is used and it becomes idle. +\item After each read or write error on the drive. +\end{itemize} + +At each of the above times, Bacula will call the new \textbf{tapealert} +script, which uses the \textbf{tapeinfo} program. The tapeinfo utility is +part of the apt sg3-utils and rpm sg3\_utils packages that must be +installed on your systems. Then after each alert that Bacula finds for +that drive, Bacula will emit a Job message that is either INFO, WARNING, or +FATAL depending on the designation in the Tape Alert published by the T10 +Technical Committee on SCSI Storage Interfaces (www.t10.org). For the +specification, please see: www.t10.org/ftp/t10/document.02/02-142r0.pdf + +\smallskip +As a somewhat extreme example, if tape alerts 3, 5, and 39 are set, you +will get the following output in your backup job. + +{\small + \begin{verbatim} + 17-Nov 13:37 rufus-sd JobId 1: Error: block.c:287 + Write error at 0:17 on device "tape" + (/home/kern/bacula/k/regress/working/ach/drive0) + Vol=TestVolume001. ERR=Input/output error. + + 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert: + Volume="TestVolume001" alert=3: ERR=The operation has stopped because + an error has occurred while reading or writing data which the drive + cannot correct. The drive had a hard read or write error + + 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert: + Volume="TestVolume001" alert=5: ERR=The tape is damaged or the drive + is faulty. Call the tape drive supplier helpline. The drive can no + longer read data from the tape + + 17-Nov 13:37 rufus-sd JobId 1: Warning: Disabled Device "tape" + (/home/kern/bacula/k/regress/working/ach/drive0) due to tape alert=39. + + 17-Nov 13:37 rufus-sd JobId 1: Warning: Alert: Volume="TestVolume001" + alert=39: ERR=The tape drive may have a fault. Check for availability + of diagnostic information and run extended diagnostics if applicable. + The drive may have had a failure which may be identified by stored + diagnostic information or by running extended diagnostics (eg Send + Diagnostic). Check the tape drive users manual for instructions on + running extended diagnostic tests and retrieving diagnostic data. + + \end{verbatim} +} + +Without the tape alert feature enabled, you would only get the first error +message above, which is the error return Bacula received when it gets the +error. Notice also, that in the above output the alert number 5 is a +critical error, which causes two things to happen. First the tape drive is +disabled, and second the Job is failed. + +\smallskip +If you attempt to run another Job using the Device that has been disabled, +you will get a message similar to the following: + +\begin{verbatim} +17-Nov 15:08 rufus-sd JobId 2: Warning: + Device "tape" requested by DIR is disabled. +\end{verbatim} + +and the Job may be failed if no other drive can be found. + +\smallskip +Once the problem with the tape drive has been corrected, you can +clear the tape alerts and re-enable the device with the Bacula bconsole +command such as the following: + +\begin{verbatim} + enable Storage=Tape +\end{verbatim} + +Note, when you enable the device, the list of prior tape alerts for that +drive will be discarded. + +\smallskip +Since is is possible to miss tape alerts, Bacula maintains a temporary list +of the last 8 alerts, and each time Bacula calls the \textbf{tapealert} +script, it will keep up to 10 alert status codes. Normally there will only +be one or two alert errors for each call to the tapealert script. + +\smallskip +Once a drive has one or more tape alerts, you can see them by using the +bconsole status command as follows: +\begin{verbatim} +status storage=Tape +\end{verbatim} +which produces the following output: +\begin{verbatim} +Device Vtape is "tape" (/home/kern/bacula/k/regress/working/ach/drive0) +mounted with: + Volume: TestVolume001 + Pool: Default + Media type: tape + Device is disabled. User command. + Total Bytes Read=0 Blocks Read=1 Bytes/block=0 + Positioned at File=1 Block=0 + Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" + alert=Hard Error + Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" + alert=Read Failure + Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" + alert=Diagnostics Required +\end{verbatim} +if you want to see the long message associated with each of the alerts, +simply set the debug level to 10 or more and re-issue the status command: +\begin{verbatim} +setdebug storage=Tape level=10 +status storage=Tape +\end{verbatim} +\begin{verbatim} + ... + Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" + flags=0x0 alert=The operation has stopped because an error has occurred + while reading or writing data which the drive cannot correct. The drive had + a hard read or write error + Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" + flags=0x0 alert=The tape is damaged or the drive is faulty. Call the tape + drive supplier helpline. The drive can no longer read data from the tape + Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" flags=0x1 + alert=The tape drive may have a fault. Check for availability of diagnostic + information and run extended diagnostics if applicable. The drive may + have had a failure which may be identified by stored diagnostic information + or by running extended diagnostics (eg Send Diagnostic). Check the tape + drive users manual for instructions on running extended diagnostic tests + and retrieving diagnostic data. + ... +\end{verbatim} +The next time you \textbf{enable} the Device by either using +\textbf{bconsole} or you restart the Storage Daemon, all the saved alert +messages will be discarded. + +\subsubsection{Handling of Alerts} +Tape Alerts numbered 7,8,13,14,20,22,52,53, and 54 will cause Bacula to +disable the current Volume. + +\smallskip +Tape Alerts numbered 14,20,29,30,31,38, and 39 will cause Bacula to disable +the drive. + +\smallskip +Please note certain tape alerts such as 14 have multiple effects (disable +the Volume and disable the drive). + +\subsection{New Console ACL Directives} +By default, if a Console ACL directive is not set, Bacula will assume that the +ACL list is empty. If the current Bacula Director configuration uses restricted +Consoles and allows restore jobs, it is mandatory to configure the new +directives. + +\subsubsection{DirectoryACL} +\index[dir]{Directive!DirectoryACL} + +This directive is used to specify a list of directories that can be +accessed by a restore session. Without this directive, a restricted +console cannot restore any file. Multiple directories names may be +specified by separating them with commas, and/or by specifying multiple +DirectoryACL directives. For example, the directive may be specified as: + +\footnotesize +\begin{verbatim} + DirectoryACL = /home/bacula/, "/etc/", "/home/test/*" +\end{verbatim} +\normalsize + +With the above specification, the console can access the following +directories: +\begin{itemize} +\item \texttt{/etc/password} +\item \texttt{/etc/group} +\item \texttt{/home/bacula/.bashrc} +\item \texttt{/home/test/.ssh/config} +\item \texttt{/home/test/Desktop/Images/something.png} +\end{itemize} + +But not to the following files or directories: +\begin{itemize} +\item \texttt{/etc/security/limits.conf} +\item \texttt{/home/bacula/.ssh/id\_dsa.pub} +\item \texttt{/home/guest/something} +\item \texttt{/usr/bin/make} +\end{itemize} + +If a directory starts with a Windows pattern (ex: c:/), Bacula will +automatically ignore the case when checking directory names. + +\subsection{New Bconsole ``list'' Command Behavior} + +The bconsole \texttt{list} commands can now be used safely from a +restricted bconsole session. The information displayed will respect the +ACL configured for the Console session. For example, if a restricted +Console has access to JobA, JobB and JobC, information about JobD will not +appear in the \texttt{list jobs} command. + +\subsection{New Console ACL Directives} +\index[dir]{Directive!BackupClientACL} +It is now possible to configure a restricted Console to distinguish Backup +and Restore job permissions. The \texttt{BackupClientACL} can restrict +backup jobs on a specific set of clients, while the +\texttt{RestoreClientACL} can restrict restore jobs. + +{\small +\begin{verbatim} +# cat /opt/bacula/etc/bacula-dir.conf +... + +Console { + Name = fd-cons # Name of the FD Console + Password = yyy +... + ClientACL = localhost-fd # everything allowed + RestoreClientACL = test-fd # restore only + BackupClientACL = production-fd # backup only +} +\end{verbatim} +} + +The \texttt{ClientACL} directive takes precedence over the +\texttt{RestoreClientACL} and the \texttt{BackupClientACL}. In the Console +resource resource above, it means that the bconsole linked to the Console{} +named "fd-cons" will be able to run: + +\begin{itemize} +\item backup and restore for ``localhost-fd'' +\item backup for ``production-fd'' +\item restore for ``test-fd'' +\end{itemize} + +At the restore time, jobs for client ``localhost-fd'', ``test-fd'' and +``production-fd'' will be available. + +If \texttt{*all*} is set for \texttt{ClientACL}, backup and restore will be +allowed for all clients, despite the use of \texttt{RestoreClientACL} or +\texttt{"BackupClientACL}. + +\subsection{Client Initiated Backup} +\label{sec:featurecib} +A console program such as the new \texttt{tray-monitor} or +\texttt{bconsole} can now be configured to connect a File Daemon. There +are many new features available (see the New Tray Monitor section below), +but probably the most important is the ability for the user to initiate a +backup of his own machine. The connection established by the FD to the +Director for the backup will be used by the Director for the backup, thus +not only can clients (users) initiate backups, but a File Daemon that is +NATed (cannot be reached by the Director) can now be backed up without +using advanced tunneling techniques providing that the File Daemon can +connect to the Director. + +\smallskip +The flow of information is shown in the picture below: +\bsysimageH{nat}{Client Initiated Backup Network Flow}{fig:nat3} + +\newpage +\subsection{Configuring Client Initiated Backup} +\smallskip +In order to ensure security, there are a number of new directives +that must be enabled in the new \texttt{tray-monitor}, the File +Daemon and in the Director. +A typical configuration might look like the following: + +{\small +\begin{verbatim} +# cat /opt/bacula/etc/bacula-dir.conf +... + +Console { + Name = fd-cons # Name of the FD Console + Password = yyy + + # These commands are used by the tray-monitor, it is possible to restrict + CommandACL = run, restore, wait, .status, .jobs, .clients + CommandACL = .storages, .pools, .filesets, .defaults, .estimate + + # Adapt for your needs + jobacl = *all* + poolacl = *all* + clientacl = *all* + storageacl = *all* + catalogacl = *all* + filesetacl = *all* +} +\end{verbatim} +} + +{\small +\begin{verbatim} +# cat /opt/bacula/etc/bacula-fd.conf +... + +Console { # Console to connect the Director + Name = fd-cons + DIRPort = 9101 + address = localhost + Password = "yyy" +} + +Director { + Name = remote-cons # Name of the tray monitor/bconsole + Password = "xxx" # Password of the tray monitor/bconsole + Remote = yes # Allow to use send commands to the Console defined +} +\end{verbatim} +} + +{\small +\begin{verbatim} +cat /opt/bacula/etc/bconsole-remote.conf +.... + +Director { + Name = localhost-fd + address = localhost # Specify the FD address + DIRport = 9102 # Specify the FD Port + Password = "notused" +} + +Console { + Name = remote-cons # Name used in the auth process + Password = "xxx" +} +\end{verbatim} +} + +{\small +\begin{verbatim} +cat ~/.bacula-tray-monitor.conf +Monitor { + Name = remote-cons +} + +Client { + Name = localhost-fd + address = localhost # Specify the FD address + Port = 9102 # Specify the FD Port + Password = "xxx" + Remote = yes +} +\end{verbatim} +} + +\bsysimageH{conf-nat}{Relation Between Resources (bconsole)}{fig:nat} +\bsysimageH{conf-nat2}{Relation Between Resources (tray-monitor)}{fig:nat2} + +\medskip +A more detailed description with complete examples is available in +chapter~\ref{TrayMonitorChapter}. + +\subsection{New Tray Monitor} + +A new tray monitor has been added to the 9.0 release, the tray monitor offers +the following features: + +\begin{itemize} +\item Director, File and Storage Daemon status page +\item Support for the Client Initiated Backup protocol (See + \vref{sec:featurecib}). To use the Client Initiated Backup option from the + tray monitor, the Client option ``Remote'' should be checked in the + configuration (Fig \vref{fig:tray2}). +\item Wizard to run new job (Fig \vref{fig:tray4}) +\item Display an estimation of the number of files and the size of the next + backup job (Fig \vref{fig:tray4}) +\item Ability to configure the tray monitor configuration file directly from + the GUI (Fig \vref{fig:tray2}) +\item Ability to monitor a component and adapt the tray monitor task bar icon + if a jobs are running. +\item TLS Support +\item Better network connection handling +\item Default configuration file is stored under \texttt{\$HOME/.bacula-tray-monitor.conf} +\item Ability to ``schedule'' jobs +\item Available on Linux and Windows platforms +\end{itemize} + +% \medskip +% Please see chapter \ref{TrayMonitorChapter} for more details about this new +% functionality. + + +\begin{figure}[htbp] + \centering + \includegraphics[width=0.8\linewidth]{tray-monitor-status} + \caption{Tray Monitor Status} + \label{fig:tray0} +\end{figure} + +\begin{figure}[htbp] + \centering + \includegraphics[width=0.9\linewidth]{tray-monitor-conf-fd} + \caption{Tray Monitor Client Configuration} + \label{fig:tray2} +\end{figure} + +\begin{figure}[htbp] + \centering + \includegraphics[width=0.8\linewidth]{tray-monitor-run1} + \smallskip + \includegraphics[width=0.8\linewidth]{tray-monitor-run2} + \caption{Tray Monitor Run a Job} + \label{fig:tray4} +\end{figure} + +\subsection{Schedule Jobs via the Tray Monitor} + +The Tray Monitor can scan periodically a specific directory ``Command +Directory'' and process ``*.bcmd'' files to find jobs to run. + +The format of the ``file.bcmd'' command file is the following: +\begin{verbatim} +: +: +... + + = string + = string (bconsole command line) +\end{verbatim} + +For example: +\begin{verbatim} +localhost-fd: run job=backup-localhost-fd level=full +localhost-dir: run job=BackupCatalog +\end{verbatim} + +The command file should contain at least one command. The component specified +in the first part of the command line should be defined in the tray +monitor. Once the command file is detected by the tray monitor, a popup is +displayed to the user and it is possible for the user to cancel the job directly. + +\smallskip{} + +The file can be created with tools such as ``cron'' or the ``task scheduler'' +on Windows. It is possible to verify the network connection at that time to +avoid network errors. + +\begin{verbatim} +#!/bin/sh +if ping -c 1 director &> /dev/null +then + echo "my-dir: run job=backup" > /path/to/commands/backup.bcmd +fi +\end{verbatim} + +%\bsysimageH{tray-monitor-status}{Tray Monitor Status}{fig:tray0} +%\bsysimageH{tray-monitor1}{Tray Monitor Configuration}{fig:tray1} +%\bsysimageH{tray-monitor-conf-fd}{Tray Monitor Client Configuration}{fig:tray2} +%\bsysimageH{tray-monitor-conf-dir}{Tray Monitor Director Configuration}{fig:tray3} +%\bsysimageH{tray-monitor-run1}{Tray Monitor Run new Job}{fig:tray4} +% find a way to group them together +%\bsysimageH{tray-monitor-run2}{Tray Monitor Setup new Job}{fig:tray5} + + +\subsection{Accurate Option for Verify ``Volume Data'' Job} + +Since Bacula version 8.4.1, it has been possible to have a Verify Job +configured with \texttt{level=Data} that will reread all records from a job +and optionally check the size and the checksum of all files. Starting with + +\smallskip +Bacula version 9.0, it is now possible to use the \texttt{accurate} option to check +catalog records at the same time. When using a Verify job with +\texttt{level=Data} and \texttt{accurate=yes} can replace the +\texttt{level=VolumeToCatalog} option. + +For more information on how to setup a Verify Data job, see +\vref{label:verifyvolumedata}. + +To run a Verify Job with the \texttt{accurate} option, it is possible to set +the option in the Job definition or set use the \texttt{accurate=yes} on the +command line. + +\begin{verbatim} +* run job=VerifyData jobid=10 accurate=yes +\end{verbatim} + +\subsection{FileDaemon Saved Messages Resource Destination} + +It is now possible to send the list of all saved files to a Messages +resource with the \texttt{saved} message type. It is not recommended to +send this flow of information to the director and/or the catalog when the +client FileSet is pretty large. To avoid side effects, the \texttt{all} +keyword doesn't include the \texttt{saved} message type. The +\texttt{saved} message type should be explicitely set. + +\begin{verbatim} +# cat /opt/bacula/etc/bacula-fd.conf +... +Messages { + Name = Standard + director = mydirector-dir = all, !terminate, !restored, !saved + append = /opt/bacula/working/bacula-fd.log = all, saved, restored +} +\end{verbatim} + +\subsection{Minor Enhancements} + +\subsubsection{New Bconsole ".estimate" Command} + +The new \texttt{.estimate} command can be used to get statistics about a +job to run. The command uses the database to approximate the size and the +number of files of the next job. On a PostgreSQL database, the command +uses regression slope to compute values. On SQLite or MySQL, where these +statistical functions are not available, the command uses a simple +``average'' estimation. The correlation number is given for each value. + +{\small +\begin{verbatim} +*.estimate job=backup +level=I +nbjob=0 +corrbytes=0 +jobbytes=0 +corrfiles=0 +jobfiles=0 +duration=0 +job=backup + +*.estimate job=backup level=F +level=F +nbjob=1 +corrbytes=0 +jobbytes=210937774 +corrfiles=0 +jobfiles=2545 +duration=0 +job=backup +\end{verbatim} +} + +\subsubsection{Traceback and Lockdump} + +After the reception of a signal, \texttt{traceback} and \texttt{lockdump} +information are now stored in the same file. + +\subsection{Bconsole ``list jobs'' command options} + +The \texttt{list jobs} bconsole command now accepts new command line options: + +\begin{itemize} +\item \textbf{joberrors} Display jobs with JobErrors +\item \textbf{jobstatus=T} Display jobs with the specified status code +\item \textbf{client=cli} Display jobs for a specified client +\item \textbf{order=asc/desc} Change the output format of the job list. The + jobs are sorted by start time and JobId, the sort can use ascendant (asc) or + descendant (desc) (default) value. +\end{itemize} + +\subsection{Minor Enhancements} + +\subsubsection{New Bconsole "Tee All" Command} + +The ``@tall'' command allows logging all input/output from a console session. + +\begin{verbatim} +*@tall /tmp/log +*st dir +... +\end{verbatim} + +\subsection{Bconsole ``list jobs'' command options} + +The \texttt{list jobs} bconsole command now accepts new command line options: + +\begin{itemize} +\item \textbf{joberrors} Display jobs with JobErrors +\item \textbf{jobstatus=T} Display jobs with the specified status code +\item \textbf{client=cli} Display jobs for a specified client +\item \textbf{order=asc/desc} Change the output format of the job list. The + jobs are sorted by start time and JobId, the sort can use ascendant (asc) or + descendant (desc) (default) value. +\end{itemize} + +\subsection{New Bconsole "Tee All" Command} + +The ``@tall'' command allows logging all input/output from a console session. + +\begin{verbatim} +*@tall /tmp/log +*st dir +... +\end{verbatim} + +\subsection{New Job Edit Codes \%I} +In various places such as RunScripts, you have now access to \%I to get the +JobId of the copy or migration job started by a migrate job. + +\begin{verbatim} +Job { + Name = Migrate-Job + Type = Migrate + ... + RunAfter = "echo New JobId is %I" +} +\end{verbatim} + + +\subsection*{.api version 2} + +In Bacula version 9.0 and later, we introduced a new .api version +to help external tools to parse various Bacula bconsole output. + +% waa - 20150317 - this section needs just a little more to explain what the "43" in "s43" mean. Perhaps +% if it is not a good place to list the possibilities here, then list where a reference +% is. Also, I think .api 2 ... Means "use API version 2" but that should be stated too + +The \texttt{api\_opts} option can use the following arguments: +\begin{itemize} +\item [C] Clear current options +\item [tn] Use a specific time format (1 ISO format, 2 Unix Timestamp, 3 Default Bacula time format) +\item [sn] Use a specific separator between items (new line by default). +\item [Sn] Use a specific separator between objects (new line by default). +\item [o] Convert all keywords to lowercase and convert all non \textsl{isalpha} characters to \_ +\end{itemize} + +% waa - 20150317 - I think there should either be more output listed here to give a better feeling +% or, perhaps another output listing for different .status commands + +\begin{verbatim} + .api 2 api_opts=t1s43S35 + .status dir running +================================== +jobid=10 +job=AJob +... +\end{verbatim} + +\subsection*{New Debug Options} + +In Bacula version 9.0 and later, we introduced a new \texttt{options} parameter for +the \texttt{setdebug} bconsole command. + +\smallskip{} + +The following arguments to the new \texttt{option} parameter are available to control debug functions. + +\begin{itemize} +\item [0] Clear debug flags +\item [i] Turn off, ignore bwrite() errors on restore on File Daemon +\item [d] Turn off decomp of BackupRead() streams on File Daemon +\item [t] Turn on timestamps in traces +\item [T] Turn off timestamps in traces + +% waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it" ??? +\item [c] Truncate trace file if trace file is activated + +\item [l] Turn on recoding events on P() and V() +\item [p] Turn on the display of the event ring when doing a bactrace +\end{itemize} + +\smallskip{} + +The following command will enable debugging for the File Daemon, truncate an existing trace file, +and turn on timestamps when writing to the trace file. + +\begin{verbatim} +* setdebug level=10 trace=1 options=ct fd +\end{verbatim} + +\smallskip{} + +It is now possible to use a \textsl{class} of debug messages called \texttt{tags} +to control the debug output of Bacula daemons. + +\begin{itemize} +\item [all] Display all debug messages +\item [bvfs] Display BVFS debug messages +\item [sql] Display SQL related debug messages +\item [memory] Display memory and poolmem allocation messages +\item [scheduler] Display scheduler related debug messages +\end{itemize} + +\begin{verbatim} +* setdebug level=10 tags=bvfs,sql,memory +* setdebug level=10 tags=!bvfs + +# bacula-dir -t -d 200,bvfs,sql +\end{verbatim} + +The \texttt{tags} option is composed of a list of tags. Tags are separated by +``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!'' +in front of the tag. Note that more tags are planned for future versions. + +%\LTXtable{\linewidth}{table_debugtags} + +\subsection{Communication Line Compression} +Bacula version 9.0.0 and later now includes communication +line compression. It is turned on by default, and if the +two Bacula components (Dir, FD, SD, bconsole) are both +version 6.6.0 or greater, communication line compression) +will be enabled, by default. If for some reason, you do not want +communication line compression, you may disable it with the +following directive: + +\begin{verbatim} +Comm Compression = no +\end{verbatim} + +This directive can appear in the following resources: +\begin{verbatim} +bacula-dir.conf: Director resource +bacula-fd.conf Client (or FileDaemon) resource +bacula-sd.conf: Storage resource +bconsole.conf: Console resource +bat.conf: Console resource +\end{verbatim} + +\smallskip +In many cases, the volume of data transmitted across the +communications line can be reduced by a factor of three when +this directive is enabled (default) In the case that the compression is not +effective, Bacula turns it off on a. record by record basis. + +\smallskip +If you are backing up data that is already compressed the comm line +compression will not be effective, and you are likely +to end up with an average compression ratio that is very small. +In this case, Bacula reports {\bf None} in the Job report. + +\subsection{Deduplication Optimized Volumes} +This version of Bacula includes a new alternative (or additional) +volume format that optimizes the placement of files so +that an underlying deduplicating filesystem such as ZFS +can optimally deduplicate the backup data that is written +by Bacula. These are called Deduplication Optimized Volumes +or Aligned Volumes for short. The details of how to use this +feature and its considerations are in the +Deduplication Optimized Volumes whitepaper. + +\smallskip +This feature is available if you have Bacula Community produced binaries +and the Aligned Volumes plugin. + +\subsection{New Message Identification Format} +We are starting to add unique message indentifiers to each message (other +than debug and the Job report) that Bacula prints. At the current time +only two files in the Storage Daemon have these message identifiers and +over time with subsequent releases we will modify all messages. + +\smallskip +The message identifier will be kept unique for each message and once +assigned to a message it will not change even if the text of the message +changes. This means that the message identifier will be the same no matter +what language the text is displayed in, and more importantly, it will allow +us to make listing of the messages with in some cases, additional +explanation or instructions on how to correct the problem. All this will +take several years since it is a lot of work and requires some new programs +that are not yet written to manage these message identifiers. + +\smallskip +The format of the message identifier is: + +\begin{verbatim} + [AAnnnn] +\end{verbatim} +where A is an upper case character and nnnn is a four digit number, where +the first charcter indicates the software component (daemon); the second +letter indicates the severity, and the number is unique for a given +componet and severity. + +\smallskip +For example: + +\begin{verbatim} + [SF0001] +\end{verbatim} + +The first charcter representing the componend at the current time one of +the following: + +\begin{verbatim} + S Storage daemon + D Director + F File daemon +\end{verbatim} + +\smallskip +The second character representing the severity or level can be: + +\begin{verbatim} + A Abort + F Fatal + E Erropr + W Warning + S Security + I Info + D Debug + O OK (i.e. operation completed normallly) +\end{verbatim} + +So in the example above [SF0001] indicates it is a message id, because of +the brackets and because it is at the beginning of the message, and that +it was generated by the Storage daemon as a fatal error. +\smallskip +As mentioned above it will take some time to implement these message ids +everywhere, and over time we may add more component letters and more +severity levels as needed. + + \chapter{New Features in 7.4.0} This chapter presents the new features that have been added to the various versions of Bacula. +\section{New Features in 7.4.3} +\subsection{RunScripts} +There are two new RunScript short cut directives implemented in +the Director. They are: + +\begin{verbatim} +Job { + ... + ConsoleRunBeforeJob = "console-command" + ... +} +\end{verbatim} + +\begin{verbatim} +Job { + ... + ConsoleRunAfterJob = "console-command" + ... +} +\end{verbatim} + +As with other RunScript commands, you may have multiple copies +of either the {\bf ConsoleRunBeforeJob} or the {\bf ConsoleRunAfterJob} +in the same Job resource definition. +\smallskip +Please note that not all console commands are permitted, and that +if you run a console command that requires a response, the results +are not determined (i.e. it will probably fail). + + + \section{New Features in 7.4.0} \subsection{Verify Volume Data} @@ -320,8 +1283,6 @@ Job { \smallskip{} - - In RunScripts, the \texttt{AfterSnapshot} keyword for the \texttt{RunsWhen} directive will allow a command to be run just after the Snapshot creation. \texttt{AfterSnapshot} is a synonym for the \texttt{AfterVSS} keyword. @@ -538,7 +1499,7 @@ refer to the OpenSSL documentation to understand the pros and cons regarding the } \end{verbatim} -\subsubsection*{New Option Letter ``M'' for Accurate Directive in FileSet} +\subsubsection*{New Option Letter ``M'' for Accurate Directive in FileSet} % waa - 20150317 - is 8.0.5 correct here? Added in version 8.0.5, the new ``M'' option letter for the Accurate directive @@ -593,64 +1554,6 @@ causing files to be needlessly backed up. } \end{verbatim} -\subsubsection*{New Debug Options} - -In Bacula Enterprise version 8.0 and later, we introduced a new \texttt{options} parameter for -the \texttt{setdebug} bconsole command. - -\smallskip{} - -The following arguments to the new \texttt{option} parameter are available to control debug functions. - -\begin{itemize} -\item [0] Clear debug flags -\item [i] Turn off, ignore bwrite() errors on restore on File Daemon -\item [d] Turn off decomp of BackupRead() streams on File Daemon -\item [t] Turn on timestamps in traces -\item [T] Turn off timestamps in traces - -% waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it" ??? -\item [c] Truncate trace file if trace file is activated - -\item [l] Turn on recoding events on P() and V() -\item [p] Turn on the display of the event ring when doing a bactrace -\end{itemize} - -\smallskip{} - -The following command will enable debugging for the File Daemon, truncate an existing trace file, -and turn on timestamps when writing to the trace file. - -\begin{verbatim} -* setdebug level=10 trace=1 options=ct fd -\end{verbatim} - -\smallskip{} - -It is now possible to use a \textsl{class} of debug messages called \texttt{tags} -to control the debug output of Bacula daemons. - -\begin{itemize} -\item [all] Display all debug messages -\item [bvfs] Display BVFS debug messages -\item [sql] Display SQL related debug messages -\item [memory] Display memory and poolmem allocation messages -\item [scheduler] Display scheduler related debug messages -\end{itemize} - -\begin{verbatim} -* setdebug level=10 tags=bvfs,sql,memory -* setdebug level=10 tags=!bvfs - -# bacula-dir -t -d 200,bvfs,sql -\end{verbatim} - -The \texttt{tags} option is composed of a list of tags. Tags are separated by -``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!'' -in front of the tag. Note that more tags are planned for future versions. - -%%\LTXtable{\linewidth}{table_debugtags} - \subsection{Read Only Storage Devices} This version of Bacula allows you to define a Storage deamon device to be read-only. If the {\bf Read Only} directive is specified and @@ -663,24 +1566,6 @@ drives for restores. An example is: Read Only = yes \end{verbatim} -\subsection{New Truncate Command} -We have added a new truncate command to bconsole which -will truncate a volume if the volume is purged, and if -the volume is also marked {\bf Action On Purge = Truncate}. -This feature was originally added in Bacula version 5.0.1, -but the mechanism for actually doing the truncate required -the user to enter a complicated command such as: - -\begin{verbatim} -purge volume action=truncate storage=File pool=Default -\end{verbatim} - -The above command is now simplified to be: - -\begin{verbatim} -truncate storage=File pool=Default -\end{verbatim} - \subsection{New Resume Command} The new \texttt{resume} command does exactly the same thing as a {\bf restart} command, but for some users the @@ -1089,10 +1974,10 @@ by Bacula Systems. \subsection{Changes in the Pruning Algorithm} -We rewrote the job pruning algorithm in this version. Previously, in some users -reported that the pruning process at the end of jobs was very long. It should -not be longer the case. Now, Bacula won't prune automatically a Job if this -particular Job is needed to restore data. Example: +We rewrote the job pruning algorithm in this version. Previously, in some +users reported that the pruning process at the end of jobs was very long. +It should not be longer the case. Now, Bacula won't prune automatically a +Job if this particular Job is needed to restore data. Example: \begin{verbatim} JobId: 1 Level: Full @@ -1107,9 +1992,10 @@ resource causes that Jobs with Jobid in 1,2,3,4 can be pruned, Bacula will detect that JobId 1 and 4 are essential to restore data at the current state and will prune only JobId 2 and 3. -\texttt{Important}, this change affect only the automatic pruning step after a -Job and the \texttt{prune jobs} Bconsole command. If a volume expires after the -\texttt{VolumeRetention} period, important jobs can be pruned. +\texttt{Important}, this change affect only the automatic pruning step +after a Job and the \texttt{prune jobs} Bconsole command. If a volume +expires after the \texttt{VolumeRetention} period, important jobs can be +pruned. \subsection{Ability to Verify any specified Job} You now have the ability to tell Bacula which Job should verify instead of @@ -1596,8 +2482,8 @@ especially useful for specifying more complicated Migration/Copy SQL statements and in creating long restricted console ACL lists. \subsection{Maximum Spawned Jobs} -The Job resource now permits specifying a number of {\bf Maximum Spawn -Jobs}. The default is 300. This directive can be useful if you have +The Job resource now permits specifying a number of {\bf Maximum Spawned +Jobs}. The default is 600. This directive can be useful if you have big hardware and you do a lot of Migration/Copy jobs which start at the same time. In prior versions of Bacula, Migration/Copy was limited to spawning a maximum of 100 jobs at a time.