+\chapter{New Features in 9.0.0}
+Note: The first beta versions are released as version 7.9.0, and the first
+production release will be 9.0.0.
+
+\subsection{Maximum Virtual Full Interval Option}
+Two new director directives have been added:
+
+\begin{verbatim}
+ Max Virtual Full Interval
+and
+ Virtual Full Backup Pool
+\end(verbatime)
+
+The {\bf Max Virtual Full Interval} directive should behave similar to the
+{\bf Max Full Interval}, but for Virtual Full jobs. If Bacula sees that
+there has not been a Full backup in Max Virtual Full Interval time then it
+will upgrade the job to Virtual Full. If you have both {\bf Max Full
+Interval} and {\bf Max Virtual Full Interval} set then Max Full Interval
+should take precedence.
+
+The {\bf Virtual Full Backup Pool} directive allows one to change the pool
+as well. You probably want to use these two directives in
+conjunction with each other but that may depend on the specifics of one's
+setup. If you set the {\bf Max Full Interval} without setting {\bf Max
+Virtual Full Interval} then Bacula will use whatever the "default" pool is
+set to which is the same behavior as with the Max Full Interval.
+
+\subsection{Progressive Virtual Full}
+
+In Bacula version 9.0.0, we have added a new Directive named {\bf Backups To Keep} that
+permits you to implement Progressive Virtual Fulls within Bacula. Sometimes
+this feature is known as Incremental Forever with Consolidation.
+
+\smallskip
+
+\begin{figure}[htbp]
+ \centering
+ \includegraphics[width=.8\linewidth]{pvf-slidingbackups}
+ \caption{Backup Sequence Slides Forward One Day, Each Day}
+ \label{fig:slidingbackups}
+\end{figure}
+
+To implement the Progressive Virtual Full feature, simply add the
+{\bf Backups To Keep} directive to your Virtual Full backup Job resource.
+The value specified on the directive indicates the number of backup jobs
+that should not be merged into the Virtual Full (i.e. the number of backup
+jobs that should remain after the Virtual Full has completed. The default
+is zero, which reverts to a standard Virtual Full than consolidates all the
+backup jobs that it finds.
+
+\subsubsection{Backups To Keep Directive}
+The new {\bf BackupsToKeep} directive is specified in the Job Resource and
+has the form:
+
+\begin{verbatim}
+ Backups To Keep = 30
+\end{verbatim}
+
+where the value (30 in the above figure and example) is the number of
+backups to retain. When this directive is present during a Virtual Full
+(it is ignored for other Job types), it will look for the most recent Full
+backup that has more subsequent backups than the value specified. In the
+above example the Job will simply terminate unless there is a Full back
+followed by at least 31 backups of either level Differential or
+Incremental.
+
+\smallskip
+Assuming that the last Full backup is followed by 32 Incremental backups, a
+Virtual Full will be run that consolidates the Full with the first two
+Incrementals that were run after the Full. The result is that you will end
+up with a Full followed by 30 Incremental backups. The Job Resource
+in {\bf bacula-dir.conf} to accomplish this would be:
+
+\begin{verbatim}
+ Job {
+ Name = "VFull"
+ Type = Backup
+ Level = VirtualFull
+ Client = "my-fd"
+ File Set = "FullSet"
+ Accurate = Yes
+ Backups To Keep = 10
+ }
+\end{verbatim}
+
+\subsubsection{Delete Consolidated Jobs}
+The new directive {\bf Delete Consolidated Jobs} expects a {\bf yes}
+or {\bf no} value that if set to {\bf yes} will cause any old Job that is
+consolidated during a Virtual Full to be deleted. In the example above
+we saw that a Full plus one other job (either an Incremental or
+Differential) were consolidated into a new Full backup. The original Full
+plus the other Job consolidated will be deleted. The default value is
+{\bf no}.
+
+\subsubsection{Virtual Full Compatibility}
+Virtual Full as well as Progressive Virtual Full works with any
+standard backup Job.
+
+\smallskip
+However, it should be noted that Virtual Full jobs are not compatible with
+any plugins that you may be using.
+
+\subsection{TapeAlert Enhancements}
+There are some significant enhancements to the TapeAlert feature of Bacula.
+Several directives are used slightly differently, which unfortunately
+causes a compatibility problem with the old TapeAlert implementation.
+Consequently, if you are already using TapeAlert, you must modify your
+{\bf bacula-sd.conf} in order for Tape Alerts to work. See below
+for the details ...
+
+\subsubsection{What is New}
+First, you must define a \textbf{Alert Command} directive in the Device
+resource that calls the new \textbf{tapealert} script that is installed in
+the scripts directory (normally: /opt/bacula/scripts). It is defined as
+follows:
+
+\begin{verbatim}
+Device {
+ Name = ...
+ Archive Device = /dev/nst0
+ Alert Command = "/opt/bacula/scripts/tapealert %l"
+ Control Device = /dev/sg1 # must be SCSI ctl for /dev/nst0
+ ...
+}
+\end{verbatim}
+
+In addition the \textbf{Control Device} directive in the Storage Daemon's
+conf file must be specified in each Device resource to permit Bacula to
+detect tape alerts on a specific devices (normally only tape devices).
+
+Once the above mentioned two directives (Alert Command and Control Device)
+are in place in each of your Device resources, Bacula will check for tape
+alerts at two points:
+
+\begin{itemize}
+\item After the Drive is used and it becomes idle.
+\item After each read or write error on the drive.
+\end{itemize}
+
+At each of the above times, Bacula will call the new \textbf{tapealert}
+script, which uses the \textbf{tapeinfo} program. The tapeinfo utility is
+part of the apt sg3-utils and rpm sg3\_utils packages that must be
+installed on your systems. Then after each alert that Bacula finds for
+that drive, Bacula will emit a Job message that is either INFO, WARNING, or
+FATAL depending on the designation in the Tape Alert published by the T10
+Technical Committee on SCSI Storage Interfaces (www.t10.org). For the
+specification, please see: www.t10.org/ftp/t10/document.02/02-142r0.pdf
+
+\smallskip
+As a somewhat extreme example, if tape alerts 3, 5, and 39 are set, you
+will get the following output in your backup job.
+
+{\small
+ \begin{verbatim}
+ 17-Nov 13:37 rufus-sd JobId 1: Error: block.c:287
+ Write error at 0:17 on device "tape"
+ (/home/kern/bacula/k/regress/working/ach/drive0)
+ Vol=TestVolume001. ERR=Input/output error.
+
+ 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
+ Volume="TestVolume001" alert=3: ERR=The operation has stopped because
+ an error has occurred while reading or writing data which the drive
+ cannot correct. The drive had a hard read or write error
+
+ 17-Nov 13:37 rufus-sd JobId 1: Fatal error: Alert:
+ Volume="TestVolume001" alert=5: ERR=The tape is damaged or the drive
+ is faulty. Call the tape drive supplier helpline. The drive can no
+ longer read data from the tape
+
+ 17-Nov 13:37 rufus-sd JobId 1: Warning: Disabled Device "tape"
+ (/home/kern/bacula/k/regress/working/ach/drive0) due to tape alert=39.
+
+ 17-Nov 13:37 rufus-sd JobId 1: Warning: Alert: Volume="TestVolume001"
+ alert=39: ERR=The tape drive may have a fault. Check for availability
+ of diagnostic information and run extended diagnostics if applicable.
+ The drive may have had a failure which may be identified by stored
+ diagnostic information or by running extended diagnostics (eg Send
+ Diagnostic). Check the tape drive users manual for instructions on
+ running extended diagnostic tests and retrieving diagnostic data.
+
+ \end{verbatim}
+}
+
+Without the tape alert feature enabled, you would only get the first error
+message above, which is the error return Bacula received when it gets the
+error. Notice also, that in the above output the alert number 5 is a
+critical error, which causes two things to happen. First the tape drive is
+disabled, and second the Job is failed.
+
+\smallskip
+If you attempt to run another Job using the Device that has been disabled,
+you will get a message similar to the following:
+
+\begin{verbatim}
+17-Nov 15:08 rufus-sd JobId 2: Warning:
+ Device "tape" requested by DIR is disabled.
+\end{verbatim}
+
+and the Job may be failed if no other drive can be found.
+
+\smallskip
+Once the problem with the tape drive has been corrected, you can
+clear the tape alerts and re-enable the device with the Bacula bconsole
+command such as the following:
+
+\begin{verbatim}
+ enable Storage=Tape
+\end{verbatim}
+
+Note, when you enable the device, the list of prior tape alerts for that
+drive will be discarded.
+
+\smallskip
+Since is is possible to miss tape alerts, Bacula maintains a temporary list
+of the last 8 alerts, and each time Bacula calls the \textbf{tapealert}
+script, it will keep up to 10 alert status codes. Normally there will only
+be one or two alert errors for each call to the tapealert script.
+
+\smallskip
+Once a drive has one or more tape alerts, you can see them by using the
+bconsole status command as follows:
+\begin{verbatim}
+status storage=Tape
+\end{verbatim}
+which produces the following output:
+\begin{verbatim}
+Device Vtape is "tape" (/home/kern/bacula/k/regress/working/ach/drive0)
+mounted with:
+ Volume: TestVolume001
+ Pool: Default
+ Media type: tape
+ Device is disabled. User command.
+ Total Bytes Read=0 Blocks Read=1 Bytes/block=0
+ Positioned at File=1 Block=0
+ Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
+ alert=Hard Error
+ Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
+ alert=Read Failure
+ Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
+ alert=Diagnostics Required
+\end{verbatim}
+if you want to see the long message associated with each of the alerts,
+simply set the debug level to 10 or more and re-issue the status command:
+\begin{verbatim}
+setdebug storage=Tape level=10
+status storage=Tape
+\end{verbatim}
+\begin{verbatim}
+ ...
+ Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
+ flags=0x0 alert=The operation has stopped because an error has occurred
+ while reading or writing data which the drive cannot correct. The drive had
+ a hard read or write error
+ Critical Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001"
+ flags=0x0 alert=The tape is damaged or the drive is faulty. Call the tape
+ drive supplier helpline. The drive can no longer read data from the tape
+ Warning Alert: at 17-Nov-2016 15:08:01 Volume="TestVolume001" flags=0x1
+ alert=The tape drive may have a fault. Check for availability of diagnostic
+ information and run extended diagnostics if applicable. The drive may
+ have had a failure which may be identified by stored diagnostic information
+ or by running extended diagnostics (eg Send Diagnostic). Check the tape
+ drive users manual for instructions on running extended diagnostic tests
+ and retrieving diagnostic data.
+ ...
+\end{verbatim}
+The next time you \textbf{enable} the Device by either using
+\textbf{bconsole} or you restart the Storage Daemon, all the saved alert
+messages will be discarded.
+
+\subsubsection{Handling of Alerts}
+Tape Alerts numbered 7,8,13,14,20,22,52,53, and 54 will cause Bacula to
+disable the current Volume.
+
+\smallskip
+Tape Alerts numbered 14,20,29,30,31,38, and 39 will cause Bacula to disable
+the drive.
+
+\smallskip
+Please note certain tape alerts such as 14 have multiple effects (disable
+the Volume and disable the drive).
+
+\subsection{New Console ACL Directives}
+By default, if a Console ACL directive is not set, Bacula will assume that the
+ACL list is empty. If the current Bacula Director configuration uses restricted
+Consoles and allows restore jobs, it is mandatory to configure the new
+directives.
+
+\subsubsection{DirectoryACL}
+\index[dir]{Directive!DirectoryACL}
+
+This directive is used to specify a list of directories that can be
+accessed by a restore session. Without this directive, a restricted
+console cannot restore any file. Multiple directories names may be
+specified by separating them with commas, and/or by specifying multiple
+DirectoryACL directives. For example, the directive may be specified as:
+
+\footnotesize
+\begin{verbatim}
+ DirectoryACL = /home/bacula/, "/etc/", "/home/test/*"
+\end{verbatim}
+\normalsize
+
+With the above specification, the console can access the following
+directories:
+\begin{itemize}
+\item \texttt{/etc/password}
+\item \texttt{/etc/group}
+\item \texttt{/home/bacula/.bashrc}
+\item \texttt{/home/test/.ssh/config}
+\item \texttt{/home/test/Desktop/Images/something.png}
+\end{itemize}
+
+But not to the following files or directories:
+\begin{itemize}
+\item \texttt{/etc/security/limits.conf}
+\item \texttt{/home/bacula/.ssh/id\_dsa.pub}
+\item \texttt{/home/guest/something}
+\item \texttt{/usr/bin/make}
+\end{itemize}
+
+If a directory starts with a Windows pattern (ex: c:/), Bacula will
+automatically ignore the case when checking directory names.
+
+\subsection{New Bconsole ``list'' Command Behavior}
+
+The bconsole \texttt{list} commands can now be used safely from a
+restricted bconsole session. The information displayed will respect the
+ACL configured for the Console session. For example, if a restricted
+Console has access to JobA, JobB and JobC, information about JobD will not
+appear in the \texttt{list jobs} command.
+
+\subsection{New Console ACL Directives}
+\index[dir]{Directive!BackupClientACL}
+It is now possible to configure a restricted Console to distinguish Backup
+and Restore job permissions. The \texttt{BackupClientACL} can restrict
+backup jobs on a specific set of clients, while the
+\texttt{RestoreClientACL} can restrict restore jobs.
+
+{\small
+\begin{verbatim}
+# cat /opt/bacula/etc/bacula-dir.conf
+...
+
+Console {
+ Name = fd-cons # Name of the FD Console
+ Password = yyy
+...
+ ClientACL = localhost-fd # everything allowed
+ RestoreClientACL = test-fd # restore only
+ BackupClientACL = production-fd # backup only
+}
+\end{verbatim}
+}
+
+The \texttt{ClientACL} directive takes precedence over the
+\texttt{RestoreClientACL} and the \texttt{BackupClientACL}. In the Console
+resource resource above, it means that the bconsole linked to the Console{}
+named "fd-cons" will be able to run:
+
+\begin{itemize}
+\item backup and restore for ``localhost-fd''
+\item backup for ``production-fd''
+\item restore for ``test-fd''
+\end{itemize}
+
+At the restore time, jobs for client ``localhost-fd'', ``test-fd'' and
+``production-fd'' will be available.
+
+If \texttt{*all*} is set for \texttt{ClientACL}, backup and restore will be
+allowed for all clients, despite the use of \texttt{RestoreClientACL} or
+\texttt{"BackupClientACL}.
+
+\subsection{Client Initiated Backup}
+\label{sec:featurecib}
+A console program such as the new \texttt{tray-monitor} or
+\texttt{bconsole} can now be configured to connect a File Daemon. There
+are many new features available (see the New Tray Monitor section below),
+but probably the most important is the ability for the user to initiate a
+backup of his own machine. The connection established by the FD to the
+Director for the backup will be used by the Director for the backup, thus
+not only can clients (users) initiate backups, but a File Daemon that is
+NATed (cannot be reached by the Director) can now be backed up without
+using advanced tunneling techniques providing that the File Daemon can
+connect to the Director.
+
+\smallskip
+The flow of information is shown in the picture below:
+\bsysimageH{nat}{Client Initiated Backup Network Flow}{fig:nat3}
+
+\newpage
+\subsection{Configuring Client Initiated Backup}
+\smallskip
+In order to ensure security, there are a number of new directives
+that must be enabled in the new \texttt{tray-monitor}, the File
+Daemon and in the Director.
+A typical configuration might look like the following:
+
+{\small
+\begin{verbatim}
+# cat /opt/bacula/etc/bacula-dir.conf
+...
+
+Console {
+ Name = fd-cons # Name of the FD Console
+ Password = yyy
+
+ # These commands are used by the tray-monitor, it is possible to restrict
+ CommandACL = run, restore, wait, .status, .jobs, .clients
+ CommandACL = .storages, .pools, .filesets, .defaults, .estimate
+
+ # Adapt for your needs
+ jobacl = *all*
+ poolacl = *all*
+ clientacl = *all*
+ storageacl = *all*
+ catalogacl = *all*
+ filesetacl = *all*
+}
+\end{verbatim}
+}
+
+{\small
+\begin{verbatim}
+# cat /opt/bacula/etc/bacula-fd.conf
+...
+
+Console { # Console to connect the Director
+ Name = fd-cons
+ DIRPort = 9101
+ address = localhost
+ Password = "yyy"
+}
+
+Director {
+ Name = remote-cons # Name of the tray monitor/bconsole
+ Password = "xxx" # Password of the tray monitor/bconsole
+ Remote = yes # Allow to use send commands to the Console defined
+}
+\end{verbatim}
+}
+
+{\small
+\begin{verbatim}
+cat /opt/bacula/etc/bconsole-remote.conf
+....
+
+Director {
+ Name = localhost-fd
+ address = localhost # Specify the FD address
+ DIRport = 9102 # Specify the FD Port
+ Password = "notused"
+}
+
+Console {
+ Name = remote-cons # Name used in the auth process
+ Password = "xxx"
+}
+\end{verbatim}
+}
+
+{\small
+\begin{verbatim}
+cat ~/.bacula-tray-monitor.conf
+Monitor {
+ Name = remote-cons
+}
+
+Client {
+ Name = localhost-fd
+ address = localhost # Specify the FD address
+ Port = 9102 # Specify the FD Port
+ Password = "xxx"
+ Remote = yes
+}
+\end{verbatim}
+}
+
+\bsysimageH{conf-nat}{Relation Between Resources (bconsole)}{fig:nat}
+\bsysimageH{conf-nat2}{Relation Between Resources (tray-monitor)}{fig:nat2}
+
+\medskip
+A more detailed description with complete examples is available in
+chapter~\ref{TrayMonitorChapter}.
+
+\subsection{New Tray Monitor}
+
+A new tray monitor has been added to the 9.0 release, the tray monitor offers
+the following features:
+
+\begin{itemize}
+\item Director, File and Storage Daemon status page
+\item Support for the Client Initiated Backup protocol (See
+ \vref{sec:featurecib}). To use the Client Initiated Backup option from the
+ tray monitor, the Client option ``Remote'' should be checked in the
+ configuration (Fig \vref{fig:tray2}).
+\item Wizard to run new job (Fig \vref{fig:tray4})
+\item Display an estimation of the number of files and the size of the next
+ backup job (Fig \vref{fig:tray4})
+\item Ability to configure the tray monitor configuration file directly from
+ the GUI (Fig \vref{fig:tray2})
+\item Ability to monitor a component and adapt the tray monitor task bar icon
+ if a jobs are running.
+\item TLS Support
+\item Better network connection handling
+\item Default configuration file is stored under \texttt{\$HOME/.bacula-tray-monitor.conf}
+\item Ability to ``schedule'' jobs
+\item Available on Linux and Windows platforms
+\end{itemize}
+
+% \medskip
+% Please see chapter \ref{TrayMonitorChapter} for more details about this new
+% functionality.
+
+
+\begin{figure}[htbp]
+ \centering
+ \includegraphics[width=0.8\linewidth]{tray-monitor-status}
+ \caption{Tray Monitor Status}
+ \label{fig:tray0}
+\end{figure}
+
+\begin{figure}[htbp]
+ \centering
+ \includegraphics[width=0.9\linewidth]{tray-monitor-conf-fd}
+ \caption{Tray Monitor Client Configuration}
+ \label{fig:tray2}
+\end{figure}
+
+\begin{figure}[htbp]
+ \centering
+ \includegraphics[width=0.8\linewidth]{tray-monitor-run1}
+ \smallskip
+ \includegraphics[width=0.8\linewidth]{tray-monitor-run2}
+ \caption{Tray Monitor Run a Job}
+ \label{fig:tray4}
+\end{figure}
+
+\subsection{Schedule Jobs via the Tray Monitor}
+
+The Tray Monitor can scan periodically a specific directory ``Command
+Directory'' and process ``*.bcmd'' files to find jobs to run.
+
+The format of the ``file.bcmd'' command file is the following:
+\begin{verbatim}
+<component name>:<run command>
+<component name>:<run command>
+...
+
+<component name> = string
+<run command> = string (bconsole command line)
+\end{verbatim}
+
+For example:
+\begin{verbatim}
+localhost-fd: run job=backup-localhost-fd level=full
+localhost-dir: run job=BackupCatalog
+\end{verbatim}
+
+The command file should contain at least one command. The component specified
+in the first part of the command line should be defined in the tray
+monitor. Once the command file is detected by the tray monitor, a popup is
+displayed to the user and it is possible for the user to cancel the job directly.
+
+\smallskip{}
+
+The file can be created with tools such as ``cron'' or the ``task scheduler''
+on Windows. It is possible to verify the network connection at that time to
+avoid network errors.
+
+\begin{verbatim}
+#!/bin/sh
+if ping -c 1 director &> /dev/null
+then
+ echo "my-dir: run job=backup" > /path/to/commands/backup.bcmd
+fi
+\end{verbatim}
+
+%\bsysimageH{tray-monitor-status}{Tray Monitor Status}{fig:tray0}
+%\bsysimageH{tray-monitor1}{Tray Monitor Configuration}{fig:tray1}
+%\bsysimageH{tray-monitor-conf-fd}{Tray Monitor Client Configuration}{fig:tray2}
+%\bsysimageH{tray-monitor-conf-dir}{Tray Monitor Director Configuration}{fig:tray3}
+%\bsysimageH{tray-monitor-run1}{Tray Monitor Run new Job}{fig:tray4}
+% find a way to group them together
+%\bsysimageH{tray-monitor-run2}{Tray Monitor Setup new Job}{fig:tray5}
+
+
+\subsection{Accurate Option for Verify ``Volume Data'' Job}
+
+Since Bacula version 8.4.1, it has been possible to have a Verify Job
+configured with \texttt{level=Data} that will reread all records from a job
+and optionally check the size and the checksum of all files. Starting with
+
+\smallskip
+Bacula version 9.0, it is now possible to use the \texttt{accurate} option to check
+catalog records at the same time. When using a Verify job with
+\texttt{level=Data} and \texttt{accurate=yes} can replace the
+\texttt{level=VolumeToCatalog} option.
+
+For more information on how to setup a Verify Data job, see
+\vref{label:verifyvolumedata}.
+
+To run a Verify Job with the \texttt{accurate} option, it is possible to set
+the option in the Job definition or set use the \texttt{accurate=yes} on the
+command line.
+
+\begin{verbatim}
+* run job=VerifyData jobid=10 accurate=yes
+\end{verbatim}
+
+\subsection{FileDaemon Saved Messages Resource Destination}
+
+It is now possible to send the list of all saved files to a Messages
+resource with the \texttt{saved} message type. It is not recommended to
+send this flow of information to the director and/or the catalog when the
+client FileSet is pretty large. To avoid side effects, the \texttt{all}
+keyword doesn't include the \texttt{saved} message type. The
+\texttt{saved} message type should be explicitely set.
+
+\begin{verbatim}
+# cat /opt/bacula/etc/bacula-fd.conf
+...
+Messages {
+ Name = Standard
+ director = mydirector-dir = all, !terminate, !restored, !saved
+ append = /opt/bacula/working/bacula-fd.log = all, saved, restored
+}
+\end{verbatim}
+
+\subsection{Minor Enhancements}
+
+\subsubsection{New Bconsole ".estimate" Command}
+
+The new \texttt{.estimate} command can be used to get statistics about a
+job to run. The command uses the database to approximate the size and the
+number of files of the next job. On a PostgreSQL database, the command
+uses regression slope to compute values. On SQLite or MySQL, where these
+statistical functions are not available, the command uses a simple
+``average'' estimation. The correlation number is given for each value.
+
+{\small
+\begin{verbatim}
+*.estimate job=backup
+level=I
+nbjob=0
+corrbytes=0
+jobbytes=0
+corrfiles=0
+jobfiles=0
+duration=0
+job=backup
+
+*.estimate job=backup level=F
+level=F
+nbjob=1
+corrbytes=0
+jobbytes=210937774
+corrfiles=0
+jobfiles=2545
+duration=0
+job=backup
+\end{verbatim}
+}
+
+\subsubsection{Traceback and Lockdump}
+
+After the reception of a signal, \texttt{traceback} and \texttt{lockdump}
+information are now stored in the same file.
+
+\subsection{Bconsole ``list jobs'' command options}
+
+The \texttt{list jobs} bconsole command now accepts new command line options:
+
+\begin{itemize}
+\item \textbf{joberrors} Display jobs with JobErrors
+\item \textbf{jobstatus=T} Display jobs with the specified status code
+\item \textbf{client=cli} Display jobs for a specified client
+\item \textbf{order=asc/desc} Change the output format of the job list. The
+ jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
+ descendant (desc) (default) value.
+\end{itemize}
+
+\subsection{Minor Enhancements}
+
+\subsubsection{New Bconsole "Tee All" Command}
+
+The ``@tall'' command allows logging all input/output from a console session.
+
+\begin{verbatim}
+*@tall /tmp/log
+*st dir
+...
+\end{verbatim}
+
+\subsection{Bconsole ``list jobs'' command options}
+
+The \texttt{list jobs} bconsole command now accepts new command line options:
+
+\begin{itemize}
+\item \textbf{joberrors} Display jobs with JobErrors
+\item \textbf{jobstatus=T} Display jobs with the specified status code
+\item \textbf{client=cli} Display jobs for a specified client
+\item \textbf{order=asc/desc} Change the output format of the job list. The
+ jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
+ descendant (desc) (default) value.
+\end{itemize}
+
+\subsection{New Bconsole "Tee All" Command}
+
+The ``@tall'' command allows logging all input/output from a console session.
+
+\begin{verbatim}
+*@tall /tmp/log
+*st dir
+...
+\end{verbatim}
+
+\subsection{New Job Edit Codes \%I}
+In various places such as RunScripts, you have now access to \%I to get the
+JobId of the copy or migration job started by a migrate job.
+
+\begin{verbatim}
+Job {
+ Name = Migrate-Job
+ Type = Migrate
+ ...
+ RunAfter = "echo New JobId is %I"
+}
+\end{verbatim}
+
+
+\subsection*{.api version 2}
+
+In Bacula version 9.0 and later, we introduced a new .api version
+to help external tools to parse various Bacula bconsole output.
+
+% waa - 20150317 - this section needs just a little more to explain what the "43" in "s43" mean. Perhaps
+% if it is not a good place to list the possibilities here, then list where a reference
+% is. Also, I think .api 2 ... Means "use API version 2" but that should be stated too
+
+The \texttt{api\_opts} option can use the following arguments:
+\begin{itemize}
+\item [C] Clear current options
+\item [tn] Use a specific time format (1 ISO format, 2 Unix Timestamp, 3 Default Bacula time format)
+\item [sn] Use a specific separator between items (new line by default).
+\item [Sn] Use a specific separator between objects (new line by default).
+\item [o] Convert all keywords to lowercase and convert all non \textsl{isalpha} characters to \_
+\end{itemize}
+
+% waa - 20150317 - I think there should either be more output listed here to give a better feeling
+% or, perhaps another output listing for different .status commands
+
+\begin{verbatim}
+ .api 2 api_opts=t1s43S35
+ .status dir running
+==================================
+jobid=10
+job=AJob
+...
+\end{verbatim}
+
+\subsection*{New Debug Options}
+
+In Bacula version 9.0 and later, we introduced a new \texttt{options} parameter for
+the \texttt{setdebug} bconsole command.
+
+\smallskip{}
+
+The following arguments to the new \texttt{option} parameter are available to control debug functions.
+
+\begin{itemize}
+\item [0] Clear debug flags
+\item [i] Turn off, ignore bwrite() errors on restore on File Daemon
+\item [d] Turn off decomp of BackupRead() streams on File Daemon
+\item [t] Turn on timestamps in traces
+\item [T] Turn off timestamps in traces
+
+% waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it" ???
+\item [c] Truncate trace file if trace file is activated
+
+\item [l] Turn on recoding events on P() and V()
+\item [p] Turn on the display of the event ring when doing a bactrace
+\end{itemize}
+
+\smallskip{}
+
+The following command will enable debugging for the File Daemon, truncate an existing trace file,
+and turn on timestamps when writing to the trace file.
+
+\begin{verbatim}
+* setdebug level=10 trace=1 options=ct fd
+\end{verbatim}
+
+\smallskip{}
+
+It is now possible to use a \textsl{class} of debug messages called \texttt{tags}
+to control the debug output of Bacula daemons.
+
+\begin{itemize}
+\item [all] Display all debug messages
+\item [bvfs] Display BVFS debug messages
+\item [sql] Display SQL related debug messages
+\item [memory] Display memory and poolmem allocation messages
+\item [scheduler] Display scheduler related debug messages
+\end{itemize}
+
+\begin{verbatim}
+* setdebug level=10 tags=bvfs,sql,memory
+* setdebug level=10 tags=!bvfs
+
+# bacula-dir -t -d 200,bvfs,sql
+\end{verbatim}
+
+The \texttt{tags} option is composed of a list of tags. Tags are separated by
+``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!''
+in front of the tag. Note that more tags are planned for future versions.
+
+%\LTXtable{\linewidth}{table_debugtags}
+
+\subsection{Communication Line Compression}
+Bacula version 9.0.0 and later now includes communication
+line compression. It is turned on by default, and if the
+two Bacula components (Dir, FD, SD, bconsole) are both
+version 6.6.0 or greater, communication line compression)
+will be enabled, by default. If for some reason, you do not want
+communication line compression, you may disable it with the
+following directive:
+
+\begin{verbatim}
+Comm Compression = no
+\end{verbatim}
+
+This directive can appear in the following resources:
+\begin{verbatim}
+bacula-dir.conf: Director resource
+bacula-fd.conf Client (or FileDaemon) resource
+bacula-sd.conf: Storage resource
+bconsole.conf: Console resource
+bat.conf: Console resource
+\end{verbatim}
+
+\smallskip
+In many cases, the volume of data transmitted across the
+communications line can be reduced by a factor of three when
+this directive is enabled (default) In the case that the compression is not
+effective, Bacula turns it off on a. record by record basis.
+
+\smallskip
+If you are backing up data that is already compressed the comm line
+compression will not be effective, and you are likely
+to end up with an average compression ratio that is very small.
+In this case, Bacula reports {\bf None} in the Job report.
+
+\subsection{Deduplication Optimized Volumes}
+This version of Bacula includes a new alternative (or additional)
+volume format that optimizes the placement of files so
+that an underlying deduplicating filesystem such as ZFS
+can optimally deduplicate the backup data that is written
+by Bacula. These are called Deduplication Optimized Volumes
+or Aligned Volumes for short. The details of how to use this
+feature and its considerations are in the
+Deduplication Optimized Volumes whitepaper.
+
+\smallskip
+This feature is available if you have Bacula Community produced binaries
+and the Aligned Volumes plugin.
+
+\subsection{New Message Identification Format}
+We are starting to add unique message indentifiers to each message (other
+than debug and the Job report) that Bacula prints. At the current time
+only two files in the Storage Daemon have these message identifiers and
+over time with subsequent releases we will modify all messages.
+
+\smallskip
+The message identifier will be kept unique for each message and once
+assigned to a message it will not change even if the text of the message
+changes. This means that the message identifier will be the same no matter
+what language the text is displayed in, and more importantly, it will allow
+us to make listing of the messages with in some cases, additional
+explanation or instructions on how to correct the problem. All this will
+take several years since it is a lot of work and requires some new programs
+that are not yet written to manage these message identifiers.
+
+\smallskip
+The format of the message identifier is:
+
+\begin{verbatim}
+ [AAnnnn]
+\end{verbatim}
+where A is an upper case character and nnnn is a four digit number, where
+the first charcter indicates the software component (daemon); the second
+letter indicates the severity, and the number is unique for a given
+componet and severity.
+
+\smallskip
+For example:
+
+\begin{verbatim}
+ [SF0001]
+\end{verbatim}
+
+The first charcter representing the componend at the current time one of
+the following:
+
+\begin{verbatim}
+ S Storage daemon
+ D Director
+ F File daemon
+\end{verbatim}
+
+\smallskip
+The second character representing the severity or level can be:
+
+\begin{verbatim}
+ A Abort
+ F Fatal
+ E Erropr
+ W Warning
+ S Security
+ I Info
+ D Debug
+ O OK (i.e. operation completed normallly)
+\end{verbatim}
+
+So in the example above [SF0001] indicates it is a message id, because of
+the brackets and because it is at the beginning of the message, and that
+it was generated by the Storage daemon as a fatal error.
+\smallskip
+As mentioned above it will take some time to implement these message ids
+everywhere, and over time we may add more component letters and more
+severity levels as needed.
+
+
+\chapter{New Features in 7.4.0}
+This chapter presents the new features that have been added to
+the various versions of Bacula.
+
+\section{New Features in 7.4.3}
+\subsection{RunScripts}
+There are two new RunScript short cut directives implemented in
+the Director. They are:
+
+\begin{verbatim}
+Job {
+ ...
+ ConsoleRunBeforeJob = "console-command"
+ ...
+}
+\end{verbatim}
+
+\begin{verbatim}
+Job {
+ ...
+ ConsoleRunAfterJob = "console-command"
+ ...
+}
+\end{verbatim}
+
+As with other RunScript commands, you may have multiple copies
+of either the {\bf ConsoleRunBeforeJob} or the {\bf ConsoleRunAfterJob}
+in the same Job resource definition.
+\smallskip
+Please note that not all console commands are permitted, and that
+if you run a console command that requires a response, the results
+are not determined (i.e. it will probably fail).
+
+
+
+\section{New Features in 7.4.0}
+\subsection{Verify Volume Data}
+
+It is now possible to have a Verify Job configured with \texttt{level=Data} to
+reread all records from a job and optionally check the size and the checksum
+of all files.
+
+\begin{verbatim}
+# Verify Job definition
+Job {
+ Name = VerifyData
+ Level = Data
+ Client = 127.0.0.1-fd # Use local file daemon
+ FileSet = Dummy # Will be adapted during the job
+ Storage = File # Should be the right one
+ Messages = Standard
+ Pool = Default
+}
+
+# Backup Job definition
+Job {
+ Name = MyBackupJob
+ Type = Backup
+ Client = windows1
+ FileSet = MyFileSet
+ Pool = 1Month
+ Storage = File
+}
+
+FileSet {
+ Name = MyFileSet
+ Include {
+ Options {
+ Verify = s5
+ Signature = MD5
+ }
+ File = /
+}
+\end{verbatim}
+
+To run the Verify job, it is possible to use the ``jobid'' parameter of the ``run'' command.
+
+\begin{verbatim}
+*run job=VerifyData jobid=10
+Run Verify Job
+JobName: VerifyData
+Level: Data
+Client: 127.0.0.1-fd
+FileSet: Dummy
+Pool: Default (From Job resource)
+Storage: File (From Job resource)
+Verify Job: MyBackupJob.2015-11-11_09.41.55_03
+Verify List: /opt/bacula/working/working/VerifyVol.bsr
+When: 2015-11-11 09:47:38
+Priority: 10
+OK to run? (yes/mod/no): yes
+Job queued. JobId=14
+
+...
+
+11-Nov 09:46 my-dir JobId 13: Bacula 7.4.0 (13Nov15):
+ Build OS: x86_64-unknown-linux-gnu archlinux
+ JobId: 14
+ Job: VerifyData.2015-11-11_09.46.29_03
+ FileSet: MyFileSet
+ Verify Level: Data
+ Client: 127.0.0.1-fd
+ Verify JobId: 10
+ Verify Job:q
+ Start time: 11-Nov-2015 09:46:31
+ End time: 11-Nov-2015 09:46:32
+ Files Expected: 1,116
+ Files Examined: 1,116
+ Non-fatal FD errors: 0
+ SD Errors: 0
+ FD termination status: Verify differences
+ SD termination status: OK
+ Termination: Verify Differences
+\end{verbatim}
+
+The current Verify Data implementation requires specifying the correct Storage
+resource in the Verify job. The Storage resource can be changed with the bconsole
+command line and with the menu.
+
+\subsection{Bconsole ``list jobs'' command options}
+
+The \texttt{list jobs} bconsole command now accepts new command line options:
+
+\begin{itemize}
+\item \textbf{joberrors} Display jobs with JobErrors
+\item \textbf{jobstatus=T} Display jobs with the specified status code
+\item \textbf{client=cli} Display jobs for a specified client
+\item \textbf{order=asc/desc} Change the output format of the job list. The
+ jobs are sorted by start time and JobId, the sort can use ascendant (asc) or
+ descendant (desc) (default) value.
+\end{itemize}
+
+\subsection{Minor Enhancements}
+
+\subsubsection{New Bconsole "Tee All" Command}
+
+The ``@tall'' command allows logging all input/output from a console session.
+
+\begin{verbatim}
+*@tall /tmp/log
+*st dir
+...
+\end{verbatim}
+
+\subsection{Windows Encrypted File System (EFS) Support}
+
+The Bacula Enterprise Windows File Daemon for the community version
+7.4.0 now automatically supports files and
+directories that are encrypted on Windows filesystem.
+
+\subsection{SSL Connections to MySQL}
+
+There are five new Directives for the Catalog resource in the
+{\bf bacula-dir.conf} file that you can use to encrypt the
+communications between Bacula and MySQL for additional
+security.
+
+\begin{description}
+\item [dbsslkey] takes a string variable that specifies the filename of an
+SSL key file.
+\item [dbsslcert] takes a string variable that specifies the filename of an
+SSL certificate file.
+\item [dbsslca] takes a string variable that specifies the filename of a
+SSL CA (certificate authority) certificate.
+\item [dbsslcipher] takes a string variable that specifies the cipher
+to be used.
+\end{description}
+
+\subsection{Max Virtual Full Interval}
+This is a new Job resource directive that specifies the time in seconds
+that is a maximum time between Virtual Full jobs. It is much like the
+Max Full Interval directive but applies to Virtual Full jobs rather
+that Full jobs.
+
+\subsection{New List Volumes Output}
+The {\bf list} and {\bf llist} commands have been modified so that when
+listing Volumes a new pseudo field {\bf expiresin} will be printed. This
+field is the number of seconds in which the retention period will expire.
+If the retention period has already expired the value will be zero. Any
+non-zero value means that the retention period is still in effect.
+
+An example with many columns shorted for display purpose is:
+
+\begin{verbatim}
+*list volumes
+Pool: Default
+*list volumes
+Pool: Default
++----+---------------+-----------+---------+-------------+-----------+
+| id | volumename | volstatus | enabled | volbytes | expiresin |
++----+---------------+-----------+---------+-------------+-----------+
+| 1 | TestVolume001 | Full | 1 | 249,940,696 | 0 |
+| 2 | TestVolume002 | Full | 1 | 249,961,704 | 1 |
+| 3 | TestVolume003 | Full | 1 | 249,961,704 | 2 |
+| 4 | TestVolume004 | Append | 1 | 127,367,896 | 3 |
++----+---------------+-----------+---------+-------------+-----------+
+\end{verbatim}
+
+%%
+%%
\chapter{New Features in 7.2.0}
This chapter presents the new features that have been added to
the various versions of Bacula.
also possible to manage Snapshots from Bacula's \texttt{bconsole} tool through a
unique interface.
+\subsubsection{Snapshot Backends}
+
+The following Snapshot backends are supported with Bacula Enterprise 8.2:
+
+\begin{itemize}
+\item BTRFS
+\item ZFS
+\item LVM\footnote{Some restrictions described in \vref{LVMBackend} applies to
+ the LVM backend}
+\end{itemize}
+
+By default, Snapshots are mounted (or directly available) under
+\textbf{.snapshots} directory on the root filesystem. (On ZFS, the default
+is \textbf{.zfs/snapshots}).
+
+\smallskip{}
+
+The Snapshot backend program is called \textbf{bsnapshot} and is available in
+the \textbf{bacula-enterprise-snapshot} package. In order to use the Snapshot
+Management feature, the package must be installed on the Client.
+
+\smallskip{}
+\label{bsnapshotconf}
+The \textbf{bsnapshot} program can be configured using
+\texttt{/opt/bacula/etc/bsnapshot.conf} file. The following parameters can
+be adjusted in the configuration file:
+
+\begin{itemize}
+\item \texttt{trace=<file>} Specify a trace file
+\item \texttt{debug=<num>} Specify a debug level
+\item \texttt{sudo=<yes/no>} Use sudo to run commands
+\item \texttt{disabled=<yes/no>} Disable snapshot support
+\item \texttt{retry=<num>} Configure the number of retries for some operations
+\item \texttt{snapshot\_dir=<dirname>} Use a custom name for the Snapshot directory. (\textbf{.SNAPSHOT}, \textbf{.snapdir}, etc...)
+\item \texttt{lvm\_snapshot\_size=<lvpath:size>} Specify a custom snapshot size for a given LVM volume
+\end{itemize}
+
+\begin{verbatim}
+# cat /opt/bacula/etc/bsnapshot.conf
+trace=/tmp/snap.log
+debug=10
+lvm_snapshot_size=/dev/ubuntu-vg/root:5%
+\end{verbatim}
+
+
\subsubsection{Application Quiescing}
When using Snapshots, it is very important to quiesce applications that are
\smallskip{}
-
-
In RunScripts, the \texttt{AfterSnapshot} keyword for the \texttt{RunsWhen} directive will
allow a command to be run just after the Snapshot creation. \texttt{AfterSnapshot} is a
synonym for the \texttt{AfterVSS} keyword.
}
\end{verbatim}
-\subsubsection*{New Option Letter ``M'' for Accurate Directive in FileSet}
+\subsubsection*{New Option Letter ``M'' for Accurate Directive in FileSet}
% waa - 20150317 - is 8.0.5 correct here?
Added in version 8.0.5, the new ``M'' option letter for the Accurate directive
}
\end{verbatim}
-\subsubsection*{New Debug Options}
-
-In Bacula Enterprise version 8.0 and later, we introduced a new \texttt{options} parameter for
-the \texttt{setdebug} bconsole command.
-
-\smallskip{}
-
-The following arguments to the new \texttt{option} parameter are available to control debug functions.
-
-\begin{itemize}
-\item [0] Clear debug flags
-\item [i] Turn off, ignore bwrite() errors on restore on File Daemon
-\item [d] Turn off decomp of BackupRead() streams on File Daemon
-\item [t] Turn on timestamps in traces
-\item [T] Turn off timestamps in traces
-
-% waa - 20150306 - does this "c" item mean to say "Truncate trace file if one exists, otherwise append to it" ???
-\item [c] Truncate trace file if trace file is activated
-
-\item [l] Turn on recoding events on P() and V()
-\item [p] Turn on the display of the event ring when doing a bactrace
-\end{itemize}
-
-\smallskip{}
-
-The following command will enable debugging for the File Daemon, truncate an existing trace file,
-and turn on timestamps when writing to the trace file.
-
-\begin{verbatim}
-* setdebug level=10 trace=1 options=ct fd
-\end{verbatim}
-
-\smallskip{}
-
-It is now possible to use a \textsl{class} of debug messages called \texttt{tags}
-to control the debug output of Bacula daemons.
-
-\begin{itemize}
-\item [all] Display all debug messages
-\item [bvfs] Display BVFS debug messages
-\item [sql] Display SQL related debug messages
-\item [memory] Display memory and poolmem allocation messages
-\item [scheduler] Display scheduler related debug messages
-\end{itemize}
-
-\begin{verbatim}
-* setdebug level=10 tags=bvfs,sql,memory
-* setdebug level=10 tags=!bvfs
-
-# bacula-dir -t -d 200,bvfs,sql
-\end{verbatim}
-
-The \texttt{tags} option is composed of a list of tags. Tags are separated by
-``,'' or ``+'' or ``-'' or ``!''. To disable a specific tag, use ``-'' or ``!''
-in front of the tag. Note that more tags are planned for future versions.
-
-%%\LTXtable{\linewidth}{table_debugtags}
-
\subsection{Read Only Storage Devices}
This version of Bacula allows you to define a Storage deamon device
to be read-only. If the {\bf Read Only} directive is specified and
Read Only = yes
\end{verbatim}
-\subsection{Catalog Performance Improvements}
-There is a new Bacula database format (schema) in this version
-of Bacula that eliminates the FileName table by placing the
-Filename into the File record of the File table.
-This substantiallly improves performance,
-particularly for large (1GB or greater) databases.
-
-% waa - 20150317 - Is 1GB _really_ considered to be a large database? Do we mean to say 100GB??
-
-The \texttt{update\_xxx\_catalog} script will automatically update the
-Bacula database format, but you should realize that for
-very large databases (greater than 1GB), it may take some
-time, and there are several different options for doing the
-update: 1. Shudown the database and update it. 2. Update the
-database while production jobs are running. See the Bacula Systems
-White Paper ``Migration-to-6.6'' on this subject.
-
-\smallskip
-This database format change can provide very significant improvements in
-the speed of metadata insertion into the database, and in some cases
-(backup of large email servers) can significantly reduce the size of the
-database.
-
-\subsection{New Truncate Command}
-We have added a new truncate command to bconsole which
-will truncate a volume if the volume is purged, and if
-the volume is also marked {\bf Action On Purge = Truncate}.
-This feature was originally added in Bacula version 5.0.1,
-but the mechanism for actually doing the truncate required
-the user to enter a complicated command such as:
-
-\begin{verbatim}
-purge volume action=truncate storage=File pool=Default
-\end{verbatim}
-
-The above command is now simplified to be:
-
-\begin{verbatim}
-truncate storage=File pool=Default
-\end{verbatim}
-
\subsection{New Resume Command}
The new \texttt{resume} command does exactly the same thing as a
{\bf restart} command, but for some users the
\subsection{Changes in the Pruning Algorithm}
-We rewrote the job pruning algorithm in this version. Previously, in some users
-reported that the pruning process at the end of jobs was very long. It should
-not be longer the case. Now, Bacula won't prune automatically a Job if this
-particular Job is needed to restore data. Example:
+We rewrote the job pruning algorithm in this version. Previously, in some
+users reported that the pruning process at the end of jobs was very long.
+It should not be longer the case. Now, Bacula won't prune automatically a
+Job if this particular Job is needed to restore data. Example:
\begin{verbatim}
JobId: 1 Level: Full
detect that JobId 1 and 4 are essential to restore data at the current state
and will prune only JobId 2 and 3.
-\texttt{Important}, this change affect only the automatic pruning step after a
-Job and the \texttt{prune jobs} Bconsole command. If a volume expires after the
-\texttt{VolumeRetention} period, important jobs can be pruned.
+\texttt{Important}, this change affect only the automatic pruning step
+after a Job and the \texttt{prune jobs} Bconsole command. If a volume
+expires after the \texttt{VolumeRetention} period, important jobs can be
+pruned.
\subsection{Ability to Verify any specified Job}
You now have the ability to tell Bacula which Job should verify instead of
projects / bacula / docs / blobdiff