-\chapter{Enterprise Bacula New Features}
-This chapter presents the new features that will be added to the
-current Enterprise version of Bacula that is under development. This version will be
-released at some later date, probably near the end of June 2011. These
-features are available only with a Bacula Systems subscription.
+\chapter{New Features in Bacula Enterprise}
+This chapter presents the new features that have been added to the
+current Enterprise version of Bacula.
+These features are available only with a Bacula Systems subscription.
In addition to the features in this chapter, the Enterprise version
will include the Community features described in the Community new Features
chapter.
-\section{Restart Incomplete Job}
+\section{Bacula Enterprise 6.0.6}
+\subsection{Incremental Accelerator Plugin for NetApp}
-\medskip
-This project was funded by Bacula Systems and is available with Bacula
+The Incremental Accelerator for NetApp Plugin is designed to simplify the
+backup and restore procedure of your NetApp NAS hosting a huge number of files.
+
+\smallskip{} When using the NetApp HFC Plugin, Bacula Enterprise will query the
+NetApp device to get the list of all files modified since the last backup
+instead of having to walk through the entire filesystem. Once Bacula have the
+list of all files to back's up, it will use a standard network share (such as
+NFS or CIFS) to access files.
+
+\smallskip
+This project was funded by Bacula Systems and is available with the Bacula
Enterprise Edition.
-\section{Support for MSSQL Block Level Backup}
+\subsection{PostgreSQL Plugin}
+
+The PostgreSQL plugin is designed to simplify the backup and restore procedure
+of your PostgreSQL cluster, the backup administrator doesn't need to learn about
+internals of Postgres backup techniques or write complex scripts. The plugin
+will automatically take care for you to backup essential information such as
+configuration, users definition or tablespaces. The PostgreSQL plugin supports
+both dump and Point In Time Recovery (PITR) backup techniques.
+\smallskip
This project was funded by Bacula Systems and is available with the Bacula
Enterprise Edition.
-\section{Support for NDMP protocol}
+\subsection{Maximum Reload Requests}
-The new \texttt{ndmp} Plugin is able to backup a NAS through NDMP protocol
-using \textbf{Filer to server} approach, where the Filer is backing up across
-the LAN to your Bacula server.
+The new Director directive \texttt{Maximum Reload Requests} permits to
+configure the number of reload requests that can be done while jobs are
+running.
-Accurate option should be turned on in the Job resource.
-\begin{verbatim}
-Job {
- Accurate = yes
- FileSet = NDMPFS
- ...
-}
+\begin{lstlisting}
+Director {
+ Name = localhost-dir
+ Maximum Reload Requests = 64
+ ...
-FileSet {
- Name = NDMPFS
- ...
- Include {
- Plugin = "ndmp:host=nasbox user=root pass=root file=/vol/vol1"
- }
}
-\end{verbatim}
+\end{lstlisting}
+
+\subsection{FD Storage Address}
+
+When the Director is behind a NAT, in a WAN area, to connect to
+% the FileDaemon or
+the StorageDaemon, the Director uses an ``external'' ip address,
+and the FileDaemon should use an ``internal'' IP address to contact the
+StorageDaemon.
+
+The normal way to handle this situation is to use a canonical name such as
+``storage-server'' that will be resolved on the Director side as the WAN address
+and on the Client side as the LAN address. This is now possible to configure
+this parameter using the new \texttt{FDStorageAddress} Storage
+% or Client
+directive.
+
+\bsysimageH{BackupOverWan1}{Backup Over WAN}{figbs6:fdstorageaddress}
+% \label{fig:fdstorageaddress}
+
+\begin{lstlisting}
+Storage {
+ Name = storage1
+ Address = 65.1.1.1
+ FD Storage Address = 10.0.0.1
+ SD Port 9103
+ ...
+}
+\end{lstlisting}
+
+% # or in the Client resouce
+%
+% Client {
+% Name = client1
+% Address = 65.1.1.2
+% FD Storage Address = 10.0.0.1
+% FD Port 9103
+% ...
+% }
+% \end{lstlisting}
+%
+% Note that using the Client \texttt{FDStorageAddress} directive will not allow
+% to use multiple Storage Daemon, all Backup or Restore requests will be sent to
+% the specified \texttt{FDStorageAddress}.
+
+\subsection{Maximum Concurrent Read Jobs}
+This is a new directive that can be used in the {\bf bacula-dir.conf} file
+in the Storage resource. The main purpose is to limit the number
+of concurrent Copy, Migration, and VirtualFull jobs so that
+they don't monopolize all the Storage drives causing a deadlock situation
+where all the drives are allocated for reading but none remain for
+writing. This deadlock situation can occur when running multiple
+simultaneous Copy, Migration, and VirtualFull jobs.
+
+\smallskip
+The default value is set to 0 (zero), which means there is no
+limit on the number of read jobs. Note, limiting the read jobs
+does not apply to Restore jobs, which are normally started by
+hand. A reasonable value for this directive is one half the number
+of drives that the Storage resource has rounded down. Doing so,
+will leave the same number of drives for writing and will generally
+avoid over commiting drives and a deadlock.
+
+
+\section{Bacula Enterprise 6.0.4}
+
+\subsection{VMWare vSphere VADP Plugin}
+
+The Bacula Enterprise vSphere plugin provides virtual
+machine bare metal recovery, while the backup at the guest level simplify data
+protection of critical applications.
+
+The plugin integrates the VMware's Changed Block Tracking (CBT) technology to
+ensure only blocks that have changed since the initial Full, and/or the last
+Incremental or Differential Backup are sent to the current Incremental or
+Differential backup stream to give you more efficient backups and reduced
+network load.
+
+\subsection{Oracle RMAN Plugin}
+
+The Bacula Enterprise Oracle Plugin is designed to simplify the backup and
+restore procedure of your Oracle Database instance, the backup administrator
+don't need to learn about internals of Oracle backup techniques or write
+complex scripts. The Bacula Enterprise Oracle plugin supports both dump and
+Point In Time Recovery (PITR) with RMAN backup techniques.
+
+\section{Bacula Enterprise 6.0.0}
+
+\subsection{Incomplete Jobs}
+During a backup, if the Storage daemon experiences disconnection
+with the File daemon during backup (normally a comm line problem
+or possibly an FD failure), under conditions that the SD determines
+to be safe it will make the failed job as Incomplete rather than
+failed. This is done only if there is sufficient valid backup
+data that was written to the Volume. The advantage of an Incomplete
+job is that it can be restarted by the new bconsole {\bf restart}
+command from the point where it left off rather than from the
+beginning of the jobs as is the case with a cancel.
+
+\subsection{The Stop Command}
+Bacula has been enhanced to provide a {\bf stop} command,
+very similar to the {\bf cancel} command with the main difference
+that the Job that is stopped is marked as Incomplete so that
+it can be restarted later by the {\bf restart} command where
+it left off (see below). The {\bf stop} command with no
+arguments, will like the cancel command, prompt you with the
+list of running jobs allowing you to select one, which might
+look like the following:
+
+\begin{lstlisting}
+*stop
+Select Job:
+ 1: JobId=3 Job=Incremental.2012-03-26_12.04.26_07
+ 2: JobId=4 Job=Incremental.2012-03-26_12.04.30_08
+ 3: JobId=5 Job=Incremental.2012-03-26_12.04.36_09
+Choose Job to stop (1-3): 2
+2001 Job "Incremental.2012-03-26_12.04.30_08" marked to be stopped.
+3000 JobId=4 Job="Incremental.2012-03-26_12.04.30_08" marked to be stopped.
+\end{lstlisting}
+
+\subsection{The Restart Command}
+The new {\bf Restart command} allows console users to restart
+a canceled, failed, or incomplete Job. For canceled and failed
+Jobs, the Job will restart from the beginning. For incomplete
+Jobs the Job will restart at the point that it was stopped either
+by a stop command or by some recoverable failure.
+
+\smallskip
+If you enter the {\bf restart} command in bconsole, you will get the
+following prompts:
+
+\begin{lstlisting}
+*restart
+You have the following choices:
+ 1: Incomplete
+ 2: Canceled
+ 3: Failed
+ 4: All
+Select termination code: (1-4):
+\end{lstlisting}
+
+If you select the {\bf All} option, you may see something like:
+
+\begin{lstlisting}
+Select termination code: (1-4): 4
++-------+-------------+---------------------+------+-------+----------+-----------+-----------+
+| jobid | name | starttime | type | level | jobfiles |
+jobbytes | jobstatus |
++-------+-------------+---------------------+------+-------+----------+-----------+-----------+
+| 1 | Incremental | 2012-03-26 12:15:21 | B | F | 0 |
+ 0 | A |
+| 2 | Incremental | 2012-03-26 12:18:14 | B | F | 350 |
+4,013,397 | I |
+| 3 | Incremental | 2012-03-26 12:18:30 | B | F | 0 |
+ 0 | A |
+| 4 | Incremental | 2012-03-26 12:18:38 | B | F | 331 |
+3,548,058 | I |
++-------+-------------+---------------------+------+-------+----------+-----------+-----------+
+Enter the JobId list to select:
+\end{lstlisting}
+
+Then you may enter one or more JobIds to be restarted, which may
+take the form of a list of JobIds separated by commas, and/or JobId
+ranges such as {\bf 1-4}, which indicates you want to restart JobIds
+1 through 4, inclusive.
+
+\subsection{Support for Exchange Incremental Backups}
+The Bacula Enterprise version 6.0 VSS plugin now supports
+Full and Incremental backups for Exchange. We strongly
+recommend that you do not attempt to run Differential jobs with
+Exchange as it is likely to produce a situation where restores
+will no longer select the correct jobs, and thus the
+Windows Exchange VSS writer will fail when applying log files.
+There is a Bacula Systems Enterprise white paper that provides
+the details of backup and restore of Exchange 2010 with the
+Bacula VSS plugin.
+
+\smallskip
+Restores can be done while Exchange is running, but you
+must first unmount (dismount in Microsoft terms) any database
+you wish to restore and explicitly mark them to permit a
+restore operation (see the white paper for details).
+
+\smallskip
+This project was funded by Bacula Systems and is available with the Bacula
+Enterprise Edition.
+\subsection{Support for MSSQL Block Level Backups}
+With the addition of block level backup support to the
+Bacula Enterprise VSS MSSQL component, you can now do
+Differential backups in addition to Full backups.
+Differential backups use Microsoft's partial block backup
+(a block differencing or deduplication that we call Delta).
+This partial block backup permits backing up only those
+blocks that have changed. Database restores can be made while
+the MSSQL server is running, but any databases selected for
+restore will be automatically taken offline by the MSSQL
+server during the restore process.
+
+Incremental backups for MSSQL are not support by
+Microsoft. We strongly recommend that you not perform Incremental
+backups with MSSQL as they will probably produce a situation
+where restore will no longer work correctly.
+
+\smallskip
+ We are currently working on producing a white paper that will give more
+details of backup and restore with MSSQL. One point to note is that during
+a restore, you will normally not want to restore the {\bf master} database.
+You must exclude it from the backup selections that you have made or the
+restore will fail.
+
+\smallskip
+It is possible to restore the {\bf master} database, but you must
+first shutdown the MSSQL server, then you must perform special
+recovery commands. Please see Microsoft documentation on how
+to restore the master database.
+
+\smallskip
This project was funded by Bacula Systems and is available with the Bacula
Enterprise Edition.
-\section{Job Bandwidth Limitation}
+
+\subsection{Job Bandwidth Limitation}
The new {\bf Job Bandwidth Limitation} directive may be added to the File
daemon's and/or Director's configuration to limit the bandwidth used by a Job
that File daemon, or it can be set for each Job in the Director's conf file.
For example:
-\begin{verbatim}
+\begin{lstlisting}
FileDaemon {
Name = localhost-fd
Working Directory = /some/path
...
Maximum Bandwidth Per Job = 5Mb/s
}
-\end{verbatim}
+\end{lstlisting}
The above example would cause any jobs running with the FileDaemon to not
exceed 5Mb/s of throughput when sending data to the Storage Daemon.
You can specify the speed parameter in k/s, Kb/s, m/s, Mb/s.
For example:
-\begin{verbatim}
+\begin{lstlisting}
Job {
Name = locahost-data
FileSet = FS_localhost
Maximum Bandwidth = 5Mb/s
...
}
-\end{verbatim}
+\end{lstlisting}
The above example would cause Job \texttt{localhost-data} to not exceed 5MB/s
of throughput when sending data from the File daemon to the Storage daemon.
A new console command \texttt{setbandwidth} permits to set dynamically the
maximum throughput of a running Job or for future jobs of a Client.
-\begin{verbatim}
+\begin{lstlisting}
* setbandwidth limit=1000000 jobid=10
-\end{verbatim}
+\end{lstlisting}
The \texttt{limit} parameter is in Kb/s.
This project was funded by Bacula Systems and is available in
the Enterprise Edition.
-\section{Incremental/Differential Block Level Difference Backup}
+\subsection{Incremental/Differential Block Level Difference Backup}
The new \texttt{delta} Plugin is able to compute and apply signature-based file
differences. It can be used to backup only changes in a big binary file like
-Outlook PST, VirtualBox/VmWare images or database files.
+Outlook PST, VirtualBox/VMware images or database files.
It supports both Incremental and Differential backups and stores signatures
database in the File Daemon working directory. This plugin is available on all
platform including Windows 32 and 64bit.
Accurate option should be turned on in the Job resource.
-\begin{verbatim}
+\begin{lstlisting}
Job {
Accurate = yes
FileSet = DeltaFS
Name = DeltaFS
...
Include {
+ # Specify one file
Plugin = "delta:/home/eric/.VirtualBox/HardDisks/lenny-i386.vdi"
}
}
-\end{verbatim}
-Note: the core code features to implement this functionality are included
-in the Community plugin interface as will as the core Community code.
+FileSet {
+ Name = DeltaFS-Include
+ ...
+ Include {
+ Options {
+ Compression = GZIP1
+ Signature = MD5
+ Plugin = delta
+ }
+ # Use the Options{} filtering and options
+ File = /home/user/.VirtualBox
+ }
+}
+\end{lstlisting}
+
+Please contact Bacula Systems support to get Delta Plugin specific
+documentation.
+
+\medskip
This project was funded by Bacula Systems and is available with the Bacula
Enterprise Edition.
-\section{Include All Windows Drives in FileSet}
+\subsection{SAN Shared Tape Storage Plugin}
-The \texttt{alldrives} Windows Plugin allows you to include all local drives
-with a simple directive. This plugin is available for Windows 64 and 32 bit.
+The problem with backing up multiple servers at the same time to the
+same tape library (or autoloader) is that if both servers access the
+same tape drive same time, you will very likely get data corruption.
+This is where the Bacula Systems shared tape storage plugin comes into play. The
+plugin ensures that only one server at a time can connect to each device
+(tape drive) by using the SPC-3 SCSI reservation protocol. Please contact
+Bacula Systems support to get SAN Shared Storage Plugin specific
+documentation.
-\begin{verbatim}
-FileSet {
- Name = EverythingFS
- ...
- Include {
- Plugin = "alldrives"
- }
-}
-\end{verbatim}
+\medskip
+This project was funded by Bacula Systems and is available with Bacula
+Enterprise Edition.
+
+\subsection{Advanced Autochanger Usage}
+
+The new \texttt{Shared Storage} Director's directive is a Bacula Enterprise
+feature that allows you to share volumes between different Storage
+resources. This directive should be used \textbf{only} if all \texttt{Media
+ Type} are correctly set across all Devices.
+
+The \texttt{Shared Storage} directive should be used when using the SAN
+Shared Storage plugin or when accessing from the Director Storage resources
+directly to Devices of an Autochanger.
+
+When sharing volumes between different Storage resources, you will
+need also to use the \texttt{reset-storageid} script before using the
+\texttt{update slots} command. This script can be scheduled once a day in
+an Admin job.
-You exclude some specific drives with the \texttt{exclude} option.
+\begin{lstlisting}
+ $ /opt/bacula/scripts/reset-storageid MediaType StorageName
+ $ bconsole
+ * update slots storage=StorageName drive=0
+\end{lstlisting}
-\begin{verbatim}
+Please contact Bacula Systems support to get help on this advanced
+configuration.
+
+\medskip
+This project was funded by Bacula Systems and is available with Bacula
+Enterprise Edition.
+
+\subsection{Enhancement of the NDMP Plugin}
+
+The previous NDMP Plugin 4.0 was fully supporting only the NetApp hardware, the
+new NDMP Plugin should now be able to support all NAS vendors with the
+\texttt{volume\_format} plugin command option.
+
+On some NDMP devices such as Celera or Blueray, the administrator can use arbitrary
+volume structure name, ex:
+
+\begin{lstlisting}
+ /dev/volume_home
+ /rootvolume/volume_tmp
+ /VG/volume_var
+\end{lstlisting}
+
+The NDMP plugin should be aware of the structure organization in order to
+detect if the administrator wants to restore in a new volume
+(\texttt{where=/dev/vol\_tmp}) or inside a subdirectory of the targeted volume
+(\texttt{where=/tmp}).
+
+\begin{lstlisting}
FileSet {
- Name = EverythingFS
+ Name = NDMPFS
...
Include {
- Plugin = "alldrives: exclude=D,E"
+ Plugin = "ndmp:host=nasbox user=root pass=root file=/dev/vol1 volume_format=/dev/"
}
}
-\end{verbatim}
+\end{lstlisting}
+Please contact Bacula Systems support to get NDMP Plugin specific
+documentation.
+\medskip
This project was funded by Bacula Systems and is available with the Bacula
Enterprise Edition.
-
-\section{Always Backup a File}
+\subsection{Always Backup a File}
When the Accurate mode is turned on, you can decide to always backup a file
-by using the following option:
+by using then new {\bf A} Accurate option in your FileSet. For example:
-\begin{verbatim}
+\begin{lstlisting}
Job {
Name = ...
FileSet = FS_Example
}
...
}
-\end{verbatim}
+\end{lstlisting}
This project was funded by Bacula Systems based on an idea of James Harper and
is available with the Bacula Enterprise Edition.
-\section{Setting Accurate Mode During at Runtime}
+\subsection{Setting Accurate Mode During at Runtime}
You are now able to specify the Accurate mode on the \texttt{run} command and
in the Schedule resource.
-\begin{verbatim}
+\begin{lstlisting}
* run accurate=yes job=Test
-\end{verbatim}
+\end{lstlisting}
-\begin{verbatim}
+\begin{lstlisting}
Schedule {
Name = WeeklyCycle
Run = Full 1st sun at 23:05
Run = Differential accurate=yes 2nd-5th sun at 23:05
Run = Incremental accurate=no mon-sat at 23:05
}
-\end{verbatim}
+\end{lstlisting}
It can allow you to save memory and and CPU resources on the catalog server in
some cases.
+\medskip
These advanced tuning options are available with the Bacula Enterprise Edition.
+
+% Common with community
+\subsection{Additions to RunScript variables}
+You can have access to JobBytes, JobFiles and Director name using \%b, \%F and \%D
+in your runscript command. The Client address is now available through \%h.
+
+\begin{lstlisting}
+RunAfterJob = "/bin/echo Job=%j JobBytes=%b JobFiles=%F ClientAddress=%h Dir=%D"
+\end{lstlisting}
+
+\subsection{LZO Compression}
+
+LZO compression was added in the Unix File Daemon. From the user point of view,
+it works like the GZIP compression (just replace {\bf compression=GZIP} with
+{\bf compression=LZO}).
+
+For example:
+\begin{lstlisting}
+Include {
+ Options { compression=LZO }
+ File = /home
+ File = /data
+}
+\end{lstlisting}
+
+LZO provides much faster compression and decompression speed but lower
+compression ratio than GZIP. It is a good option when you backup to disk. For
+tape, the built-in compression may be a better option.
+
+LZO is a good alternative for GZIP1 when you don't want to slow down your
+backup. On a modern CPU it should be able to run almost as fast as:
+
+\begin{bsysitemize}
+\item your client can read data from disk. Unless you have very fast disks like
+ SSD or large/fast RAID array.
+\item the data transfers between the file daemon and the storage daemon even on
+ a 1Gb/s link.
+\end{bsysitemize}
+
+Note that bacula only use one compression level LZO1X-1.
+
+\medskip
+The code for this feature was contributed by Laurent Papier.
+
+\subsection{New Tray Monitor}
+
+Since the old integrated Windows tray monitor doesn't work with
+recent Windows versions, we have written a new Qt Tray Monitor that is available
+for both Linux and Windows. In addition to all the previous features,
+this new version allows you to run Backups from
+the tray monitor menu.
+
+%\begin{figure}[htbp]
+% \centering
+% \includegraphics[width=10cm]{\idir
+\bsysimageH{tray-monitor}{New tray monitor}{figbs6:traymonitor}
+% \label{fig:traymonitor}
+% \caption{New tray monitor}
+%\end{figure}
+
+%\begin{figure}[htbp]
+% \centering
+\bsysimageH{tray-monitor1}{Run a Job through the new tray monitor}{figbs6:traymonitor1}
+% \includegraphics[width=10cm]{\idir tray-monitor1}
+% \label{fig:traymonitor1}
+% \caption{Run a Job through the new tray monitor}
+%\end{figure}
+
+
+To be able to run a job from the tray monitor, you need to
+allow specific commands in the Director monitor console:
+\begin{lstlisting}
+Console {
+ Name = win2003-mon
+ Password = "xxx"
+ CommandACL = status, .clients, .jobs, .pools, .storage, .filesets, .messages, run
+ ClientACL = *all* # you can restrict to a specific host
+ CatalogACL = *all*
+ JobACL = *all*
+ StorageACL = *all*
+ ScheduleACL = *all*
+ PoolACL = *all*
+ FileSetACL = *all*
+ WhereACL = *all*
+}
+\end{lstlisting}
+
+\medskip
+This project was funded by Bacula Systems and is available with Bacula
+the Enterprise Edition and the Community Edition.
+
+\subsection{Purge Migration Job}
+
+The new {\bf Purge Migration Job} directive may be added to the Migration
+Job definition in the Director's configuration file. When it is enabled
+the Job that was migrated during a migration will be purged at
+the end of the migration job.
+
+For example:
+\begin{lstlisting}
+Job {
+ Name = "migrate-job"
+ Type = Migrate
+ Level = Full
+ Client = localhost-fd
+ FileSet = "Full Set"
+ Messages = Standard
+ Storage = DiskChanger
+ Pool = Default
+ Selection Type = Job
+ Selection Pattern = ".*Save"
+...
+ Purge Migration Job = yes
+}
+\end{lstlisting}
+
+\medskip
+
+This project was submitted by Dunlap Blake; testing and documentation was funded
+by Bacula Systems.
+
+\subsection{Changes in the Pruning Algorithm}
+
+We rewrote the job pruning algorithm in this version. Previously, in some users
+reported that the pruning process at the end of jobs was very long. It should
+not be longer the case. Now, Bacula won't prune automatically a Job if this
+particular Job is needed to restore data. Example:
+
+\begin{lstlisting}
+JobId: 1 Level: Full
+JobId: 2 Level: Incremental
+JobId: 3 Level: Incremental
+JobId: 4 Level: Differential
+.. Other incrementals up to now
+\end{lstlisting}
+
+In this example, if the Job Retention defined in the Pool or in the Client
+resource causes that Jobs with Jobid in 1,2,3,4 can be pruned, Bacula will
+detect that JobId 1 and 4 are essential to restore data at the current state
+and will prune only JobId 2 and 3.
+
+\texttt{Important}, this change affect only the automatic pruning step after a
+Job and the \texttt{prune jobs} Bconsole command. If a volume expires after the
+\texttt{VolumeRetention} period, important jobs can be pruned.
+
+\subsection{Ability to Verify any specified Job}
+You now have the ability to tell Bacula which Job should verify instead of
+automatically verify just the last one.
+
+This feature can be used with VolumeToCatalog, DiskToCatalog and Catalog level.
+
+To verify a given job, just specify the Job jobid in argument when starting the
+job.
+\begin{lstlisting}
+*run job=VerifyVolume jobid=1 level=VolumeToCatalog
+Run Verify job
+JobName: VerifyVolume
+Level: VolumeToCatalog
+Client: 127.0.0.1-fd
+FileSet: Full Set
+Pool: Default (From Job resource)
+Storage: File (From Job resource)
+Verify Job: VerifyVol.2010-09-08_14.17.17_03
+Verify List: /tmp/regress/working/VerifyVol.bsr
+When: 2010-09-08 14:17:31
+Priority: 10
+OK to run? (yes/mod/no):
+\end{lstlisting}
+
+\medskip
+This project was funded by Bacula Systems and is available with Bacula
+Enterprise Edition and Community Edition.
projects / bacula / docs / blobdiff