-%%
-%%
-
\chapter{Basic Volume Management}
\label{DiskChapter}
\index[general]{Basic Volume Management}
Most of the concepts apply equally well to both tape and disk Volumes.
However, the chapter was originally written to explain backing up to disk, so
you will see it is slanted in that direction, but all the directives
-presented here apply equally well whether your volume is disk or tape.
+presented here apply equally well whether your volume is disk or tape.
If you have a lot of hard disk storage or you absolutely must have your
backups run within a small time window, you may want to direct Bacula to
backup to disk Volumes rather than tape Volumes. This chapter is intended to
give you some of the options that are available to you so that you can manage
-either disk or tape volumes.
+either disk or tape volumes.
\label{Concepts}
\section{Key Concepts and Resource Records}
rather easy. In the Storage daemon's configuration file, you simply define an
{\bf Archive Device} to be a directory. For example, if you want your disk
backups to go into the directory {\bf /home/bacula/backups}, you could use the
-following:
+following:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Device {
Name = FileBackup
Media Type = File
RemovableMedia = no;
AlwaysOpen = no;
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
Assuming you have the appropriate {\bf Storage} resource in your Director's
-configuration file that references the above Device resource,
+configuration file that references the above Device resource,
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Storage {
Name = FileStorage
Address = ...
Device = FileBackup
Media Type = File
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
Bacula will then write the archive to the file {\bf
another directory, you should not rename it or it will become unreadable by
Bacula. This is because each archive has the filename as part of the internal
label, and the internal label must agree with the system filename before
-Bacula will use it.
+Bacula will use it.
Although this is quite simple, there are a number of problems. The first is
that unless you specify otherwise, Bacula will always write to the same volume
\index[general]{Pool Options to Limit the Volume Usage }
Some of the options you have, all of which are specified in the Pool record,
-are:
+are:
-\begin{itemize}
+\begin{bsysitemize}
\item To write each Volume only once (i.e. one Job per Volume or file in this
case), use:
-{\bf UseVolumeOnce = yes}.
+{\bf UseVolumeOnce = yes}.
\item To write nnn Jobs to each Volume, use:
- {\bf Maximum Volume Jobs = nnn}.
+ {\bf Maximum Volume Jobs = nnn}.
\item To limit the maximum size of each Volume, use:
- {\bf Maximum Volume Bytes = mmmm}.
+ {\bf Maximum Volume Bytes = mmmm}.
Note, if you use disk volumes, with all versions up to and including
- 1.39.28, you should probably limit the Volume size to some reasonable
+ 1.39.28, you should probably limit the Volume size to some reasonable
value such as say 5GB. This is because during a restore, Bacula is
currently unable to seek to the proper place in a disk volume to restore
a file, which means that it must read all records up to where the
\item To limit the use time (i.e. write the Volume for a maximum of five days),
use:
-{\bf Volume Use Duration = ttt}.
-\end{itemize}
+{\bf Volume Use Duration = ttt}.
+\end{bsysitemize}
Note that although you probably would not want to limit the number of bytes on
a tape as you would on a disk Volume, the other options can be very useful in
limiting the time Bacula will use a particular Volume (be it tape or disk).
For example, the above directives can allow you to ensure that you rotate
-through a set of daily Volumes if you wish.
+through a set of daily Volumes if you wish.
As mentioned above, each of those directives is specified in the Pool or
Pools that you use for your Volumes. In the case of {\bf Maximum Volume Job},
changing the Pool will have no effect on existing Volumes. You can either
manually change the Volume values, or refresh them from the Pool defaults using
the {\bf update volume} command in the Console. As an example
-of the use of one of the above, suppose your Pool resource contains:
+of the use of one of the above, suppose your Pool resource contains:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Pool {
Name = File
Pool Type = Backup
Volume Use Duration = 23h
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
then if you run a backup once a day (every 24 hours), Bacula will use a new
Volume for each backup, because each Volume it writes can only be used for 23 hours
-after the first write. Note, setting the use duration to 23 hours is not a very
+after the first write. Note, setting the use duration to 23 hours is not a very
good solution for tapes unless you have someone on-site during the weekends,
because Bacula will want a new Volume and no one will be present to mount it,
so no weekend backups will be done until Monday morning.
needed. While, the automatic Volume labeling in version 1.30 and prior is a
bit simplistic, but it does allow for automation, the features added in
version 1.31 permit automatic creation of a wide variety of labels including
-information from environment variables and special Bacula Counter variables.
-In version 1.37 and later, it is probably much better to use Python scripting
+information from environment variables and special Bacula Counter variables.
+In version 1.37 and later, it is probably much better to use Python scripting
and the NewVolume event since generating Volume labels in a Python script is
much easier than trying to figure out Counter variables. See the
-\ilink{Python Scripting}{PythonChapter} chapter of this manual for more
-details.
+\bsysxrlink{Python Scripting}{PythonChapter}{misc}{chapter} of the \miscman{}
+ for more details.
Please note that automatic Volume labeling can also be used with tapes, but
it is not nearly so practical since the tapes must be pre-mounted. This
requires some user interaction. Automatic labeling from templates does NOT
work with autochangers since Bacula will not access unknown slots. There
are several methods of labeling all volumes in an autochanger magazine.
-For more information on this, please see the \ilink{
-Autochanger}{AutochangersChapter} chapter of this manual.
+For more information on this, please see the \ilink{Autochanger}
+{AutochangersChapter} chapter of this manual.
Automatic Volume labeling is enabled by making a change to both the Pool
resource (Director) and to the Device resource (Storage daemon) shown above.
that it will use to create new names. In the simplest form, the label format
is simply the Volume name, to which Bacula will append a four digit number.
This number starts at 0001 and is incremented for each Volume the catalog
-contains. Thus if you modify your Pool resource to be:
+contains. Thus if you modify your Pool resource to be:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Pool {
Name = File
Pool Type = Backup
Volume Use Duration = 23h
LabelFormat = "Vol"
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
Bacula will create Volume names Vol0001, Vol0002, and so on when new Volumes
are needed. Much more complex and elaborate labels can be created using
-variable expansion defined in the
-\ilink{Variable Expansion}{VarsChapter} chapter of this manual.
+variable expansion defined in the
+\bsysxrlink{Variable Expansion}{VarsChapter}{misc}{chapter} of the \miscman{}.
The second change that is necessary to make automatic labeling work is to give
the Storage daemon permission to automatically label Volumes. Do so by adding
-{\bf LabelMedia = yes} to the Device resource as follows:
+{\bf LabelMedia = yes} to the Device resource as follows:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Device {
Name = File
Media Type = File
AlwaysOpen = no;
LabelMedia = yes
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
-You can find more details of the {\bf Label Format} Pool record in
+You can find more details of the {\bf Label Format} Pool record in
\ilink{Label Format}{Label} description of the Pool resource
-records.
+records.
\label{Recycling1}
\subsection{Restricting the Number of Volumes and Recycling}
With the above scheme, a new Volume will be created every day. If you have not
specified Retention periods, your Catalog will continue to fill keeping track
of all the files Bacula has backed up, and this procedure will create one new
-archive file (Volume) every day.
+archive file (Volume) every day.
The tools Bacula gives you to help automatically manage these problems are the
-following:
+following:
\begin{enumerate}
-\item Catalog file record retention periods, the
+\item Catalog file record retention periods, the
\ilink{File Retention = ttt}{FileRetention} record in the Client
- resource.
-\item Catalog job record retention periods, the
+ resource.
+\item Catalog job record retention periods, the
\ilink{Job Retention = ttt}{JobRetention} record in the Client
- resource.
-\item The
+ resource.
+\item The
\ilink{ AutoPrune = yes}{AutoPrune} record in the Client resource
- to permit application of the above two retention periods.
-\item The
+ to permit application of the above two retention periods.
+\item The
\ilink{ Volume Retention = ttt}{VolRetention} record in the Pool
- resource.
-\item The
+ resource.
+\item The
\ilink{ AutoPrune = yes}{PoolAutoPrune} record in the Pool
- resource to permit application of the Volume retention period.
-\item The
+ resource to permit application of the Volume retention period.
+\item The
\ilink{ Recycle = yes}{PoolRecycle} record in the Pool resource
- to permit automatic recycling of Volumes whose Volume retention period has
- expired.
-\item The
+ to permit automatic recycling of Volumes whose Volume retention period has
+ expired.
+\item The
\ilink{ Recycle Oldest Volume = yes}{RecycleOldest} record in the
Pool resource tells Bacula to Prune the oldest volume in the Pool, and if all
- files were pruned to recycle this volume and use it.
-\item The
+ files were pruned to recycle this volume and use it.
+\item The
\ilink{ Recycle Current Volume = yes}{RecycleCurrent} record in
the Pool resource tells Bacula to Prune the currently mounted volume in the
- Pool, and if all files were pruned to recycle this volume and use it.
-\item The
+ Pool, and if all files were pruned to recycle this volume and use it.
+\item The
\ilink{ Purge Oldest Volume = yes}{PurgeOldest} record in the
Pool resource permits a forced recycling of the oldest Volume when a new one
is needed. {\bf N.B. This record ignores retention periods! We highly
- recommend not to use this record, but instead use Recycle Oldest Volume}
-\item The
+ recommend not to use this record, but instead use Recycle Oldest Volume}
+\item The
\ilink{ Maximum Volumes = nnn}{MaxVolumes} record in the Pool
- resource to limit the number of Volumes that can be created.
+ resource to limit the number of Volumes that can be created.
\end{enumerate}
The first three records (File Retention, Job Retention, and AutoPrune)
determine the amount of time that Job and File records will remain in your
-Catalog, and they are discussed in detail in the
+Catalog, and they are discussed in detail in the
\ilink{Automatic Volume Recycling}{RecyclingChapter} chapter of
-this manual.
+this manual.
Volume Retention, AutoPrune, and Recycle determine how long Bacula will keep
your Volumes before reusing them, and they are also discussed in detail in the
\ilink{Automatic Volume Recycling}{RecyclingChapter} chapter of
-this manual.
+this manual.
The Maximum Volumes record can also be used in conjunction with the Volume
Retention period to limit the total number of archive Volumes (files) that
through a fixed set of Volumes. Cycling through a fixed set of Volumes can
also be done by setting {\bf Recycle Oldest Volume = yes} or {\bf Recycle
Current Volume = yes}. In this case, when Bacula needs a new Volume, it will
-prune the specified volume.
+prune the specified volume.
\label{ConcurrentDiskJobs}
\section{Concurrent Disk Jobs}
\index[general]{Concurrent Disk Jobs}
Above, we discussed how you could have a single device named {\bf
-FileBackup} that writes to volumes in {\bf /home/bacula/backups}.
-You can, in fact, run multiple concurrent jobs using the
+FileBackup} that writes to volumes in {\bf /home/bacula/backups}.
+You can, in fact, run multiple concurrent jobs using the
Storage definition given with this example, and all the jobs will
simultaneously write into the Volume that is being written.
Storage resources in your bacula-dir.conf.
OK, so now you should understand that you need multiple Device definitions
-in the case of different directories or different Pools, but you also
+in the case of different directories or different Pools, but you also
need to know that the catalog data that Bacula keeps contains only
-the Media Type and not the specific storage device. This permits a tape
+the Media Type and not the specific storage device. This permits a tape
for example to be re-read on any compatible tape drive. The compatibility
being determined by the Media Type. The same applies to disk storage.
Since a volume that is written by a Device in say directory {\bf
/home/bacula/backups} cannot be read by a Device with an Archive Device
definition of {\bf /home/bacula/client1}, you will not be able to
-restore all your files if you give both those devices
-{\bf Media Type = File}. During the restore, Bacula will simply choose
+restore all your files if you give both those devices
+{\bf Media Type = File}. During the restore, Bacula will simply choose
the first available device, which may not be the correct one. If this
is confusing, just remember that the Directory has only the Media Type
and the Volume name. It does not know the {\bf Archive Device} (or the
The example shown below shows a case where there are two clients, each
using its own Pool and storing their Volumes in different directories.
-
+
\label{Example2}
\section{An Example}
machine. Each Volume is used (written) only once, and there are four Full
saves done every hour (so the whole thing cycles around after three hours).
-What is key here is that each physical device on the Storage daemon
+What is key here is that each physical device on the Storage daemon
has a different Media Type. This allows the Director to choose the
correct device for restores ...
-The Director's configuration file is as follows:
+The Director's configuration file is as follows:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Director {
Name = my-dir
QueryFile = "~/bacula/bin/query.sql"
Recycle = yes
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
-and the Storage daemon's configuration file is:
+and the Storage daemon's configuration file is:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Storage {
Name = my-sd
WorkingDirectory = "~/bacula/working"
Name = Standard
director = my-dir = all
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
With a little bit of work, you can change the above example into a weekly or
-monthly cycle (take care about the amount of archive disk space used).
+monthly cycle (take care about the amount of archive disk space used).
\label{MultipleDisks}
\section{Backing up to Multiple Disks}
For example, assume that you have two disks named {\bf /disk1} and {\bf
/disk2}. If you then create a standard Storage daemon Device resource for
-backing up to the first disk, it will look like the following:
+backing up to the first disk, it will look like the following:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Device {
Name = client1
Media Type = File
RemovableMedia = no;
AlwaysOpen = no;
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
Since there is no way to get the above Device resource to reference both {\bf
/disk1} and {\bf /disk2} we do it by pre-creating Volumes on /disk2 with the
-following:
+following:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
ln -s /disk2/Disk2-vol001 /disk1/Disk2-vol001
ln -s /disk2/Disk2-vol002 /disk1/Disk2-vol002
ln -s /disk2/Disk2-vol003 /disk1/Disk2-vol003
...
-\end{verbatim}
+\end{lstlisting}
\normalsize
At this point, you can label the Volumes as Volume {\bf Disk2-vol001}, {\bf
actually write the data to /disk2. The only minor inconvenience with this
method is that you must explicitly name the disks and cannot use automatic
labeling unless you arrange to have the labels exactly match the links you
-have created.
+have created.
An important thing to know is that Bacula treats disks like tape drives
as much as it can. This means that you can only have a single Volume
mounted at one time on a disk as defined in your Device resource in
-the Storage daemon's conf file. You can have multiple concurrent
+the Storage daemon's conf file. You can have multiple concurrent
jobs running that all write to the one Volume that is being used, but
if you want to have multiple concurrent jobs that are writing to
-separate disks drives (or partitions), you will need to define
+separate disks drives (or partitions), you will need to define
separate Device resources for each one, exactly as you would do for
two different tape drives. There is one fundamental difference, however.
The Volumes that you create on the two drives cannot be easily exchanged
An example would be the following:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Device {
Name = Disk1
Media Type = File1
RemovableMedia = no;
AlwaysOpen = no;
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
With the above device definitions, you can run two concurrent
\index[general]{Multiple Clients}
If we take the above example and add a second Client, here are a few
-considerations:
+considerations:
-\begin{itemize}
+\begin{bsysitemize}
\item Although the second client can write to the same set of Volumes, you
- will probably want to write to a different set.
+ will probably want to write to a different set.
\item You can write to a different set of Volumes by defining a second Pool,
- which has a different name and a different {\bf LabelFormat}.
+ which has a different name and a different {\bf LabelFormat}.
\item If you wish the Volumes for the second client to go into a different
directory (perhaps even on a different filesystem to spread the load), you
would do so by defining a second Device resource in the Storage daemon. The
{\bf Name} must be different, and the {\bf Archive Device} could be
different. To ensure that Volumes are never mixed from one pool to another,
-you might also define a different MediaType (e.g. {\bf File1}).
-\end{itemize}
+you might also define a different MediaType (e.g. {\bf File1}).
+\end{bsysitemize}
In this example, we have two clients, each with a different Pool and a
different number of archive files retained. They also write to different
-directories with different Volume labeling.
+directories with different Volume labeling.
-The Director's configuration file is as follows:
+The Director's configuration file is as follows:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Director {
Name = my-dir
QueryFile = "~/bacula/bin/query.sql"
Include {
Options {
compression=GZIP
- signature=SHA1
+ signature=SHA1
}
File = /home/kern/bacula/bin
}
Maximum Volumes = 8
Recycle = yes
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
-and the Storage daemon's configuration file is:
+and the Storage daemon's configuration file is:
\footnotesize
-\begin{verbatim}
+\begin{lstlisting}
Storage {
Name = my-sd
WorkingDirectory = "~/bacula/working"
Name = Standard
director = my-dir = all
}
-\end{verbatim}
+\end{lstlisting}
\normalsize
projects / bacula / docs / blobdiff