Update FOSDEM talk

[bacula/docs] / docs / manual / tips.tex

%%
%%

\chapter{Tips and Suggestions}
\label{TipsChapter}
\index[general]{Tips and Suggestions }
\index[general]{Suggestions!Tips and }
\label{examples}
\index[general]{Examples }

There are a number of example scripts for various things that can be found in
the {\bf example} subdirectory and its subdirectories of the Bacula source
distribution. 

\section{Upgrading Bacula Versions}
\label{upgrading}
\index[general]{Upgrading Bacula Versions }
\index[general]{Versions!Upgrading Bacula }

The first thing to do before upgrading from one version to another is to
ensure that you don't overwrite or delete your production (current) version of Bacula until
you have tested that the new version works. 

If you have installed Bacula into a single directory, this is simple: simply
make a copy of your Bacula directory. 

If you have done a more typical Unix installation where the binaries are
placed in one directory and the configuration files are placed in another,
then the simplest way is to configure your new Bacula to go into a single
file. Alternatively, make copies of all your binaries and especially your 
conf files.

Whatever your situation may be (one of the two just described), you should
probably start with the {\bf defaultconf} script that can be found in the {\bf
examples} subdirectory. Copy this script to the main Bacula directory, modify
it as necessary (there should not need to be many modifications), configure
Bacula, build it, install it, then stop your production Bacula, copy all the
{\bf *.conf} files from your production Bacula directory to the test Bacula
directory, start the test version, and run a few test backups. If all seems
good, then you can proceed to install the new Bacula in place of or possibly
over the old Bacula. 

When installing a new Bacula you need not worry about losing the changes you
made to your configuration files as the installation process will not
overwrite them providing that you do not do a {\bf make uninstall}.

If the new version of Bacula requires an upgrade to the database,
you can upgrade it with the script {\bf update\_bacula\_tables}, which
will be installed in your scripts directory (default {\bf /etc/bacula}),
or alternatively, you can find it in the  
{\bf \lt{}bacula-source\gt{}/src/cats} directory.

\section{Getting Notified of Job Completion}
\label{notification}
\index[general]{Getting Notified of Job Completion }
\index[general]{Completion!Getting Notified of Job }

One of the first things you should do is to ensure that you are being properly
notified of the status of each Job run by Bacula, or at a minimum of each Job
that terminates with an error. 

Until you are completely comfortable with {\bf Bacula}, we recommend that you
send an email to yourself for each Job that is run. This is most easily
accomplished by adding an email notification address in the {\bf Messages}
resource of your Director's configuration file. An email is automatically
configured in the default configuration files, but you must ensure that the
default {\bf root} address is replaced by your email address. 

For additional examples of how to configure a Bacula, please take a look at the
{\bf .conf} files found in the {\bf examples} sub-directory. We recommend the
following configuration (where you change the paths and email address to
correspond to your setup). Note, the {\bf mailcommand} and {\bf
operatorcommand} should be on a single line. They were split here for
presentation: 

\footnotesize
\begin{verbatim}
Messages {
  Name = Standard
  mailcommand = "/home/bacula/bin/bsmtp -h localhost
                -f \"\(Bacula\) %r\"
                -s \"Bacula: %t %e of %c %l\" %r"
  operatorcommand = "/home/bacula/bin/bsmtp -h localhost
                -f \"\(Bacula\) %r\"
                -s \"Bacula: Intervention needed for %j\" %r"
  Mail = your-email-address = all, !skipped, !terminate
  append = "/home/bacula/bin/log" = all, !skipped, !terminate
  operator = your-email-address = mount
  console = all, !skipped, !saved
}
\end{verbatim}
\normalsize

You will need to ensure that the {\bf /home/bacula/bin} path on the {\bf
mailcommand} and the {\bf operatorcommand} lines point to your {\bf Bacula}
binary directory where the {\bf bsmtp} program will be installed. You will
also want to ensure that the {\bf your-email-address} is replaced by your
email address, and finally, you will also need to ensure that the {\bf
/home/bacula/bin/log} points to the file where you want to log all messages. 

With the above Messages resource, you will be notified by email of every Job
that ran, all the output will be appended to the {\bf log} file you specify,
all output will be directed to the console program, and all mount messages
will be emailed to you. Note, some messages will be sent to multiple
destinations. 

The form of the mailcommand is a bit complicated, but it allows you to
distinguish whether the Job terminated in error or terminated normally. Please
see the 
\ilink{Mail Command}{mailcommand} section of the Messages
Resource chapter of this manual for the details of the substitution characters
used above. 

Once you are totally comfortable with Bacula as I am, or if you have a large
number of nightly Jobs as I do (eight), you will probably want to change the
{\bf Mail} command to {\bf Mail On Error} which will generate an email message
only if the Job terminates in error. If the Job terminates normally, no email
message will be sent, but the output will still be appended to the log file as
well as sent to the Console program. 

\section{Getting Email Notification to Work}
\label{email}
\index[general]{Work!Getting Email Notification to }
\index[general]{Getting Email Notification to Work }

The section above describes how to get email notification of job status.
Occasionally, however, users have problems receiving any email at all. In that
case, the things to check are the following: 

\begin{itemize}
\item Ensure that you have a valid email address specified on your  {\bf Mail}
   record in the Director's Messages resource. The email  address should be fully
   qualified. Simply using {\bf root} generally  will not work, rather you should
use {\bf root@localhost} or better  yet your full domain.  
\item Ensure that you do not have a {\bf Mail} record in the Storage  daemon's
   or File daemon's configuration files. The only record  you should have is {\bf
   director}:  

\footnotesize
\begin{verbatim}
      director = director-name = all
      
\end{verbatim}
\normalsize

\item If all else fails, try replacing the {\bf mailcommand} with 

   \footnotesize
\begin{verbatim}
mailcommand = "mail -s test your@domain.com"
\end{verbatim}
\normalsize

\item Once the above is working, assuming you want to use {\bf bsmtp},  submit
   the desired bsmtp command by hand and ensure that the email  is delivered,
   then put that command into {\bf Bacula}. Small  differences in things such as
the parenthesis around the word  Bacula can make a big difference to some
bsmtp programs.  For example, you might start simply by using: 

\footnotesize
\begin{verbatim}
mailcommand = "/home/bacula/bin/bsmtp -f \"root@localhost\" %r"
\end{verbatim}
\normalsize

\end{itemize}

\section{Getting Notified that Bacula is Running}
\label{JobNotification}
\index[general]{Running!Getting Notified that Bacula is }
\index[general]{Getting Notified that Bacula is Running }

If like me, you have setup Bacula so that email is sent only when a Job has
errors, as described in the previous section of this chapter, inevitably, one
day, something will go wrong and {\bf Bacula} can stall. This could be because
Bacula crashes, which is vary rare, or more likely the network has caused {\bf
Bacula} to {\bf hang} for some unknown reason. 

To avoid this, you can use the {\bf RunAfterJob} command in the Job resource
to schedule a Job nightly, or weekly that simply emails you a message saying
that Bacula is still running. For example, I have setup the following Job in
my Director's configuration file: 

\footnotesize
\begin{verbatim}
Schedule {
  Name = "Watchdog"
  Run = Level=Full sun-sat at 6:05
}
Job {
  Name = "Watchdog"
  Type = Admin
  Client=Watchdog
  FileSet="Verify Set"
  Messages = Standard
  Storage = DLTDrive
  Pool = Default
  Schedule = "Watchdog"
  RunAfterJob = "/home/kern/bacula/bin/watchdog %c %d"
}
Client {
  Name = Watchdog
  Address = rufus
  FDPort = 9102
  Catalog = Verify
  Password = ""
  File Retention = 1day
  Job Retention = 1 month
  AutoPrune = yes
}
\end{verbatim}
\normalsize

Where I established a schedule to run the Job nightly. The Job itself is type
{\bf Admin} which means that it doesn't actually do anything, and I've defined
a FileSet, Pool, Storage, and Client, all of which are not really used (and
probably don't need to be specified). The key aspect of this Job is the
command: 

\footnotesize
\begin{verbatim}
  RunAfterJob = "/home/kern/bacula/bin/watchdog %c %d"
\end{verbatim}
\normalsize

which runs my "watchdog" script. As an example, I have added the Job codes
\%c and \%d which will cause the Client name and the Director's name to be
passed to the script. For example, if the Client's name is {\bf Watchdog} and
the Director's name is {\bf main-dir} then referencing \$1 in the script would
get {\bf Watchdog} and referencing \$2 would get {\bf main-dir}. In this case,
having the script know the Client and Director's name is not really useful,
but in other situations it may be. 

You can put anything in the watchdog script. In my case, I like to monitor the
size of my catalog to be sure that {\bf Bacula} is really pruning it. The
following is my watchdog script: 

\footnotesize
\begin{verbatim}
#!/bin/sh
cd /home/kern/mysql/var/bacula
du . * |
/home/kern/bacula/bin/bsmtp  \
   -f "\(Bacula\) abuse@whitehouse.com" -h mail.yyyy.com \
   -s "Bacula running" abuse@whitehouse.com
\end{verbatim}
\normalsize

If you just wish to send yourself a message, you can do it with: 

\footnotesize
\begin{verbatim}
#!/bin/sh
cd /home/kern/mysql/var/bacula
/home/kern/bacula/bin/bsmtp  \
   -f "\(Bacula\) abuse@whitehouse.com" -h mail.yyyy.com \
   -s "Bacula running" abuse@whitehouse.com <<END-OF-DATA
Bacula is still running!!!
END-OF-DATA
\end{verbatim}
\normalsize

\section{Maintaining a Valid Bootstrap File}
\label{bootstrap}
\index[general]{Maintaining a Valid Bootstrap File }
\index[general]{File!Maintaining a Valid Bootstrap }

By using a 
\ilink{ WriteBootstrap}{writebootstrap} record in each of your
Director's Job resources, you can constantly maintain a 
\ilink{bootstrap}{BootstrapChapter} file that will enable you to
recover the state of your system as of the last backup without having the
Bacula catalog. This permits you to more easily recover from a disaster that
destroys your Bacula catalog. 

When a Job resource has a {\bf WriteBootstrap} record, Bacula will maintain
the designated file (normally on another system but mounted by NSF) with up to
date information necessary to restore your system. For example, in my
Director's configuration file, I have the following record: 

\footnotesize
\begin{verbatim}
 Write Bootstrap = "/mnt/deuter/files/backup/client-name.bsr"
\end{verbatim}
\normalsize

where I replace {\bf client-name} by the actual name of the client that is
being backed up. Thus, Bacula automatically maintains one file for each of my
clients. The necessary bootstrap information is appended to this file during
each {\bf Incremental} backup, and the file is totally rewritten during each
{\bf Full} backup. 

Note, one disadvantage of writing to an NFS mounted volume as I do is
that if the other machine goes down, the OS will wait forever on the fopen()
call that Bacula makes. As a consequence, Bacula will completely stall until
the machine exporting the NFS mounts comes back up. A possible solution to this
problem was provided by Andrew Hilborne, and consists of using the {\bf soft}
option instead of the {\bf hard} option when mounting the NFS volume, which is
typically done in {\bf /etc/fstab}/. The NFS documentation explains these
options in detail. However, I found that with the {\bf soft} option 
NFS disconnected frequently causing even more problems.

If you are starting off in the middle of a cycle (i.e. with Incremental
backups) rather than at the beginning (with a Full backup), the {\bf
bootstrap} file will not be immediately valid as it must always have the
information from a Full backup as the first record. If you wish to synchronize
your bootstrap file immediately, you can do so by running a {\bf restore}
command for the client and selecting a full restore, but when the restore
command asks for confirmation to run the restore Job, you simply reply no,
then copy the bootstrap file that was written to the location specified on the
{\bf Write Bootstrap} record. The restore bootstrap file can be found in {\bf
restore.bsr} in the working directory that you defined. In the example given
below for the client {\bf rufus}, my input is shown in bold. Note, the JobId
output has been partially truncated to fit on the page here: 

\footnotesize
\begin{verbatim}
(in the Console program)
*restore
First you select one or more JobIds that contain files
to be restored. You will then be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.
To select the JobIds, you have the following choices:
     1: List last 20 Jobs run
     2: List Jobs where a given File is saved
     3: Enter list of JobIds to select
     4: Enter SQL list command
     5: Select the most recent backup for a client
     6: Cancel
Select item:  (1-6): 5
The defined Client resources are:
     1: Minimatou
     2: Rufus
     3: Timmy
Select Client (File daemon) resource (1-3): 2
The defined FileSet resources are:
     1: Other Files
Item 1 selected automatically.
+-------+------+-------+---------+---------+------+-------+------------+
| JobId | Levl | Files | StrtTim | VolName | File | SesId | VolSesTime |
+-------+------+-------+---------+---------+------+-------+------------+
| 2     | F    | 84    |  ...    | test1   | 0    | 1     | 1035645259 |
+-------+------+-------+---------+---------+------+-------+------------+
You have selected the following JobId: 2
Building directory tree for JobId 2 ...
The defined Storage resources are:
     1: File
Item 1 selected automatically.
You are now entering file selection mode where you add and
remove files to be restored. All files are initially added.
Enter "done" to leave this mode.
cwd is: /
$ done
84 files selected to restore.
Run Restore job
JobName:    kernsrestore
Bootstrap:  /home/kern/bacula/working/restore.bsr
Where:      /tmp/bacula-restores
FileSet:    Other Files
Client:     Rufus
Storage:    File
JobId:      *None*
OK to run? (yes/mod/no): no
quit
(in a shell window)
cp ../working/restore.bsr /mnt/deuter/files/backup/rufus.bsr
\end{verbatim}
\normalsize

\section{Rejected Volumes After a Crash}
\label{RejectedVolumes}
\index[general]{Crash!Rejected Volumes After a }
\index[general]{Rejected Volumes After a Crash }

Bacula keeps a count of the number of files on each Volume in its Catalog
database so that before appending to a tape, it can verify that the number of
files are correct, and thus prevent overwriting valid data. If the Director or
the Storage daemon crashes before the job has completed, the tape will contain
one more file than is noted in the Catalog, and the next time you attempt to
use the same Volume, Bacula will reject it due to a mismatch between the
physical tape (Volume) and the catalog. 

The easiest solution to this problem is to label a new tape and start fresh.
If you wish to continue appending to the current tape, you can do so by using
the {\bf update} command in the console program to change the {\bf Volume
Files} entry in the catalog. A typical sequence of events would go like the
following: 

\footnotesize
\begin{verbatim}
- Bacula crashes
- You restart Bacula
\end{verbatim}
\normalsize

Bacula then prints: 

\footnotesize
\begin{verbatim}
17-Jan-2003 16:45 rufus-dir: Start Backup JobId 13,
                  Job=kernsave.2003-01-17_16.45.46
17-Jan-2003 16:45 rufus-sd: Volume test01 previously written,
                  moving to end of data.
17-Jan-2003 16:46 rufus-sd: kernsave.2003-01-17_16.45.46 Error:
                  I cannot write on this volume because:
                  The number of files mismatch! Volume=11 Catalog=10
17-Jan-2003 16:46 rufus-sd: Job kernsave.2003-01-17_16.45.46 waiting.
                   Cannot find any appendable volumes.
Please use the "label"  command to create a new Volume for:
    Storage:      SDT-10000
    Media type:   DDS-4
    Pool:         Default
\end{verbatim}
\normalsize

(note, lines wrapped for presentation)
The key here is the line that reads: 

\footnotesize
\begin{verbatim}
  The number of files mismatch! Volume=11 Catalog=10
\end{verbatim}
\normalsize

It says that Bacula found eleven files on the volume, but that the catalog
says there should be ten. When you see this, you can be reasonably sure that
the SD was interrupted while writing before it had a chance to update the
catalog. As a consequence, you can just modify the catalog count to eleven,
and even if the catalog contains references to files saved in file 11,
everything will be OK and nothing will be lost. Note that if the SD had
written several file marks to the volume, the difference between the Volume
count and the Catalog count could be larger than one, but this is unusual. 

If on the other hand the catalog is marked as having more files than Bacula
found on the tape, you need to consider the possible negative consequences of
modifying the catalog. Please see below for a more complete discussion of
this. 

Continuing with the example of {\bf Volume = 11 Catalog = 10}, to enable to
Bacula to append to the tape, you do the following: 

\footnotesize
\begin{verbatim}
update
Update choice:
     1: Volume parameters
     2: Pool from resource
     3: Slots from autochanger
Choose catalog item to update (1-3): 1
Defined Pools:
     1: Default
     2: File
Select the Pool (1-2):
+-------+---------+--------+---------+-----------+------+----------+------+-----+
| MedId | VolName | MedTyp | VolStat | VolBytes  | Last | VolReten | Recy | Slt |
+-------+---------+--------+---------+-----------+------+----------+------+-----+
| 1     | test01  | DDS-4  | Error   | 352427156 | ...  | 31536000 | 1    | 0   |
+-------+---------+--------+---------+-----------+------+----------+------+-----+
Enter MediaId or Volume name: 1
\end{verbatim}
\normalsize

(note table output truncated for presentation) First, you chose to update the
Volume parameters by entering a {\bf 1}. In the volume listing that follows,
notice how the VolStatus is {\bf Error}. We will correct that after changing
the Volume Files. Continuing, you respond 1, 

\footnotesize
\begin{verbatim}
Updating Volume "test01"
Parameters to modify:
     1: Volume Status
     2: Volume Retention Period
     3: Volume Use Duration
     4: Maximum Volume Jobs
     5: Maximum Volume Files
     6: Maximum Volume Bytes
     7: Recycle Flag
     8: Slot
     9: Volume Files
    10: Pool
    11: Done
Select parameter to modify (1-11): 9
Warning changing Volume Files can result
in loss of data on your Volume
Current Volume Files is: 10
Enter new number of Files for Volume: 11
New Volume Files is: 11
Updating Volume "test01"
Parameters to modify:
     1: Volume Status
     2: Volume Retention Period
     3: Volume Use Duration
     4: Maximum Volume Jobs
     5: Maximum Volume Files
     6: Maximum Volume Bytes
     7: Recycle Flag
     8: Slot
     9: Volume Files
    10: Pool
    11: Done
Select parameter to modify (1-10): 1
\end{verbatim}
\normalsize

Here, you have selected {\bf 9} in order to update the Volume Files, then you
changed it from {\bf 10} to {\bf 11}, and you now answer {\bf 1} to change the
Volume Status. 

\footnotesize
\begin{verbatim}
Current Volume status is: Error
Possible Values are:
     1: Append
     2: Archive
     3: Disabled
     4: Full
     5: Used
     6: Read-Only
Choose new Volume Status (1-6): 1
New Volume status is: Append
Updating Volume "test01"
Parameters to modify:
     1: Volume Status
     2: Volume Retention Period
     3: Volume Use Duration
     4: Maximum Volume Jobs
     5: Maximum Volume Files
     6: Maximum Volume Bytes
     7: Recycle Flag
     8: Slot
     9: Volume Files
    10: Pool
    11: Done
Select parameter to modify (1-11): 11
Selection done.
\end{verbatim}
\normalsize

At this point, you have changed the Volume Files from {\bf 10} to {\bf 11} to
account for the last file that was written but not updated in the database,
and you changed the Volume Status back to {\bf Append}. 

This was a lot of words to describe something quite simple. 

The {\bf Volume Files} option exists only in version 1.29 and later, and you
should be careful using it. Generally, if you set the value to that which
Bacula said is on the tape, you will be OK, especially if the value is one
more than what is in the catalog. 

Now lets consider the case: 

\footnotesize
\begin{verbatim}
  The number of files mismatch! Volume=10 Catalog=12
\end{verbatim}
\normalsize

Here the Bacula found fewer files on the volume than what is marked in the
catalog. Now, in this case, you should hesitate a lot before modifying the count
in the catalog, because if you force the catalog from 12 to 10, Bacula will
start writing after the file 10 on the tape, possibly overwriting valid data,
and if you ever try to restore any of the files that the catalog has marked as
saved on Files 11 and 12, all chaos will break out. In this case, you will
probably be better off using a new tape. In fact, you might want to see what
files the catalog claims are actually stored on that Volume, and back them up
to another tape and recycle this tape. 

\section{Security Considerations}
\label{security}
\index[general]{Considerations!Security }
\index[general]{Security Considerations }

Only the File daemon needs to run with root permission (so that it can access
all files). As a consequence, you may run your Director, Storage daemon, and
MySQL or PostgreSQL database server as non-root processes. Version 1.30 has
the {\bf -u} and the {\bf -g} options that allow you to specify a userid and
groupid on the command line to be used after Bacula starts. 

As of version 1.33, thanks to Dan Langille, it is easier to configure the
Bacula Director and Storage daemon to run as non-root. 

You should protect the Bacula port addresses (normally 9101, 9102, and 9103)
from outside access by a firewall or other means of protection to prevent
unauthorized use of your daemons. 

You should ensure that the configuration files are not world readable since
they contain passwords that allow access to the daemons. Anyone who can access
the Director using a console program can restore any file from a backup
Volume. 

You should protect your Catalog database. If you are using SQLite, make sure
that the working directory is readable only by root (or your Bacula userid),
and ensure that {\bf bacula.db} has permissions {\bf -rw-r\verb:--:r\verb:--:} (i.e. 640) or
more strict. If you are using MySQL or PostgreSQL, please note that the Bacula
setup procedure leaves the database open to anyone. At a minimum, you should
assign the user {\bf bacula} a userid and add it to your Director's
configuration file in the appropriate Catalog resource. 

\section{Creating Holiday Schedules}
\label{holiday}
\index[general]{Schedules!Creating Holiday }
\index[general]{Creating Holiday Schedules }

If you normally change tapes every day or at least every Friday, but Thursday
is a holiday, you can use a trick proposed by Lutz Kittler to ensure that no
job runs on Thursday so that you can insert Friday's tape and be sure it will
be used on Friday. To do so, define a {\bf RunJobBefore} script that normally
returns zero, so that the Bacula job will normally continue. You can then
modify the script to return non-zero on any day when you do not want Bacula to
run the job. 

\section{Automatic Labeling Using Your Autochanger}
\label{autolabel}
\index[general]{Automatic Labeling Using Your Autochanger }
\index[general]{Autochanger!Automatic Labeling Using Your }

If you have an autochanger but it does not support barcodes, using a "trick"
you can make Bacula automatically label all the volumes in your autochanger's
magazine. 

First create a file containing one line for each slot in your autochanger that
has a tape to be labeled. The line will contain the slot number a colon (:)
then the Volume name you want to use. For example, create a file named {\bf
volume-list}, which contains: 

\footnotesize
\begin{verbatim}
1:Volume001
2:TestVolume02
5:LastVolume
\end{verbatim}
\normalsize

The records do not need to be in any order and you don't need to mention all
the slots. Normally, you will have a consistent set of Volume names and a
sequential set of numbers for each slot you want labeled. In the example
above, I've left out slots 3 and 4 just as an example. Now, modify your {\bf
mtx-changer} script and comment out all the lines in the {\bf list)} case by
putting a \# in column 1. Then add the following two lines: 

\footnotesize
\begin{verbatim}
  cat <absolute-path>/volume-list
  exit 0
\end{verbatim}
\normalsize

so that the whole case looks like: 

\footnotesize
\begin{verbatim}
  list)
#
# commented out lines
   cat <absolute-path>/volume-list
   exit 0
   ;;
\end{verbatim}
\normalsize

where you replace \lt{}absolute-path\gt{} with the full path to the
volume-list file. Then using the console, you enter the following command: 

\footnotesize
\begin{verbatim}
   label barcodes
\end{verbatim}
\normalsize

and Bacula will proceed to mount the autochanger Volumes in the list and label
them with the Volume names you have supplied. Bacula will think that the list
was provided by the autochanger barcodes, but in reality, it was you who
supplied the \lt{}barcodes\gt{}. 

If it seems to work, when it finishes, enter: 

\footnotesize
\begin{verbatim}
   list volumes
\end{verbatim}
\normalsize

and you should see all the volumes nicely created. 

\section{Backing Up Portables Using DHCP}
\label{DNS}
\index[general]{DHCP!Backing Up Portables Using }
\index[general]{Backing Up Portables Using DHCP }

You may want to backup laptops or portables that are not always connected to
the network. If you are using DHCP to assign an IP address to those machines
when they connect, you will need to use the Dynamic Update capability of DNS
to assign a name to those machines that can be used in the Address field of
the Client resource in the Director's conf file. 

\section{Going on Vacation}
\label{Vacation}
\index[general]{Vacation!Going on }
\index[general]{Going on Vacation }

At some point, you may want to be absent for a week or two and you want to
make sure Bacula has enough tape left so that the backups will complete. You
start by doing a {\bf list volumes} in the Console program: 

\footnotesize
\begin{verbatim}
list volumes
 
Using default Catalog name=BackupDB DB=bacula
Pool: Default
+---------+---------------+-----------+-----------+----------------+-
| MediaId | VolumeName    | MediaType | VolStatus |       VolBytes |
+---------+---------------+-----------+-----------+----------------+-
|      23 | DLT-30Nov02   | DLT8000   | Full      | 54,739,278,128 |
|      24 | DLT-21Dec02   | DLT8000   | Full      | 56,331,524,629 |
|      25 | DLT-11Jan03   | DLT8000   | Full      | 67,863,514,895 |
|      26 | DLT-02Feb03   | DLT8000   | Full      | 63,439,314,216 |
|      27 | DLT-03Mar03   | DLT8000   | Full      | 66,022,754,598 |
|      28 | DLT-04Apr03   | DLT8000   | Full      | 60,792,559,924 |
|      29 | DLT-28Apr03   | DLT8000   | Full      | 62,072,494,063 |
|      30 | DLT-17May03   | DLT8000   | Full      | 65,901,767,839 |
|      31 | DLT-07Jun03   | DLT8000   | Used      | 56,558,490,015 |
|      32 | DLT-28Jun03   | DLT8000   | Full      | 64,274,871,265 |
|      33 | DLT-19Jul03   | DLT8000   | Full      | 64,648,749,480 |
|      34 | DLT-08Aug03   | DLT8000   | Full      | 64,293,941,255 |
|      35 | DLT-24Aug03   | DLT8000   | Append    |  9,999,216,782 |
+---------+---------------+-----------+-----------+----------------+
\end{verbatim}
\normalsize

Note, I have truncated the output for presentation purposes. What is
significant, is that I can see that my current tape has almost 10 Gbytes of
data, and that the average amount of data I get on my tapes is about 60
Gbytes. So if I go on vacation now, I don't need to worry about tape capacity
(at least not for short absences). 

Equally significant is the fact that I did go on vacation the 28th of June
2003, and when I did the {\bf list volumes} command, my current tape at that
time, DLT-07Jun03 MediaId 31, had 56.5 Gbytes written. I could see that the
tape would fill shortly. Consequently, I manually marked it as {\bf Used} and
replaced it with a fresh tape that I labeled as DLT-28Jun03, thus assuring
myself that the backups would all complete without my intervention. 

\section{Exclude Files on Windows Regardless of Case}
\label{Case}
\index[general]{Exclude Files on Windows Regardless of Case}
% TODO: should this be put in the win32 chapter?
% TODO: should all these tips be placed in other chapters?

This tip was submitted by Marc Brueckner who wasn't sure of the case of some
of his files on Win32, which is case insensitive. The problem is that Bacula
thinks that {\bf /UNIMPORTANT FILES} is different from {\bf /Unimportant
Files}. Marc was aware that the file exclusion permits wild-cards. So, he
specified: 

\footnotesize
\begin{verbatim}
"/[Uu][Nn][Ii][Mm][Pp][Oo][Rr][Tt][Aa][Nn][Tt] [Ff][Ii][Ll][Ee][Ss]"
\end{verbatim}
\normalsize

As a consequence, the above exclude works for files of any case. 

Please note that this works only in Bacula Exclude statement and not in
Include. 

\section{Executing Scripts on a Remote Machine}
\label{RemoteExecution}
\index[general]{Machine!Executing Scripts on a Remote }
\index[general]{Executing Scripts on a Remote Machine }

This tip also comes from Marc Brueckner. (Note, this tip is probably outdated
by the addition of {\bf ClientRunBeforJob} and {\bf ClientRunAfterJob} Job
records, but the technique still could be useful.) First I thought the "Run
Before Job" statement in the Job-resource is for executing a script on the
remote machine(the machine to be backed up). It could be useful to execute
scripts on the remote machine e.g. for stopping databases or other services
while doing the backup. (Of course I have to start the services again when the
backup has finished) I found the following solution: Bacula could execute
scripts on the remote machine by using ssh. The authentication is done
automatically using a private key. First you have to generate a keypair. I've
done this by: 

\footnotesize
\begin{verbatim}
ssh-keygen -b 4096 -t dsa -f Bacula_key
\end{verbatim}
\normalsize

This statement may take a little time to run. It creates a public/private key
pair with no passphrase. You could save the keys in /etc/bacula. Now you have
two new files : Bacula\_key which contains the private key and Bacula\_key.pub
which contains the public key. 

Now you have to append the Bacula\_key.pub file to the file authorized\_keys
in the \textbackslash{}root\textbackslash{}.ssh directory of the remote
machine. Then you have to add (or uncomment) the line 

\footnotesize
\begin{verbatim}
AuthorizedKeysFile           %h/.ssh/authorized_keys
\end{verbatim}
\normalsize

to the sshd\_config file on the remote machine. Where the \%h stands for the
home-directory of the user (root in this case). 

Assuming that your sshd is already running on the remote machine, you can now
enter the following on the machine where Bacula runs: 

\footnotesize
\begin{verbatim}
ssh -i Bacula_key  -l root "ls -la"
\end{verbatim}
\normalsize

This should execute the "ls -la" command on the remote machine. 

Now you could add lines like the following to your Director's conf file: 

\footnotesize
\begin{verbatim}
...
Run Before Job = ssh -i /etc/bacula/Bacula_key 192.168.1.1 \
                 "/etc/init.d/database stop"
Run After Job = ssh -i /etc/bacula/Bacula_key 192.168.1.1 \
                 "/etc/init.d/database start"
...
\end{verbatim}
\normalsize

Even though Bacula version 1.32 has a ClientRunBeforeJob, the ssh method still
could be useful for updating all the Bacula clients on several remote machines
in a single script. 

\section{Recycling All Your Volumes}
\label{recycle}
\index[general]{Recycling All Your Volumes }
\index[general]{Volumes!Recycling All Your }

This tip comes from Phil Stracchino. 

If you decide to blow away your catalog and start over, the simplest way to
re-add all your prelabeled tapes with a minimum of fuss (provided you don't
care about the data on the tapes) is to add the tape labels using the console
{\bf add} command, then go into the catalog and manually set the VolStatus of
every tape to {\bf Recycle}. 

The SQL command to do this is very simple: 

\footnotesize
\begin{verbatim}
update Media set VolStatus = "Recycle";
\end{verbatim}
\normalsize

Bacula will then ignore the data already stored on the tapes and just re-use
each tape without further objection. 

\section{Backing up ACLs on ext3 or XFS filesystems}
\label{ACLs}
\index[general]{Filesystems!Backing up ACLs on ext3 or XFS }
\index[general]{Backing up ACLs on ext3 or XFS filesystems }

This tip comes from Volker Sauer. 

Note, this tip was given prior to implementation of ACLs in Bacula (version
1.34.5). It is left here because dumping/displaying ACLs can still be useful
in testing/verifying that Bacula is backing up and restoring your ACLs
properly. Please see the 
\ilink{aclsupport}{ACLSupport} FileSet option in the
configuration chapter of this manual. 

For example, you could dump the ACLs to a file with a script similar to the
following: 

\footnotesize
\begin{verbatim}
#!/bin/sh
BACKUP_DIRS="/foo /bar"
STORE_ACL=/root/acl-backup
umask 077
for i in $BACKUP_DIRS; do
 cd $i /usr/bin/getfacl -R --skip-base .>$STORE_ACL/${i//\//_}
done
\end{verbatim}
\normalsize

Then use Bacula to backup {\bf /root/acl-backup}. 

The ACLs could be restored using Bacula to the {\bf /root/acl-backup} file,
then restored to your system using: 

\footnotesize
\begin{verbatim}
setfacl --restore/root/acl-backup
\end{verbatim}
\normalsize

\section{Total Automation of Bacula Tape Handling}
\label{automate}
\index[general]{Handling!Total Automation of Bacula Tape }
\index[general]{Total Automation of Bacula Tape Handling }

This tip was provided by Alexander Kuehn. 

\elink{Bacula}{http://www.bacula.org/} is a really nice backup program except
that the manual tape changing requires user interaction with the bacula
console. 

Fortunately I can fix this.
NOTE!!! This suggestion applies for people who do *NOT* have tape autochangers
and must change tapes manually.!!!!! 

Bacula supports a variety of tape changers through the use of mtx-changer
scripts/programs. This highly flexible approach allowed me to create 
\ilink{this shell script}{mtx-changer.txt} which does the following:
% TODO: This link doesn't work; also need to include this in book appendix
% TODO: and point to it.
Whenever a new tape is required it sends a mail to the operator to insert the
new tape. Then it waits until a tape has been inserted, sends a mail again to
say thank you and let's bacula continue its backup.
So you can schedule and run backups without ever having to log on or see the
console.
To make the whole thing work you need to create a Device resource which looks
something like this ("Archive Device", "Maximum Changer Wait", "Media
Type" and "Label media" may have different values): 

\footnotesize
\begin{verbatim}
Device {
   Name=DDS3
   Archive Device = # use yours not mine! ;)/dev/nsa0
   Changer Device = # not really required/dev/nsa0
   Changer Command = "# use this (maybe change the path)!
         /usr/local/bin/mtx-changer %o %a %S"
   Maximum Changer Wait = 3d          # 3 days in seconds
   AutomaticMount = yes;              # mount on start
   AlwaysOpen = yes;                  # keep device locked
   Media Type = DDS3                  # it's just a name
   RemovableMedia = yes;              #
   Offline On Unmount = Yes;          # keep this too
   Label media = Yes;                 #
}
\end{verbatim}
\normalsize

As the script has to emulate the complete wisdom of a mtx-changer it has an
internal "database" containing where which tape is stored, you can see this on
the following line:

\footnotesize
\begin{verbatim}
labels="VOL-0001 VOL-0002 VOL-0003 VOL-0004 VOL-0005 VOL-0006
VOL-0007 VOL-0008 VOL-0009 VOL-0010 VOL-0011 VOL-0012"
\end{verbatim}
\normalsize

The above should be all on one line, and it effectively tells Bacula that
volume "VOL-0001" is located in slot 1 (which is our lowest slot), that
volume "VOL-0002" is located in slot 2 and so on..
The script also maintains a logfile (/var/log/mtx.log) where you can monitor
its operation.

\section{Running Concurrent Jobs}
\label{ConcurrentJobs}
\index[general]{Jobs!Running Concurrent}
\index[general]{Running Concurrent Jobs}
\index[general]{Concurrent Jobs}

Bacula can run multiple concurrent jobs, but the default configuration files
are not set to do so. Using the {\bf Maximum Concurrent Jobs} directive, you
have a lot of control over how many jobs can run at the same time, and which
jobs can run simultaneously. The downside is that it can be a bit tricky to
set it up for the first time as you need to set the concurrency in at least
five different places. 

The Director, the File daemon, and the Storage daemon each have a {\bf Maximum
Concurrent Jobs} directive that determines overall number of concurrent jobs
the daemon will run. The default is one for the Director and ten for both the
File daemon and the Storage daemon, so assuming you will not be running more
than ten concurrent jobs, the only changes that are needed are in the
Director's conf file (bacula-dir.conf). 

Within the Director's configuration file, {\bf Maximum Concurrent Jobs} can be
set in the Direct, Job, Client, and Storage resources. Each one must be set
properly, according to your needs, otherwise your jobs may be run one at a
time. 

For example, if you want two different jobs to run simultaneously backing up
the same Client to the same Storage device, they will run concurrently only if
you have set {\bf Maximum Concurrent Jobs} greater than one in the Director
resource, the Client resource, and the Storage resource in bacula-dir.conf. 

We recommend that you carefully read the \ilink{Data
Spooling}{SpoolingChapter} of this manual first, then carefully
test your multiple concurrent backup including
restore testing before you put it into production.      

Below is a super stripped down bacula-dir.conf file showing you the four
places where the the file has been modified to allow the same job {\bf
NightlySave} to run up to four times concurrently. The change to the Job
resource is not necessary if you want different Jobs to run at the same time,
which is the normal case. 

\footnotesize
\begin{verbatim}
#
# Bacula Director Configuration file -- bacula-dir.conf
#
Director {
  Name = rufus-dir
  Maximum Concurrent Jobs = 4
  ...
}
Job {
  Name = "NightlySave"
  Maximum Concurrent Jobs = 4
  Client = rufus-fd
  Storage = File
  ...
}
Client {
  Name = rufus-fd
  Maximum Concurrent Jobs = 4
  ...
}
Storage {
  Name = File
  Maximum Concurrent Jobs = 4
  ...
}
\end{verbatim}
\normalsize