Updates

[bacula/docs] / docs / manual / tapetesting.tex

%%
%%

\chapter{Testing Your Tape Drive With Bacula}
\label{TapeTestingChapter}
\index[general]{Testing Your Tape Drive With Bacula}

This chapter is concerned with testing and configuring your tape drive to make
sure that it will work properly with Bacula using the {\bf btape} program. 
\label{summary}

\section{Get Your Tape Drive Working}

In general, you should follow the following steps to get your tape drive to
work with Bacula. Start with a tape mounted in your drive. If you have an
autochanger, load a tape into the drive. We use {\bf /dev/nst0} as the tape
drive name, you will need to adapt it according to your system. 

Do not proceed to the next item until you have succeeded with the previous
one. 

\begin{enumerate}
\item Make sure that Bacula (the Storage daemon) is not running
  or that you have {\bf unmount}ed the drive you will use 
  for testing.

\item Use tar to write to, then read from your drive:  

   \footnotesize
\begin{verbatim}
   mt -f /dev/nst0 rewind
   tar cvf /dev/nst0 .
   mt -f /dev/nst0 rewind
   tar tvf /dev/nst0
   
\end{verbatim}
\normalsize

\item Make sure you have a valid and correct Device resource  corresponding to
   your drive. For Linux users, generally,  the default one works. For FreeBSD
   users, there are two  possible Device configurations (see below). 

\item Run the btape {\bf test} command:  

   \footnotesize
\begin{verbatim}
   ./btape -c bacula-sd.conf /dev/nst0
   test
   
\end{verbatim}
\normalsize

It isn't necessary to run the autochanger part of the  test at this time,  but
do not go past this point until the basic test succeeds. If you do have 
an autochanger, please be sure to read the
\ilink{Autochanger chapter}{AutochangersChapter} of this manual.

\item Run the btape {\bf fill} command, preferably with two volumes.  This
   can take a long time. If you have an autochanger and it  is configured, Bacula
   will automatically use it. If you do  not have it configured, you can manually
   issue the appropriate  {\bf mtx} command, or press the autochanger buttons to
   change  the tape when requested to do so. 

\item FreeBSD users, if you have a pre-5.0 system run the {\bf tapetest}
   program, and make sure your system is patched if necessary. The tapetest
   program can be found in the platform/freebsd directory. The instructions
   for its use are at the top of the file.

\item Run Bacula, and backup a reasonably small directory,  say 60 Megabytes.
   Do three successive backups of this  directory. 

\item Stop Bacula, then restart it. Do another full backup  of the same
   directory. Then stop and restart Bacula. 

\item Do a restore of the directory backed up, by entering the  following
   restore command, being careful to restore it to  an alternate location:  

\footnotesize
\begin{verbatim}
   restore select all done
   yes
   
\end{verbatim}
\normalsize

Do a {\bf diff} on the restored directory to ensure it is identical  to the
original directory. If you are going to backup multiple different systems
(Linux, Windows, Mac, Solaris, FreeBSD, ...), be sure you test the restore
on each system type.

\item If you have an autochanger, you should now go back to the  btape program
   and run the autochanger test:  

\footnotesize
\begin{verbatim}
     ./btape -c bacula-sd.conf /dev/nst0
     auto
     
\end{verbatim}
\normalsize

Adjust your autochanger as necessary to ensure that it works  correctly. See
the Autochanger chapter of this manual  for a complete discussion of testing
your autochanger.  

\item We strongly recommend that you use a dedicated SCSI
controller for your tape drives. Scanners are known to induce
serious problems with the SCSI bus, causing it to reset. If the
SCSI bus is reset while Bacula has the tape drive open, it will
most likely be fatal to your tape since the drive will rewind.
These kinds of problems show up in the system log. For example,
the following was most likely caused by a scanner:

\footnotesize
\begin{verbatim}
Feb 14 17:29:55 epohost kernel: (scsi0:A:2:0): No or incomplete CDB sent to device.
Feb 14 17:29:55 epohost kernel: scsi0: Issued Channel A Bus Reset. 1 SCBs aborted
\end{verbatim}
\normalsize

\end{enumerate}

If you have reached this point, you stand a good chance of having everything
work. If you get into trouble at any point, {\bf carefully} read the
documentation given below. If you cannot get past some point, ask the {\bf
bacula-users} email list, but specify which of the steps you have successfully
completed. In particular, you may want to look at the 
\ilink{ Tips for Resolving Problems}{problems1} section below. 


\label{NoTapeInDrive}
\subsection{Problems When no Tape in Drive}
\index[general]{Problems When no Tape in Drive}
When Bacula was first written the Linux 2.4 kernel permitted opening the
drive whether or not there was a tape in the drive. Thus the Bacula code is
based on the concept that if the drive cannot be opened, there is a serious
problem, and the job is failed.

With version 2.6 of the Linux kernel, if there is no tape in the drive, the
OS will wait two minutes (default) and then return a failure, and consequently,
Bacula version 1.36 and below will fail the job.  This is important to keep
in mind, because if you use an option such as {\bf Offline on Unmount =
yes}, there will be a point when there is no tape in the drive, and if
another job starts or if Bacula asks the operator to mount a tape, when
Bacula attempts to open the drive (about a 20 minute delay), it will fail
and Bacula will fail the job.

In version 1.38.x, the Bacula code partially gets around this problem -- at
least in the initial open of the drive.  However, functions like Polling
the drive do not work correctly if there is no tape in the drive.
Providing you do not use {\bf Offline on Unmount = yes}, you should not
experience job failures as mentioned above.  If you do experience such
failures, you can also increase the {\bf Maximum Open Wait} time interval,
which will give you more time to mount the next tape before the job is
failed.

\subsection{Specifying the Configuration File}
\index[general]{File!Specifying the Configuration}
\index[general]{Specifying the Configuration File}

Starting with version 1.27, each of the tape utility programs including the
{\bf btape} program requires a valid Storage daemon configuration file
(actually, the only part of the configuration file that {\bf btape} needs is
the {\bf Device} resource definitions). This permits {\bf btape} to find the
configuration parameters for your archive device (generally a tape drive).
Without those parameters, the testing and utility programs do not know how to
properly read and write your drive. By default, they use {\bf bacula-sd.conf}
in the current directory, but you may specify a different configuration file
using the {\bf -c} option. 

\subsection{Specifying a Device Name For a Tape}
\index[general]{Tape!Specifying a Device Name For a}
\index[general]{Specifying a Device Name For a Tape}

{\bf btape} {\bf device-name} where the Volume can be found. In the case of a
tape, this is the physical device name such as {\bf /dev/nst0} or {\bf
/dev/rmt/0ubn} depending on your system that you specify on the Archive Device
directive. For the program to work, it must find the identical name in the
Device resource of the configuration file. If the name is not found in the
list of physical names, the utility program will compare the name you entered
to the Device names (rather than the Archive device names). 

When specifying a tape device, it is preferable that the "non-rewind"
variant of the device file name be given.  In addition, on systems such as
Sun, which have multiple tape access methods, you must be sure to specify
to use Berkeley I/O conventions with the device.  The
{\bf b} in the Solaris (Sun) archive specification {\bf /dev/rmt/0mbn} is
what is needed in this case.  Bacula does not support SysV tape drive
behavior.

See below for specifying Volume names. 

\subsection{Specifying a Device Name For a File}
\index[general]{File!Specifying a Device Name For a}
\index[general]{Specifying a Device Name For a File}

If you are attempting to read or write an archive file rather than a tape, the
{\bf device-name} should be the full path to the archive location including
the filename. The filename (last part of the specification) will be stripped
and used as the Volume name, and the path (first part before the filename)
must have the same entry in the configuration file. So, the path is equivalent
to the archive device name, and the filename is equivalent to the volume name.


\section{btape}
\label{btape1}
\index[general]{Btape}

This program permits a number of elementary tape operations via a tty command
interface. The {\bf test} command, described below, can be very useful for
testing tape drive compatibility problems. Aside from initial testing of tape
drive compatibility with {\bf Bacula}, {\bf btape} will be mostly used by
developers writing new tape drivers. 

{\bf btape} can be dangerous to use with existing {\bf Bacula} tapes because
it will relabel a tape or write on the tape if so requested regardless of
whether or not the tape contains valuable data, so please be careful and use
it only on blank tapes. 

To work properly, {\bf btape} needs to read the Storage daemon's configuration
file. As a default, it will look for {\bf bacula-sd.conf} in the current
directory. If your configuration file is elsewhere, please use the {\bf -c}
option to specify where. 

The physical device name or the Device resource name must be specified on the
command line, and this same device name must be present in the Storage
daemon's configuration file read by {\bf btape} 

\footnotesize
\begin{verbatim}
Usage: btape [options] device_name
       -b <file>   specify bootstrap file
       -c <file>   set configuration file to file
       -d <nn>     set debug level to nn
       -p          proceed inspite of I/O errors
       -s          turn off signals
       -v          be verbose
       -?          print this message.
\end{verbatim}
\normalsize

\subsection{Using btape to Verify your Tape Drive}
\index[general]{Using btape to Verify your Tape Drive}
\index[general]{Drive!Using btape to Verify your Tape}

An important reason for this program is to ensure that a Storage daemon
configuration file is defined so that Bacula will correctly read and write
tapes. 

It is highly recommended that you run the {\bf test} command before running
your first Bacula job to ensure that the parameters you have defined for your
storage device (tape drive) will permit {\bf Bacula} to function properly. You
only need to mount a blank tape, enter the command, and the output should be
reasonably self explanatory. For example: 

\footnotesize
\begin{verbatim}
(ensure that Bacula is not running)
./btape -c /usr/bin/bacula/bacula-sd.conf /dev/nst0
\end{verbatim}
\normalsize

The output will be: 

\footnotesize
\begin{verbatim}
Tape block granularity is 1024 bytes.
btape: btape.c:376 Using device: /dev/nst0
*
\end{verbatim}
\normalsize

Enter the test command: 

\footnotesize
\begin{verbatim}
test
\end{verbatim}
\normalsize

The output produced should be something similar to the following: I've cut the
listing short because it is frequently updated to have new tests. 

\footnotesize
\begin{verbatim}
=== Append files test ===
This test is essential to Bacula.
I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2
btape: btape.c:387 Rewound /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:855 Wrote one record of 64412 bytes.
btape: btape.c:857 Wrote block to device.
btape: btape.c:410 Wrote EOF to /dev/nst0
btape: btape.c:387 Rewound /dev/nst0
btape: btape.c:693 Now moving to end of media.
btape: btape.c:427 Moved to end of media
We should be in file 3. I am at file 3. This is correct!
Now the important part, I am going to attempt to append to the tape.
...
=== End Append files test ===
\end{verbatim}
\normalsize

If you do not successfully complete the above test, please resolve the
problem(s) before attempting to use {\bf Bacula}. Depending on your tape
drive, the test may recommend that you add certain records to your
configuration. We strongly recommend that you do so and then re-run the above
test to insure it works the first time. 

Some of the suggestions it provides for resolving the problems may or may not
be useful. If at all possible avoid using fixed blocking. If the test suddenly
starts to print a long series of: 

\footnotesize
\begin{verbatim}
Got EOF on tape.
Got EOF on tape.
...
\end{verbatim}
\normalsize

then almost certainly, you are running your drive in fixed block mode rather
than variable block mode. See below for more help of resolving fix
versus variable block problems.

It is also possible that you have your drive
set in SysV tape drive mode. The drive must use BSD tape conventions.
See the section above on setting your {\bf Archive device} correctly.

For FreeBSD users, please see the notes below for doing further testing of
your tape drive. 

\subsection{Linux SCSI Tricks}
\index[general]{Tricks!Linux SCSI}
\index[general]{Linux SCSI Tricks}

You can find out what SCSI devices you have by doing: 

\footnotesize
\begin{verbatim}
cat /proc/scsi/scsi
\end{verbatim}
\normalsize

For example, I get the following: 

\footnotesize
\begin{verbatim}
Attached devices:
Host: scsi2 Channel: 00 Id: 01 Lun: 00
  Vendor: HP       Model: C5713A           Rev: H107
  Type:   Sequential-Access                ANSI SCSI revision: 02
Host: scsi2 Channel: 00 Id: 04 Lun: 00
  Vendor: SONY     Model: SDT-10000        Rev: 0110
  Type:   Sequential-Access                ANSI SCSI revision: 02
\end{verbatim}
\normalsize

The above represents first an autochanger and second a simple
tape drive. The HP changer (the first entry) uses the same SCSI channel
for data and for control, so in Bacula, you would use: 
\footnotesize
\begin{verbatim}
Archive Device = /dev/nst0
Changer Device = /dev/sg0
\end{verbatim}
\normalsize

If you want to remove the SDT-10000 device, you can do so as root with: 

\footnotesize
\begin{verbatim}
echo "scsi remove-single-device 2 0 4 0">/proc/scsi/scsi
\end{verbatim}
\normalsize

and you can put add it back with: 

\footnotesize
\begin{verbatim}
echo "scsi add-single-device 2 0 4 0">/proc/scsi/scsi
\end{verbatim}
\normalsize

where the 2 0 4 0 are the Host, Channel, Id, and Lun as seen on the output
from {\bf cat /proc/scsi/scsi}. Note, the Channel must be specified as
numeric. 

Below is a slightly more complicated output, which is a single autochanger
with two drives, and which operates the changer on a different channel
from from the drives:

\footnotesize
\begin{verbatim}
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: WDC WD1600JD-75H Rev: 08.0
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi2 Channel: 00 Id: 04 Lun: 00
  Vendor: HP       Model: Ultrium 2-SCSI   Rev: F6CH
  Type:   Sequential-Access                ANSI SCSI revision: 03
Host: scsi2 Channel: 00 Id: 05 Lun: 00
  Vendor: HP       Model: Ultrium 2-SCSI   Rev: F6CH
  Type:   Sequential-Access                ANSI SCSI revision: 03
Host: scsi2 Channel: 00 Id: 06 Lun: 00
  Vendor: OVERLAND Model: LXB              Rev: 0106
  Type:   Medium Changer                   ANSI SCSI revision: 02
\end{verbatim}
\normalsize

The above tape drives are accessed on /dev/nst0 and /dev/nst1, while
the control channel for those two drives is /dev/sg3.



\label{problems1}
\section{Tips for Resolving Problems}
\index[general]{Problems!Tips for Resolving}
\index[general]{Tips for Resolving Problems}

\label{CannotRestore}
\subsection{Bacula Saves But Cannot Restore Files}
\index[general]{Files!Bacula Saves But Cannot Restore}
\index[general]{Bacula Saves But Cannot Restore Files}

If you are getting error messages such as: 

\footnotesize
\begin{verbatim}
Volume data error at 0:1! Wanted block-id: "BB02", got "". Buffer discarded
\end{verbatim}
\normalsize

It is very likely that Bacula has tried to do block positioning and ended up
at an invalid block. This can happen if your tape drive is in fixed block mode
while Bacula's default is variable blocks. Note that in such cases, Bacula is
perfectly able to write to your Volumes (tapes), but cannot position to read
them. 

There are two possible solutions. 

\begin{enumerate}
\item The first and  best is to always ensure that your drive is in  variable
   block mode. Note, it can switch back to  fixed block mode on a reboot or if
   another program  uses the drive. So on such systems you  need to modify the
   Bacula startup files  to explicitly set: 

\footnotesize
\begin{verbatim}
mt -f /dev/nst0 defblksize 0
\end{verbatim}
\normalsize

or whatever is appropriate on your system. Note, if you are running a Linux
system, and the above command does not work, it is most likely because you
have not loaded the appropriate {\bf mt} package, which is often called
{\bf mt\_st}, but may differ according to your distribution.

\item The second possibility, especially, if Bacula wrote  while the drive was
   in fixed block mode, is to turn  off block positioning in Bacula. This is done
   by  adding: 

\footnotesize
\begin{verbatim}
Block Positioning = no
\end{verbatim}
\normalsize

to the Device resource. This is not the recommended  procedure because it can
enormously slow down  recovery of files, but it may help where all else 
fails. This directive is available in version 1.35.5  or later (and not yet
tested).  
\end{enumerate}

If you are getting error messages such as:
\footnotesize
\begin{verbatim}
Volume data error at 0:0!
Block checksum mismatch in block=0 len=32625 calc=345678 blk=123456
\end{verbatim}
\normalsize

You are getting tape read errors, and this is most likely due to 
one of the following things:
\begin{enumerate}
\item An old or bad tape.
\item A dirty drive that needs cleaning (particularly for DDS drives).
\item A loose SCSI cable.
\item Old firmware in your drive. Make sure you have the latest firmware
      loaded.
\item Computer memory errors.
\item Over-clocking your CPU.
\item A bad SCSI card.
\end{enumerate}


\label{opendevice}
\subsection{Bacula Cannot Open the Device}
\index[general]{Device!Bacula Cannot Open the}
\index[general]{Bacula Cannot Open the Device}

If you get an error message such as: 

\footnotesize
\begin{verbatim}
dev open failed: dev.c:265 stored: unable to open
device /dev/nst0:> ERR=No such device or address
\end{verbatim}
\normalsize

the first time you run a job, it is most likely due to the fact that you
specified the incorrect device name on your {\bf Archive Device}. 

If Bacula works fine with your drive, then all off a sudden you get error
messages similar to the one shown above, it is quite possible that your driver
module is being removed because the kernel deems it idle. This is done via
{\bf crontab} with the use of {\bf rmmod -a}. To fix the problem, you can
remove this entry from {\bf crontab}, or you can manually {\bf modprob} your
driver module (or add it to the local startup script). Thanks to Alan Brown
for this tip. 
\label{IncorrectFiles}

\subsection{Incorrect File Number}
\index[general]{Number!Incorrect File}
\index[general]{Incorrect File Number}

When Bacula moves to the end of the medium, it normally uses the {\bf
ioctl(MTEOM)} function. Then Bacula uses the {\bf ioctl(MTIOCGET)} function to
retrieve the current file position from the {\bf mt\_fileno} field. Some SCSI
tape drivers will use a fast means of seeking to the end of the medium and in
doing so, they will not know the current file position and hence return a {\bf
-1}. As a consequence, if you get {\bf "This is NOT correct!"} in the
positioning tests, this may be the cause. You must correct this condition in
order for Bacula to work. 

There are two possible solutions to the above problem of incorrect file
number: 

\begin{itemize}
\item Figure out how to configure your SCSI driver to  keep track of the file
   position during the MTEOM  request. This is the preferred solution.  
\item Modify the {\bf Device} resource of your {\bf bacula-sd.conf} file  to
   include:  

\footnotesize
\begin{verbatim}
Hardware End of File = no
\end{verbatim}
\normalsize

This will cause Bacula to use the MTFSF request to  seek to the end of the
medium, and Bacula will keep  track of the file number itself. 
\end{itemize}

\label{IncorrectBlocks}
\subsection{Incorrect Number of Blocks or Positioning Errors}
\index[general]{Testing!Incorrect Number of Blocks or Positioning Errors}
\index[general]{Incorrect Number of Blocks or Positioning Errors}

{\bf Bacula's} preferred method of working with tape drives (sequential
devices) is to run in variable block mode, and this is what is set by default.
You should first ensure that your tape drive is set for variable block mode
(see below). 

If your tape drive is in fixed block mode and you have told Bacula to use
different fixed block sizes or variable block sizes (default), you will get
errors when Bacula attempts to forward space to the correct block (the kernel
driver's idea of tape blocks will not correspond to Bacula's). 

All modern tape drives support variable tape blocks, but some older drives (in
particular the QIC drives) as well as the ATAPI ide-scsi driver run only in
fixed block mode. The Travan tape drives also apparently must run in fixed
block mode (to be confirmed). 

Even in variable block mode, with the exception of the first record on the
second or subsequent volume of a multi-volume backup, Bacula will write blocks
of a fixed size. However, in reading a tape, Bacula will assume that for each
read request, exactly one block from the tape will be transferred. This the
most common way that tape drives work and is well supported by {\bf Bacula}. 

Drives that run in fixed block mode can cause serious problems for Bacula if
the drive's block size does not correspond exactly to {\bf Bacula's} block
size. In fixed block size mode, drivers may transmit a partial block or
multiple blocks for a single read request. From {\bf Bacula's} point of view,
this destroys the concept of tape blocks. It is much better to run in variable
block mode, and almost all modern drives (the OnStream is an exception) run in
variable block mode. In order for Bacula to run in fixed block mode, you must
include the following records in the Storage daemon's Device resource
definition: 

\footnotesize
\begin{verbatim}
Minimum Block Size = nnn
Maximum Block Size = nnn
\end{verbatim}
\normalsize

where {\bf nnn} must be the same for both records and must be identical to the
driver's fixed block size. 

We recommend that you avoid this configuration if at all possible by using
variable block sizes. 

If you must run with fixed size blocks, make sure they are not 512 bytes. This
is too small and the overhead that Bacula has with each record will become
excessive. If at all possible set any fixed block size to something like
64,512 bytes or possibly 32,768 if 64,512 is too large for your drive. See
below for the details on checking and setting the default drive block size. 

To recover files from tapes written in fixed block mode, see below. 

\label{TapeModes}
\subsection{Ensuring that the Tape Modes Are Properly Set -- {\bf Linux
Only}}
\index[general]{Ensuring that the Tape Modes Are Properly Set -- Linux Only}

If you have a modern SCSI tape drive and you are having problems with the {\bf
test} command as noted above, it may be that some program has set one or more
of your SCSI driver's options to non-default values. For example, if your
driver is set to work in SysV manner, Bacula will not work correctly because
it expects BSD behavior. To reset your tape drive to the default values, you
can try the following, but {\bf ONLY} if you have a SCSI tape drive on a {\bf
Linux} system: 

\footnotesize
\begin{verbatim}
become super user
mt -f /dev/nst0 rewind
mt -f /dev/nst0 stoptions buffer-writes async-writes read-ahead
\end{verbatim}
\normalsize

The above commands will clear all options and then set those specified. None
of the specified options are required by Bacula, but a number of other options
such as SysV behavior must not be set. Bacula does not support SysV tape
behavior. On systems other than Linux, you will need to consult your {\bf mt}
man pages or documentation to figure out how to do the same thing. This should
not really be necessary though -- for example, on both Linux and Solaris
systems, the default tape driver options are compatible with Bacula. 
On Solaris systems, you must take care to specify the correct device
name on the {\bf Archive device} directive. See above for more details.

You may also want to ensure that no prior program has set the default block
size, as happened to one user, by explicitly turning it off with: 

\footnotesize
\begin{verbatim}
mt -f /dev/nst0 defblksize 0
\end{verbatim}
\normalsize

If you are running a Linux
system, and the above command does not work, it is most likely because you
have not loaded the appropriate {\bf mt} package, which is often called
{\bf mt\_st}, but may differ according to your distribution.

If you would like to know what options you have set before making any of the
changes noted above, you can now view them on Linux systems, thanks to a tip
provided by Willem Riede. Do the following: 

\footnotesize
\begin{verbatim}
become super user
mt -f /dev/nst0 stsetoptions 0
grep st0 /var/log/messages
\end{verbatim}
\normalsize

and you will get output that looks something like the following: 

\footnotesize
\begin{verbatim}
kernel: st0: Mode 0 options: buffer writes: 1, async writes: 1, read ahead: 1
kernel: st0:    can bsr: 0, two FMs: 0, fast mteom: 0, auto lock: 0,
kernel: st0:    defs for wr: 0, no block limits: 0, partitions: 0, s2 log: 0
kernel: st0:    sysv: 0 nowait: 0
\end{verbatim}
\normalsize

Note, I have chopped off the beginning of the line with the date and machine
name for presentation purposes. 

Some people find that the above settings only last until the next reboot, so
please check this otherwise you may have unexpected problems. 

Beginning with Bacula version 1.35.8, if Bacula detects that you are running
in variable block mode, it will attempt to set your drive appropriately. All
OSes permit setting variable block mode, but some OSes do not permit setting
the other modes that Bacula needs to function properly. 

\label{compression}
\subsection{Tape Hardware Compression and Blocking Size}
\index[general]{Tape Hardware Compression and Blocking Size}
\index[general]{Size!Tape Hardware Compression and Blocking Size}

As far as I can tell, there is no way with the {\bf mt} program to check if
your tape hardware compression is turned on or off. You can, however, turn it
on by using (on Linux): 

\footnotesize
\begin{verbatim}
become super user
mt -f /dev/nst0 defcompression 1
\end{verbatim}
\normalsize

and of course, if you use a zero instead of the one at the end, you will turn
it off. 

If you have built the {\bf mtx} program in the {\bf depkgs} package, you can
use tapeinfo to get quite a bit of information about your tape drive even if
it is not an autochanger. This program is called using the SCSI control
device. On Linux for tape drive /dev/nst0, this is usually /dev/sg0, while on
FreeBSD for /dev/nsa0, the control device is often /dev/pass2. For example on
my DDS-4 drive (/dev/nst0), I get the following: 

\footnotesize
\begin{verbatim}
tapeinfo -f /dev/sg0
Product Type: Tape Drive
Vendor ID: 'HP      '
Product ID: 'C5713A          '
Revision: 'H107'
Attached Changer: No
MinBlock:1
MaxBlock:16777215
SCSI ID: 5
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x26
BlockSize: 0             
\end{verbatim}
\normalsize

where the {\bf DataCompEnabled: yes} means that tape hardware compression is
turned on. You can turn it on and off (yes|no) by using the {\bf mt}
commands given above. Also, this output will tell you if the {\bf BlockSize}
is non-zero and hence set for a particular block size. Bacula is not likely to
work in such a situation because it will normally attempt to write blocks of
64,512 bytes, except the last block of the job which will generally be
shorter. The first thing to try is setting the default block size to zero
using the {\bf mt -f /dev/nst0 defblksize 0} command as shown above.
On FreeBSD, this would be something like: {\bf mt -f /dev/nsa0 blocksize 0}. 

On some operating systems with some tape drives, the amount of data that
can be written to the tape and whether or not compression is enabled is
determined by the density usually the {\bf mt -f /dev/nst0 setdensity xxx} command.
Often  {\bf mt -f /dev/nst0 status} will print out the current
density code that is used with the drive.  Most systems, but unfortunately
not all, set the density to the maximum by default. On some systems, you
can also get a list of all available density codes with:
{\bf mt -f /dev/nst0 densities} or a similar {\bf mt} command.
Note, for DLT and SDLT devices, no-compression versus compression is very 
often controlled by the density code.  On FreeBSD systems, the compression
mode is set using {\bf mt -f /dev/nsa0 comp xxx} where xxx is the
mode you want.  In general, see {\bf man mt}  for the options available on
your system.

Note, some of the above {\bf mt} commands may not be persistent depending
on your system configuration. That is they may be reset if a program  
other than Bacula uses the drive or, as is frequently the case, on reboot
of your system.
                   
If your tape drive requires fixed block sizes (very unusual), you can use the
following records: 

\footnotesize
\begin{verbatim}
Minimum Block Size = nnn
Maximum Block Size = nnn
\end{verbatim}
\normalsize

in your Storage daemon's Device resource to force Bacula to write fixed size
blocks (where you sent nnn to be the same for both of the above records). This
should be done only if your drive does not support variable block sizes, or
you have some other strong reasons for using fixed block sizes. As mentioned
above, a small fixed block size of 512 or 1024 bytes will be very inefficient.
Try to set any fixed block size to something like 64,512 bytes or larger if
your drive will support it. 

Also, note that the {\bf Medium Type} field of the output of {\bf tapeinfo}
reports {\bf Not Loaded}, which is not correct. As a consequence, you should
ignore that field as well as the {\bf Attached Changer} field. 

To recover files from tapes written in fixed block mode, see below. 
\label{FreeBSDTapes}

\subsection{Tape Modes on FreeBSD}
\index[general]{FreeBSD!Tape Modes on}
\index[general]{Tape Modes on FreeBSD}

On most FreeBSD systems such as 4.9 and most tape drives, Bacula should run
with: 

\footnotesize
\begin{verbatim}
mt  -f  /dev/nsa0  seteotmodel  2
mt  -f  /dev/nsa0  blocksize   0
mt  -f  /dev/nsa0  comp  enable
\end{verbatim}
\normalsize

You might want to put those commands in a startup script to make sure your
tape driver is properly initialized before running Bacula, because
depending on your system configuration, these modes may be reset if a      
program other than Bacula uses the drive or when your system is rebooted.

Then according to what the {\bf btape test} command returns, you will probably
need to set the following (see below for an alternative): 

\footnotesize
\begin{verbatim}
  Hardware End of Medium = no
  BSF at EOM = yes
  Backward Space Record = no
  Backward Space File = no
  Fast Forward Space File = no
  TWO EOF = yes
\end{verbatim}
\normalsize

Then be sure to run some append tests with Bacula where you start and stop
Bacula between appending to the tape, or use {\bf btape} version 1.35.1 or
greater, which includes simulation of stopping/restarting Bacula. 

Please see the file {\bf platforms/freebsd/pthreads-fix.txt} in the main
Bacula directory concerning {\bf important} information concerning
compatibility of Bacula and your system. A much more optimal Device
configuration is shown below, but does not work with all tape drives. Please
test carefully before putting either into production. 

Note, for FreeBSD 4.10-RELEASE, using a Sony TSL11000 L100 DDS4 with an
autochanger set to variable block size and DCLZ compression, Brian McDonald
reports that to get Bacula to append correctly between Bacula executions,
the correct values to use are:

\footnotesize
\begin{verbatim}
mt  -f  /dev/nsa0  seteotmodel  1
mt  -f  /dev/nsa0  blocksize  0
mt  -f /dev/nsa0  comp  enable
\end{verbatim}
\normalsize

and 

\footnotesize
\begin{verbatim}
  Hardware End of Medium = no
  BSF at EOM = no
  Backward Space Record = no
  Backward Space File = no
  Fast Forward Space File = yes
  TWO EOF = no
\end{verbatim}
\normalsize

This has been confirmed by several other people using different hardware. This
configuration is the preferred one because it uses one EOF and no backspacing
at the end of the tape, which works much more efficiently and reliably with
modern tape drives. 

Finally, here is a Device configuration that Danny Butroyd reports to work
correctly with the Overland Powerloader tape library using LT0-2 and
FreeBSD 5.4-Stable:

\footnotesize
\begin{verbatim}
# Overland Powerloader LT02 - 17 slots single drive
Device {
  Name = Powerloader
  Media Type = LT0-2
  Archive Device = /dev/nsa0
  AutomaticMount = yes;              
  AlwaysOpen = yes;
  RemovableMedia = yes;
  RandomAccess = no;
  Changer Command = "/usr/local/sbin/mtx-changer %c %o %S %a %d"
  Changer Device = /dev/pass2
  AutoChanger = yes
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  # FreeBSD Specific Settings
  Offline On Unmount = no
  Hardware End of Medium = no
  BSF at EOM = yes
  Backward Space Record = no
  Fast Forward Space File = no
  TWO EOF = yes
}

The following Device resource works fine with Dell PowerVault 110T and
120T devices on both FreeBSD 5.3 and on NetBSD 3.0.  It also works
with Sony AIT-2 drives on FreeBSD.
\footnotesize
\begin{verbatim}
Device {
  ...
  # FreeBSD/NetBSD Specific Settings
  Hardware End of Medium = no
  BSF at EOM = yes
  Backward Space Record = no
  Fast Forward Space File = yes
  TWO EOF = yes
}
\end{verbatim}
\normalsize

On FreeBSD version 6.0, it is reported that you can even set
Backward Space Record = yes.



\subsection{Finding your Tape Drives and Autochangers on FreeBSD}
\index[general]{FreeBSD!Finding Tape Drives and Autochangers}
\index[general]{Finding Tape Drives and Autochangers on FreeBSD}

On FreeBSD, you can do a {\bf camcontrol devlist} as root to determine what
drives and autochangers you have. For example, 

\footnotesize
\begin{verbatim}
undef# camcontrol devlist
    at scbus0 target 2 lun 0 (pass0,sa0)
    at scbus0 target 4 lun 0 (pass1,sa1)
    at scbus0 target 4 lun 1 (pass2)
\end{verbatim}
\normalsize

from the above, you can determine that there is a tape drive on {\bf /dev/sa0}
and another on {\bf /dev/sa1} in addition since there is a second line for the
drive on {\bf /dev/sa1}, you know can assume that it is the control device for
the autochanger (i.e. {\bf /dev/pass2}). It is also the control device name to
use when invoking the tapeinfo program. E.g. 

\footnotesize
\begin{verbatim}
tapeinfo -f /dev/pass2
\end{verbatim}
\normalsize

\label{onstream}

\subsection{Using the OnStream driver on Linux Systems}
\index[general]{Using the OnStream driver on Linux Systems}
\index[general]{Systems!Using the OnStream driver on Linux}

Bacula version 1.33 (not 1.32x) is now working and ready for testing with the
OnStream kernel osst driver version 0.9.14 or above. Osst is available from: 
\elink{http://sourceforge.net/projects/osst/}
{http://sourceforge.net/projects/osst/}. 

To make Bacula work you must first load the new driver then, as root, do: 

\footnotesize
\begin{verbatim}
  mt -f /dev/nosst0 defblksize 32768
\end{verbatim}
\normalsize

Also you must add the following to your Device resource in your Storage
daemon's conf file: 

\footnotesize
\begin{verbatim}
 Minimum Block Size = 32768
 Maximum Block Size = 32768
\end{verbatim}
\normalsize

Here is a Device specification provided by Michel Meyers that is known to
work: 

\footnotesize
\begin{verbatim}
Device {
  Name = "Onstream DI-30"
  Media Type = "ADR-30"
  Archive Device = /dev/nosst0
  Minimum Block Size = 32768
  Maximum Block Size = 32768
  Hardware End of Medium = yes
  BSF at EOM = no
  Backward Space File = yes
  Fast Forward Space File = yes
  Two EOF = no
  AutomaticMount = yes
  AlwaysOpen = yes
  Removable Media = yes
}
\end{verbatim}
\normalsize

\section{Hardware Compression on EXB-8900}
\index[general]{Hardware Compression on EXB-8900}
\index[general]{EXB-8900!Hardware Compression}

To active, check, or disable the hardware compression feature
on an EXB-8900, use the exabyte MammothTool. You can get it here:
\elink{http://www.exabyte.com/support/online/downloads/index.cfm}
{http://www.exabyte.com/support/online/downloads/index.cfm}.
There is a Solaris version of this tool. With option -C 0 or 1 you
can disable or activate compression. Start this tool without any
options for a small reference.

\label{fill}
\subsection{Using btape to Simulate Filling a Tape}
\index[general]{Using btape to Simulate Filling a Tape}
\index[general]{Tape!Using btape to Simulate Filling}

Because there are often problems with certain tape drives or systems when end
of tape conditions occur, {\bf btape} has a special command {\bf fill} that
causes it to write random data to a tape until the tape fills. It then writes
at least one more Bacula block to a second tape. Finally, it reads back both
tapes to ensure that the data has been written in a way that Bacula can
recover it. Note, there is also a single tape option as noted below, which you
should use rather than the two tape test. See below for more details. 

This can be an extremely time consuming process (here it is about 6 hours) to
fill a full tape. Note, that btape writes random data to the tape when it is
filling it. This has two consequences: 1. it takes a bit longer to generate
the data, especially on slow CPUs. 2. the total amount of data is
approximately the real physical capacity of your tape, regardless of whether
or not the tape drive compression is on or off. This is because random data
does not compress very much. 

To begin this test, you enter the {\bf fill} command and follow the
instructions. There are two options: the simple single tape option and the
multiple tape option. Please use only the simple single tape option because
the multiple tape option still doesn't work totally correctly. If the single
tape option does not succeed, you should correct the problem before using
Bacula. 
\label{RecoveringFiles}

\section{Recovering Files Written With Fixed Block Sizes}
\index[general]{Recovering Files Written With Fixed Block Sizes}

If you have been previously running your tape drive in fixed block mode
(default 512) and Bacula with variable blocks (default), then in version
1.32f-x and 1.34 and above, Bacula will fail to recover files because it does
block spacing, and because the block sizes don't agree between your tape drive
and Bacula it will not work. 

The long term solution is to run your drive in variable block mode as
described above. However, if you have written tapes using fixed block sizes,
this can be a bit of a pain. The solution to the problem is: while you are
doing a restore command using a tape written in fixed block size, ensure that
your drive is set to the fixed block size used while the tape was written.
Then when doing the {\bf restore} command in the Console program, do not
answer the prompt {\bf yes/mod/no}. Instead, edit the bootstrap file (the
location is listed in the prompt) using any ASCII editor. Remove all {\bf
VolBlock} lines in the file. When the file is re-written, answer the question,
and Bacula will run without using block positioning, and it should recover
your files. 

\label{BlockModes}
\section{Tape Blocking Modes}
\index[general]{Modes!Tape Blocking}
\index[general]{Tape Blocking Modes}

SCSI tapes may either be written in {\bf variable} or {\bf fixed} block sizes.
Newer drives support both modes, but some drives such as the QIC devices
always use fixed block sizes. Bacula attempts to fill and write complete
blocks (default 65K), so that in normal mode (variable block size), Bacula
will always write blocks of the same size except the last block of a Job. If
Bacula is configured to write fixed block sizes, it will pad the last block of
the Job to the correct size. Bacula expects variable tape block size drives to
behave as follows: Each write to the drive results in a single record being
written to the tape. Each read returns a single record. If you request less
bytes than are in the record, only those number of bytes will be returned, but
the entire logical record will have been read (the next read will retrieve the
next record). Thus data from a single write is always returned in a single
read, and sequentially written records are returned by sequential reads. 

Bacula expects fixed block size tape drives to behave as follows: If a write
length is greater than the physical block size of the drive, the write will be
written as two blocks each of the fixed physical size. This single write may
become multiple physical records on the tape. (This is not a good situation).
According to the documentation, one may never write an amount of data that is
not the exact multiple of the blocksize (it is not specified if an error
occurs or if the the last record is padded). When reading, it is my
understanding that each read request reads one physical record from the tape.
Due to the complications of fixed block size tape drives, you should avoid
them if possible with Bacula, or you must be ABSOLUTELY certain that you use
fixed block sizes within Bacula that correspond to the physical block size of
the tape drive. This will ensure that Bacula has a one to one correspondence
between what it writes and the physical record on the tape. 

Please note that Bacula will not function correctly if it writes a block and
that block is split into two or more physical records on the tape. Bacula
assumes that each write causes a single record to be written, and that it can
sequentially recover each of the blocks it has written by using the same
number of sequential reads as it had written. 

\section{Details of Tape Modes}
\index[general]{Modes!Details}
\index[general]{Details of Tape Modes}
Rudolf Cejka has provided the following information concerning
certain tape modes and MTEOM.

\begin{description}
\item[Tape level]
  It is always possible to position filemarks or blocks, whereas
  positioning to the end-of-data is only optional feature, however it is
  implemented very often.  SCSI specification also talks about optional
  sequential filemarks, setmarks and sequential setmarks, but these are not
  implemented so often.  Modern tape drives keep track of file positions in
  built-in chip (AIT, LTO) or at the beginning of the tape (SDLT), so there
  is not any speed difference, if end-of-data or filemarks is used (I have
  heard, that LTO-1 from all 3 manufacturers do not use its chip for file
  locations, but a tape as in SDLT case, and I'm not sure about LTO-2 and
  LTO-3 case).  However there is a big difference, that end-of-data ignores
  file position, whereas filemarks returns the real number of skipped
  files, so OS can track current file number just in filemarks case.

\item[OS level]
  Solaris does use just SCSI SPACE Filemarks, it does not support SCSI
  SPACE End-of-data.  When MTEOM is called, Solaris does use SCSI SPACE
  Filemarks with count = 1048576 for fast mode, and combination of SCSI
  SPACE Filemarks with count = 1 with SCSI SPACE Blocks with count = 1 for
  slow mode, so EOD mark on the tape on some older tape drives is not
  skipped.  File number is always tracked for MTEOM.

  Linux does support both SCSI SPACE Filemarks and End-of-data: When MTEOM
  is called in MT\_ST\_FAST\_MTEOM mode, SCSI SPACE End-of-data is used.
  In the other case, SCSI SPACE Filemarks with count =
  8388607 is used.  
  There is no real slow mode like in Solaris - I just expect, that for
  older tape drives Filemarks may be slower than End-of-data, but not so
  much as in Solaris slow mode.  File number is tracked for MTEOM just
  without MT\_ST\_FAST\_MTEOM - when MT\_ST\_FAST\_MTEOM is used, it is not.

  FreeBSD does support both SCSI SPACE Filemarks and End-of-data, but when
  MTEOD (MTEOM) is called, SCSI SPACE End-of-data is always used.  FreeBSD
  never use SCSI SPACE Filemarks for MTEOD. File number is never tracked
  for MTEOD.

\item[Bacula level]
  When {\bf Hardware End of Medium = Yes} is used, MTEOM is called, but it
  does not mean, that hardware End-of-data must be used.  When Hardware End
  of Medium = No, if Fast Forward Space File = Yes, MTFSF with count =
  32767 is used, else Block Read with count = 1 with Forward Space File
  with count = 1 is used, which is really very slow.

\item [Hardware End of Medium = Yes|No]
  The name of this option is misleading and is the source of confusion,
  because it is not the hardware EOM, what is really switched here.

  If I use Yes, OS must not use SCSI SPACE End-of-data, because Bacula
  expects, that there is tracked file number, which is not supported by
  SCSI specification.  Instead, the OS have to use SCSI SPACE Filemarks.

  If I use No, an action depends on Fast Forward Space File.

  When I set {\bf Hardware End of Medium = no}
  and {\bf Fast Forward Space File = no}
  file positioning was very slow
  on my LTO-3 (about ten to 100 minutes), but

  with {\bf Hardware End of Medium = no} and
{\bf Fast Forward Space File = yes}, the time is ten to
100 times faster (about one to two minutes).

\end{description}

\section{Autochanger Errors}
\index[general]{Errors!Autochanger}
\index[general]{Autochanger Errors}

If you are getting errors such as:

\footnotesize
\begin{verbatim}
3992 Bad autochanger "load slot 1, drive 1": ERR=Child exited with code 1.
\end{verbatim}
\normalsize

and you are running your Storage daemon as non-root, then most likely
you are having permissions problems with the control channel. Running
as root, set permissions on /dev/sgX so that the userid and group of
your Storage daemon can access the device. You need to ensure that you
all access to the proper control device, and if you don't have any
SCSI disk drives (including SATA drives), you might want to change
the permissions on /dev/sg*.

\section{Syslog Errors}
\index[general]{Errors!Syslog}
\index[general]{Syslog Errors}

If you are getting errors such as:

\footnotesize
\begin{verbatim}
: kernel: st0: MTSETDRVBUFFER only allowed for root
\end{verbatim}
\normalsize

you are most likely running your Storage daemon as non-root, and
Bacula is attempting to set the correct OS buffering to correspond
to your Device resource. Most OSes allow only root to issue this
ioctl command. In general, the message can be ignored providing 
you are sure that your OS parameters are properly configured as
described earlier in this manual.  If you are running your Storage daemon 
as root, you should not be getting these system log messages, and if
you are, something is probably wrong.