4 \chapter{Testing Your Tape Drive With Bacula}
5 \label{TapeTestingChapter}
6 \index[general]{Testing Your Tape Drive With Bacula}
8 This chapter is concerned with testing and configuring your tape drive to make
9 sure that it will work properly with Bacula using the {\bf btape} program.
12 \section{Get Your Tape Drive Working}
14 In general, you should follow the following steps to get your tape drive to
15 work with Bacula. Start with a tape mounted in your drive. If you have an
16 autochanger, load a tape into the drive. We use {\bf /dev/nst0} as the tape
17 drive name, you will need to adapt it according to your system.
19 Do not proceed to the next item until you have succeeded with the previous
23 \item Make sure that Bacula (the Storage daemon) is not running
24 or that you have {\bf unmount}ed the drive you will use
27 \item Use tar to write to, then read from your drive:
31 mt -f /dev/nst0 rewind
33 mt -f /dev/nst0 rewind
39 \item Make sure you have a valid and correct Device resource corresponding
40 to your drive. For Linux users, generally, the default one works. For
41 FreeBSD users, there are two possible Device configurations (see below).
42 For other drives and/or OSes, you will need to first ensure that your
43 system tape modes are properly setup (see below), then possibly modify
44 you Device resource depending on the output from the btape program (next
45 item). When doing this, you should consult the \borgxrlink{Storage Daemon
46 Configuration}{StoredConfChapter}{main}{chapter} of the \mainman{}.
48 \item If you are using a Fibre Channel to connect your tape drive to
49 Bacula, please be sure to disable any caching in the NSR (network
50 storage router, which is a Fibre Channel to SCSI converter).
52 \item Run the btape {\bf test} command:
56 ./btape -c bacula-sd.conf /dev/nst0
62 It isn't necessary to run the autochanger part of the test at this time,
63 but do not go past this point until the basic test succeeds. If you do
64 have an autochanger, please be sure to read the \borgxrlink{Autochanger
65 chapter}{AutochangersChapter}{main}{chapter} of the \mainman{}.
67 \item Run the btape {\bf fill} command, preferably with two volumes. This
68 can take a long time. If you have an autochanger and it is configured, Bacula
69 will automatically use it. If you do not have it configured, you can manually
70 issue the appropriate {\bf mtx} command, or press the autochanger buttons to
71 change the tape when requested to do so.
73 \item FreeBSD users, if you have a pre-5.0 system run the {\bf tapetest}
74 program, and make sure your system is patched if necessary. The tapetest
75 program can be found in the platform/freebsd directory. The instructions
76 for its use are at the top of the file.
78 \item Run Bacula, and backup a reasonably small directory, say 60
79 Megabytes. Do three successive backups of this directory.
81 \item Stop Bacula, then restart it. Do another full backup of the same
82 directory. Then stop and restart Bacula.
84 \item Do a restore of the directory backed up, by entering the following
85 restore command, being careful to restore it to an alternate location:
90 restore select all done
96 Do a {\bf diff} on the restored directory to ensure it is identical to the
97 original directory. If you are going to backup multiple different systems
98 (Linux, Windows, Mac, Solaris, FreeBSD, ...), be sure you test the restore
101 \item If you have an autochanger, you should now go back to the btape program
102 and run the autochanger test:
106 ./btape -c bacula-sd.conf /dev/nst0
112 Adjust your autochanger as necessary to ensure that it works correctly. See
113 the Autochanger chapter of this manual for a complete discussion of testing
116 \item We strongly recommend that you use a dedicated SCSI
117 controller for your tape drives. Scanners are known to induce
118 serious problems with the SCSI bus, causing it to reset. If the
119 SCSI bus is reset while Bacula has the tape drive open, it will
120 most likely be fatal to your tape since the drive will rewind.
121 These kinds of problems show up in the system log. For example,
122 the following was most likely caused by a scanner:
126 Feb 14 17:29:55 epohost kernel: (scsi0:A:2:0): No or incomplete CDB sent to device.
127 Feb 14 17:29:55 epohost kernel: scsi0: Issued Channel A Bus Reset. 1 SCBs aborted
133 If you have reached this point, you stand a good chance of having everything
134 work. If you get into trouble at any point, {\bf carefully} read the
135 documentation given below. If you cannot get past some point, ask the {\bf
136 bacula-users} email list, but specify which of the steps you have successfully
137 completed. In particular, you may want to look at the
138 \ilink{ Tips for Resolving Problems}{problems1} section below.
141 \label{NoTapeInDrive}
142 \subsection{Problems When no Tape in Drive}
143 \index[general]{Problems When no Tape in Drive}
144 When Bacula was first written the Linux 2.4 kernel permitted opening the
145 drive whether or not there was a tape in the drive. Thus the Bacula code is
146 based on the concept that if the drive cannot be opened, there is a serious
147 problem, and the job is failed.
149 With version 2.6 of the Linux kernel, if there is no tape in the drive, the
150 OS will wait two minutes (default) and then return a failure, and consequently,
151 Bacula version 1.36 and below will fail the job. This is important to keep
152 in mind, because if you use an option such as {\bf Offline on Unmount =
153 yes}, there will be a point when there is no tape in the drive, and if
154 another job starts or if Bacula asks the operator to mount a tape, when
155 Bacula attempts to open the drive (about a 20 minute delay), it will fail
156 and Bacula will fail the job.
158 In version 1.38.x, the Bacula code partially gets around this problem -- at
159 least in the initial open of the drive. However, functions like Polling
160 the drive do not work correctly if there is no tape in the drive.
161 Providing you do not use {\bf Offline on Unmount = yes}, you should not
162 experience job failures as mentioned above. If you do experience such
163 failures, you can also increase the {\bf Maximum Open Wait} time interval,
164 which will give you more time to mount the next tape before the job is
167 \subsection{Specifying the Configuration File}
168 \index[general]{File!Specifying the Configuration}
169 \index[general]{Specifying the Configuration File}
171 Starting with version 1.27, each of the tape utility programs including the
172 {\bf btape} program requires a valid Storage daemon configuration file
173 (actually, the only part of the configuration file that {\bf btape} needs is
174 the {\bf Device} resource definitions). This permits {\bf btape} to find the
175 configuration parameters for your archive device (generally a tape drive).
176 Without those parameters, the testing and utility programs do not know how to
177 properly read and write your drive. By default, they use {\bf bacula-sd.conf}
178 in the current directory, but you may specify a different configuration file
179 using the {\bf -c} option.
181 \subsection{Specifying a Device Name For a Tape}
182 \index[general]{Tape!Specifying a Device Name For a}
183 \index[general]{Specifying a Device Name For a Tape}
185 {\bf btape} {\bf device-name} where the Volume can be found. In the case of a
186 tape, this is the physical device name such as {\bf /dev/nst0} or {\bf
187 /dev/rmt/0ubn} depending on your system that you specify on the Archive Device
188 directive. For the program to work, it must find the identical name in the
189 Device resource of the configuration file. If the name is not found in the
190 list of physical names, the utility program will compare the name you entered
191 to the Device names (rather than the Archive device names).
193 When specifying a tape device, it is preferable that the "non-rewind"
194 variant of the device file name be given. In addition, on systems such as
195 Sun, which have multiple tape access methods, you must be sure to specify
196 to use Berkeley I/O conventions with the device. The
197 {\bf b} in the Solaris (Sun) archive specification {\bf /dev/rmt/0mbn} is
198 what is needed in this case. Bacula does not support SysV tape drive
201 See below for specifying Volume names.
203 \subsection{Specifying a Device Name For a File}
204 \index[general]{File!Specifying a Device Name For a}
205 \index[general]{Specifying a Device Name For a File}
207 If you are attempting to read or write an archive file rather than a tape, the
208 {\bf device-name} should be the full path to the archive location including
209 the filename. The filename (last part of the specification) will be stripped
210 and used as the Volume name, and the path (first part before the filename)
211 must have the same entry in the configuration file. So, the path is equivalent
212 to the archive device name, and the filename is equivalent to the volume name.
217 \index[general]{Btape}
219 This program permits a number of elementary tape operations via a tty command
220 interface. The {\bf test} command, described below, can be very useful for
221 testing tape drive compatibility problems. Aside from initial testing of tape
222 drive compatibility with {\bf Bacula}, {\bf btape} will be mostly used by
223 developers writing new tape drivers.
225 {\bf btape} can be dangerous to use with existing {\bf Bacula} tapes because
226 it will relabel a tape or write on the tape if so requested regardless of
227 whether or not the tape contains valuable data, so please be careful and use
228 it only on blank tapes.
230 To work properly, {\bf btape} needs to read the Storage daemon's configuration
231 file. As a default, it will look for {\bf bacula-sd.conf} in the current
232 directory. If your configuration file is elsewhere, please use the {\bf -c}
233 option to specify where.
235 The physical device name or the Device resource name must be specified on the
236 command line, and this same device name must be present in the Storage
237 daemon's configuration file read by {\bf btape}
241 Usage: btape [options] device_name
242 -b <file> specify bootstrap file
243 -c <file> set configuration file to file
244 -d <nn> set debug level to nn
245 -p proceed inspite of I/O errors
248 -? print this message.
252 \subsection{Using btape to Verify your Tape Drive}
253 \index[general]{Using btape to Verify your Tape Drive}
254 \index[general]{Drive!Using btape to Verify your Tape}
256 An important reason for this program is to ensure that a Storage daemon
257 configuration file is defined so that Bacula will correctly read and write
260 It is highly recommended that you run the {\bf test} command before running
261 your first Bacula job to ensure that the parameters you have defined for your
262 storage device (tape drive) will permit {\bf Bacula} to function properly. You
263 only need to mount a blank tape, enter the command, and the output should be
264 reasonably self explanatory. For example:
268 (ensure that Bacula is not running)
269 ./btape -c /usr/bin/bacula/bacula-sd.conf /dev/nst0
277 Tape block granularity is 1024 bytes.
278 btape: btape.c:376 Using device: /dev/nst0
283 Enter the test command:
291 The output produced should be something similar to the following: I've cut the
292 listing short because it is frequently updated to have new tests.
296 === Append files test ===
297 This test is essential to Bacula.
298 I'm going to write one record in file 0,
299 two records in file 1,
300 and three records in file 2
301 btape: btape.c:387 Rewound /dev/nst0
302 btape: btape.c:855 Wrote one record of 64412 bytes.
303 btape: btape.c:857 Wrote block to device.
304 btape: btape.c:410 Wrote EOF to /dev/nst0
305 btape: btape.c:855 Wrote one record of 64412 bytes.
306 btape: btape.c:857 Wrote block to device.
307 btape: btape.c:855 Wrote one record of 64412 bytes.
308 btape: btape.c:857 Wrote block to device.
309 btape: btape.c:410 Wrote EOF to /dev/nst0
310 btape: btape.c:855 Wrote one record of 64412 bytes.
311 btape: btape.c:857 Wrote block to device.
312 btape: btape.c:855 Wrote one record of 64412 bytes.
313 btape: btape.c:857 Wrote block to device.
314 btape: btape.c:855 Wrote one record of 64412 bytes.
315 btape: btape.c:857 Wrote block to device.
316 btape: btape.c:410 Wrote EOF to /dev/nst0
317 btape: btape.c:387 Rewound /dev/nst0
318 btape: btape.c:693 Now moving to end of media.
319 btape: btape.c:427 Moved to end of media
320 We should be in file 3. I am at file 3. This is correct!
321 Now the important part, I am going to attempt to append to the tape.
323 === End Append files test ===
327 If you do not successfully complete the above test, please resolve the
328 problem(s) before attempting to use {\bf Bacula}. Depending on your tape
329 drive, the test may recommend that you add certain records to your
330 configuration. We strongly recommend that you do so and then re-run the above
331 test to insure it works the first time.
333 Some of the suggestions it provides for resolving the problems may or may not
334 be useful. If at all possible avoid using fixed blocking. If the test suddenly
335 starts to print a long series of:
345 then almost certainly, you are running your drive in fixed block mode rather
346 than variable block mode. See below for more help of resolving fix
347 versus variable block problems.
349 It is also possible that you have your drive
350 set in SysV tape drive mode. The drive must use BSD tape conventions.
351 See the section above on setting your {\bf Archive device} correctly.
353 For FreeBSD users, please see the notes below for doing further testing of
356 \subsection{Testing tape drive speed}
357 \label{sec:btapespeed}
359 To determine the best configuration of your tape drive, you can run the
360 \texttt{speed} command available in the \texttt{btape} program.
362 This command can have the following arguments:
364 \item[\texttt{file\_size=n}] Specify the Maximum File Size for this test
365 (between 1 and 5GB). This counter is in GB.
366 \item[\texttt{nb\_file=n}] Specify the number of file to be written. The amount
367 of data should be greater than your memory ($file\_size*nb\_file$).
368 \item[\texttt{skip\_zero}] This flag permits to skip tests with constant
370 \item[\texttt{skip\_random}] This flag permits to skip tests with random
372 \item[\texttt{skip\_raw}] This flag permits to skip tests with raw access.
373 \item[\texttt{skip\_block}] This flag permits to skip tests with Bacula block
378 *speed file_size=3 skip_raw
379 btape.c:1078 Test with zero data and bacula block structure.
380 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
381 ++++++++++++++++++++++++++++++++++++++++++
382 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
383 btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s
385 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s
387 btape.c:1090 Test with random data, should give the minimum throughput.
388 btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
389 +++++++++++++++++++++++++++++++++++++++++++
390 btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
391 btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s
392 +++++++++++++++++++++++++++++++++++++++++++
394 btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s
398 When using compression, the random test will give your the minimum throughput
399 of your drive . The test using constant string will give you the maximum speed
400 of your hardware chain. (cpu, memory, scsi card, cable, drive, tape).
402 You can change the block size in the Storage Daemon configuration file.
405 \subsection{Linux SCSI Tricks}
406 \index[general]{Tricks!Linux SCSI}
407 \index[general]{Linux SCSI Tricks}
409 You can find out what SCSI devices you have by doing:
421 [0:0:0:0] disk ATA ST3160812AS 3.AD /dev/sda
422 [2:0:4:0] tape HP Ultrium 2-SCSI F6CH /dev/st0
423 [2:0:5:0] tape HP Ultrium 2-SCSI F6CH /dev/st1
424 [2:0:6:0] mediumx OVERLAND LXB 0107 -
425 [2:0:9:0] tape HP Ultrium 1-SCSI E50H /dev/st2
426 [2:0:10:0] mediumx OVERLAND LXB 0107 -
430 There are two drives in one autochanger: /dev/st0 and /dev/st1
431 and a third tape drive at /dev/st2. For using them with Bacula, one
432 would normally reference them as /dev/nst0 ... /dev/nst2. Not also,
433 there are two different autochangers identified as "mediumx OVERLAND LXB".
434 They can be addressed via their /dev/sgN designation, which can be
435 obtained by counting from the beginning as 0 to each changer. In the
436 above case, the two changers are located on /dev/sg3 and /dev/sg5. The one
437 at /dev/sg3, controls drives /dev/nst0 and /dev/nst1; and the one at
438 /dev/sg5 controles drive /dev/nst2.
440 If you do not have the {\bf lsscsi} command, you can obtain the same
441 information as follows:
449 For the above example with the three drives and two autochangers,
455 Host: scsi0 Channel: 00 Id: 00 Lun: 00
456 Vendor: ATA Model: ST3160812AS Rev: 3.AD
457 Type: Direct-Access ANSI SCSI revision: 05
458 Host: scsi2 Channel: 00 Id: 04 Lun: 00
459 Vendor: HP Model: Ultrium 2-SCSI Rev: F6CH
460 Type: Sequential-Access ANSI SCSI revision: 03
461 Host: scsi2 Channel: 00 Id: 05 Lun: 00
462 Vendor: HP Model: Ultrium 2-SCSI Rev: F6CH
463 Type: Sequential-Access ANSI SCSI revision: 03
464 Host: scsi2 Channel: 00 Id: 06 Lun: 00
465 Vendor: OVERLAND Model: LXB Rev: 0107
466 Type: Medium Changer ANSI SCSI revision: 02
467 Host: scsi2 Channel: 00 Id: 09 Lun: 00
468 Vendor: HP Model: Ultrium 1-SCSI Rev: E50H
469 Type: Sequential-Access ANSI SCSI revision: 03
470 Host: scsi2 Channel: 00 Id: 10 Lun: 00
471 Vendor: OVERLAND Model: LXB Rev: 0107
472 Type: Medium Changer ANSI SCSI revision: 02
477 As an additional example, I get the following (on a different machine from the
483 Host: scsi2 Channel: 00 Id: 01 Lun: 00
484 Vendor: HP Model: C5713A Rev: H107
485 Type: Sequential-Access ANSI SCSI revision: 02
486 Host: scsi2 Channel: 00 Id: 04 Lun: 00
487 Vendor: SONY Model: SDT-10000 Rev: 0110
488 Type: Sequential-Access ANSI SCSI revision: 02
492 The above represents first an autochanger and second a simple
493 tape drive. The HP changer (the first entry) uses the same SCSI channel
494 for data and for control, so in Bacula, you would use:
497 Archive Device = /dev/nst0
498 Changer Device = /dev/sg0
502 If you want to remove the SDT-10000 device, you can do so as root with:
506 echo "scsi remove-single-device 2 0 4 0">/proc/scsi/scsi
510 and you can put add it back with:
514 echo "scsi add-single-device 2 0 4 0">/proc/scsi/scsi
518 where the 2 0 4 0 are the Host, Channel, Id, and Lun as seen on the output
519 from {\bf cat /proc/scsi/scsi}. Note, the Channel must be specified as
522 Below is a slightly more complicated output, which is a single autochanger
523 with two drives, and which operates the changer on a different channel
524 from from the drives:
529 Host: scsi0 Channel: 00 Id: 00 Lun: 00
530 Vendor: ATA Model: WDC WD1600JD-75H Rev: 08.0
531 Type: Direct-Access ANSI SCSI revision: 05
532 Host: scsi2 Channel: 00 Id: 04 Lun: 00
533 Vendor: HP Model: Ultrium 2-SCSI Rev: F6CH
534 Type: Sequential-Access ANSI SCSI revision: 03
535 Host: scsi2 Channel: 00 Id: 05 Lun: 00
536 Vendor: HP Model: Ultrium 2-SCSI Rev: F6CH
537 Type: Sequential-Access ANSI SCSI revision: 03
538 Host: scsi2 Channel: 00 Id: 06 Lun: 00
539 Vendor: OVERLAND Model: LXB Rev: 0106
540 Type: Medium Changer ANSI SCSI revision: 02
544 The above tape drives are accessed on /dev/nst0 and /dev/nst1, while
545 the control channel for those two drives is /dev/sg3.
550 \section{Tips for Resolving Problems}
551 \index[general]{Problems!Tips for Resolving}
552 \index[general]{Tips for Resolving Problems}
554 \label{CannotRestore}
555 \subsection{Bacula Saves But Cannot Restore Files}
556 \index[general]{Files!Bacula Saves But Cannot Restore}
557 \index[general]{Bacula Saves But Cannot Restore Files}
559 If you are getting error messages such as:
563 Volume data error at 0:1! Wanted block-id: "BB02", got "". Buffer discarded
567 It is very likely that Bacula has tried to do block positioning and ended up
568 at an invalid block. This can happen if your tape drive is in fixed block mode
569 while Bacula's default is variable blocks. Note that in such cases, Bacula is
570 perfectly able to write to your Volumes (tapes), but cannot position to read
573 There are two possible solutions.
576 \item The first and best is to always ensure that your drive is in variable
577 block mode. Note, it can switch back to fixed block mode on a reboot or if
578 another program uses the drive. So on such systems you need to modify the
579 Bacula startup files to explicitly set:
583 mt -f /dev/nst0 defblksize 0
587 or whatever is appropriate on your system. Note, if you are running a Linux
588 system, and the above command does not work, it is most likely because you
589 have not loaded the appropriate {\bf mt} package, which is often called
590 {\bf mt\_st}, but may differ according to your distribution.
592 \item The second possibility, especially, if Bacula wrote while the drive was
593 in fixed block mode, is to turn off block positioning in Bacula. This is done
598 Block Positioning = no
602 to the Device resource. This is not the recommended procedure because it can
603 enormously slow down recovery of files, but it may help where all else
604 fails. This directive is available in version 1.35.5 or later (and not yet
608 If you are getting error messages such as:
611 Volume data error at 0:0!
612 Block checksum mismatch in block=0 len=32625 calc=345678 blk=123456
616 You are getting tape read errors, and this is most likely due to
617 one of the following things:
619 \item An old or bad tape.
620 \item A dirty drive that needs cleaning (particularly for DDS drives).
621 \item A loose SCSI cable.
622 \item Old firmware in your drive. Make sure you have the latest firmware
624 \item Computer memory errors.
625 \item Over-clocking your CPU.
626 \item A bad SCSI card.
631 \subsection{Bacula Cannot Open the Device}
632 \index[general]{Device!Bacula Cannot Open the}
633 \index[general]{Bacula Cannot Open the Device}
635 If you get an error message such as:
639 dev open failed: dev.c:265 stored: unable to open
640 device /dev/nst0:> ERR=No such device or address
644 the first time you run a job, it is most likely due to the fact that you
645 specified the incorrect device name on your {\bf Archive Device}.
647 If Bacula works fine with your drive, then all off a sudden you get error
648 messages similar to the one shown above, it is quite possible that your driver
649 module is being removed because the kernel deems it idle. This is done via
650 {\bf crontab} with the use of {\bf rmmod -a}. To fix the problem, you can
651 remove this entry from {\bf crontab}, or you can manually {\bf modprob} your
652 driver module (or add it to the local startup script). Thanks to Alan Brown
654 \label{IncorrectFiles}
656 \subsection{Incorrect File Number}
657 \index[general]{Number!Incorrect File}
658 \index[general]{Incorrect File Number}
660 When Bacula moves to the end of the medium, it normally uses the {\bf
661 ioctl(MTEOM)} function. Then Bacula uses the {\bf ioctl(MTIOCGET)} function to
662 retrieve the current file position from the {\bf mt\_fileno} field. Some SCSI
663 tape drivers will use a fast means of seeking to the end of the medium and in
664 doing so, they will not know the current file position and hence return a {\bf
665 -1}. As a consequence, if you get {\bf "This is NOT correct!"} in the
666 positioning tests, this may be the cause. You must correct this condition in
667 order for Bacula to work.
669 There are two possible solutions to the above problem of incorrect file
673 \item Figure out how to configure your SCSI driver to keep track of the file
674 position during the MTEOM request. This is the preferred solution.
675 \item Modify the {\bf Device} resource of your {\bf bacula-sd.conf} file to
680 Hardware End of File = no
684 This will cause Bacula to use the MTFSF request to seek to the end of the
685 medium, and Bacula will keep track of the file number itself.
688 \label{IncorrectBlocks}
689 \subsection{Incorrect Number of Blocks or Positioning Errors}
690 \index[general]{Testing!Incorrect Number of Blocks or Positioning Errors}
691 \index[general]{Incorrect Number of Blocks or Positioning Errors}
693 {\bf Bacula's} preferred method of working with tape drives (sequential
694 devices) is to run in variable block mode, and this is what is set by default.
695 You should first ensure that your tape drive is set for variable block mode
698 If your tape drive is in fixed block mode and you have told Bacula to use
699 different fixed block sizes or variable block sizes (default), you will get
700 errors when Bacula attempts to forward space to the correct block (the kernel
701 driver's idea of tape blocks will not correspond to Bacula's).
703 All modern tape drives support variable tape blocks, but some older drives (in
704 particular the QIC drives) as well as the ATAPI ide-scsi driver run only in
705 fixed block mode. The Travan tape drives also apparently must run in fixed
706 block mode (to be confirmed).
708 Even in variable block mode, with the exception of the first record on the
709 second or subsequent volume of a multi-volume backup, Bacula will write blocks
710 of a fixed size. However, in reading a tape, Bacula will assume that for each
711 read request, exactly one block from the tape will be transferred. This the
712 most common way that tape drives work and is well supported by {\bf Bacula}.
714 Drives that run in fixed block mode can cause serious problems for Bacula if
715 the drive's block size does not correspond exactly to {\bf Bacula's} block
716 size. In fixed block size mode, drivers may transmit a partial block or
717 multiple blocks for a single read request. From {\bf Bacula's} point of view,
718 this destroys the concept of tape blocks. It is much better to run in variable
719 block mode, and almost all modern drives (the OnStream is an exception) run in
720 variable block mode. In order for Bacula to run in fixed block mode, you must
721 include the following records in the Storage daemon's Device resource
726 Minimum Block Size = nnn
727 Maximum Block Size = nnn
731 where {\bf nnn} must be the same for both records and must be identical to the
732 driver's fixed block size.
734 We recommend that you avoid this configuration if at all possible by using
735 variable block sizes.
737 If you must run with fixed size blocks, make sure they are not 512 bytes. This
738 is too small and the overhead that Bacula has with each record will become
739 excessive. If at all possible set any fixed block size to something like
740 64,512 bytes or possibly 32,768 if 64,512 is too large for your drive. See
741 below for the details on checking and setting the default drive block size.
743 To recover files from tapes written in fixed block mode, see below.
746 \subsection{Ensuring that the Tape Modes Are Properly Set -- {\bf Linux
748 \index[general]{Ensuring that the Tape Modes Are Properly Set -- Linux Only}
750 If you have a modern SCSI tape drive and you are having problems with the {\bf
751 test} command as noted above, it may be that some program has set one or more
752 of your SCSI driver's options to non-default values. For example, if your
753 driver is set to work in SysV manner, Bacula will not work correctly because
754 it expects BSD behavior. To reset your tape drive to the default values, you
755 can try the following, but {\bf ONLY} if you have a SCSI tape drive on a {\bf
761 mt -f /dev/nst0 rewind
762 mt -f /dev/nst0 stoptions buffer-writes async-writes read-ahead
766 The above commands will clear all options and then set those specified. None
767 of the specified options are required by Bacula, but a number of other options
768 such as SysV behavior must not be set. Bacula does not support SysV tape
769 behavior. On systems other than Linux, you will need to consult your {\bf mt}
770 man pages or documentation to figure out how to do the same thing. This should
771 not really be necessary though -- for example, on both Linux and Solaris
772 systems, the default tape driver options are compatible with Bacula.
773 On Solaris systems, you must take care to specify the correct device
774 name on the {\bf Archive device} directive. See above for more details.
776 You may also want to ensure that no prior program has set the default block
777 size, as happened to one user, by explicitly turning it off with:
781 mt -f /dev/nst0 defblksize 0
785 If you are running a Linux
786 system, and the above command does not work, it is most likely because you
787 have not loaded the appropriate {\bf mt} package, which is often called
788 {\bf mt\_st}, but may differ according to your distribution.
790 If you would like to know what options you have set before making any of the
791 changes noted above, you can now view them on Linux systems, thanks to a tip
792 provided by Willem Riede. Do the following:
797 mt -f /dev/nst0 stsetoptions 0
798 grep st0 /var/log/messages
802 and you will get output that looks something like the following:
806 kernel: st0: Mode 0 options: buffer writes: 1, async writes: 1, read ahead: 1
807 kernel: st0: can bsr: 0, two FMs: 0, fast mteom: 0, auto lock: 0,
808 kernel: st0: defs for wr: 0, no block limits: 0, partitions: 0, s2 log: 0
809 kernel: st0: sysv: 0 nowait: 0
813 Note, I have chopped off the beginning of the line with the date and machine
814 name for presentation purposes.
816 Some people find that the above settings only last until the next reboot, so
817 please check this otherwise you may have unexpected problems.
819 Beginning with Bacula version 1.35.8, if Bacula detects that you are running
820 in variable block mode, it will attempt to set your drive appropriately. All
821 OSes permit setting variable block mode, but some OSes do not permit setting
822 the other modes that Bacula needs to function properly.
825 \subsection{Tape Hardware Compression and Blocking Size}
826 \index[general]{Tape Hardware Compression and Blocking Size}
827 \index[general]{Size!Tape Hardware Compression and Blocking Size}
829 You should be able to verify the tape compression status with sysfs on Linux.
831 cat /sys/class/scsi_tape/nst0/default_compression
834 You can, turn it on by using (on Linux):
839 mt -f /dev/nst0 defcompression 1
843 and of course, if you use a zero instead of the one at the end, you will turn
846 If you have built the {\bf mtx} program in the {\bf depkgs} package, you can
847 use tapeinfo to get quite a bit of information about your tape drive even if
848 it is not an autochanger. This program is called using the SCSI control
849 device. On Linux for tape drive /dev/nst0, this is usually /dev/sg0, while on
850 FreeBSD for /dev/nsa0, the control device is often /dev/pass2. For example on
851 my DDS-4 drive (/dev/nst0), I get the following:
856 Product Type: Tape Drive
858 Product ID: 'C5713A '
867 Medium Type: Not Loaded
873 where the {\bf DataCompEnabled: yes} means that tape hardware compression is
874 turned on. You can turn it on and off (yes|no) by using the {\bf mt}
875 commands given above. Also, this output will tell you if the {\bf BlockSize}
876 is non-zero and hence set for a particular block size. Bacula is not likely to
877 work in such a situation because it will normally attempt to write blocks of
878 64,512 bytes, except the last block of the job which will generally be
879 shorter. The first thing to try is setting the default block size to zero
880 using the {\bf mt -f /dev/nst0 defblksize 0} command as shown above.
881 On FreeBSD, this would be something like: {\bf mt -f /dev/nsa0 blocksize 0}.
883 On some operating systems with some tape drives, the amount of data that
884 can be written to the tape and whether or not compression is enabled is
885 determined by the density usually the {\bf mt -f /dev/nst0 setdensity xxx} command.
886 Often {\bf mt -f /dev/nst0 status} will print out the current
887 density code that is used with the drive. Most systems, but unfortunately
888 not all, set the density to the maximum by default. On some systems, you
889 can also get a list of all available density codes with:
890 {\bf mt -f /dev/nst0 densities} or a similar {\bf mt} command.
891 Note, for DLT and SDLT devices, no-compression versus compression is very
892 often controlled by the density code. On FreeBSD systems, the compression
893 mode is set using {\bf mt -f /dev/nsa0 comp xxx} where xxx is the
894 mode you want. In general, see {\bf man mt} for the options available on
897 Note, some of the above {\bf mt} commands may not be persistent depending
898 on your system configuration. That is they may be reset if a program
899 other than Bacula uses the drive or, as is frequently the case, on reboot
902 If your tape drive requires fixed block sizes (very unusual), you can use the
907 Minimum Block Size = nnn
908 Maximum Block Size = nnn
912 in your Storage daemon's Device resource to force Bacula to write fixed size
913 blocks (where you sent nnn to be the same for both of the above records). This
914 should be done only if your drive does not support variable block sizes, or
915 you have some other strong reasons for using fixed block sizes. As mentioned
916 above, a small fixed block size of 512 or 1024 bytes will be very inefficient.
917 Try to set any fixed block size to something like 64,512 bytes or larger if
918 your drive will support it.
920 Also, note that the {\bf Medium Type} field of the output of {\bf tapeinfo}
921 reports {\bf Not Loaded}, which is not correct. As a consequence, you should
922 ignore that field as well as the {\bf Attached Changer} field.
924 To recover files from tapes written in fixed block mode, see below.
927 \subsection{Tape Modes on FreeBSD}
928 \index[general]{FreeBSD!Tape Modes on}
929 \index[general]{Tape Modes on FreeBSD}
931 On most FreeBSD systems such as 4.9 and most tape drives, Bacula should run
936 mt -f /dev/nsa0 seteotmodel 2
937 mt -f /dev/nsa0 blocksize 0
938 mt -f /dev/nsa0 comp enable
942 You might want to put those commands in a startup script to make sure your
943 tape driver is properly initialized before running Bacula, because
944 depending on your system configuration, these modes may be reset if a
945 program other than Bacula uses the drive or when your system is rebooted.
947 Then according to what the {\bf btape test} command returns, you will probably
948 need to set the following (see below for an alternative):
952 Hardware End of Medium = no
954 Backward Space Record = no
955 Backward Space File = no
956 Fast Forward Space File = no
961 Then be sure to run some append tests with Bacula where you start and stop
962 Bacula between appending to the tape, or use {\bf btape} version 1.35.1 or
963 greater, which includes simulation of stopping/restarting Bacula.
965 Please see the file {\bf platforms/freebsd/pthreads-fix.txt} in the main
966 Bacula directory concerning {\bf important} information concerning
967 compatibility of Bacula and your system. A much more optimal Device
968 configuration is shown below, but does not work with all tape drives. Please
969 test carefully before putting either into production.
971 Note, for FreeBSD 4.10-RELEASE, using a Sony TSL11000 L100 DDS4 with an
972 autochanger set to variable block size and DCLZ compression, Brian McDonald
973 reports that to get Bacula to append correctly between Bacula executions,
974 the correct values to use are:
978 mt -f /dev/nsa0 seteotmodel 1
979 mt -f /dev/nsa0 blocksize 0
980 mt -f /dev/nsa0 comp enable
988 Hardware End of Medium = no
990 Backward Space Record = no
991 Backward Space File = no
992 Fast Forward Space File = yes
997 This has been confirmed by several other people using different hardware. This
998 configuration is the preferred one because it uses one EOF and no backspacing
999 at the end of the tape, which works much more efficiently and reliably with
1002 Finally, here is a Device configuration that Danny Butroyd reports to work
1003 correctly with the Overland Powerloader tape library using LT0-2 and
1008 # Overland Powerloader LT02 - 17 slots single drive
1012 Archive Device = /dev/nsa0
1013 AutomaticMount = yes;
1015 RemovableMedia = yes;
1017 Changer Command = "/usr/local/sbin/mtx-changer %c %o %S %a %d"
1018 Changer Device = /dev/pass2
1020 Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
1022 # FreeBSD Specific Settings
1023 Offline On Unmount = no
1024 Hardware End of Medium = no
1026 Backward Space Record = no
1027 Fast Forward Space File = no
1031 The following Device resource works fine with Dell PowerVault 110T and
1032 120T devices on both FreeBSD 5.3 and on NetBSD 3.0. It also works
1033 with Sony AIT-2 drives on FreeBSD.
1038 # FreeBSD/NetBSD Specific Settings
1039 Hardware End of Medium = no
1041 Backward Space Record = no
1042 Fast Forward Space File = yes
1048 On FreeBSD version 6.0, it is reported that you can even set
1049 Backward Space Record = yes.
1053 \subsection{Finding your Tape Drives and Autochangers on FreeBSD}
1054 \index[general]{FreeBSD!Finding Tape Drives and Autochangers}
1055 \index[general]{Finding Tape Drives and Autochangers on FreeBSD}
1057 On FreeBSD, you can do a {\bf camcontrol devlist} as root to determine what
1058 drives and autochangers you have. For example,
1062 undef# camcontrol devlist
1063 at scbus0 target 2 lun 0 (pass0,sa0)
1064 at scbus0 target 4 lun 0 (pass1,sa1)
1065 at scbus0 target 4 lun 1 (pass2)
1069 from the above, you can determine that there is a tape drive on {\bf /dev/sa0}
1070 and another on {\bf /dev/sa1} in addition since there is a second line for the
1071 drive on {\bf /dev/sa1}, you know can assume that it is the control device for
1072 the autochanger (i.e. {\bf /dev/pass2}). It is also the control device name to
1073 use when invoking the tapeinfo program. E.g.
1077 tapeinfo -f /dev/pass2
1083 \subsection{Using the OnStream driver on Linux Systems}
1084 \index[general]{Using the OnStream driver on Linux Systems}
1085 \index[general]{Systems!Using the OnStream driver on Linux}
1087 Bacula version 1.33 (not 1.32x) is now working and ready for testing with the
1088 OnStream kernel osst driver version 0.9.14 or above. Osst is available from:
1089 \elink{http://sourceforge.net/projects/osst/}
1090 {http://sourceforge.net/projects/osst/}.
1092 To make Bacula work you must first load the new driver then, as root, do:
1096 mt -f /dev/nosst0 defblksize 32768
1100 Also you must add the following to your Device resource in your Storage
1105 Minimum Block Size = 32768
1106 Maximum Block Size = 32768
1110 Here is a Device specification provided by Michel Meyers that is known to
1116 Name = "Onstream DI-30"
1117 Media Type = "ADR-30"
1118 Archive Device = /dev/nosst0
1119 Minimum Block Size = 32768
1120 Maximum Block Size = 32768
1121 Hardware End of Medium = yes
1123 Backward Space File = yes
1124 Fast Forward Space File = yes
1126 AutomaticMount = yes
1128 Removable Media = yes
1133 \section{Hardware Compression on EXB-8900}
1134 \index[general]{Hardware Compression on EXB-8900}
1135 \index[general]{EXB-8900!Hardware Compression}
1137 To active, check, or disable the hardware compression feature
1138 on an EXB-8900, use the exabyte MammothTool. You can get it here:
1139 \elink{http://www.exabyte.com/support/online/downloads/index.cfm}
1140 {http://www.exabyte.com/support/online/downloads/index.cfm}.
1141 There is a Solaris version of this tool. With option -C 0 or 1 you
1142 can disable or activate compression. Start this tool without any
1143 options for a small reference.
1146 \subsection{Using btape to Simulate Filling a Tape}
1147 \index[general]{Using btape to Simulate Filling a Tape}
1148 \index[general]{Tape!Using btape to Simulate Filling}
1150 Because there are often problems with certain tape drives or systems when end
1151 of tape conditions occur, {\bf btape} has a special command {\bf fill} that
1152 causes it to write random data to a tape until the tape fills. It then writes
1153 at least one more Bacula block to a second tape. Finally, it reads back both
1154 tapes to ensure that the data has been written in a way that Bacula can
1155 recover it. Note, there is also a single tape option as noted below, which you
1156 should use rather than the two tape test. See below for more details.
1158 This can be an extremely time consuming process (here it is about 6 hours) to
1159 fill a full tape. Note, that btape writes random data to the tape when it is
1160 filling it. This has two consequences: 1. it takes a bit longer to generate
1161 the data, especially on slow CPUs. 2. the total amount of data is
1162 approximately the real physical capacity of your tape, regardless of whether
1163 or not the tape drive compression is on or off. This is because random data
1164 does not compress very much.
1166 To begin this test, you enter the {\bf fill} command and follow the
1167 instructions. There are two options: the simple single tape option and the
1168 multiple tape option. Please use only the simple single tape option because
1169 the multiple tape option still doesn't work totally correctly. If the single
1170 tape option does not succeed, you should correct the problem before using
1172 \label{RecoveringFiles}
1174 \section{Recovering Files Written With Fixed Block Sizes}
1175 \index[general]{Recovering Files Written With Fixed Block Sizes}
1177 If you have been previously running your tape drive in fixed block mode
1178 (default 512) and Bacula with variable blocks (default), then in version
1179 1.32f-x and 1.34 and above, Bacula will fail to recover files because it does
1180 block spacing, and because the block sizes don't agree between your tape drive
1181 and Bacula it will not work.
1183 The long term solution is to run your drive in variable block mode as
1184 described above. However, if you have written tapes using fixed block sizes,
1185 this can be a bit of a pain. The solution to the problem is: while you are
1186 doing a restore command using a tape written in fixed block size, ensure that
1187 your drive is set to the fixed block size used while the tape was written.
1188 Then when doing the {\bf restore} command in the Console program, do not
1189 answer the prompt {\bf yes/mod/no}. Instead, edit the bootstrap file (the
1190 location is listed in the prompt) using any ASCII editor. Remove all {\bf
1191 VolBlock} lines in the file. When the file is re-written, answer the question,
1192 and Bacula will run without using block positioning, and it should recover
1196 \section{Tape Blocking Modes}
1197 \index[general]{Modes!Tape Blocking}
1198 \index[general]{Tape Blocking Modes}
1200 SCSI tapes may either be written in {\bf variable} or {\bf fixed} block sizes.
1201 Newer drives support both modes, but some drives such as the QIC devices
1202 always use fixed block sizes. Bacula attempts to fill and write complete
1203 blocks (default 65K), so that in normal mode (variable block size), Bacula
1204 will always write blocks of the same size except the last block of a Job. If
1205 Bacula is configured to write fixed block sizes, it will pad the last block of
1206 the Job to the correct size. Bacula expects variable tape block size drives to
1207 behave as follows: Each write to the drive results in a single record being
1208 written to the tape. Each read returns a single record. If you request less
1209 bytes than are in the record, only those number of bytes will be returned, but
1210 the entire logical record will have been read (the next read will retrieve the
1211 next record). Thus data from a single write is always returned in a single
1212 read, and sequentially written records are returned by sequential reads.
1214 Bacula expects fixed block size tape drives to behave as follows: If a write
1215 length is greater than the physical block size of the drive, the write will be
1216 written as two blocks each of the fixed physical size. This single write may
1217 become multiple physical records on the tape. (This is not a good situation).
1218 According to the documentation, one may never write an amount of data that is
1219 not the exact multiple of the blocksize (it is not specified if an error
1220 occurs or if the the last record is padded). When reading, it is my
1221 understanding that each read request reads one physical record from the tape.
1222 Due to the complications of fixed block size tape drives, you should avoid
1223 them if possible with Bacula, or you must be ABSOLUTELY certain that you use
1224 fixed block sizes within Bacula that correspond to the physical block size of
1225 the tape drive. This will ensure that Bacula has a one to one correspondence
1226 between what it writes and the physical record on the tape.
1228 Please note that Bacula will not function correctly if it writes a block and
1229 that block is split into two or more physical records on the tape. Bacula
1230 assumes that each write causes a single record to be written, and that it can
1231 sequentially recover each of the blocks it has written by using the same
1232 number of sequential reads as it had written.
1234 \section{Details of Tape Modes}
1235 \index[general]{Modes!Details}
1236 \index[general]{Details of Tape Modes}
1237 Rudolf Cejka has provided the following information concerning
1238 certain tape modes and MTEOM.
1242 It is always possible to position filemarks or blocks, whereas
1243 positioning to the end-of-data is only optional feature, however it is
1244 implemented very often. SCSI specification also talks about optional
1245 sequential filemarks, setmarks and sequential setmarks, but these are not
1246 implemented so often. Modern tape drives keep track of file positions in
1247 built-in chip (AIT, LTO) or at the beginning of the tape (SDLT), so there
1248 is not any speed difference, if end-of-data or filemarks is used (I have
1249 heard, that LTO-1 from all 3 manufacturers do not use its chip for file
1250 locations, but a tape as in SDLT case, and I'm not sure about LTO-2 and
1251 LTO-3 case). However there is a big difference, that end-of-data ignores
1252 file position, whereas filemarks returns the real number of skipped
1253 files, so OS can track current file number just in filemarks case.
1256 Solaris does use just SCSI SPACE Filemarks, it does not support SCSI
1257 SPACE End-of-data. When MTEOM is called, Solaris does use SCSI SPACE
1258 Filemarks with count = 1048576 for fast mode, and combination of SCSI
1259 SPACE Filemarks with count = 1 with SCSI SPACE Blocks with count = 1 for
1260 slow mode, so EOD mark on the tape on some older tape drives is not
1261 skipped. File number is always tracked for MTEOM.
1263 Linux does support both SCSI SPACE Filemarks and End-of-data: When MTEOM
1264 is called in MT\_ST\_FAST\_MTEOM mode, SCSI SPACE End-of-data is used.
1265 In the other case, SCSI SPACE Filemarks with count =
1267 There is no real slow mode like in Solaris - I just expect, that for
1268 older tape drives Filemarks may be slower than End-of-data, but not so
1269 much as in Solaris slow mode. File number is tracked for MTEOM just
1270 without MT\_ST\_FAST\_MTEOM - when MT\_ST\_FAST\_MTEOM is used, it is not.
1272 FreeBSD does support both SCSI SPACE Filemarks and End-of-data, but when
1273 MTEOD (MTEOM) is called, SCSI SPACE End-of-data is always used. FreeBSD
1274 never use SCSI SPACE Filemarks for MTEOD. File number is never tracked
1278 When {\bf Hardware End of Medium = Yes} is used, MTEOM is called, but it
1279 does not mean, that hardware End-of-data must be used. When Hardware End
1280 of Medium = No, if Fast Forward Space File = Yes, MTFSF with count =
1281 32767 is used, else Block Read with count = 1 with Forward Space File
1282 with count = 1 is used, which is really very slow.
1284 \item [Hardware End of Medium = Yes|No]
1285 The name of this option is misleading and is the source of confusion,
1286 because it is not the hardware EOM, what is really switched here.
1288 If I use Yes, OS must not use SCSI SPACE End-of-data, because Bacula
1289 expects, that there is tracked file number, which is not supported by
1290 SCSI specification. Instead, the OS have to use SCSI SPACE Filemarks.
1292 If I use No, an action depends on Fast Forward Space File.
1294 When I set {\bf Hardware End of Medium = no}
1295 and {\bf Fast Forward Space File = no}
1296 file positioning was very slow
1297 on my LTO-3 (about ten to 100 minutes), but
1299 with {\bf Hardware End of Medium = no} and
1300 {\bf Fast Forward Space File = yes}, the time is ten to
1301 100 times faster (about one to two minutes).
1305 \section{Tape Performance Problems}
1306 \index[general]{Tape Performance}
1307 If you have LTO-3 or LTO-4 drives, you should be able to
1308 fairly good transfer rates; from 60 to 150 MB/second, providing
1309 you have fast disks; GigaBit Ethernet connections (probably 2); you are
1310 running multiple simultaneous jobs; you have Bacula data spooling
1311 enabled; your tape block size is set to 131072 or 262144; and
1312 you have set {\bf Maximum File Size = 5G}.
1314 If you are not getting good performance, consider some of the following
1315 suggestions from the Allen Balck on the Bacula Users email list:
1318 \item You are using an old HBA (i.e. SCSI-1, which only does 5 MB/s)
1320 \item There are other, slower, devices on the SCSI bus. The HBA will
1321 negotiate the speed of every device down to the speed of the
1324 \item There is a termination problem on the bus (either too much or
1325 too little termination). The HBA will drop the bus speed in an
1326 attempt to increase the reliability of the bus.
1328 \item Loose or damaged cabling - this will probably make the HBA "think"
1329 you have a termination problem and it will react as in 3 above.
1332 See if /var/adm/messages (or /var/log/messages) tells you what the sync
1333 rate of the SCSI devices/bus are. Also, the next time you reboot, the
1334 BIOS may be able to tell you what the rate of each device is.
1337 \section{Autochanger Errors}
1338 \index[general]{Errors!Autochanger}
1339 \index[general]{Autochanger Errors}
1341 If you are getting errors such as:
1345 3992 Bad autochanger "load slot 1, drive 1": ERR=Child exited with code 1.
1349 and you are running your Storage daemon as non-root, then most likely
1350 you are having permissions problems with the control channel. Running
1351 as root, set permissions on /dev/sgX so that the userid and group of
1352 your Storage daemon can access the device. You need to ensure that you
1353 all access to the proper control device, and if you don't have any
1354 SCSI disk drives (including SATA drives), you might want to change
1355 the permissions on /dev/sg*.
1357 \section{Syslog Errors}
1358 \index[general]{Errors!Syslog}
1359 \index[general]{Syslog Errors}
1361 If you are getting errors such as:
1365 : kernel: st0: MTSETDRVBUFFER only allowed for root
1369 you are most likely running your Storage daemon as non-root, and
1370 Bacula is attempting to set the correct OS buffering to correspond
1371 to your Device resource. Most OSes allow only root to issue this
1372 ioctl command. In general, the message can be ignored providing
1373 you are sure that your OS parameters are properly configured as
1374 described earlier in this manual. If you are running your Storage daemon
1375 as root, you should not be getting these system log messages, and if
1376 you are, something is probably wrong.