1 \chapter{Basic Volume Management}
3 \index[general]{Basic Volume Management}
4 \index[general]{Management!Basic Volume}
5 \index[general]{Disk Volumes}
7 This chapter presents most all the features needed to do Volume management.
8 Most of the concepts apply equally well to both tape and disk Volumes.
9 However, the chapter was originally written to explain backing up to disk, so
10 you will see it is slanted in that direction, but all the directives
11 presented here apply equally well whether your volume is disk or tape.
13 If you have a lot of hard disk storage or you absolutely must have your
14 backups run within a small time window, you may want to direct Bacula to
15 backup to disk Volumes rather than tape Volumes. This chapter is intended to
16 give you some of the options that are available to you so that you can manage
17 either disk or tape volumes.
20 \section{Key Concepts and Resource Records}
21 \index[general]{Key Concepts and Resource Records }
22 \index[general]{Records!Key Concepts and Resource }
24 Getting Bacula to write to disk rather than tape in the simplest case is
25 rather easy. In the Storage daemon's configuration file, you simply define an
26 {\bf Archive Device} to be a directory. For example, if you want your disk
27 backups to go into the directory {\bf /home/bacula/backups}, you could use the
35 Archive Device = /home/bacula/backups
44 Assuming you have the appropriate {\bf Storage} resource in your Director's
45 configuration file that references the above Device resource,
59 Bacula will then write the archive to the file {\bf
60 /home/bacula/backups/\lt{}volume-name\gt{}} where \lt{}volume-name\gt{} is the
61 volume name of a Volume defined in the Pool. For example, if you have labeled
62 a Volume named {\bf Vol001}, Bacula will write to the file {\bf
63 /home/bacula/backups/Vol001}. Although you can later move the archive file to
64 another directory, you should not rename it or it will become unreadable by
65 Bacula. This is because each archive has the filename as part of the internal
66 label, and the internal label must agree with the system filename before
69 Although this is quite simple, there are a number of problems. The first is
70 that unless you specify otherwise, Bacula will always write to the same volume
71 until you run out of disk space. This problem is addressed below.
73 In addition, if you want to use concurrent jobs that write to several
74 different volumes at the same time, you will need to understand a number
75 of other details. An example of such a configuration is given
76 at the end of this chapter under \ilink{Concurrent Disk
77 Jobs}{ConcurrentDiskJobs}.
79 \subsection{Pool Options to Limit the Volume Usage}
80 \index[general]{Usage!Pool Options to Limit the Volume }
81 \index[general]{Pool Options to Limit the Volume Usage }
83 Some of the options you have, all of which are specified in the Pool record,
87 \item To write each Volume only once (i.e. one Job per Volume or file in this
90 {\bf UseVolumeOnce = yes}.
92 \item To write nnn Jobs to each Volume, use:
94 {\bf Maximum Volume Jobs = nnn}.
96 \item To limit the maximum size of each Volume, use:
98 {\bf Maximum Volume Bytes = mmmm}.
100 Note, if you use disk volumes, with all versions up to and including
101 1.39.28, you should probably limit the Volume size to some reasonable
102 value such as say 5GB. This is because during a restore, Bacula is
103 currently unable to seek to the proper place in a disk volume to restore
104 a file, which means that it must read all records up to where the
105 restore begins. If your Volumes are 50GB, reading half or more of the
106 volume could take quite a bit of time. Also, if you ever have a partial
107 hard disk failure, you are more likely to be able to recover more data
108 if they are in smaller Volumes.
110 \item To limit the use time (i.e. write the Volume for a maximum of five days),
113 {\bf Volume Use Duration = ttt}.
116 Note that although you probably would not want to limit the number of bytes on
117 a tape as you would on a disk Volume, the other options can be very useful in
118 limiting the time Bacula will use a particular Volume (be it tape or disk).
119 For example, the above directives can allow you to ensure that you rotate
120 through a set of daily Volumes if you wish.
122 As mentioned above, each of those directives is specified in the Pool or
123 Pools that you use for your Volumes. In the case of {\bf Maximum Volume Job},
124 {\bf Maximum Volume Bytes}, and {\bf Volume Use Duration}, you can actually
125 specify the desired value on a Volume by Volume basis. The value specified in
126 the Pool record becomes the default when labeling new Volumes. Once a Volume
127 has been created, it gets its own copy of the Pool defaults, and subsequently
128 changing the Pool will have no effect on existing Volumes. You can either
129 manually change the Volume values, or refresh them from the Pool defaults using
130 the {\bf update volume} command in the Console. As an example
131 of the use of one of the above, suppose your Pool resource contains:
138 Volume Use Duration = 23h
143 then if you run a backup once a day (every 24 hours), Bacula will use a new
144 Volume for each backup, because each Volume it writes can only be used for 23 hours
145 after the first write. Note, setting the use duration to 23 hours is not a very
146 good solution for tapes unless you have someone on-site during the weekends,
147 because Bacula will want a new Volume and no one will be present to mount it,
148 so no weekend backups will be done until Monday morning.
150 \label{AutomaticLabeling}
151 \subsection{Automatic Volume Labeling}
152 \index[general]{Automatic Volume Labeling }
153 \index[general]{Labeling!Automatic Volume }
155 Use of the above records brings up another problem -- that of labeling your
156 Volumes. For automated disk backup, you can either manually label each of your
157 Volumes, or you can have Bacula automatically label new Volumes when they are
158 needed. While, the automatic Volume labeling in version 1.30 and prior is a
159 bit simplistic, but it does allow for automation, the features added in
160 version 1.31 permit automatic creation of a wide variety of labels including
161 information from environment variables and special Bacula Counter variables.
162 In version 1.37 and later, it is probably much better to use Python scripting
163 and the NewVolume event since generating Volume labels in a Python script is
164 much easier than trying to figure out Counter variables. See the
165 \bsysxrlink{Python Scripting}{PythonChapter}{misc}{chapter} of the \miscman{}
168 Please note that automatic Volume labeling can also be used with tapes, but
169 it is not nearly so practical since the tapes must be pre-mounted. This
170 requires some user interaction. Automatic labeling from templates does NOT
171 work with autochangers since Bacula will not access unknown slots. There
172 are several methods of labeling all volumes in an autochanger magazine.
173 For more information on this, please see the \ilink{Autochanger}
174 {AutochangersChapter} chapter of this manual.
176 Automatic Volume labeling is enabled by making a change to both the Pool
177 resource (Director) and to the Device resource (Storage daemon) shown above.
178 In the case of the Pool resource, you must provide Bacula with a label format
179 that it will use to create new names. In the simplest form, the label format
180 is simply the Volume name, to which Bacula will append a four digit number.
181 This number starts at 0001 and is incremented for each Volume the catalog
182 contains. Thus if you modify your Pool resource to be:
189 Volume Use Duration = 23h
195 Bacula will create Volume names Vol0001, Vol0002, and so on when new Volumes
196 are needed. Much more complex and elaborate labels can be created using
197 variable expansion defined in the
198 \bsysxrlink{Variable Expansion}{VarsChapter}{misc}{chapter} of the \miscman{}.
200 The second change that is necessary to make automatic labeling work is to give
201 the Storage daemon permission to automatically label Volumes. Do so by adding
202 {\bf LabelMedia = yes} to the Device resource as follows:
209 Archive Device = /home/bacula/backups
211 AutomaticMount = yes;
219 You can find more details of the {\bf Label Format} Pool record in
220 \ilink{Label Format}{Label} description of the Pool resource
224 \subsection{Restricting the Number of Volumes and Recycling}
225 \index[general]{Recycling!Restricting the Number of Volumes and Recycling}
226 \index[general]{Restricting the Number of Volumes and Recycling}
228 Automatic labeling discussed above brings up the problem of Volume management.
229 With the above scheme, a new Volume will be created every day. If you have not
230 specified Retention periods, your Catalog will continue to fill keeping track
231 of all the files Bacula has backed up, and this procedure will create one new
232 archive file (Volume) every day.
234 The tools Bacula gives you to help automatically manage these problems are the
238 \item Catalog file record retention periods, the
239 \ilink{File Retention = ttt}{FileRetention} record in the Client
241 \item Catalog job record retention periods, the
242 \ilink{Job Retention = ttt}{JobRetention} record in the Client
245 \ilink{ AutoPrune = yes}{AutoPrune} record in the Client resource
246 to permit application of the above two retention periods.
248 \ilink{ Volume Retention = ttt}{VolRetention} record in the Pool
251 \ilink{ AutoPrune = yes}{PoolAutoPrune} record in the Pool
252 resource to permit application of the Volume retention period.
254 \ilink{ Recycle = yes}{PoolRecycle} record in the Pool resource
255 to permit automatic recycling of Volumes whose Volume retention period has
258 \ilink{ Recycle Oldest Volume = yes}{RecycleOldest} record in the
259 Pool resource tells Bacula to Prune the oldest volume in the Pool, and if all
260 files were pruned to recycle this volume and use it.
262 \ilink{ Recycle Current Volume = yes}{RecycleCurrent} record in
263 the Pool resource tells Bacula to Prune the currently mounted volume in the
264 Pool, and if all files were pruned to recycle this volume and use it.
266 \ilink{ Purge Oldest Volume = yes}{PurgeOldest} record in the
267 Pool resource permits a forced recycling of the oldest Volume when a new one
268 is needed. {\bf N.B. This record ignores retention periods! We highly
269 recommend not to use this record, but instead use Recycle Oldest Volume}
271 \ilink{ Maximum Volumes = nnn}{MaxVolumes} record in the Pool
272 resource to limit the number of Volumes that can be created.
275 The first three records (File Retention, Job Retention, and AutoPrune)
276 determine the amount of time that Job and File records will remain in your
277 Catalog, and they are discussed in detail in the
278 \ilink{Automatic Volume Recycling}{RecyclingChapter} chapter of
281 Volume Retention, AutoPrune, and Recycle determine how long Bacula will keep
282 your Volumes before reusing them, and they are also discussed in detail in the
283 \ilink{Automatic Volume Recycling}{RecyclingChapter} chapter of
286 The Maximum Volumes record can also be used in conjunction with the Volume
287 Retention period to limit the total number of archive Volumes (files) that
288 Bacula will create. By setting an appropriate Volume Retention period, a
289 Volume will be purged just before it is needed and thus Bacula can cycle
290 through a fixed set of Volumes. Cycling through a fixed set of Volumes can
291 also be done by setting {\bf Recycle Oldest Volume = yes} or {\bf Recycle
292 Current Volume = yes}. In this case, when Bacula needs a new Volume, it will
293 prune the specified volume.
295 \label{ConcurrentDiskJobs}
296 \section{Concurrent Disk Jobs}
297 \index[general]{Concurrent Disk Jobs}
298 Above, we discussed how you could have a single device named {\bf
299 FileBackup} that writes to volumes in {\bf /home/bacula/backups}.
300 You can, in fact, run multiple concurrent jobs using the
301 Storage definition given with this example, and all the jobs will
302 simultaneously write into the Volume that is being written.
304 Now suppose you want to use multiple Pools, which means multiple
305 Volumes, or suppose you want each client to have its own Volume
306 and perhaps its own directory such as {\bf /home/bacula/client1}
307 and {\bf /home/bacula/client2} ... With the single Storage and Device
308 definition above, neither of these two is possible. Why? Because
309 Bacula disk storage follows the same rules as tape devices. Only
310 one Volume can be mounted on any Device at any time. If you want
311 to simultaneously write multiple Volumes, you will need multiple
312 Device resources in your bacula-sd.conf file, and thus multiple
313 Storage resources in your bacula-dir.conf.
315 OK, so now you should understand that you need multiple Device definitions
316 in the case of different directories or different Pools, but you also
317 need to know that the catalog data that Bacula keeps contains only
318 the Media Type and not the specific storage device. This permits a tape
319 for example to be re-read on any compatible tape drive. The compatibility
320 being determined by the Media Type. The same applies to disk storage.
321 Since a volume that is written by a Device in say directory {\bf
322 /home/bacula/backups} cannot be read by a Device with an Archive Device
323 definition of {\bf /home/bacula/client1}, you will not be able to
324 restore all your files if you give both those devices
325 {\bf Media Type = File}. During the restore, Bacula will simply choose
326 the first available device, which may not be the correct one. If this
327 is confusing, just remember that the Directory has only the Media Type
328 and the Volume name. It does not know the {\bf Archive Device} (or the
329 full path) that is specified in the Storage daemon. Thus you must
330 explicitly tie your Volumes to the correct Device by using the Media Type.
332 The example shown below shows a case where there are two clients, each
333 using its own Pool and storing their Volumes in different directories.
338 \index[general]{Example }
340 The following example is not very practical, but can be used to demonstrate
341 the proof of concept in a relatively short period of time. The example
342 consists of a two clients that are backed up to a set of 12 archive files
343 (Volumes) for each client into different directories on the Storage
344 machine. Each Volume is used (written) only once, and there are four Full
345 saves done every hour (so the whole thing cycles around after three hours).
347 What is key here is that each physical device on the Storage daemon
348 has a different Media Type. This allows the Director to choose the
349 correct device for restores ...
351 The Director's configuration file is as follows:
357 QueryFile = "~/bacula/bin/query.sql"
358 PidDirectory = "~/bacula/working"
359 WorkingDirectory = "~/bacula/working"
360 Password = dir_password
364 Run = Level=Full hourly at 0:05
365 Run = Level=Full hourly at 0:20
366 Run = Level=Full hourly at 0:35
367 Run = Level=Full hourly at 0:50
370 Name = "RecycleExample"
374 FileSet= "Example FileSet"
376 Storage = FileStorage
378 Schedule = FourPerHour
382 Name = "RecycleExample2"
386 FileSet= "Example FileSet"
388 Storage = FileStorage1
390 Schedule = FourPerHour
394 Name = "Example FileSet"
400 File = /home/kern/bacula/bin
408 Password = client_password
415 Password = client1_password
421 Password = local_storage_password
429 Password = local_storage_password
436 dbname = bacula; user = bacula; password = ""
444 Use Volume Once = yes
446 LabelFormat = "Recycle-"
455 Use Volume Once = yes
457 LabelFormat = "Recycle1-"
467 and the Storage daemon's configuration file is:
473 WorkingDirectory = "~/bacula/working"
474 Pid Directory = "~/bacula/working"
475 MaximumConcurrentJobs = 10
479 Password = local_storage_password
484 Archive Device = /home/bacula/backups
487 AutomaticMount = yes;
495 Archive Device = /home/bacula/backups1
498 AutomaticMount = yes;
505 director = my-dir = all
510 With a little bit of work, you can change the above example into a weekly or
511 monthly cycle (take care about the amount of archive disk space used).
513 \label{MultipleDisks}
514 \section{Backing up to Multiple Disks}
515 \index[general]{Disks!Backing up to Multiple }
516 \index[general]{Backing up to Multiple Disks }
518 Bacula can, of course, use multiple disks, but in general, each disk must be a
519 separate Device specification in the Storage daemon's conf file, and you must
520 then select what clients to backup to each disk. You will also want to
521 give each Device specification a different Media Type so that during
522 a restore, Bacula will be able to find the appropriate drive.
524 The situation is a bit more complicated if you want to treat two different
525 physical disk drives (or partitions) logically as a single drive, which
526 Bacula does not directly support. However, it is possible to back up your
527 data to multiple disks as if they were a single drive by linking the
528 Volumes from the first disk to the second disk.
530 For example, assume that you have two disks named {\bf /disk1} and {\bf
531 /disk2}. If you then create a standard Storage daemon Device resource for
532 backing up to the first disk, it will look like the following:
539 Archive Device = /disk1
542 AutomaticMount = yes;
549 Since there is no way to get the above Device resource to reference both {\bf
550 /disk1} and {\bf /disk2} we do it by pre-creating Volumes on /disk2 with the
555 ln -s /disk2/Disk2-vol001 /disk1/Disk2-vol001
556 ln -s /disk2/Disk2-vol002 /disk1/Disk2-vol002
557 ln -s /disk2/Disk2-vol003 /disk1/Disk2-vol003
562 At this point, you can label the Volumes as Volume {\bf Disk2-vol001}, {\bf
563 Disk2-vol002}, ... and Bacula will use them as if they were on /disk1 but
564 actually write the data to /disk2. The only minor inconvenience with this
565 method is that you must explicitly name the disks and cannot use automatic
566 labeling unless you arrange to have the labels exactly match the links you
569 An important thing to know is that Bacula treats disks like tape drives
570 as much as it can. This means that you can only have a single Volume
571 mounted at one time on a disk as defined in your Device resource in
572 the Storage daemon's conf file. You can have multiple concurrent
573 jobs running that all write to the one Volume that is being used, but
574 if you want to have multiple concurrent jobs that are writing to
575 separate disks drives (or partitions), you will need to define
576 separate Device resources for each one, exactly as you would do for
577 two different tape drives. There is one fundamental difference, however.
578 The Volumes that you create on the two drives cannot be easily exchanged
579 as they can for a tape drive, because they are physically resident (already
580 mounted in a sense) on the particular drive. As a consequence, you will
581 probably want to give them different Media Types so that Bacula can
582 distinguish what Device resource to use during a restore.
583 An example would be the following:
590 Archive Device = /disk1
593 AutomaticMount = yes;
601 Archive Device = /disk2
604 AutomaticMount = yes;
611 With the above device definitions, you can run two concurrent
612 jobs each writing at the same time, one to {\bf /disk1} and the
613 other to {\bf /disk2}. The fact that you have given them different
614 Media Types will allow Bacula to quickly choose the correct
615 Storage resource in the Director when doing a restore.
617 \label{MultipleClients}
618 \section{Considerations for Multiple Clients}
619 \index[general]{Clients!Considerations for Multiple }
620 \index[general]{Multiple Clients}
622 If we take the above example and add a second Client, here are a few
626 \item Although the second client can write to the same set of Volumes, you
627 will probably want to write to a different set.
628 \item You can write to a different set of Volumes by defining a second Pool,
629 which has a different name and a different {\bf LabelFormat}.
630 \item If you wish the Volumes for the second client to go into a different
631 directory (perhaps even on a different filesystem to spread the load), you
632 would do so by defining a second Device resource in the Storage daemon. The
633 {\bf Name} must be different, and the {\bf Archive Device} could be
634 different. To ensure that Volumes are never mixed from one pool to another,
635 you might also define a different MediaType (e.g. {\bf File1}).
638 In this example, we have two clients, each with a different Pool and a
639 different number of archive files retained. They also write to different
640 directories with different Volume labeling.
642 The Director's configuration file is as follows:
648 QueryFile = "~/bacula/bin/query.sql"
649 PidDirectory = "~/bacula/working"
650 WorkingDirectory = "~/bacula/working"
651 Password = dir_password
653 # Basic weekly schedule
655 Name = "WeeklySchedule"
656 Run = Level=Full fri at 1:30
657 Run = Level=Incremental sat-thu at 1:30
660 Name = "Example FileSet"
666 File = /home/kern/bacula/bin
670 Name = "Backup-client1"
674 FileSet= "Example FileSet"
678 Schedule = "WeeklySchedule"
681 Name = "Backup-client2"
685 FileSet= "Example FileSet"
689 Schedule = "WeeklySchedule"
695 Password = client1_password
702 Password = client2_password
704 # Two Storage definitions with different Media Types
705 # permits different directories
709 Password = local_storage_password
716 Password = local_storage_password
722 dbname = bacula; user = bacula; password = ""
728 # Two pools permits different cycling periods and Volume names
729 # Cycle through 15 Volumes (two weeks)
732 Use Volume Once = yes
734 LabelFormat = "Client1-"
736 VolumeRetention = 13d
740 # Cycle through 8 Volumes (1 week)
743 Use Volume Once = yes
745 LabelFormat = "Client2-"
754 and the Storage daemon's configuration file is:
760 WorkingDirectory = "~/bacula/working"
761 Pid Directory = "~/bacula/working"
762 MaximumConcurrentJobs = 10
766 Password = local_storage_password
768 # Archive directory for Client1
772 Archive Device = /home/bacula/client1
775 AutomaticMount = yes;
779 # Archive directory for Client2
783 Archive Device = /home/bacula/client2
786 AutomaticMount = yes;
792 director = my-dir = all