4 \chapter{Basic Volume Management}
6 \index[general]{Basic Volume Management}
7 \index[general]{Management!Basic Volume}
8 \index[general]{Disk Volumes}
10 This chapter presents most all the features needed to do Volume management.
11 Most of the concepts apply equally well to both tape and disk Volumes.
12 However, the chapter was originally written to explain backing up to disk, so
13 you will see it is slanted in that direction, but all the directives
14 presented here apply equally well whether your volume is disk or tape.
16 If you have a lot of hard disk storage or you absolutely must have your
17 backups run within a small time window, you may want to direct Bacula to
18 backup to disk Volumes rather than tape Volumes. This chapter is intended to
19 give you some of the options that are available to you so that you can manage
20 either disk or tape volumes.
23 \section{Key Concepts and Resource Records}
24 \index[general]{Key Concepts and Resource Records }
25 \index[general]{Records!Key Concepts and Resource }
27 Getting Bacula to write to disk rather than tape in the simplest case is
28 rather easy. In the Storage daemon's configuration file, you simply define an
29 {\bf Archive Device} to be a directory. For example, if you want your disk
30 backups to go into the directory {\bf /home/bacula/backups}, you could use the
38 Archive Device = /home/bacula/backups
47 Assuming you have the appropriate {\bf Storage} resource in your Director's
48 configuration file that references the above Device resource,
62 Bacula will then write the archive to the file {\bf
63 /home/bacula/backups/\lt{}volume-name\gt{}} where \lt{}volume-name\gt{} is the
64 volume name of a Volume defined in the Pool. For example, if you have labeled
65 a Volume named {\bf Vol001}, Bacula will write to the file {\bf
66 /home/bacula/backups/Vol001}. Although you can later move the archive file to
67 another directory, you should not rename it or it will become unreadable by
68 Bacula. This is because each archive has the filename as part of the internal
69 label, and the internal label must agree with the system filename before
72 Although this is quite simple, there are a number of problems. The first is
73 that unless you specify otherwise, Bacula will always write to the same volume
74 until you run out of disk space. This problem is addressed below.
76 In addition, if you want to use concurrent jobs that write to several
77 different volumes at the same time, you will need to understand a number
78 of other details. An example of such a configuration is given
79 at the end of this chapter under \ilink{Concurrent Disk Jobs}{ConcurrentDiskJobs}.
81 \subsection{Pool Options to Limit the Volume Usage}
82 \index[general]{Usage!Pool Options to Limit the Volume }
83 \index[general]{Pool Options to Limit the Volume Usage }
85 Some of the options you have, all of which are specified in the Pool record,
89 \item To write each Volume only once (i.e. one Job per Volume or file in this
92 {\bf UseVolumeOnce = yes}.
94 \item To write nnn Jobs to each Volume, use:
96 {\bf Maximum Volume Jobs = nnn}.
98 \item To limit the maximum size of each Volume, use:
100 {\bf Maximum Volume Bytes = mmmm}.
102 Note, if you use disk volumes and do not specify a Maximum Volume Bytes,
103 Bacula will write to your first Volume until your whole disk fills.
104 This is probably not what you want. We recommend keeping your total
105 number of Volumes created in the Catalog down to less than 5,000 for
106 performance reasons when pruning. For a small site (10-20 clients), you
107 might choose a Maximum Volume Bytes of 5G, for a medium site (20-100
108 clients) you might choose as size of 50G, and for large sites (greater
109 than 100 clients) 200G.
111 The advantage of bigger Volumes is that it improves performance during
112 pruning (smaller number of Volumes to prune). The advantage of smaller
113 Volumes is that you waste less disk space by being able to recycle them
114 more often than big Volumes.
116 As of Bacula version 2.0 and greater large Volumes present little
117 performance penality during restores, because Bacula is able to seek to
118 the start of each Job (and often within a Job).
120 \item To limit the use time (i.e. write the Volume for a maximum of five days),
123 {\bf Volume Use Duration = ttt}.
126 Note that although you probably would not want to limit the number of bytes on
127 a tape as you would on a disk Volume, the other options can be very useful in
128 limiting the time Bacula will use a particular Volume (be it tape or disk).
129 For example, the above directives can allow you to ensure that you rotate
130 through a set of daily Volumes if you wish.
132 As mentioned above, each of those directives is specified in the Pool or
133 Pools that you use for your Volumes. In the case of {\bf Maximum Volume Job},
134 {\bf Maximum Volume Bytes}, and {\bf Volume Use Duration}, you can actually
135 specify the desired value on a Volume by Volume basis. The value specified in
136 the Pool record becomes the default when labeling new Volumes. Once a Volume
137 has been created, it gets its own copy of the Pool defaults, and subsequently
138 changing the Pool will have no effect on existing Volumes. You can either
139 manually change the Volume values, or refresh them from the Pool defaults using
140 the {\bf update volume} command in the Console. As an example
141 of the use of one of the above, suppose your Pool resource contains:
148 Volume Use Duration = 23h
153 then if you run a backup once a day (every 24 hours), Bacula will use a new
154 Volume for each backup, because each Volume it writes can only be used for 23 hours
155 after the first write. Note, setting the use duration to 23 hours is not a very
156 good solution for tapes unless you have someone on-site during the weekends,
157 because Bacula will want a new Volume and no one will be present to mount it,
158 so no weekend backups will be done until Monday morning.
160 \label{AutomaticLabeling}
161 \subsection{Automatic Volume Labeling}
162 \index[general]{Automatic Volume Labeling }
163 \index[general]{Labeling!Automatic Volume }
165 Use of the above records brings up another problem -- that of labeling your
166 Volumes. For automated disk backup, you can either manually label each of your
167 Volumes, or you can have Bacula automatically label new Volumes when they are
168 needed. While, the automatic Volume labeling in version 1.30 and prior is a
169 bit simplistic, but it does allow for automation, the features added in
170 version 1.31 permit automatic creation of a wide variety of labels including
171 information from environment variables and special Bacula Counter variables.
172 In version 1.37 and later, it is probably much better to use Python scripting
173 and the NewVolume event since generating Volume labels in a Python script is
174 much easier than trying to figure out Counter variables. See the
175 \borgxrlink{Python Scripting}{PythonChapter}{misc}{chapter} of the \miscman{}
178 Please note that automatic Volume labeling can also be used with tapes, but
179 it is not nearly so practical since the tapes must be pre-mounted. This
180 requires some user interaction. Automatic labeling from templates does NOT
181 work with autochangers since Bacula will not access unknown slots. There
182 are several methods of labeling all volumes in an autochanger magazine.
183 For more information on this, please see the \ilink{Autochanger}{AutochangersChapter} chapter of this manual.
185 Automatic Volume labeling is enabled by making a change to both the Pool
186 resource (Director) and to the Device resource (Storage daemon) shown above.
187 In the case of the Pool resource, you must provide Bacula with a label format
188 that it will use to create new names. In the simplest form, the label format
189 is simply the Volume name, to which Bacula will append a four digit number.
190 This number starts at 0001 and is incremented for each Volume the catalog
191 contains. Thus if you modify your Pool resource to be:
198 Volume Use Duration = 23h
204 Bacula will create Volume names Vol0001, Vol0002, and so on when new Volumes
205 are needed. Much more complex and elaborate labels can be created using
206 variable expansion defined in the
207 \borgxrlink{Variable Expansion}{VarsChapter}{misc}{chapter} of the \miscman{}.
209 The second change that is necessary to make automatic labeling work is to give
210 the Storage daemon permission to automatically label Volumes. Do so by adding
211 {\bf LabelMedia = yes} to the Device resource as follows:
218 Archive Device = /home/bacula/backups
220 AutomaticMount = yes;
228 You can find more details of the {\bf Label Format} Pool record in
229 \ilink{Label Format}{Label} description of the Pool resource
233 \subsection{Restricting the Number of Volumes and Recycling}
234 \index[general]{Recycling!Restricting the Number of Volumes and Recycling}
235 \index[general]{Restricting the Number of Volumes and Recycling}
237 Automatic labeling discussed above brings up the problem of Volume
238 management. With the above scheme, a new Volume will be created every day.
239 If you have not specified Retention periods, your Catalog will continue to
240 fill keeping track of all the files Bacula has backed up, and this
241 procedure will create one new archive file (Volume) every day.
243 The tools Bacula gives you to help automatically manage these problems are
247 \item Catalog file record retention periods, the
248 \ilink{File Retention = ttt}{FileRetention} record in the Client
250 \item Catalog job record retention periods, the
251 \ilink{Job Retention = ttt}{JobRetention} record in the Client
254 \ilink{ AutoPrune = yes}{AutoPrune} record in the Client resource
255 to permit application of the above two retention periods.
257 \ilink{ Volume Retention = ttt}{VolRetention} record in the Pool
260 \ilink{ AutoPrune = yes}{PoolAutoPrune} record in the Pool
261 resource to permit application of the Volume retention period.
263 \ilink{ Recycle = yes}{PoolRecycle} record in the Pool resource
264 to permit automatic recycling of Volumes whose Volume retention period has
267 \ilink{ Recycle Oldest Volume = yes}{RecycleOldest} record in the
268 Pool resource tells Bacula to Prune the oldest volume in the Pool, and if all
269 files were pruned to recycle this volume and use it.
271 \ilink{ Recycle Current Volume = yes}{RecycleCurrent} record in
272 the Pool resource tells Bacula to Prune the currently mounted volume in the
273 Pool, and if all files were pruned to recycle this volume and use it.
275 \ilink{ Purge Oldest Volume = yes}{PurgeOldest} record in the
276 Pool resource permits a forced recycling of the oldest Volume when a new one
277 is needed. {\bf N.B. This record ignores retention periods! We highly
278 recommend not to use this record, but instead use Recycle Oldest Volume}
280 \ilink{ Maximum Volumes = nnn}{MaxVolumes} record in the Pool
281 resource to limit the number of Volumes that can be created.
284 The first three records (File Retention, Job Retention, and AutoPrune)
285 determine the amount of time that Job and File records will remain in your
286 Catalog, and they are discussed in detail in the
287 \ilink{Automatic Volume Recycling}{RecyclingChapter} chapter of
290 Volume Retention, AutoPrune, and Recycle determine how long Bacula will
291 keep your Volumes before reusing them, and they are also discussed in
292 detail in the \ilink{Automatic Volume Recycling}{RecyclingChapter} chapter
295 The Maximum Volumes record can also be used in conjunction with the Volume
296 Retention period to limit the total number of archive Volumes (files) that
297 Bacula will create. By setting an appropriate Volume Retention period, a
298 Volume will be purged just before it is needed and thus Bacula can cycle
299 through a fixed set of Volumes. Cycling through a fixed set of Volumes can
300 also be done by setting {\bf Recycle Oldest Volume = yes} or {\bf Recycle
301 Current Volume = yes}. In this case, when Bacula needs a new Volume, it will
302 prune the specified volume.
304 \label{ConcurrentDiskJobs}
305 \section{Concurrent Disk Jobs}
306 \index[general]{Concurrent Disk Jobs}
307 Above, we discussed how you could have a single device named {\bf
308 FileBackup} that writes to volumes in {\bf /home/bacula/backups}.
309 You can, in fact, run multiple concurrent jobs using the
310 Storage definition given with this example, and all the jobs will
311 simultaneously write into the Volume that is being written.
313 Now suppose you want to use multiple Pools, which means multiple
314 Volumes, or suppose you want each client to have its own Volume
315 and perhaps its own directory such as {\bf /home/bacula/client1}
316 and {\bf /home/bacula/client2} ... With the single Storage and Device
317 definition above, neither of these two is possible. Why? Because
318 Bacula disk storage follows the same rules as tape devices. Only
319 one Volume can be mounted on any Device at any time. If you want
320 to simultaneously write multiple Volumes, you will need multiple
321 Device resources in your bacula-sd.conf file, and thus multiple
322 Storage resources in your bacula-dir.conf.
324 OK, so now you should understand that you need multiple Device definitions
325 in the case of different directories or different Pools, but you also
326 need to know that the catalog data that Bacula keeps contains only
327 the Media Type and not the specific storage device. This permits a tape
328 for example to be re-read on any compatible tape drive. The compatibility
329 being determined by the Media Type. The same applies to disk storage.
330 Since a volume that is written by a Device in say directory {\bf
331 /home/bacula/backups} cannot be read by a Device with an Archive Device
332 definition of {\bf /home/bacula/client1}, you will not be able to
333 restore all your files if you give both those devices
334 {\bf Media Type = File}. During the restore, Bacula will simply choose
335 the first available device, which may not be the correct one. If this
336 is confusing, just remember that the Directory has only the Media Type
337 and the Volume name. It does not know the {\bf Archive Device} (or the
338 full path) that is specified in the Storage daemon. Thus you must
339 explicitly tie your Volumes to the correct Device by using the Media Type.
341 The example shown below shows a case where there are two clients, each
342 using its own Pool and storing their Volumes in different directories.
347 \index[general]{Example }
349 The following example is not very practical, but can be used to demonstrate
350 the proof of concept in a relatively short period of time. The example
351 consists of a two clients that are backed up to a set of 12 archive files
352 (Volumes) for each client into different directories on the Storage
353 machine. Each Volume is used (written) only once, and there are four Full
354 saves done every hour (so the whole thing cycles around after three hours).
356 What is key here is that each physical device on the Storage daemon
357 has a different Media Type. This allows the Director to choose the
358 correct device for restores ...
360 The Director's configuration file is as follows:
366 QueryFile = "~/bacula/bin/query.sql"
367 PidDirectory = "~/bacula/working"
368 WorkingDirectory = "~/bacula/working"
369 Password = dir_password
373 Run = Level=Full hourly at 0:05
374 Run = Level=Full hourly at 0:20
375 Run = Level=Full hourly at 0:35
376 Run = Level=Full hourly at 0:50
379 Name = "RecycleExample"
383 FileSet= "Example FileSet"
385 Storage = FileStorage
387 Schedule = FourPerHour
391 Name = "RecycleExample2"
395 FileSet= "Example FileSet"
397 Storage = FileStorage1
399 Schedule = FourPerHour
403 Name = "Example FileSet"
409 File = /home/kern/bacula/bin
417 Password = client_password
424 Password = client1_password
430 Password = local_storage_password
438 Password = local_storage_password
445 dbname = bacula; user = bacula; password = ""
453 Use Volume Once = yes
455 LabelFormat = "Recycle-"
464 Use Volume Once = yes
466 LabelFormat = "Recycle1-"
476 and the Storage daemon's configuration file is:
482 WorkingDirectory = "~/bacula/working"
483 Pid Directory = "~/bacula/working"
484 MaximumConcurrentJobs = 10
488 Password = local_storage_password
493 Archive Device = /home/bacula/backups
496 AutomaticMount = yes;
504 Archive Device = /home/bacula/backups1
507 AutomaticMount = yes;
514 director = my-dir = all
519 With a little bit of work, you can change the above example into a weekly or
520 monthly cycle (take care about the amount of archive disk space used).
522 \label{MultipleDisks}
523 \section{Backing up to Multiple Disks}
524 \index[general]{Disks!Backing up to Multiple }
525 \index[general]{Backing up to Multiple Disks }
527 Bacula can, of course, use multiple disks, but in general, each disk must be a
528 separate Device specification in the Storage daemon's conf file, and you must
529 then select what clients to backup to each disk. You will also want to
530 give each Device specification a different Media Type so that during
531 a restore, Bacula will be able to find the appropriate drive.
533 The situation is a bit more complicated if you want to treat two different
534 physical disk drives (or partitions) logically as a single drive, which
535 Bacula does not directly support. However, it is possible to back up your
536 data to multiple disks as if they were a single drive by linking the
537 Volumes from the first disk to the second disk.
539 For example, assume that you have two disks named {\bf /disk1} and {\bf
540 /disk2}. If you then create a standard Storage daemon Device resource for
541 backing up to the first disk, it will look like the following:
548 Archive Device = /disk1
551 AutomaticMount = yes;
558 Since there is no way to get the above Device resource to reference both {\bf
559 /disk1} and {\bf /disk2} we do it by pre-creating Volumes on /disk2 with the
564 ln -s /disk2/Disk2-vol001 /disk1/Disk2-vol001
565 ln -s /disk2/Disk2-vol002 /disk1/Disk2-vol002
566 ln -s /disk2/Disk2-vol003 /disk1/Disk2-vol003
571 At this point, you can label the Volumes as Volume {\bf Disk2-vol001}, {\bf
572 Disk2-vol002}, ... and Bacula will use them as if they were on /disk1 but
573 actually write the data to /disk2. The only minor inconvenience with this
574 method is that you must explicitly name the disks and cannot use automatic
575 labeling unless you arrange to have the labels exactly match the links you
578 An important thing to know is that Bacula treats disks like tape drives
579 as much as it can. This means that you can only have a single Volume
580 mounted at one time on a disk as defined in your Device resource in
581 the Storage daemon's conf file. You can have multiple concurrent
582 jobs running that all write to the one Volume that is being used, but
583 if you want to have multiple concurrent jobs that are writing to
584 separate disks drives (or partitions), you will need to define
585 separate Device resources for each one, exactly as you would do for
586 two different tape drives. There is one fundamental difference, however.
587 The Volumes that you create on the two drives cannot be easily exchanged
588 as they can for a tape drive, because they are physically resident (already
589 mounted in a sense) on the particular drive. As a consequence, you will
590 probably want to give them different Media Types so that Bacula can
591 distinguish what Device resource to use during a restore.
592 An example would be the following:
599 Archive Device = /disk1
602 AutomaticMount = yes;
610 Archive Device = /disk2
613 AutomaticMount = yes;
620 With the above device definitions, you can run two concurrent
621 jobs each writing at the same time, one to {\bf /disk1} and the
622 other to {\bf /disk2}. The fact that you have given them different
623 Media Types will allow Bacula to quickly choose the correct
624 Storage resource in the Director when doing a restore.
626 \label{MultipleClients}
627 \section{Considerations for Multiple Clients}
628 \index[general]{Clients!Considerations for Multiple }
629 \index[general]{Multiple Clients}
631 If we take the above example and add a second Client, here are a few
635 \item Although the second client can write to the same set of Volumes, you
636 will probably want to write to a different set.
637 \item You can write to a different set of Volumes by defining a second Pool,
638 which has a different name and a different {\bf LabelFormat}.
639 \item If you wish the Volumes for the second client to go into a different
640 directory (perhaps even on a different filesystem to spread the load), you
641 would do so by defining a second Device resource in the Storage daemon. The
642 {\bf Name} must be different, and the {\bf Archive Device} could be
643 different. To ensure that Volumes are never mixed from one pool to another,
644 you might also define a different MediaType (e.g. {\bf File1}).
647 In this example, we have two clients, each with a different Pool and a
648 different number of archive files retained. They also write to different
649 directories with different Volume labeling.
651 The Director's configuration file is as follows:
657 QueryFile = "~/bacula/bin/query.sql"
658 PidDirectory = "~/bacula/working"
659 WorkingDirectory = "~/bacula/working"
660 Password = dir_password
662 # Basic weekly schedule
664 Name = "WeeklySchedule"
665 Run = Level=Full fri at 1:30
666 Run = Level=Incremental sat-thu at 1:30
669 Name = "Example FileSet"
675 File = /home/kern/bacula/bin
679 Name = "Backup-client1"
683 FileSet= "Example FileSet"
687 Schedule = "WeeklySchedule"
690 Name = "Backup-client2"
694 FileSet= "Example FileSet"
698 Schedule = "WeeklySchedule"
704 Password = client1_password
711 Password = client2_password
713 # Two Storage definitions with different Media Types
714 # permits different directories
718 Password = local_storage_password
725 Password = local_storage_password
731 dbname = bacula; user = bacula; password = ""
737 # Two pools permits different cycling periods and Volume names
738 # Cycle through 15 Volumes (two weeks)
741 Use Volume Once = yes
743 LabelFormat = "Client1-"
745 VolumeRetention = 13d
749 # Cycle through 8 Volumes (1 week)
752 Use Volume Once = yes
754 LabelFormat = "Client2-"
763 and the Storage daemon's configuration file is:
769 WorkingDirectory = "~/bacula/working"
770 Pid Directory = "~/bacula/working"
771 MaximumConcurrentJobs = 10
775 Password = local_storage_password
777 # Archive directory for Client1
781 Archive Device = /home/bacula/client1
784 AutomaticMount = yes;
788 # Archive directory for Client2
792 Archive Device = /home/bacula/client2
795 AutomaticMount = yes;
801 director = my-dir = all