4 \section*{Automatic Volume Recycling}
5 \label{_ChapterStart22}
6 \index[general]{Recycling!Automatic Volume }
7 \index[general]{Automatic Volume Recycling }
8 \addcontentsline{toc}{section}{Automatic Volume Recycling}
10 By default, once Bacula starts writing a Volume, it can
11 append to the volume, but it will not overwrite the existing
12 data thus destroying it.
13 However when Bacula {\bf recycles} a Volume, the Volume becomes
14 available for being reused, and Bacula can at some later time
15 over write the previous contents of that Volume.
16 Thus all previous data will be lost. If the Volume is a tape,
17 the tape will be rewritten from the beginning. If the Volume is
18 a disk file, the file will be truncated before being rewritten.
20 You may not want Bacula to automatically recycle (reuse) tapes. This would
21 require a large number of tapes though, and in such a case, it is possible
22 to manually recycle tapes. For more on manual recycling, see the section
23 entitled \ilink{ Manually Recycling Volumes}{manualrecycling} below in this
26 Most people prefer to have a Pool of tapes that are used for daily backups and
27 recycled once a week, another Pool of tapes that are used for Full backups
28 once a week and recycled monthly, and finally a Pool of tapes that are used
29 once a month and recycled after a year or two. With a scheme like this, the
30 number of tapes in your pool or pools remains constant.
32 By properly defining your Volume Pools with appropriate Retention periods,
33 Bacula can manage the recycling (such as defined above) automatically.
35 Automatic recycling of Volumes is controlled by three records in the {\bf
36 Pool} resource definition in the Director's configuration file. These three
41 \item VolumeRetention = \lt{}time\gt{}
45 Automatic recycling of Volumes is performed by Bacula only when it wants a
46 new Volume and no appendable Volumes are available in the Pool. It will then
47 search the Pool for any Volumes with the {\bf Recycle} flag set and whose
48 Volume Status is {\bf Full}. At that point, the recycling occurs in two steps.
49 The first is that the Catalog for a Volume must be purged of all Jobs and
50 Files contained on that Volume, and the second step is the actual recycling of
51 the Volume. The Volume will be purged if the VolumeRetention period has
52 expired. When a Volume is marked as Purged, it means that no Catalog records
53 reference that Volume, and the Volume can be recycled. Until recycling
54 actually occurs, the Volume data remains intact. If no Volumes can be found
55 for recycling for any of the reasons stated above, Bacula will request
56 operator intervention (i.e. it will ask you to label a new volume).
58 A key point mentioned above, that can be a source of frustration, is that Bacula
59 will only recycle purged Volumes if there is no other appendable Volume
60 available, otherwise, it will always write to an appendable Volume before
61 recycling even if there are Volume marked as Purged. This preserves your data
62 as long as possible. So, if you wish to "force" Bacula to use a purged
63 Volume, you must first ensure that no other Volume in the Pool is marked {\bf
64 Append}. If necessary, you can manually set a volume to {\bf Full}. The reason
65 for this is that Bacula wants to preserve the data on your old tapes (even
66 though purged from the catalog) as long as absolutely possible before
70 \subsection*{Automatic Pruning}
71 \index[general]{Automatic Pruning }
72 \index[general]{Pruning!Automatic }
73 \addcontentsline{toc}{subsection}{Automatic Pruning}
75 As Bacula writes files to tape, it keeps a list of files, jobs, and volumes in
76 a database called the catalog. Among other things, the database helps Bacula
77 to decide which files to back up in an incremental or differential backup, and
78 helps you locate files on past backups when you want to restore something.
79 However, the catalog will grow larger and larger as time goes on, and
80 eventually it can become unacceptably large.
82 Bacula's process for removing entries from the catalog is called Pruning. The
83 default is Automatic Pruning, which means that once an entry reaches a certain
84 age (e.g. 30 days old) it is removed from the catalog. Once a job has been
85 pruned, you can still restore it from the backup tape, but one additional step
86 is required: scanning the volume with bscan. The alternative to Automatic
87 Pruning is Manual Pruning, in which you explicitly tell Bacula to erase the
88 catalog entries for a volume. You'd usually do this when you want to reuse a
89 Bacula volume, because there's no point in keeping a list of files that USED
90 TO BE on a tape. Or, if the catalog is starting to get too big, you could
91 prune the oldest jobs to save space. Manual pruning is done with the
92 \ilink{ prune command}{ManualPruning} in the console. (thanks to
93 Bryce Denney for the above explanation).
95 \subsection*{Prunning Directives}
96 \index[general]{Prunning Directives }
97 \index[general]{Directives!Prunning }
98 \addcontentsline{toc}{subsection}{Prunning Directives}
100 There are three pruning durations. All apply to catalog database records and
101 not to the actual data in a Volume. The pruning (or retention) durations are
102 for: Volumes (Media records), Jobs (Job records), and Files (File records).
103 The durations inter-depend a bit because if Bacula prunes a Volume, it
104 automatically removes all the Job records, and all the File records. Also when
105 a Job record is pruned, all the File records for that Job are also pruned
106 (deleted) from the catalog.
108 Having the File records in the database means that you can examine all the
109 files backed up for a particular Job. They take the most space in the catalog
110 (probably 90-95\% of the total). When the File records are pruned, the Job
111 records can remain, and you can still examine what Jobs ran, but not the
112 details of the Files backed up. In addition, without the File records, you
113 cannot use the Console restore command to restore the files.
115 When a Job record is pruned, the Volume (Media record) for that Job can still
116 remain in the database, and if you do a "list volumes", you will see the
117 volume information, but the Job records (and its File records) will no longer
120 In each case, pruning removes information about where older files are, but it
121 also prevents the catalog from growing to be too large. You choose the
122 retention periods in function of how many files you are backing up and the
123 time periods you want to keep those records online, and the size of the
124 database. You can always re-insert the records (with 98\% of the original data)
125 by using "bscan" to scan in a whole Volume or any part of the volume that
128 By setting {\bf AutoPrune} to {\bf yes} you will permit {\bf Bacula} to
129 automatically prune all Volumes in the Pool when a Job needs another Volume.
130 Volume pruning means removing records from the catalog. It does not shrink the
131 size of the Volume or affect the Volume data until the Volume gets
132 overwritten. When a Job requests another volume and there are no Volumes with
133 Volume Status {\bf Append} available, Bacula will begin volume pruning. This
134 means that all Jobs that are older than the {\bf VolumeRetention} period will
135 be pruned from every Volume that has Volume Status {\bf Full} or {\bf Used}
136 and has Recycle set to {\bf yes}. Pruning consists of deleting the
137 corresponding Job, File, and JobMedia records from the catalog database. No
138 change to the physical data on the Volume occurs during the pruning process.
139 When all files are pruned from a Volume (i.e. no records in the catalog), the
140 Volume will be marked as {\bf Purged} implying that no Jobs remain on the
141 volume. The Pool records that control the pruning are described below.
145 \item [AutoPrune = \lt{}yes|no\gt{}]
146 \index[console]{AutoPrune }
147 If AutoPrune is set to {\bf yes} (default), Bacula
148 will automatically apply the Volume retention period when running a Job and
149 it needs a new Volume but no appendable volumes are available. At that point,
150 Bacula will prune all Volumes that can be pruned (i.e. AutoPrune set) in an
151 attempt to find a usable volume. If during the autoprune, all files are
152 pruned from the Volume, it will be marked with VolStatus {\bf Purged}. The
153 default is {\bf yes}. Note, that although the File and Job records may be
154 pruned from the catalog, a Volume will be marked Purged (and hence
155 ready for recycling) if the Volume status is Append, Full, Used, or Error.
156 If the Volume has another status, such as Archive, Read-Only, Disabled,
157 Busy, or Cleaning, the Volume status will not be changed to Purged.
159 \item [Volume Retention = \lt{}time-period-specification\gt{}]
160 \index[console]{Volume Retention }
161 The Volume Retention record defines the length of time that Bacula will
162 guarantee that the Volume is not reused counting from the time the last job
163 stored on the Volume terminated.
165 When this time period expires, and if {\bf AutoPrune} is set to {\bf yes},
166 and a new Volume is needed, but no appendable Volume is available, Bacula
167 will prune (remove) Job records that are older than the specified Volume
170 The Volume Retention period takes precedence over any Job Retention period
171 you have specified in the Client resource. It should also be noted, that the
172 Volume Retention period is obtained by reading the Catalog Database Media
173 record rather than the Pool resource record. This means that if you change
174 the VolumeRetention in the Pool resource record, you must ensure that the
175 corresponding change is made in the catalog by using the {\bf update pool}
176 command. Doing so will insure that any new Volumes will be created with the
177 changed Volume Retention period. Any existing Volumes will have their own
178 copy of the Volume Retention period that can only be changed on a Volume by
179 Volume basis using the {\bf update volume} command.
181 When all file catalog entries are removed from the volume, its VolStatus is
182 set to {\bf Purged}. The files remain physically on the Volume until the
183 volume is overwritten.
185 Retention periods are specified in seconds, minutes, hours, days, weeks,
186 months, quarters, or years on the record. See the
187 \ilink{Configuration chapter}{Time} of this manual for
188 additional details of time specification.
190 The default is 1 year.
192 \item [Recycle = \lt{}yes|no\gt{}]
194 This statement tells Bacula whether or not the particular Volume can be
195 recycled (i.e. rewritten). If Recycle is set to {\bf no} (the default), then
196 even if Bacula prunes all the Jobs on the volume and it is marked {\bf
197 Purged}, it will not consider the tape for recycling. If Recycle is set to
198 {\bf yes} and all Jobs have been pruned, the volume status will be set to
199 {\bf Purged} and the volume may then be reused when another volume is needed.
200 If the volume is reused, it is relabeled with the same Volume Name, however
201 all previous data will be lost.
204 It is also possible to "force" pruning of all Volumes in the Pool
205 associated with a Job by adding {\bf Prune Files = yes} to the Job resource.
208 \subsection*{Recycling Algorithm}
209 \index[general]{Algorithm!Recycling }
210 \index[general]{Recycling Algorithm }
211 \addcontentsline{toc}{subsection}{Recycling Algorithm}
213 After all Volumes of a Pool have been pruned (as mentioned above, this happens
214 when a Job needs a new Volume and no appendable Volumes are available), Bacula
215 will look for the oldest Volume that is Purged (all Jobs and Files expired),
216 and if the {\bf Recycle} flag is on (Recycle=yes) for that Volume, Bacula will
217 relabel it and write new data on it.
219 The full algorithm that Bacula uses when it needs a new Volume is:
222 \item Search the Pool for a Volume with VolStatus=Append (if there is more
223 than one, the Volume with the oldest date last written is chosen. If
224 two have the same date then the one with the lowest MediaId is chosen).
225 \item Search the Pool for a Volume with VolStatus=Recycle and the InChanger
226 flag is set true (if there is more than one, the Volume with the oldest
227 date last written is chosen. If two have the same date then the one
228 with the lowest MediaId is chosen).
229 \item Try recycling any purged Volumes.
230 \item Prune volumes applying Volume retention period (Volumes with VolStatus
231 Full, Used, or Append are pruned).
232 \item Search the Pool for a Volume with VolStatus=Purged
233 \item If InChanger was set, go back to the first step above, but
234 this second time, ignore the InChanger flag in step 2.
235 \item Attempt to create a new Volume if automatic labeling enabled
236 If Python is enabled, a Python NewVolume even is generated before
237 the Label Format check is used.
238 \item If a Pool named "Scratch" exists, search for a Volume and if found
239 move it to the current Pool for the Job and use it.
240 \item Prune the oldest Volume if RecycleOldestVolume=yes (the Volume with the
241 oldest LastWritten date and VolStatus equal to Full, Recycle, Purged, Used,
242 or Append is chosen). This record ensures that all retention periods are
244 \item Purge the oldest Volume if PurgeOldestVolume=yes (the Volume with the
245 oldest LastWritten date and VolStatus equal to Full, Recycle, Purged, Used,
246 or Append is chosen). We strongly recommend against the use of {\bf
247 PurgeOldestVolume} as it can quite easily lead to loss of current backup
249 \item Give up and ask operator.
252 The above occurs when Bacula has finished writing a Volume or when no Volume
253 is present in the drive.
255 On the other hand, if you have inserted a different Volume after the last job,
256 and Bacula recognizes the Volume as valid, it will request authorization from
257 the Director to use this Volume. In this case, if you have set {\bf Recycle
258 Current Volume = yes} and the Volume is marked as Used or Full, Bacula will
259 prune the volume and if all jobs were removed during the pruning (respecting
260 the retention periods), the Volume will be recycled and used.
261 The recycling algorithm in this case is:
264 \item If the VolStatus is {\bf Append} or {\bf Recycle} and {\bf Accept Any
265 Volume} is set, the volume will be used.
266 \item If {\bf Recycle Current Volume} is set and the volume is marked {\bf
267 Full} or {\bf Used}, Bacula will prune the volume (applying the retention
268 period). If all Jobs are pruned from the volume, it will be recycled.
271 This permits users to manually change the Volume every day and load tapes in
272 an order different from what is in the catalog, and if the volume does not
273 contain a current copy of your backup data, it will be used.
275 \subsection*{Recycle Status}
276 \index[general]{Status!Recycle }
277 \index[general]{Recycle Status }
278 \addcontentsline{toc}{subsection}{Recycle Status}
280 Each Volume inherits the Recycle status (yes or no) from the Pool resource
281 record when the Media record is created (normally when the Volume is labeled).
282 This Recycle status is stored in the Media record of the Catalog. Using
283 the Console program, you may subsequently change the Recycle status for each
284 Volume. For example in the following output from {\bf list volumes}:
288 +----------+-------+--------+---------+------------+--------+-----+
289 | VolumeNa | Media | VolSta | VolByte | LastWritte | VolRet | Rec |
290 +----------+-------+--------+---------+------------+--------+-----+
291 | File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 1 |
292 | File0002 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
293 | File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
294 | File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
295 | File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
296 | File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
297 | File0007 | File | Purged | 1896466 | 2002-05-26 | 14400 | 1 |
298 +----------+-------+--------+---------+------------+--------+-----+
302 all the volumes are marked as recyclable, and the last Volume, {\bf File0007}
303 has been purged, so it may be immediately recycled. The other volumes are all
304 marked recyclable and when their Volume Retention period (14400 seconds or 4
305 hours) expires, they will be eligible for pruning, and possibly recycling.
306 Even though Volume {\bf File0007} has been purged, all the data on the Volume
307 is still recoverable. A purged Volume simply means that there are no entries
308 in the Catalog. Even if the Volume Status is changed to {\bf Recycle}, the
309 data on the Volume will be recoverable. The data is lost only when the Volume
310 is re-labeled and re-written.
312 To modify Volume {\bf File0001} so that it cannot be recycled, you use the
313 {\bf update volume pool=File} command in the console program, or simply {\bf
314 update} and Bacula will prompt you for the information.
318 +----------+------+-------+---------+-------------+-------+-----+
319 | VolumeNa | Media| VolSta| VolByte | LastWritten | VolRet| Rec |
320 +----------+------+-------+---------+-------------+-------+-----+
321 | File0001 | File | Full | 4190055 | 2002-05-25 | 14400 | 0 |
322 | File0002 | File | Full | 1897236 | 2002-05-26 | 14400 | 1 |
323 | File0003 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
324 | File0004 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
325 | File0005 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
326 | File0006 | File | Full | 1896460 | 2002-05-26 | 14400 | 1 |
327 | File0007 | File | Purged| 1896466 | 2002-05-26 | 14400 | 1 |
328 +----------+------+-------+---------+-------------+-------+-----+
332 In this case, {\bf File0001} will never be automatically recycled. The same
333 effect can be achieved by setting the Volume Status to Read-Only.
335 \subsection*{Making Bacula Use a Single Tape}
337 \index[general]{Tape!Making Bacula Use a Single }
338 \index[general]{Making Bacula Use a Single Tape }
339 \addcontentsline{toc}{subsection}{Making Bacula Use a Single Tape}
341 Most people will want Bacula to fill a tape and when it is full, a new tape
342 will be mounted, and so on. However, as an extreme example, it is possible for
343 Bacula to write on a single tape, and every night to rewrite it. To get this
344 to work, you must do two things: first, set the VolumeRetention to less than
345 your save period (one day), and the second item is to make Bacula mark the
346 tape as full after using it once. This is done using {\bf UseVolumeOnce =
347 yes}. If this latter record is not used and the tape is not full after the
348 first time it is written, Bacula will simply append to the tape and eventually
349 request another volume. Using the tape only once, forces the tape to be marked
350 {\bf Full} after each use, and the next time {\bf Bacula} runs, it will
353 An example Pool resource that does this is:
359 Use Volume Once = yes
362 VolumeRetention = 12h # expire after 12 hours
368 \subsection*{A Daily, Weekly, Monthly Tape Usage Example}
370 \index[general]{Daily, Weekly, Monthly Tape Usage Example }
371 \index[general]{Example!Daily Weekly Monthly Tape Usage }
372 \addcontentsline{toc}{subsection}{Daily, Weekly, Monthly Tape Usage Example}
374 This example is meant to show you how one could define a fixed set of volumes
375 that Bacula will rotate through on a regular schedule. There are an infinite
376 number of such schemes, all of which have various advantages and
379 We start with the following assumptions:
382 \item A single tape has more than enough capacity to do a full save.
383 \item There are 10 tapes that are used on a daily basis for incremental
384 backups. They are prelabeled Daily1 ... Daily10.
385 \item There are 4 tapes that are used on a weekly basis for full backups.
386 They are labeled Week1 ... Week4.
387 \item There are 12 tapes that are used on a monthly basis for full backups.
388 They are numbered Month1 ... Month12
389 \item A full backup is done every Saturday evening (tape inserted Friday
390 evening before leaving work).
391 \item No backups are done over the weekend (this is easy to change).
392 \item The first Friday of each month, a Monthly tape is used for the Full
394 \item Incremental backups are done Monday - Friday (actually Tue-Fri
398 We start the system by doing a Full save to one of the weekly volumes or one
399 of the monthly volumes. The next morning, we remove the tape and insert a
400 Daily tape. Friday evening, we remove the Daily tape and insert the next tape
401 in the Weekly series. Monday, we remove the Weekly tape and re-insert the
402 Daily tape. On the first Friday of the next month, we insert the next Monthly
403 tape in the series rather than a Weekly tape, then continue. When a Daily tape
404 finally fills up, {\bf Bacula} will request the next one in the series, and
405 the next day when you notice the email message, you will mount it and {\bf
406 Bacula} will finish the unfinished incremental backup.
408 What does this give? Well, at any point, you will have the last complete
409 Full save plus several Incremental saves. For any given file you want to
410 recover (or your whole system), you will have a copy of that file every day
411 for at least the last 14 days. For older versions, you will have at least 3
412 and probably 4 Friday full saves of that file, and going back further, you
413 will have a copy of that file made on the beginning of the month for at least
416 So you have copies of any file (or your whole system) for at least a year, but
417 as you go back in time, the time between copies increases from daily to weekly
420 What would the Bacula configuration look like to implement such a scheme?
426 Run = Level=Full Pool=Monthly 1st sat at 03:05
427 Run = Level=Full Pool=Weekly 2nd-5th sat at 03:05
428 Run = Level=Incremental Pool=Daily tue-fri at 03:05
434 Client = LocalMachine
439 Schedule = "NightlySave"
441 # Definition of file storage device
446 Password = XXXXXXXXXXXXX
452 Include = signature=MD5 {
461 VolumeRetention = 10d # recycle in 10 days
467 Use Volume Once = yes
470 VolumeRetention = 30d # recycle in 30 days (default)
475 Use Volume Once = yes
478 VolumeRetention = 365d # recycle in 1 year
484 \subsection*{ Automatic Pruning and Recycling Example}
485 \label{PruningExample}
486 \index[general]{Automatic Pruning and Recycling Example }
487 \index[general]{Example!Automatic Pruning and Recycling }
488 \addcontentsline{toc}{subsection}{Automatic Pruning and Recycling Example}
490 Perhaps the best way to understand the various resource records that come into
491 play during automatic pruning and recycling is to run a Job that goes through
492 the whole cycle. If you add the following resources to your Director's
498 Name = "30 minute cycle"
499 Run = Level=Full Pool=File Messages=Standard Storage=File
501 Run = Level=Full Pool=File Messages=Standard Storage=File
513 Schedule = "30 minute cycle"
515 # Definition of file storage device
518 Address = XXXXXXXXXXX
520 Password = XXXXXXXXXXXXX
526 Include = signature=MD5 {
533 Use Volume Once = yes
544 Where you will need to replace the {\bf ffffffffff}'s by the appropriate files
545 to be saved for your configuration. For the FileSet Include, choose a
546 directory that has one or two megabytes maximum since there will probably be
547 approximately 8 copies of the directory that {\bf Bacula} will cycle through.
549 In addition, you will need to add the following to your Storage daemon's
557 Archive Device = /tmp
560 AutomaticMount = yes;
567 With the above resources, Bacula will start a Job every half hour that saves a
568 copy of the directory you chose to /tmp/File0001 ... /tmp/File0012. After 4
569 hours, Bacula will start recycling the backup Volumes (/tmp/File0001 ...). You
570 should see this happening in the output produced. Bacula will automatically
571 create the Volumes (Files) the first time it uses them.
573 To turn it off, either delete all the resources you've added, or simply
574 comment out the {\bf Schedule} record in the {\bf Job} resource.
576 \subsection*{Manually Recycling Volumes}
577 \label{manualrecycling}
578 \index[general]{Volumes!Manually Recycling }
579 \index[general]{Manually Recycling Volumes }
580 \addcontentsline{toc}{subsection}{Manually Recycling Volumes}
582 Although automatic recycling of Volumes is implemented in version 1.20 and
584 \ilink{Automatic Recycling of Volumes}{_ChapterStart22} chapter of
585 this manual), you may want to manually force reuse (recycling) of a Volume.
587 Assuming that you want to keep the Volume name, but you simply want to write
588 new data on the tape, the steps to take are:
591 \item Use the {\bf update volume} command in the Console to ensure that the
592 {\bf Recycle} field is set to {\bf 1}
593 \item Use the {\bf purge jobs volume} command in the Console to mark the
594 Volume as {\bf Purged}. Check by using {\bf list volumes}.
597 Once the Volume is marked Purged, it will be recycled the next time a Volume
600 If you wish to reuse the tape by giving it a new name, follow the following
604 \item Use the {\bf purge jobs volume} command in the Console to mark the
605 Volume as {\bf Purged}. Check by using {\bf list volumes}.
606 \item In Bacula version 1.30 or greater, use the Console {\bf relabel}
607 command to relabel the Volume.
610 Please note that the relabel command applies only to tape Volumes.
612 For Bacula versions prior to 1.30 or to manually relabel the Volume, use the
616 \item Use the {\bf delete volume} command in the Console to delete the Volume
618 \item If a different tape is mounted, use the {\bf unmount} command,
619 remove the tape, and insert the tape to be renamed.
620 \item Write an EOF mark in the tape using the following commands:
624 mt -f /dev/nst0 rewind
629 where you replace {\bf /dev/nst0} with the appropriate device name on your
631 \item Use the {\bf label} command to write a new label to the tape and to
632 enter it in the catalog.
635 Please be aware that the {\bf delete} command can be dangerous. Once it is
636 done, to recover the File records, you must either restore your database as it
637 was before the {\bf delete} command, or use the {\bf bscan} utility program to
638 scan the tape and recreate the database entries.