4 \subsection*{The FileSet Resource}
5 \label{FileSetResource}
6 \index[general]{Resource!FileSet }
7 \index[general]{FileSet Resource }
8 \addcontentsline{toc}{subsection}{FileSet Resource}
10 The FileSet resource defines what files are to be included or excluded in a
11 backup job. A {\bf FileSet} resource is required for each backup Job. It
12 consists of a list of files or directories to be included, a list of files
13 or directories to be excluded and the various backup options such as
14 compression, encryption, and signatures that are to be applied to each
17 Any change to the list of the included files will cause Bacula to
18 automatically create a new FileSet (defined by the name and an MD5 checksum
19 of the Include/Exclude contents). Each time a new FileSet is created,
20 Bacula will ensure that the next backup is always a Full save.
26 Start of the FileSet resource. One {\bf FileSet} resource must be
27 defined for each Backup job.
29 \item [Name = \lt{}name\gt{}]
31 The name of the FileSet resource. This directive is required.
33 \item [Ignore FileSet Changes = \lt{}yes|no\gt{}]
34 \index[dir]{Ignore FileSet Changes}
35 If this directive is set to {\bf yes}, any changes you make to the FileSet
36 Include or Exclude lists will be ignored and not cause Bacula to immediately
37 perform a Full backup. The default is {\bf no}, in which case, if you change
38 the Include or Exclude, Bacula will force a Full backup to ensure that
39 everything is properly backed up. It is not recommended to set this directive
40 to yes. This directive is available in Bacula version 1.35.4 or later.
42 \item [Enable VSS = \lt{}yes|no\gt{}]
43 \index[dir]{Enable VSS}
44 If this directive is set to {\bf yes} the File daemon will be notified
45 that the user wants to use a Volume Shadow Copy Service (VSS) backup
46 for this job. The default is {\bf no}. This directive is effective
47 only for VSS enabled Win32 File daemons. It permits a consistent copy
48 of open files to be made for cooperating writer applications, and for
49 applications that are not VSS away, Bacula can at least copy open files.
50 For more information, please see the
51 \ilink{Windows}{VSS} chapter of this manual.
53 \item [Include \{ Options \{\lt{}file-options\gt{}\} ...;
54 \lt{}file-list\gt{} \} ]
55 \index[dir]{Include \{ [ Options \{\lt{}file-options\gt{}\} ...]
56 \lt{}file-list\gt{} \} }
58 \item [Options \{ \lt{}file-options\gt{} \} ]
59 \index[dir]{Options \{ \lt{}file-options\gt{} \} }
61 \item [Exclude \{ \lt{}file-list\gt{} \}]
62 \index[dir]{Exclude \{ \lt{}file-list\gt{} \} }
66 The Include resource must contain a list of directories and/or files to be
67 processed in the backup job. Normally, all files found in all
68 subdirectories of any directory in the Include File list will be backed up.
69 Note, see below for the definition of \lt{}file-list\gt{}.
70 The Include resource may also contain one or more Options resources that
71 specify options such as compression to be applied to all or any subset of
72 the files found when processing the file-list for backup.
74 There can be any number of {\bf Include} resources within the FileSet, each
75 having its own list of directories or files to be backed up and the backup
76 options defined by one or more Options resources. The {\bf file-list}
77 consists of one file or directory name per line. Directory names should be
78 specified without a trailing slash with Unix path notation.
80 Windows users, please take note to specify directories (even c:/...) in
81 Unix path notation. If you use Windows conventions, you will most likely
82 not be able to restore your files due to the fact that the Windows
83 path separator was defined as an escape character long before Windows
84 existed, and Bacula adheres to that convention (i.e. \\ means the next character
87 You should always specify a full path for every directory and file that you
88 list in the FileSet. In addition, on Windows machines, you should {\bf
89 always} prefix the directory or filename with the drive specification in
90 lower case (e.g. {\bf c:/xxx}) using Unix directory name separators
93 Bacula's default for processing directories is to recursively descend in
94 the directory saving all files and subdirectories. Bacula will not by
95 default cross filesystems (or mount points in Unix parlance). This means
96 that if you specify the root partition (e.g. {\bf /}), Bacula will save
97 only the root partition and not any of the other mounted filesystems.
98 Similarly on Windows systems, you must explicitly specify each of the
99 drives you want saved (e.g.
100 {\bf c:/} and {\bf d:/} ...). In addition, at least for Windows systems, you
101 will most likely want to enclose each specification within double quotes
102 particularly if the directory (or file) name contains spaces. The {\bf df}
103 command on Unix systems will show you which mount points you must specify to
104 save everything. See below for an example.
106 Take special care not to include a directory twice or Bacula will backup
107 the same files two times wasting a lot of space on your archive device.
108 Including a directory twice is very easy to do. For example:
115 Options { compression=GZIP }
120 on a Unix system where /usr is a subdirectory (rather than a mounted
121 filesystem) will cause /usr to be backed up twice. In this case, on Bacula
122 versions prior to 1.32f-5-09Mar04 due to a bug, you will not be able to
123 restore hard linked files that were backed up twice.
125 If you have used Bacula prior to version 1.36.3, you will note three things in
126 the new FileSet syntax:
129 \item There is no equal sign (=) after the Include and before the opening
130 brace (\{). The same is true for the Exclude.
131 \item Each directory (or filename) to be included or excluded is preceded by a {\bf File
132 =}. Previously they were simply listed on separate lines.
133 \item The options that previously appeared on the Include line now must be
134 specified within their own Options resource.
135 \item The Exclude resource does not accept Options.
136 \item When using wild-cards or regular expressions, directory names are
137 always terminated with a slash (/) and filenames have no trailing slash.
140 The Options resource is optional, but when specified, it will contain a
141 list of {\bf keyword=value} options to be applied to the file-list.
142 See below for the definition of file-list.
143 Multiple Options resources may be specified one after another. As the
144 files are found in the specified directories, the Options will applied to
145 the filenames to determine if and how the file should be backed up. The
146 Options resources are applied in the order they are specified in the
147 FileSet until the first one that matches.
149 A key point is that in the absence of an Option or no other Option is
150 matched, every file is accepted for backing up. This means that if
151 you want to exclude something, you must explicitly specify an Option
152 with an {\bf exclude = yes} and some pattern matching.
154 Once Bacula determines that the Options resource matches the file under
155 consideration, that file will be saved without looking at any other Options
156 resources that may be present. This means that any wild cards must appear
157 before an Options resource without wild cards.
159 If for some reason, Bacula applies all the Options resources to a file
160 under consideration for backup, but there are no matches (generally because
161 of wild cards that don't match), Bacula as a default will then backup the
162 file. This is quite logical if you consider the case of no Options, where
163 you want everything to be backed up, and it is important to keep in
164 mind when excluding as mentioned above.
166 However, one additional point is that
167 in the case that no match was found, Bacula will use the options found in
168 the last Options resource. As a consequence, if you want a particular set
169 of "default" options, you should put them in an Options resource after
172 It is a good idea to put all your wild-card and regex expressions inside
173 double quotes to prevent conf file scanning problems.
175 This is perhaps a bit overwhelming, so there are a number of examples included
176 below to illustrate how this works.
178 The directives within an Options resource may be one of the following:
182 \item [compression=GZIP]
183 \index[fd]{compression }
184 All files saved will be software compressed using the GNU ZIP compression
185 format. The compression is done on a file by file basis by the File daemon.
186 If there is a problem reading the tape in a single record of a file, it will
187 at most affect that file and none of the other files on the tape. Normally
188 this option is {\bf not} needed if you have a modern tape drive as the drive
189 will do its own compression. In fact, if you specify software compression at
190 the same time you have hardware compression turned on, your files may
191 actually take more space on the volume.
193 Software compression is very important if you are writing your Volumes to a
194 file, and it can also be helpful if you have a fast computer but a slow
195 network, otherwise it is generally better to rely your tape drive's hardware
196 compression. As noted above, it is not generally a good idea to do both software
197 and hardware compression.
199 Specifying {\bf GZIP} uses the default compression level six (i.e. {\bf GZIP}
200 is identical to {\bf GZIP6}). If you want a different compression level (1
201 through 9), you can specify it by appending the level number with no
202 intervening spaces to {\bf GZIP}. Thus {\bf compression=GZIP1} would give
203 minimum compression but the fastest algorithm, and {\bf compression=GZIP9}
204 would give the highest level of compression, but requires more computation.
205 According to the GZIP documentation, compression levels greater than 6
206 generally give very little extra compression and are rather CPU intensive.
208 \item [signature=SHA1]
209 \index[fd]{signature }
210 An SHA1 signature will be computed for all The SHA1 algorithm is purported to
211 be some what slower than the MD5 algorithm, but at the same time is
212 significantly better from a cryptographic point of view (i.e. much fewer
213 collisions, much lower probability of being hacked.) It adds four more bytes
214 than the MD5 signature. We strongly recommend that either this option or MD5
215 be specified as a default for all files. Note, only one of the two options
216 MD5 or SHA1 can be computed for any file.
218 \item [signature=MD5]
219 \index[fd]{signature }
220 An MD5 signature will be computed for all files saved. Adding this option
221 generates about 5\% extra overhead for each file saved. In addition to the
222 additional CPU time, the MD5 signature adds 16 more bytes per file to your
223 catalog. We strongly recommend that this option or the SHA1 option be
224 specified as a default for all files.
226 \item [verify=\lt{}options\gt{}]
228 The options letters specified are used when running a {\bf Verify
229 Level=Catalog} as well as the {\bf DiskToCatalog} level job. The options
230 letters may be any combination of the following:
238 compare the permission bits
241 compare the number of links
253 compare the access time
256 compare the modification time (st\_mtime)
259 compare the change time (st\_ctime)
262 report file size decreases
265 compare the MD5 signature
268 compare the SHA1 signature
271 A useful set of general options on the {\bf Level=Catalog} or {\bf
272 Level=DiskToCatalog} verify is {\bf pins5} i.e. compare permission bits,
273 inodes, number of links, size, and MD5 changes.
277 If set to {\bf yes} (the default), {\bf Bacula} will remain on a single file
278 system. That is it will not backup file systems that are mounted on a
279 subdirectory. If you are using a *nix system, you may not even be aware
280 that there are several different filesystems as they are often
281 automatically mounted by the OS (e.g. /dev, /net, /sys, /proc, ...).
282 With Bacula 1.38.0 or later, it will inform you when it decides not
283 to traverse into another filesystem. This can be very useful if you
284 forgot to backup a particular partition. An example of the
285 informational message in the job report is:
289 rufus-fd: Filesystem change prohibited. Will not descend into /misc
290 rufus-fd: Filesystem change prohibited. Will not descend into /net
291 rufus-fd: Filesystem change prohibited. Will not descend into /var/lib/nfs/rpc_pipefs
292 rufus-fd: Filesystem change prohibited. Will not descend into /selinux
293 rufus-fd: Filesystem change prohibited. Will not descend into /sys
294 rufus-fd: Filesystem change prohibited. Will not descend into /dev
295 rufus-fd: Filesystem change prohibited. Will not descend into /home
298 If you wish to backup multiple filesystems, you can explicitly
299 list each filesystem you want saved. Otherwise, if you set the onefs option
300 to {\bf no}, Bacula will backup all mounted file systems (i.e. traverse mount
301 points) that are found within the {\bf FileSet}. Thus if you have NFS or
302 Samba file systems mounted on a directory listed in your FileSet, they will
303 also be backed up. Normally, it is preferable to set {\bf onefs=yes} and to
304 explicitly name each filesystem you want backed up. Explicitly naming the
305 filesystems you want backed up avoids the possibility of getting into a
306 infinite loop recursing filesystems. Another possiblity is to
307 use {\bf onefs=no} and to set {\bs fstype=ext2, ...}.
308 See the example below for more details.
310 If you think that Bacula should be backing up a particular directory
311 and it is not, and you have {\bf onefs=no} set, before you complain,
321 where you replace {\bf filesystem} with the one in question. If the
322 {\bf Device:} number is different for / and for your filesystem, then they
323 are on different filesystems. E.g.
328 Size: 4096 Blocks: 16 IO Block: 4096 directory
329 Device: 302h/770d Inode: 2 Links: 26
330 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
331 Access: 2005-11-10 12:28:01.000000000 +0100
332 Modify: 2005-09-27 17:52:32.000000000 +0200
333 Change: 2005-09-27 17:52:32.000000000 +0200
337 Size: 4096 Blocks: 16 IO Block: 4096 directory
338 Device: 308h/776d Inode: 2 Links: 7
339 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
340 Access: 2005-11-10 12:28:02.000000000 +0100
341 Modify: 2005-11-06 12:36:48.000000000 +0100
342 Change: 2005-11-06 12:36:48.000000000 +0100
346 Also be aware that even if you include {\bf /home} in your list
347 of files to backup, as you most likely should, you will get the
348 informational message about Filesystem change prohibited when Bacula is
349 processing the {\bf /} directory.
353 \item [portable=yes|no]
354 \index[dir]{portable }
355 If set to {\bf yes} (default is {\bf no}), the Bacula File daemon will backup
356 Win32 files in a portable format, but not all Win32 file attributes will be
357 saved and restored. By default, this option is set to {\bf no}, which means
358 that on Win32 systems, the data will be backed up using Windows API calls and
359 on WinNT/2K/XP, all the security and ownership attributes will be properly
360 backed up (and restored). However this format is not portable to other
361 systems -- e.g. Unix, Win95/98/Me. When backing up Unix systems, this option
362 is ignored, and unless you have a specific need to have portable backups, we
363 recommend accept the default ({\bf no}) so that the maximum information
364 concerning your files is saved.
366 \item [recurse=yes|no]
368 If set to {\bf yes} (the default), Bacula will recurse (or descend) into all
369 subdirectories found unless the directory is explicitly excluded using an
370 {\bf exclude} definition. If you set {\bf recurse=no}, Bacula will save the
371 subdirectory entries, but not descend into the subdirectories, and thus will
372 not save the files or directories contained in the subdirectories. Normally,
373 you will want the default ({\bf yes}).
375 \item [sparse=yes|no]
377 Enable special code that checks for sparse files such as created by
378 ndbm. The default is {\bf no}, so no checks are made for sparse files.
379 You may specify {\bf sparse=yes} even on files that are not sparse file.
380 No harm will be done, but there will be a small additional overhead to
381 check for buffers of all zero, and a small additional amount of space on
382 the output archive will be used to save the seek address of each
383 non-zero record read.
385 {\bf Restrictions:} Bacula reads files in 32K buffers. If the whole
386 buffer is zero, it will be treated as a sparse block and not written to
387 tape. However, if any part of the buffer is non-zero, the whole buffer
388 will be written to tape, possibly including some disk sectors (generally
389 4098 bytes) that are all zero. As a consequence, Bacula's detection of
390 sparse blocks is in 32K increments rather than the system block size.
391 If anyone considers this to be a real problem, please send in a request
392 for change with the reason.
394 If you are not familiar with sparse files, an example is say a file
395 where you wrote 512 bytes at address zero, then 512 bytes at address 1
396 million. The operating system will allocate only two blocks, and the
397 empty space or hole will have nothing allocated. However, when you read
398 the sparse file and read the addresses where nothing was written, the OS
399 will return all zeros as if the space were allocated, and if you backup
400 such a file, a lot of space will be used to write zeros to the volume.
401 Worse yet, when you restore the file, all the previously empty space
402 will now be allocated using much more disk space. By turning on the
403 {\bf sparse} option, Bacula will specifically look for empty space in
404 the file, and any empty space will not be written to the Volume, nor
405 will it be restored. The price to pay for this is that Bacula must
406 search each block it reads before writing it. On a slow system, this
407 may be important. If you suspect you have sparse files, you should
408 benchmark the difference or set sparse for only those files that are
412 \item [readfifo=yes|no]
413 \index[fd]{readfifo }
414 If enabled, tells the Client to read the data on a backup and write the
415 data on a restore to any FIFO (pipe) that is explicitly mentioned in the
416 FileSet. In this case, you must have a program already running that
417 writes into the FIFO for a backup or reads from the FIFO on a restore.
418 This can be accomplished with the {\bf RunBeforeJob} directive. If this
419 is not the case, Bacula will hang indefinitely on reading/writing the
420 FIFO. When this is not enabled (default), the Client simply saves the
421 directory entry for the FIFO.
423 Unfortunately, when Bacula runs a RunBeforeJob, it waits until that
424 script terminates, and if the script accesses the FIFO to write
425 into the it, the Bacula job will block and everything will stall.
426 However, Vladimir Stavrinov as supplied tip that allows this feature
427 to work correctly. He simply adds the following to the beginning
428 of the RunBeforeJob script:
435 \item [mtimeonly=yes|no]
436 \index[dir]{mtimeonly }
437 If enabled, tells the Client that the selection of files during
438 Incremental and Differential backups should based only on the st\_mtime
439 value in the stat() packet. The default is {\bf no} which means that
440 the selection of files to be backed up will be based on both the
441 st\_mtime and the st\_ctime values. In general, it is not recommended
444 \item [keepatime=yes|no]
445 \index[dir]{keepatime }
446 The default is {\bf no}. When enabled, Bacula will reset the st\_atime
447 (access time) field of files that it backs up to their value prior to
448 the backup. This option is not generally recommended as there are very
449 few programs that use st\_atime, and the backup overhead is increased
450 because of the additional system call necessary to reset the times.
451 However, for some files, such as mailboxes, when Bacula backs up the
452 file, the user will notice that someone (Bacula) has accessed the
453 file. In this, case keepatime can be useful.
454 (I'm not sure this works on Win32).
456 Note, if you use this feature, when Bacula resets the access time, the
457 change time (st\_ctime) will automatically be modified by the system,
458 so on the next incremental job, the file will be backed up even if
459 it has not changed. As a consequence, you will probably also want
460 to use {\bf mtimeonly = yes} as well as keepatime (thanks to
461 Rudolf Cejka for this tip).
463 \item [hardlinks=yes|no]
464 \index[dir]{hardlinks}
465 When enabled (default), this directive will cause hard inks to be
466 backed up. However, the File daemon keeps track of hard linked files and
467 will backup the data only once. The process of keeping track of the
468 hard links can be quite expensive if you have lots of them (tens of
469 thousands or more). This doesn't occur on normal Unix systems, but if
470 you use a program like BackupPC, it can create hundreds of thousands, or
471 even millions of hard links. Backups become very long and the File daemon
472 will consume a lot of CPU power checking hard links. In such a case,
473 set {\bf hardlinks=no} and hard links will not be backed up. Note, using
474 this option will most likely backup more data and on a restore the file
475 system will not be restored identically to the original.
477 \item [wild=\lt{}string\gt{}]
479 Specifies a wild-card string to be applied to the filenames and
480 directory names. Note, if {\bf Exclude} is not enabled, the wild-card
481 will select which files are to be included. If {\bf Exclude=yes} is
482 specified, the wild-card will select which files are to be excluded.
483 Multiple wild-card directives may be specified, and they will be applied
484 in turn until the first one that matches. Note, if you exclude a
485 directory, no files or directories below it will be matched.
486 It is recommended to enclose the string in double quotes.
488 \item [wildfile=\lt{}string\gt{}]
489 \index[dir]{wildfile }
490 Specifies a wild-card string to be applied to filenames only. No
491 directories will be matched by this directive. Note, if {\bf Exclude}
492 is not enabled, the wild-card will select which files are to be
493 included. If {\bf Exclude=yes} is specified, the wild-card will select
494 which files are to be excluded. Multiple wild-card directives may be
495 specified, and they will be applied in turn until the first one that
497 It is recommended to enclose the string in double quotes.
498 An example of excluding with the WildFile option on Win32 machines is
501 \item [wilddir=\lt{}string\gt{}]
502 \index[dir]{wilddir }
503 Specifies a wild-card string to be applied to directory names only. No
504 filenames will be matched by this directive. Note, if {\bf Exclude} is
505 not enabled, the wild-card will select directories files are to be
506 included. If {\bf Exclude=yes} is specified, the wild-card will select
507 which files are to be excluded. Multiple wild-card directives may be
508 specified, and they will be applied in turn until the first one that
509 matches. Note, if you exclude a directory, no files or directories
510 below it will be matched.
511 It is recommended to enclose the string in double quotes.
512 An example of excluding with the WildDir option on Win32 machines is
516 \item [regex=\lt{}string\gt{}]
518 Specifies a POSIX extended regular expression to be applied to the
519 filenames and directory names.
520 This directive is available in version 1.35 and later. If {\bf
521 Exclude} is not enabled, the regex will select which files are to be
522 included. If {\bf Exclude=yes} is specified, the regex will select
523 which files are to be excluded. Multiple regex directives may be
524 specified within an Options resource, and they will be applied in turn
525 until the first one that matches. Note, if you exclude a
526 directory, no files or directories below it will be matched.
527 It is recommended to enclose the string in double quotes.
529 \item [regexfile=\lt{}string\gt{}]
530 \index[dir]{regexfile }
531 Specifies a POSIX extended regular expression to be applied to filenames
532 only. No directories will be matched by this directive. Note, if {\bf
533 Exclude} is not enabled, the regex will select which files are to be
534 included. If {\bf Exclude=yes} is specified, the regex will select
535 which files are to be excluded. Multiple regex directives may be
536 specified, and they will be applied in turn until the first one that
538 It is recommended to enclose the string in double quotes.
540 \item [regexdir=\lt{}string\gt{}]
541 \index[dir]{regexdir }
542 Specifies a POSIX extended regular expression to be applied to directory
543 names only. No filenames will be matched by this directive. Note, if
544 {\bf Exclude} is not enabled, the regex will select directories
545 files are to be included. If {\bf Exclude=yes} is specified, the
546 regex will select which files are to be excluded. Multiple
547 regex directives may be specified, and they will be applied in turn
548 until the first one that matches. Note, if you exclude a directory, no
549 files or directories below it will be matched.
550 It is recommended to enclose the string in double quotes.
552 \item [exclude=yes|no]
553 \index[dir]{exclude }
554 The default is {\bf no}. When enabled, any files matched within the Options
555 will be excluded from the backup.
559 \item [aclsupport=yes|no]
560 \index[dir]{aclsupport }
561 The default is {\bf no}. If this option is set to yes, and you have the
562 POSIX {\bf libacl} installed on your system, Bacula will backup the file
563 and directory UNIX Access Control Lists (ACL) as defined in IEEE Std
564 1003.1e draft 17 and "POSIX.1e" (abandoned). This feature is
565 available on UNIX only and depends on the ACL library. Bacula is
566 automatically compiled with ACL support if the {\bf libacl} library is
567 installed on your system (shown in config.out). While restoring the
568 files Bacula will try to restore the ACLs, if there is no ACL support
569 available on the system, Bacula restores the files and directories but
570 not the ACL information. Please note, if you backup an EXT3 or XFS
571 filesystem with ACLs, then you restore them to a different filesystem
572 (perhaps reiserfs) that does not have ACLs, the ACLs will be ignored.
574 \item [ignore case=yes|no]
575 \index[dir]{ignore case }
576 The default is {\bf no}, except on Windows systems where the default
577 is {\bf yes}. When this directive is set to {\bf yes} all the case
578 of character will be ignored in wild-card and regex comparisons.
579 That is an uppercase A will match a lowercase a.
581 \item [fstype=filesystem-type]
583 This option allows you to select files and directories by the
584 filesystem type. The permitted filesystem-type names are:
586 ext2, jfs, ntfs, proc, reiserfs, xfs, usbdevfs, sysfs, smbfs,
587 iso9660. For ext3 systems, use ext2.
589 You may have multiple Fstype directives, and thus permit matching
590 of multiple filesystem types within a single Options resource. If
591 the type specified on the fstype directive does not match the
592 filesystem for a particular directive, that directory will not be
593 backed up. This directive can be used to prevent backing up
594 non-local filesystems. Normally, when you use this directive, you
595 would also set {\bf onefs=no} so that Bacula will traverse filesystems.
597 This option is not implemented in Win32 systems.
600 \item [hfsplussupport=yes|no]
601 \index[dir]{hfsplussupport }
602 This option allows you to turn on support for Mac OSX HFS plus
607 {\bf \lt{}file-list\gt{}} is a list of directory and/or filename names
608 specified with a {\bf File =} directive. To include names containing spaces,
609 enclose the name between double-quotes.
611 There are a number of special cases when specifying directories and files in a
612 {\bf file-list}. They are:
615 \item Any name preceded by an at-sign (@) is assumed to be the name of a
616 file, which contains a list of files each preceded by a "File =". The
617 named file is read once when the configuration file is parsed during the
618 Director startup. Note, that the file is read on the Director's machine
619 and not on the Client's. In fact, the @filename can appear anywhere
620 within the conf file where a token would be read, and the contents of
621 the named file will be logically inserted in the place of the @filename.
622 What must be in the file depends on the location the @filename is
623 specified in the conf file. For example:
628 Options { compression=GZIP }
629 @/home/files/my-files
634 \item Any name beginning with a vertical bar (|) is assumed to be the name of
635 a program. This program will be executed on the Director's machine at
636 the time the Job starts (not when the Director reads the configuration
637 file), and any output from that program will be assumed to be a list of
638 files or directories, one per line, to be included.
640 This allows you to have a job that, for example, includes all the local
641 partitions even if you change the partitioning by adding a disk. The
642 examples below show you how to do this. However, please note two
644 1. if you want the local filesystems, you probably should be
645 using the new {\bf fstype} directive, which was added in version 1.36.3
646 and set {\bf onefs=no}.
649 2. the exact syntax of the command needed in the examples below is very
650 system dependent. For example, on recent Linux systems, you may need to
651 add the -P option, on FreeBSD systems, the options will be different as
654 In general, you will need to prefix your command or commands with a {\bf
655 sh -c} so that they are invoked by a shell. This will not be the case
656 if you are invoking a script as in the second example below. Also, you
657 must take care to escape (precede with a \textbackslash{}) wild-cards,
658 shell character, and to ensure that any spaces in your command are
659 escaped as well. If you use a single quotes (') within a double quote
660 ("), Bacula will treat everything between the single quotes as one field
661 so it will not be necessary to escape the spaces. In general, getting
662 all the quotes and escapes correct is a real pain as you can see by the
663 next example. As a consequence, it is often easier to put everything in
664 a file and simply use the file name within Bacula. In that case the
665 {\bf sh -c} will not be necessary providing the first line of the file
674 Options { signature = SHA1 }
675 File = "|sh -c 'df -l | grep \"^/dev/hd[ab]\" | grep -v \".*/tmp\" \
676 | awk \"{print \\$6}\"'"
681 will produce a list of all the local partitions on a RedHat Linux system.
682 Note, the above line was split, but should normally be written on one line.
683 Quoting is a real problem because you must quote for Bacula which consists of
684 preceding every \textbackslash{} and every " with a \textbackslash{}, and
685 you must also quote for the shell command. In the end, it is probably easier
686 just to execute a small file with:
695 File = "|my_partitions"
700 where my\_partitions has:
705 df -l | grep "^/dev/hd[ab]" | grep -v ".*/tmp" \
710 If the vertical bar (|) in front of my\_partitions is preceded by a
711 backslash as in \textbackslash{}|, the program will be executed on the
712 Client's machine instead of on the Director's machine.
713 Please note that if the filename is given within quotes, you
714 will need to use two slashes. An example, provided by John Donagher,
715 that backs up all the local UFS partitions on a remote system is:
720 Name = "All local partitions"
722 Options { signature=SHA1; onefs=yes; }
723 File = "\\|bash -c \"df -klF ufs | tail +2 | awk '{print \$6}'\""
729 The above requires two backslash characters after the double quote (one
730 preserves the next one). If you are a Linux user, just change the {\bf ufs}
731 to {\bf ext3} (or your preferred filesystem type), and you will be in
734 If you know what filesystems you have mounted on your system, e.g.
735 for RedHat Linux normally only ext2 and ext3, you can backup
736 all local fileystems using something like:
742 Options { signature = SHA1; onfs=no; fstype=ext2 }
749 \item Any file-list item preceded by a less-than sign (\lt{}) will be taken
750 to be a file. This file will be read on the Director's machine at the time
751 the Job starts, and the data will be assumed to be a list of directories or
752 files, one per line, to be included. The names should start in column 1 and
753 should not be quoted even if they contain spaces. This feature allows you to
754 modify the external file and change what will be saved without stopping and
755 restarting Bacula as would be necessary if using the @ modifier noted above.
761 Options { signature = SHA1 }
762 File = "</home/files/local-filelist"
767 If you precede the less-than sign (\lt{}) with a backslash as in
768 \textbackslash{}\lt{}, the file-list will be read on the Client machine
769 instead of on the Director's machine. Please note that if the filename
770 is given within quotes, you will need to use two slashes.
775 Options { signature = SHA1 }
776 File = "\\</home/xxx/filelist-on-client"
781 \item If you explicitly specify a block device such as {\bf /dev/hda1}, then
782 Bacula (starting with version 1.28) will assume that this is a raw partition
783 to be backed up. In this case, you are strongly urged to specify a {\bf
784 sparse=yes} include option, otherwise, you will save the whole partition
785 rather than just the actual data that the partition contains. For example:
790 Options { signature=MD5; sparse=yes }
796 will backup the data in device /dev/hd6.
798 Ludovic Strappazon has pointed out that this feature can be used to backup a
799 full Microsoft Windows disk. Simply boot into the system using a Linux Rescue
800 disk, then load a statically linked Bacula as described in the
801 \ilink{ Disaster Recovery Using Bacula}{_ChapterStart38} chapter of
802 this manual. Then save the whole disk partition. In the case of a disaster,
803 you can then restore the desired partition by again booting with the rescue
804 disk and doing a restore of the partition.
805 \item If you explicitly specify a FIFO device name (created with mkfifo), and
806 you add the option {\bf readfifo=yes} as an option, Bacula will read the FIFO
807 and back its data up to the Volume. For example:
816 File = /home/abc/fifo
821 if {\bf /home/abc/fifo} is a fifo device, Bacula will open the fifo,
822 read it, and store all data thus obtained on the Volume. Please note,
823 you must have a process on the system that is writing into the fifo, or
824 Bacula will hang, and after one minute of waiting, Bacula will give up
825 and go on to the next file. The data read can be anything since Bacula
826 treats it as a stream.
828 This feature can be an excellent way to do a "hot" backup of a very
829 large database. You can use the {\bf RunBeforeJob} to create the fifo
830 and to start a program that dynamically reads your database and writes
831 it to the fifo. Bacula will then write it to the Volume. Be sure to
832 read the \ilink{readfifo section}{readfifo} that gives a
833 tip to ensure that the RunBeforeJob does not block Bacula.
835 During the restore operation, the inverse is true, after Bacula creates
836 the fifo if there was any data stored with it (no need to explicitly
837 list it or add any options), that data will be written back to the fifo.
838 As a consequence, if any such FIFOs exist in the fileset to be restored,
839 you must ensure that there is a reader program or Bacula will block, and
840 after one minute, Bacula will time out the write to the fifo and move on
844 \subsubsection*{FileSet Examples}
845 \index[general]{Examples!FileSet }
846 \index[general]{FileSet Examples}
847 \addcontentsline{toc}{subsection}{FileSet Examples}
849 The following is an example of a valid FileSet resource definition. Note,
850 the first Include pulls in the contents of the file {\bf /etc/backup.list}
851 when Bacula is started (i.e. the @), and that file must have each filename
852 to be backed up preceded by a {\bf File =} and on a separate line.
873 File = /usr/lib/another_file
879 In the above example, all the files contained in /etc/backup.list will
880 be compressed with GZIP compression, an SHA1 signature will be computed on the
881 file's contents (its data), and sparse file handling will apply.
883 The two directories /root/myfile and /usr/lib/another\_file will also be saved
884 without any options, but all files in those directories with the extensions
885 {\bf .o} and {\bf .exe} will be excluded.
887 Let's say that you now want to exclude the directory /tmp. The simplest way
888 to do so is to add an exclude directive that lists /tmp. The example
889 above would then become:
910 File = /usr/lib/another_file
920 You can add wild-cards to the File directives listed in the Exclude
921 directory, but you need to take care because if you exclude a directory,
922 it and all files and directories below it will also be excluded.
924 Now lets take a slight variation on the above and suppose
925 you want to save all your whole filesystem except {\bf /tmp}.
926 The problem that comes up is that Bacula will not normally
927 cross from one filesystem to another.
928 Doing a {\bf df} command, you get the following output:
933 Filesystem 1k-blocks Used Available Use% Mounted on
934 /dev/hda5 5044156 439232 4348692 10% /
935 /dev/hda1 62193 4935 54047 9% /boot
936 /dev/hda9 20161172 5524660 13612372 29% /home
937 /dev/hda2 62217 6843 52161 12% /rescue
938 /dev/hda8 5044156 42548 4745376 1% /tmp
939 /dev/hda6 5044156 2613132 2174792 55% /usr
940 none 127708 0 127708 0% /dev/shm
941 //minimatou/c$ 14099200 9895424 4203776 71% /mnt/mmatou
942 lmatou:/ 1554264 215884 1258056 15% /mnt/matou
943 lmatou:/home 2478140 1589952 760072 68% /mnt/matou/home
944 lmatou:/usr 1981000 1199960 678628 64% /mnt/matou/usr
945 lpmatou:/ 995116 484112 459596 52% /mnt/pmatou
946 lpmatou:/home 19222656 2787880 15458228 16% /mnt/pmatou/home
947 lpmatou:/usr 2478140 2038764 311260 87% /mnt/pmatou/usr
948 deuter:/ 4806936 97684 4465064 3% /mnt/deuter
949 deuter:/home 4806904 280100 4282620 7% /mnt/deuter/home
950 deuter:/files 44133352 27652876 14238608 67% /mnt/deuter/files
954 And we see that there are a number of separate filesystems (/ /boot
955 /home /rescue /tmp and /usr not to mention mounted systems).
956 If you specify only {\bf /} in your Include list, Bacula will only save the
957 Filesystem {\bf /dev/hda5}. To save all filesystems except {\bf /tmp} with
958 out including any of the Samba or NFS mounted systems, and explicitly
959 excluding a /tmp, /proc, .journal, and .autofsck, which you will not want to
960 be saved and restored, you can use the following:
965 Name = Include_example
970 wildfile = ".journal"
971 wildfile = ".autofsck"
984 Since /tmp is on its own filesystem and it was not explicitly named in the
985 Include list, it is not really needed in the exclude list. It is better to
986 list it in the Exclude list for clarity, and in case the disks are changed so
987 that it is no longer in its own partition.
989 Now, lets assume you only want to backup .Z and .gz files and nothing
990 else. This is a bit trickier because Bacula by default will select
991 everything to backup, so we must exclude everything but .Z and .gz files.
992 If we take the first example above and make the obvious modifications
993 to it, we might come up with a FileSet that looks like this:
999 Include { !!!!!!!!!!!!
1001 wildfile = "*.Z" example
1002 wildfile = "*.gz" doesn't
1011 The *.Z and *.gz files will indeed be backed up, but all other files
1012 that are not matched by the Options directives will automatically
1013 be backed up too (i.e. that is the default rule).
1015 To accomplish what we want, we must explicitly exclude all other files.
1016 We do this with the fillowing:
1037 The "trick" here was to add a RegexFile expression that matches
1038 all files. It does not match directory names, so all directories in
1039 /myfile will be backed up (the directory entry) and any *.Z and *.gz
1040 files contained in them. If you know that certain directories do
1041 not contain any *.Z or *.gz files and you do not want the directory
1042 entries backed up, you will need to explicitly exclude those directories.
1043 Backing up a directory entries is not very expensive.
1045 Bacula uses the system regex library and some of them are
1046 different on different OSes. The above has been reported not to work
1047 on FreeBSD. This can be tested by using the {\bf estimate job=job-name
1048 listing} command in the console and adapting the RegexFile expression
1049 appropriately. In a future version of Bacula, we will supply our own
1050 Regex code to avoid such system dependencies.
1052 Please be aware that allowing Bacula to traverse or change file systems can be
1053 {\bf very} dangerous. For example, with the following:
1058 Name = "Bad example"
1060 Options { onefs=no }
1067 you will be backing up an NFS mounted partition ({\bf /mnt/matou}), and since
1068 {\bf onefs} is set to {\bf no}, Bacula will traverse file systems. Now if {\bf
1069 /mnt/matou} has the current machine's file systems mounted, as is often the
1070 case, you will get yourself into a recursive loop and the backup will never
1073 As a final example, let's say that you have only one or two
1074 subdirectories of /home that you want to backup. For example,
1075 you want to backup only subdirectories beginning with the letter
1076 a and the letter b -- i.e. /home/a* and /home/b*. Now, you might first
1084 wilddir = "/home/a*"
1085 wilddir = "/home/b*"
1093 The problem is that the above will include everything in /home. To get
1094 things to work correctly, you need to start with the idea of exclusion
1095 instead of inclusion. So, you could simply exclude all directories
1096 except the two you want using:
1103 RegexDir = "^/home/[c-z]"
1112 And assuming that all subdirectories start with a lowercase letter, this
1115 An alternative would be to include the two subdirectories desired and
1116 exclude everything else:
1123 wilddir = "/home/a*"
1124 wilddir = "/home/b*"
1136 I haven't actually tried the above two examples, so you may need to
1137 tweak them to get them to work right.
1140 \subsubsection*{Backing up Raw Partitions}
1141 \index[general]{Backing up!Partitions }
1142 \index[general]{Backing up Raw Partitions }
1143 \addcontentsline{toc}{subsection}{Backing up Raw Partitions}
1145 The following FileSet definition will backup a raw partition:
1150 Name = "RawPartition"
1152 Options { sparse=yes }
1159 While backing up and restoring a raw partition, you should ensure that no
1160 other process including the system is writing to that partition. As a
1161 precaution, you are strongly urged to ensure that the raw partition is not
1162 mounted or is mounted read-only. If necessary, this can be done using the {\bf
1163 RunBeforeJob} directive.
1166 \subsubsection*{Excluding Files and Directories}
1167 \index[general]{Directories!Excluding Files and }
1168 \index[general]{Excluding Files and Directories }
1169 \addcontentsline{toc}{subsubsection}{Excluding Files and Directories}
1171 You may also include full filenames or directory names in addition to using
1172 wild-cards and {\bf Exclude=yes} in the Options resource as specified above by
1173 simply including the files to be excluded in an Exclude resource within the
1174 FileSet. For example:
1179 Name = Exclusion_example
1202 \subsubsection*{Windows FileSets}
1203 \index[general]{Windows FileSets }
1204 \index[general]{FileSets!Windows }
1205 \addcontentsline{toc}{subsection}{Windows FileSets}
1206 If you are entering Windows file names, the directory path may be preceded by
1207 the drive and a colon (as in c:). However, the path separators must be
1208 specified in Unix convention (i.e. forward slash (/)). If you wish to include
1209 a quote in a file name, precede the quote with a backslash
1210 (\textbackslash{}). For example you might use the following
1211 for a Windows machine to backup the "My Documents" directory:
1216 Name = "Windows Set"
1223 File = "c:/My Documents"
1229 For exclude lists to work correctly on Windows, you must observe the following
1233 \item Filenames are case sensitive, so you must use the correct case.
1234 \item To exclude a directory, you must not have a trailing slash on the
1236 \item If you have spaces in your filename, you must enclose the entire name
1237 in double-quote characters ("). Trying to use a backslash before the space
1239 \item If you are using the old Exclude syntax (noted below), you may not
1240 specify a drive letter in the exclude. The new syntax noted above should work
1241 fine including driver letters.
1244 Thanks to Thiago Lima for summarizing the above items for us. If you are
1245 having difficulties getting includes or excludes to work, you might want to
1246 try using the {\bf estimate job=xxx listing} command documented in the
1247 \ilink{Console chapter}{estimate} of this manual.
1249 On Win32 systems, if you move a directory or file or rename a file into the
1250 set of files being backed up, and a Full backup has already been made, Bacula
1251 will not know there are new files to be saved during an Incremental or
1252 Differential backup (blame Microsoft, not me). To avoid this problem, please
1253 {\bf copy} any new directory or files into the backup area. If you do not have
1254 enough disk to copy the directory or files, move them, but then initiate a
1258 \paragraph*{A Windows Example FileSet}
1259 \index[general]{FileSet!Windows Example }
1260 \index[general]{Windows Example FileSet }
1261 \addcontentsline{toc}{paragraph}{Windows Example FileSet}
1263 The following example was contributed by Russell Howe. Please note that
1264 for presentation purposes, the lines beginning with Data and Internet
1265 have been wrapped and should included on the previous line with one
1270 This is my Windows 2000 fileset:
1272 Name = "Windows 2000"
1278 # Exclude Mozilla-based programs' file caches
1279 WildDir = "[A-Z]:/Documents and Settings/*/Application
1280 Data/*/Profiles/*/*/Cache"
1281 WildDir = "[A-Z]:/Documents and Settings/*/Application
1282 Data/*/Profiles/*/*/Cache.Trash"
1283 WildDir = "[A-Z]:/Documents and Settings/*/Application
1284 Data/*/Profiles/*/*/ImapMail"
1286 # Exclude user's registry files - they're always in use anyway.
1287 WildFile = "[A-Z]:/Documents and Settings/*/Local Settings/Application
1288 Data/Microsoft/Windows/usrclass.*"
1289 WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
1291 # Exclude directories full of lots and lots of useless little files
1292 WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
1293 WildDir = "[A-Z]:/Documents and Settings/*/Recent"
1294 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
1295 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
1296 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary
1299 # These are always open and unable to be backed up
1300 WildFile = "[A-Z]:/Documents and Settings/All Users/Application
1301 Data/Microsoft/Network/Downloader/qmgr[01].dat"
1303 # Some random bits of Windows we want to ignore
1304 WildFile = "[A-Z]:/WINNT/security/logs/scepol.log"
1305 WildDir = "[A-Z]:/WINNT/system32/config"
1306 WildDir = "[A-Z]:/WINNT/msdownld.tmp"
1307 WildDir = "[A-Z]:/WINNT/Internet Logs"
1308 WildDir = "[A-Z]:/WINNT/$Nt*Uninstall*"
1309 WildDir = "[A-Z]:/WINNT/sysvol"
1310 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB"
1311 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB.LOG"
1312 WildFile = "[A-Z]:/WINNT/NTDS/edb.log"
1313 WildFile = "[A-Z]:/WINNT/NTDS/ntds.dit"
1314 WildFile = "[A-Z]:/WINNT/NTDS/temp.edb"
1315 WildFile = "[A-Z]:/WINNT/ntfrs/jet/log/edb.log"
1316 WildFile = "[A-Z]:/WINNT/ntfrs/jet/ntfrs.jdb"
1317 WildFile = "[A-Z]:/WINNT/ntfrs/jet/temp/tmp.edb"
1318 WildFile = "[A-Z]:/WINNT/system32/CPL.CFG"
1319 WildFile = "[A-Z]:/WINNT/system32/dhcp/dhcp.mdb"
1320 WildFile = "[A-Z]:/WINNT/system32/dhcp/j50.log"
1321 WildFile = "[A-Z]:/WINNT/system32/dhcp/tmp.edb"
1322 WildFile = "[A-Z]:/WINNT/system32/LServer/edb.log"
1323 WildFile = "[A-Z]:/WINNT/system32/LServer/TLSLic.edb"
1324 WildFile = "[A-Z]:/WINNT/system32/LServer/tmp.edb"
1325 WildFile = "[A-Z]:/WINNT/system32/wins/j50.log"
1326 WildFile = "[A-Z]:/WINNT/system32/wins/wins.mdb"
1327 WildFile = "[A-Z]:/WINNT/system32/wins/winstmp.mdb"
1329 # Temporary directories & files
1330 WildDir = "[A-Z]:/WINNT/Temp"
1331 WildDir = "[A-Z]:/temp"
1333 WildDir = "[A-Z]:/tmp"
1334 WildDir = "[A-Z]:/var/tmp"
1337 WildDir = "[A-Z]:/RECYCLER"
1340 WildFile = "[A-Z]:/pagefile.sys"
1342 # These are programs and are easier to reinstall than restore from
1344 WildDir = "[A-Z]:/cygwin"
1345 WildDir = "[A-Z]:/Program Files/Grisoft"
1346 WildDir = "[A-Z]:/Program Files/Java"
1347 WildDir = "[A-Z]:/Program Files/Java Web Start"
1348 WildDir = "[A-Z]:/Program Files/JavaSoft"
1349 WildDir = "[A-Z]:/Program Files/Microsoft Office"
1350 WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
1351 WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
1352 WildDir = "[A-Z]:/Program Files/mozilla.org"
1353 WildDir = "[A-Z]:/Program Files/OpenOffice*"
1356 # Our Win2k boxen all have C: and D: as the main hard drives.
1364 Note, the three line of the above Exclude were split to fit on the document
1365 page, they should be written on a single line in real use.
1367 \paragraph*{Windows NTFS Naming Considerations}
1368 \index[general]{Windows NTFS Naming Considerations }
1369 \index[general]{Considerations!Windows NTFS Naming }
1370 \addcontentsline{toc}{paragraph}{Windows NTFS Naming Considerations}
1372 NTFS filenames containing Unicode characters should now be supported
1373 as of version 1.37.30 or later.
1375 \subsubsection*{Testing Your FileSet}
1376 \index[general]{FileSet!Testing Your }
1377 \index[general]{Testing Your FileSet }
1378 \addcontentsline{toc}{subsection}{Testing Your FileSet}
1380 If you wish to get an idea of what your FileSet will really backup or if your
1381 exclusion rules will work correctly, you can test it by using the {\bf
1382 estimate} command in the Console program. See the
1383 \ilink{estimate command}{estimate} in the Console chapter of this