4 \subsection*{The FileSet Resource}
5 \label{FileSetResource}
6 \index[general]{Resource!FileSet }
7 \index[general]{FileSet Resource }
8 \addcontentsline{toc}{subsection}{FileSet Resource}
10 The FileSet resource defines what files are to be included or excluded in a
11 backup job. A {\bf FileSet} resource is required for each backup Job. It
12 consists of a list of files or directories to be included, a list of files
13 or directories to be excluded and the various backup options such as
14 compression, encryption, and signatures that are to be applied to each
17 Any change to the list of the included files will cause Bacula to
18 automatically create a new FileSet (defined by the name and an MD5 checksum
19 of the Include/Exclude contents). Each time a new FileSet is created,
20 Bacula will ensure that the next backup is always a Full save.
26 Start of the FileSet resource. One {\bf FileSet} resource must be
27 defined for each Backup job.
29 \item [Name = \lt{}name\gt{}]
31 The name of the FileSet resource. This directive is required.
33 \item [Ignore FileSet Changes = \lt{}yes|no\gt{}]
34 \index[dir]{Ignore FileSet Changes}
35 If this directive is set to {\bf yes}, any changes you make to the FileSet
36 Include or Exclude lists will be ignored and not cause Bacula to immediately
37 perform a Full backup. The default is {\bf no}, in which case, if you change
38 the Include or Exclude, Bacula will force a Full backup to ensure that
39 everything is properly backed up. It is not recommended to set this directive
40 to yes. This directive is available in Bacula version 1.35.4 or later.
42 \item [Enable VSS = \lt{}yes|no\gt{}]
43 \index[dir]{Enable VSS}
44 If this directive is set to {\bf yes} the File daemon will be notified
45 that the user wants to use a Volume Shadow Copy Service (VSS) backup
46 for this job. The default is {\bf no}. This directive is effective
47 only for VSS enabled Win32 File daemons. It permits a consistent copy
48 of open files to be made for cooperating writer applications, and for
49 applications that are not VSS away, Bacula can at least copy open files.
50 For more information, please see the
51 \ilink{Windows}{VSS} chapter of this manual.
53 \item [Include \{ Options \{\lt{}file-options\gt{}\} ...;
54 \lt{}file-list\gt{} \} ]
55 \index[dir]{Include \{ [ Options \{\lt{}file-options\gt{}\} ...]
56 \lt{}file-list\gt{} \} }
58 \item [Options \{ \lt{}file-options\gt{} \} ]
59 \index[dir]{Options \{ \lt{}file-options\gt{} \} }
61 \item [Exclude \{ \lt{}file-list\gt{} \}]
62 \index[dir]{Exclude \{ \lt{}file-list\gt{} \} }
66 The Include resource must contain a list of directories and/or files to be
67 processed in the backup job. Normally, all files found in all
68 subdirectories of any directory in the Include File list will be backed up.
69 Note, see below for the definition of \lt{}file-list\gt{}.
70 The Include resource may also contain one or more Options resources that
71 specify options such as compression to be applied to all or any subset of
72 the files found for backup.
74 There can be any number of {\bf Include} resources within the FileSet, each
75 having its own list of directories or files to be backed up and the backup
76 options defined by one or more Options resources. The {\bf file-list}
77 consists of one file or directory name per line. Directory names should be
78 specified without a trailing slash with Unix path notation.
80 You should always specify a full path for every directory and file that you
81 list in the FileSet. In addition, on Windows machines, you should {\bf
82 always} prefix the directory or filename with the drive specification in
83 lower case (e.g. {\bf c:/xxx}) using Unix directory name separators
86 Bacula's default for processing directories is to recursively descend in
87 the directory saving all files and subdirectories. Bacula will not by
88 default cross filesystems (or mount points in Unix parlance). This means
89 that if you specify the root partition (e.g. {\bf /}), Bacula will save
90 only the root partition and not any of the other mounted filesystems.
91 Similarly on Windows systems, you must explicitly specify each of the
92 drives you want saved (e.g.
93 {\bf c:/} and {\bf d:/} ...). In addition, at least for Windows systems, you
94 will most likely want to enclose each specification within double quotes
95 particularly if the directory (or file) name contains spaces. The {\bf df}
96 command on Unix systems will show you which mount points you must specify to
97 save everything. See below for an example.
99 Take special care not to include a directory twice or Bacula will backup
100 the same files two times wasting a lot of space on your archive device.
101 Including a directory twice is very easy to do. For example:
108 Options { compression=GZIP }
113 on a Unix system where /usr is a subdirectory (rather than a mounted
114 filesystem) will cause /usr to be backed up twice. In this case, on Bacula
115 versions prior to 1.32f-5-09Mar04 due to a bug, you will not be able to
116 restore hard linked files that were backed up twice.
118 If you have used Bacula prior to version 1.36.3, you will note three things in
119 the new FileSet syntax:
122 \item There is no equal sign (=) after the Include and before the opening
123 brace (\{). The same is true for the Exclude.
124 \item Each directory (or filename) to be included or excluded is preceded by a {\bf File
125 =}. Previously they were simply listed on separate lines.
126 \item The options that previously appeared on the Include line now must be
127 specified within their own Options resource.
128 \item The Exclude resource does not accept Options.
129 \item When using wild-cards or regular expressions, directory names are
130 always terminated with a slash (/) and filenames have no trailing slash.
133 The Options resource is optional, but when specified, it will contain a
134 list of {\bf keyword=value} options to be applied to the file-list.
135 See below for the definition of file-list.
136 Multiple Options resources may be specified one after another. As the
137 files are found in the specified directories, the Options will applied to
138 the filenames to determine if and how the file should be backed up. The
139 Options resources are applied in the order they are specified in the
140 FileSet until the first one that matches.
142 Once Bacula determines that the Options resource matches the file under
143 consideration, that file will be saved without looking at any other Options
144 resources that may be present. This means that any wild cards must appear
145 before an Options resource without wild cards.
147 If for some reason, Bacula applies all the Options resources to a file
148 under consideration for backup, but there are no matches (generally because
149 of wild cards that don't match), Bacula as a default will then backup the
150 file. This is quite logical if you consider the case of no Options, where
151 you want everything to be backed up.
153 However, one additional point is that
154 in the case that no match was found, Bacula will use the options found in
155 the last Options resource. As a consequence, if you want a particular set
156 of "default" options, you should put them in an Options resource after
159 It is a good idea to put all your wild-card and regex expressions inside
160 double quotes to prevent conf file scanning problems.
162 This is perhaps a bit overwhelming, so there are a number of examples included
163 below to illustrate how this works.
165 The directives within an Options resource may be one of the following:
169 \item [compression=GZIP]
170 \index[fd]{compression }
171 All files saved will be software compressed using the GNU ZIP compression
172 format. The compression is done on a file by file basis by the File daemon.
173 If there is a problem reading the tape in a single record of a file, it will
174 at most affect that file and none of the other files on the tape. Normally
175 this option is {\bf not} needed if you have a modern tape drive as the drive
176 will do its own compression. In fact, if you specify software compression at
177 the same time you have hardware compression turned on, your files may
178 actually take more space on the volume.
180 Software compression is very important if you are writing your Volumes to a
181 file, and it can also be helpful if you have a fast computer but a slow
182 network, otherwise it is generally better to rely your tape drive's hardware
183 compression. As noted above, it is not generally a good idea to do both software
184 and hardware compression.
186 Specifying {\bf GZIP} uses the default compression level six (i.e. {\bf GZIP}
187 is identical to {\bf GZIP6}). If you want a different compression level (1
188 through 9), you can specify it by appending the level number with no
189 intervening spaces to {\bf GZIP}. Thus {\bf compression=GZIP1} would give
190 minimum compression but the fastest algorithm, and {\bf compression=GZIP9}
191 would give the highest level of compression, but requires more computation.
192 According to the GZIP documentation, compression levels greater than 6
193 generally give very little extra compression and are rather CPU intensive.
195 \item [signature=SHA1]
196 \index[fd]{signature }
197 An SHA1 signature will be computed for all The SHA1 algorithm is purported to
198 be some what slower than the MD5 algorithm, but at the same time is
199 significantly better from a cryptographic point of view (i.e. much fewer
200 collisions, much lower probability of being hacked.) It adds four more bytes
201 than the MD5 signature. We strongly recommend that either this option or MD5
202 be specified as a default for all files. Note, only one of the two options
203 MD5 or SHA1 can be computed for any file.
205 \item [signature=MD5]
206 \index[fd]{signature }
207 An MD5 signature will be computed for all files saved. Adding this option
208 generates about 5\% extra overhead for each file saved. In addition to the
209 additional CPU time, the MD5 signature adds 16 more bytes per file to your
210 catalog. We strongly recommend that this option or the SHA1 option be
211 specified as a default for all files.
213 \item [verify=\lt{}options\gt{}]
215 The options letters specified are used when running a {\bf Verify
216 Level=Catalog} as well as the {\bf DiskToCatalog} level job. The options
217 letters may be any combination of the following:
225 compare the permission bits
228 compare the number of links
240 compare the access time
243 compare the modification time (st\_mtime)
246 compare the change time (st\_ctime)
249 report file size decreases
252 compare the MD5 signature
255 compare the SHA1 signature
258 A useful set of general options on the {\bf Level=Catalog} or {\bf
259 Level=DiskToCatalog} verify is {\bf pins5} i.e. compare permission bits,
260 inodes, number of links, size, and MD5 changes.
264 If set to {\bf yes} (the default), {\bf Bacula} will remain on a single file
265 system. That is it will not backup file systems that are mounted on a
266 subdirectory. If you are using a *nix system, you may not even be aware
267 that there are several different filesystems as they are often
268 automatically mounted by the OS (e.g. /dev, /net, /sys, /proc, ...).
269 With Bacula 1.38.0 or later, it will inform you when it decides not
270 to traverse into another filesystem. This can be very useful if you
271 forgot to backup a particular partition. An example of the
272 informational message in the job report is:
276 rufus-fd: Filesystem change prohibited. Will not descend into /misc
277 rufus-fd: Filesystem change prohibited. Will not descend into /net
278 rufus-fd: Filesystem change prohibited. Will not descend into /var/lib/nfs/rpc_pipefs
279 rufus-fd: Filesystem change prohibited. Will not descend into /selinux
280 rufus-fd: Filesystem change prohibited. Will not descend into /sys
281 rufus-fd: Filesystem change prohibited. Will not descend into /dev
282 rufus-fd: Filesystem change prohibited. Will not descend into /home
285 If you wish to backup multiple filesystems, you can explicitly
286 list each filesystem you want saved. Otherwise, if you set the onefs option
287 to {\bf no}, Bacula will backup all mounted file systems (i.e. traverse mount
288 points) that are found within the {\bf FileSet}. Thus if you have NFS or
289 Samba file systems mounted on a directory listed in your FileSet, they will
290 also be backed up. Normally, it is preferable to set {\bf onefs=yes} and to
291 explicitly name each filesystem you want backed up. Explicitly naming the
292 filesystems you want backed up avoids the possibility of getting into a
293 infinite loop recursing filesystems. Another possiblity is to
294 use {\bf onefs=no} and to set {\bs fstype=ext2, ...}.
295 See the example below for more details.
297 If you think that Bacula should be backing up a particular directory
298 and it is not, and you have {\bf onefs=no} set, before you complain,
308 where you replace {\bf filesystem} with the one in question. If the
309 {\bf Device:} number is different for / and for your filesystem, then they
310 are on different filesystems. E.g.
315 Size: 4096 Blocks: 16 IO Block: 4096 directory
316 Device: 302h/770d Inode: 2 Links: 26
317 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
318 Access: 2005-11-10 12:28:01.000000000 +0100
319 Modify: 2005-09-27 17:52:32.000000000 +0200
320 Change: 2005-09-27 17:52:32.000000000 +0200
324 Size: 4096 Blocks: 16 IO Block: 4096 directory
325 Device: 308h/776d Inode: 2 Links: 7
326 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
327 Access: 2005-11-10 12:28:02.000000000 +0100
328 Modify: 2005-11-06 12:36:48.000000000 +0100
329 Change: 2005-11-06 12:36:48.000000000 +0100
333 Also be aware that even if you include {\bf /home} in your list
334 of files to backup, as you most likely should, you will get the
335 informational message about Filesystem change prohibited when Bacula is
336 processing the {\bf /} directory.
340 \item [portable=yes|no]
341 \index[dir]{portable }
342 If set to {\bf yes} (default is {\bf no}), the Bacula File daemon will backup
343 Win32 files in a portable format, but not all Win32 file attributes will be
344 saved and restored. By default, this option is set to {\bf no}, which means
345 that on Win32 systems, the data will be backed up using Windows API calls and
346 on WinNT/2K/XP, all the security and ownership attributes will be properly
347 backed up (and restored). However this format is not portable to other
348 systems -- e.g. Unix, Win95/98/Me. When backing up Unix systems, this option
349 is ignored, and unless you have a specific need to have portable backups, we
350 recommend accept the default ({\bf no}) so that the maximum information
351 concerning your files is saved.
353 \item [recurse=yes|no]
355 If set to {\bf yes} (the default), Bacula will recurse (or descend) into all
356 subdirectories found unless the directory is explicitly excluded using an
357 {\bf exclude} definition. If you set {\bf recurse=no}, Bacula will save the
358 subdirectory entries, but not descend into the subdirectories, and thus will
359 not save the files or directories contained in the subdirectories. Normally,
360 you will want the default ({\bf yes}).
362 \item [sparse=yes|no]
364 Enable special code that checks for sparse files such as created by
365 ndbm. The default is {\bf no}, so no checks are made for sparse files.
366 You may specify {\bf sparse=yes} even on files that are not sparse file.
367 No harm will be done, but there will be a small additional overhead to
368 check for buffers of all zero, and a small additional amount of space on
369 the output archive will be used to save the seek address of each
370 non-zero record read.
372 {\bf Restrictions:} Bacula reads files in 32K buffers. If the whole
373 buffer is zero, it will be treated as a sparse block and not written to
374 tape. However, if any part of the buffer is non-zero, the whole buffer
375 will be written to tape, possibly including some disk sectors (generally
376 4098 bytes) that are all zero. As a consequence, Bacula's detection of
377 sparse blocks is in 32K increments rather than the system block size.
378 If anyone considers this to be a real problem, please send in a request
379 for change with the reason.
381 If you are not familiar with sparse files, an example is say a file
382 where you wrote 512 bytes at address zero, then 512 bytes at address 1
383 million. The operating system will allocate only two blocks, and the
384 empty space or hole will have nothing allocated. However, when you read
385 the sparse file and read the addresses where nothing was written, the OS
386 will return all zeros as if the space were allocated, and if you backup
387 such a file, a lot of space will be used to write zeros to the volume.
388 Worse yet, when you restore the file, all the previously empty space
389 will now be allocated using much more disk space. By turning on the
390 {\bf sparse} option, Bacula will specifically look for empty space in
391 the file, and any empty space will not be written to the Volume, nor
392 will it be restored. The price to pay for this is that Bacula must
393 search each block it reads before writing it. On a slow system, this
394 may be important. If you suspect you have sparse files, you should
395 benchmark the difference or set sparse for only those files that are
399 \item [readfifo=yes|no]
400 \index[fd]{readfifo }
401 If enabled, tells the Client to read the data on a backup and write the
402 data on a restore to any FIFO (pipe) that is explicitly mentioned in the
403 FileSet. In this case, you must have a program already running that
404 writes into the FIFO for a backup or reads from the FIFO on a restore.
405 This can be accomplished with the {\bf RunBeforeJob} directive. If this
406 is not the case, Bacula will hang indefinitely on reading/writing the
407 FIFO. When this is not enabled (default), the Client simply saves the
408 directory entry for the FIFO.
410 Unfortunately, when Bacula runs a RunBeforeJob, it waits until that
411 script terminates, and if the script accesses the FIFO to write
412 into the it, the Bacula job will block and everything will stall.
413 However, Vladimir Stavrinov as supplied tip that allows this feature
414 to work correctly. He simply adds the following to the beginning
415 of the RunBeforeJob script:
422 \item [mtimeonly=yes|no]
423 \index[dir]{mtimeonly }
424 If enabled, tells the Client that the selection of files during
425 Incremental and Differential backups should based only on the st\_mtime
426 value in the stat() packet. The default is {\bf no} which means that
427 the selection of files to be backed up will be based on both the
428 st\_mtime and the st\_ctime values. In general, it is not recommended
431 \item [keepatime=yes|no]
432 \index[dir]{keepatime }
433 The default is {\bf no}. When enabled, Bacula will reset the st\_atime
434 (access time) field of files that it backs up to their value prior to
435 the backup. This option is not generally recommended as there are very
436 few programs that use st\_atime, and the backup overhead is increased
437 because of the additional system call necessary to reset the times.
438 However, for some files, such as mailboxes, when Bacula backs up the
439 file, the user will notice that someone (Bacula) has accessed the
440 file. In this, case keepatime can be useful.
441 (I'm not sure this works on Win32).
443 Note, if you use this feature, when Bacula resets the access time, the
444 change time (st\_ctime) will automatically be modified by the system,
445 so on the next incremental job, the file will be backed up even if
446 it has not changed. As a consequence, you will probably also want
447 to use {\bf mtimeonly = yes} as well as keepatime (thanks to
448 Rudolf Cejka for this tip).
450 \item [hardlinks=yes|no]
451 \index[dir]{hardlinks}
452 When enabled (default), this directive will cause hard inks to be
453 backed up. However, the File daemon keeps track of hard linked files and
454 will backup the data only once. The process of keeping track of the
455 hard links can be quite expensive if you have lots of them (tens of
456 thousands or more). This doesn't occur on normal Unix systems, but if
457 you use a program like BackupPC, it can create hundreds of thousands, or
458 even millions of hard links. Backups become very long and the File daemon
459 will consume a lot of CPU power checking hard links. In such a case,
460 set {\bf hardlinks=no} and hard links will not be backed up. Note, using
461 this option will most likely backup more data and on a restore the file
462 system will not be restored identically to the original.
464 \item [wild=\lt{}string\gt{}]
466 Specifies a wild-card string to be applied to the filenames and
467 directory names. Note, if {\bf Exclude} is not enabled, the wild-card
468 will select which files are to be included. If {\bf Exclude=yes} is
469 specified, the wild-card will select which files are to be excluded.
470 Multiple wild-card directives may be specified, and they will be applied
471 in turn until the first one that matches. Note, if you exclude a
472 directory, no files or directories below it will be matched.
473 It is recommended to enclose the string in double quotes.
475 \item [wildfile=\lt{}string\gt{}]
476 \index[dir]{wildfile }
477 Specifies a wild-card string to be applied to filenames only. No
478 directories will be matched by this directive. Note, if {\bf Exclude}
479 is not enabled, the wild-card will select which files are to be
480 included. If {\bf Exclude=yes} is specified, the wild-card will select
481 which files are to be excluded. Multiple wild-card directives may be
482 specified, and they will be applied in turn until the first one that
484 It is recommended to enclose the string in double quotes.
486 \item [wilddir=\lt{}string\gt{}]
487 \index[dir]{wilddir }
488 Specifies a wild-card string to be applied to directory names only. No
489 filenames will be matched by this directive. Note, if {\bf Exclude} is
490 not enabled, the wild-card will select directories files are to be
491 included. If {\bf Exclude=yes} is specified, the wild-card will select
492 which files are to be excluded. Multiple wild-card directives may be
493 specified, and they will be applied in turn until the first one that
494 matches. Note, if you exclude a directory, no files or directories
495 below it will be matched.
496 It is recommended to enclose the string in double quotes.
499 \item [regex=\lt{}string\gt{}]
501 Specifies a POSIX extended regular expression to be applied to the
502 filenames and directory names.
503 This directive is available in version 1.35 and later. If {\bf
504 Exclude} is not enabled, the regex will select which files are to be
505 included. If {\bf Exclude=yes} is specified, the regex will select
506 which files are to be excluded. Multiple regex directives may be
507 specified within an Options resource, and they will be applied in turn
508 until the first one that matches. Note, if you exclude a
509 directory, no files or directories below it will be matched.
510 It is recommended to enclose the string in double quotes.
512 \item [regexfile=\lt{}string\gt{}]
513 \index[dir]{regexfile }
514 Specifies a POSIX extended regular expression to be applied to filenames
515 only. No directories will be matched by this directive. Note, if {\bf
516 Exclude} is not enabled, the regex will select which files are to be
517 included. If {\bf Exclude=yes} is specified, the regex will select
518 which files are to be excluded. Multiple regex directives may be
519 specified, and they will be applied in turn until the first one that
521 It is recommended to enclose the string in double quotes.
523 \item [regexdir=\lt{}string\gt{}]
524 \index[dir]{regexdir }
525 Specifies a POSIX extended regular expression to be applied to directory
526 names only. No filenames will be matched by this directive. Note, if
527 {\bf Exclude} is not enabled, the regex will select directories
528 files are to be included. If {\bf Exclude=yes} is specified, the
529 regex will select which files are to be excluded. Multiple
530 regex directives may be specified, and they will be applied in turn
531 until the first one that matches. Note, if you exclude a directory, no
532 files or directories below it will be matched.
533 It is recommended to enclose the string in double quotes.
535 \item [exclude=yes|no]
536 \index[dir]{exclude }
537 The default is {\bf no}. When enabled, any files matched within the Options
538 will be excluded from the backup.
542 \item [aclsupport=yes|no]
543 \index[dir]{aclsupport }
544 The default is {\bf no}. If this option is set to yes, and you have the
545 POSIX {\bf libacl} installed on your system, Bacula will backup the file
546 and directory UNIX Access Control Lists (ACL) as defined in IEEE Std
547 1003.1e draft 17 and "POSIX.1e" (abandoned). This feature is
548 available on UNIX only and depends on the ACL library. Bacula is
549 automatically compiled with ACL support if the {\bf libacl} library is
550 installed on your system (shown in config.out). While restoring the
551 files Bacula will try to restore the ACLs, if there is no ACL support
552 available on the system, Bacula restores the files and directories but
553 not the ACL information. Please note, if you backup an EXT3 or XFS
554 filesystem with ACLs, then you restore them to a different filesystem
555 (perhaps reiserfs) that does not have ACLs, the ACLs will be ignored.
557 \item [ignore case=yes|no]
558 \index[dir]{ignore case }
559 The default is {\bf no}, except on Windows systems where the default
560 is {\bf yes}. When this directive is set to {\bf yes} all the case
561 of character will be ignored in wild-card and regex comparisons.
562 That is an uppercase A will match a lowercase a.
564 \item [fstype=filesystem-type]
566 This option allows you to select files and directories by the
567 filesystem type. The permitted filesystem-type names are:
569 ext2, jfs, ntfs, proc, reiserfs, xfs, usbdevfs, sysfs, smbfs,
570 iso9660. For ext3 systems, use ext2.
572 You may have multiple Fstype directives, and thus permit matching
573 of multiple filesystem types within a single Options resource. If
574 the type specified on the fstype directive does not match the
575 filesystem for a particular directive, that directory will not be
576 backed up. This directive can be used to prevent backing up
577 non-local filesystems. Normally, when you use this directive, you
578 would also set {\bf onefs=no} so that Bacula will traverse filesystems.
580 This option is not implemented in Win32 systems.
583 \item [hfsplussupport=yes|no]
584 \index[dir]{hfsplussupport }
585 This option allows you to turn on support for Mac OSX HFS plus
590 {\bf \lt{}file-list\gt{}} is a list of directory and/or filename names
591 specified with a {\bf File =} directive. To include names containing spaces,
592 enclose the name between double-quotes.
594 There are a number of special cases when specifying directories and files in a
595 {\bf file-list}. They are:
598 \item Any name preceded by an at-sign (@) is assumed to be the name of a
599 file, which contains a list of files each preceded by a "File =". The
600 named file is read once when the configuration file is parsed during the
601 Director startup. Note, that the file is read on the Director's machine
602 and not on the Client's. In fact, the @filename can appear anywhere
603 within the conf file where a token would be read, and the contents of
604 the named file will be logically inserted in the place of the @filename.
605 What must be in the file depends on the location the @filename is
606 specified in the conf file. For example:
611 Options { compression=GZIP }
612 @/home/files/my-files
617 \item Any name beginning with a vertical bar (|) is assumed to be the name of
618 a program. This program will be executed on the Director's machine at
619 the time the Job starts (not when the Director reads the configuration
620 file), and any output from that program will be assumed to be a list of
621 files or directories, one per line, to be included.
623 This allows you to have a job that, for example, includes all the local
624 partitions even if you change the partitioning by adding a disk. The
625 examples below show you how to do this. However, please note two
627 1. if you want the local filesystems, you probably should be
628 using the new {\bf fstype} directive, which was added in version 1.36.3
629 and set {\bf onefs=no}.
632 2. the exact syntax of the command needed in the examples below is very
633 system dependent. For example, on recent Linux systems, you may need to
634 add the -P option, on FreeBSD systems, the options will be different as
637 In general, you will need to prefix your command or commands with a {\bf
638 sh -c} so that they are invoked by a shell. This will not be the case
639 if you are invoking a script as in the second example below. Also, you
640 must take care to escape (precede with a \textbackslash{}) wild-cards,
641 shell character, and to ensure that any spaces in your command are
642 escaped as well. If you use a single quotes (') within a double quote
643 ("), Bacula will treat everything between the single quotes as one field
644 so it will not be necessary to escape the spaces. In general, getting
645 all the quotes and escapes correct is a real pain as you can see by the
646 next example. As a consequence, it is often easier to put everything in
647 a file and simply use the file name within Bacula. In that case the
648 {\bf sh -c} will not be necessary providing the first line of the file
657 Options { signature = SHA1 }
658 File = "|sh -c 'df -l | grep \"^/dev/hd[ab]\" | grep -v \".*/tmp\" \
659 | awk \"{print \\$6}\"'"
664 will produce a list of all the local partitions on a RedHat Linux system.
665 Note, the above line was split, but should normally be written on one line.
666 Quoting is a real problem because you must quote for Bacula which consists of
667 preceding every \textbackslash{} and every " with a \textbackslash{}, and
668 you must also quote for the shell command. In the end, it is probably easier
669 just to execute a small file with:
678 File = "|my_partitions"
683 where my\_partitions has:
688 df -l | grep "^/dev/hd[ab]" | grep -v ".*/tmp" \
693 If the vertical bar (|) in front of my\_partitions is preceded by a
694 backslash as in \textbackslash{}|, the program will be executed on the
695 Client's machine instead of on the Director's machine.
696 Please note that if the filename is given within quotes, you
697 will need to use two slashes. An example, provided by John Donagher,
698 that backs up all the local UFS partitions on a remote system is:
703 Name = "All local partitions"
705 Options { signature=SHA1; onefs=yes; }
706 File = "\\|bash -c \"df -klF ufs | tail +2 | awk '{print \$6}'\""
712 The above requires two backslash characters after the double quote (one
713 preserves the next one). If you are a Linux user, just change the {\bf ufs}
714 to {\bf ext3} (or your preferred filesystem type), and you will be in
717 If you know what filesystems you have mounted on your system, e.g.
718 for RedHat Linux normally only ext2 and ext3, you can backup
719 all local fileystems using something like:
725 Options { signature = SHA1; onfs=no; fstype=ext2 }
732 \item Any file-list item preceded by a less-than sign (\lt{}) will be taken
733 to be a file. This file will be read on the Director's machine at the time
734 the Job starts, and the data will be assumed to be a list of directories or
735 files, one per line, to be included. The names should start in column 1 and
736 should not be quoted even if they contain spaces. This feature allows you to
737 modify the external file and change what will be saved without stopping and
738 restarting Bacula as would be necessary if using the @ modifier noted above.
744 Options { signature = SHA1 }
745 File = "</home/files/local-filelist"
750 If you precede the less-than sign (\lt{}) with a backslash as in
751 \textbackslash{}\lt{}, the file-list will be read on the Client machine
752 instead of on the Director's machine. Please note that if the filename
753 is given within quotes, you will need to use two slashes.
758 Options { signature = SHA1 }
759 File = "\\</home/xxx/filelist-on-client"
764 \item If you explicitly specify a block device such as {\bf /dev/hda1}, then
765 Bacula (starting with version 1.28) will assume that this is a raw partition
766 to be backed up. In this case, you are strongly urged to specify a {\bf
767 sparse=yes} include option, otherwise, you will save the whole partition
768 rather than just the actual data that the partition contains. For example:
773 Options { signature=MD5; sparse=yes }
779 will backup the data in device /dev/hd6.
781 Ludovic Strappazon has pointed out that this feature can be used to backup a
782 full Microsoft Windows disk. Simply boot into the system using a Linux Rescue
783 disk, then load a statically linked Bacula as described in the
784 \ilink{ Disaster Recovery Using Bacula}{_ChapterStart38} chapter of
785 this manual. Then save the whole disk partition. In the case of a disaster,
786 you can then restore the desired partition by again booting with the rescue
787 disk and doing a restore of the partition.
788 \item If you explicitly specify a FIFO device name (created with mkfifo), and
789 you add the option {\bf readfifo=yes} as an option, Bacula will read the FIFO
790 and back its data up to the Volume. For example:
799 File = /home/abc/fifo
804 if {\bf /home/abc/fifo} is a fifo device, Bacula will open the fifo,
805 read it, and store all data thus obtained on the Volume. Please note,
806 you must have a process on the system that is writing into the fifo, or
807 Bacula will hang, and after one minute of waiting, Bacula will give up
808 and go on to the next file. The data read can be anything since Bacula
809 treats it as a stream.
811 This feature can be an excellent way to do a "hot" backup of a very
812 large database. You can use the {\bf RunBeforeJob} to create the fifo
813 and to start a program that dynamically reads your database and writes
814 it to the fifo. Bacula will then write it to the Volume. Be sure to
815 read the \ilink{readfifo section}{readfifo} that gives a
816 tip to ensure that the RunBeforeJob does not block Bacula.
818 During the restore operation, the inverse is true, after Bacula creates
819 the fifo if there was any data stored with it (no need to explicitly
820 list it or add any options), that data will be written back to the fifo.
821 As a consequence, if any such FIFOs exist in the fileset to be restored,
822 you must ensure that there is a reader program or Bacula will block, and
823 after one minute, Bacula will time out the write to the fifo and move on
827 \subsubsection*{FileSet Examples}
828 \index[general]{Examples!FileSet }
829 \index[general]{FileSet Examples}
830 \addcontentsline{toc}{subsection}{FileSet Examples}
832 The following is an example of a valid FileSet resource definition. Note,
833 the first Include pulls in the contents of the file {\bf /etc/backup.list}
834 when Bacula is started (i.e. the @), and that file must have each filename
835 to be backed up preceded by a {\bf File =} and on a separate line.
856 File = /usr/lib/another_file
862 In the above example, all the files contained in /etc/backup.list will
863 be compressed with GZIP compression, an SHA1 signature will be computed on the
864 file's contents (its data), and sparse file handling will apply.
866 The two directories /root/myfile and /usr/lib/another\_file will also be saved
867 without any options, but all files in those directories with the extensions
868 {\bf .o} and {\bf .exe} will be excluded.
870 Let's say that you now want to exclude the directory /tmp. The simplest way
871 to do so is to add an exclude directive that lists /tmp. The example
872 above would then become:
893 File = /usr/lib/another_file
903 You can add wild-cards to the File directives listed in the Exclude
904 directory, but you need to take care because if you exclude a directory,
905 it and all files and directories below it will also be excluded.
907 Now lets take a slight variation on the above and suppose
908 you want to save all your whole filesystem except {\bf /tmp}.
909 The problem that comes up is that Bacula will not normally
910 cross from one filesystem to another.
911 Doing a {\bf df} command, you get the following output:
916 Filesystem 1k-blocks Used Available Use% Mounted on
917 /dev/hda5 5044156 439232 4348692 10% /
918 /dev/hda1 62193 4935 54047 9% /boot
919 /dev/hda9 20161172 5524660 13612372 29% /home
920 /dev/hda2 62217 6843 52161 12% /rescue
921 /dev/hda8 5044156 42548 4745376 1% /tmp
922 /dev/hda6 5044156 2613132 2174792 55% /usr
923 none 127708 0 127708 0% /dev/shm
924 //minimatou/c$ 14099200 9895424 4203776 71% /mnt/mmatou
925 lmatou:/ 1554264 215884 1258056 15% /mnt/matou
926 lmatou:/home 2478140 1589952 760072 68% /mnt/matou/home
927 lmatou:/usr 1981000 1199960 678628 64% /mnt/matou/usr
928 lpmatou:/ 995116 484112 459596 52% /mnt/pmatou
929 lpmatou:/home 19222656 2787880 15458228 16% /mnt/pmatou/home
930 lpmatou:/usr 2478140 2038764 311260 87% /mnt/pmatou/usr
931 deuter:/ 4806936 97684 4465064 3% /mnt/deuter
932 deuter:/home 4806904 280100 4282620 7% /mnt/deuter/home
933 deuter:/files 44133352 27652876 14238608 67% /mnt/deuter/files
937 And we see that there are a number of separate filesystems (/ /boot
938 /home /rescue /tmp and /usr not to mention mounted systems).
939 If you specify only {\bf /} in your Include list, Bacula will only save the
940 Filesystem {\bf /dev/hda5}. To save all filesystems except {\bf /tmp} with
941 out including any of the Samba or NFS mounted systems, and explicitly
942 excluding a /tmp, /proc, .journal, and .autofsck, which you will not want to
943 be saved and restored, you can use the following:
948 Name = Include_example
953 wildfile = ".journal"
954 wildfile = ".autofsck"
967 Since /tmp is on its own filesystem and it was not explicitly named in the
968 Include list, it is not really needed in the exclude list. It is better to
969 list it in the Exclude list for clarity, and in case the disks are changed so
970 that it is no longer in its own partition.
972 Now, lets assume you only want to backup .Z and .gz files and nothing
973 else. This is a bit trickier because Bacula by default will select
974 everything to backup, so we must exclude everything but .Z and .gz files.
975 If we take the first example above and make the obvious modifications
976 to it, we might come up with a FileSet that looks like this:
982 Include { !!!!!!!!!!!!
984 wildfile = "*.Z" example
985 wildfile = "*.gz" doesn't
994 The *.Z and *.gz files will indeed be backed up, but all other files
995 that are not matched by the Options directives will automatically
996 be backed up too (i.e. that is the default rule).
998 To accomplish what we want, we must explicitly exclude all other files.
999 We do this with the fillowing:
1020 The "trick" here was to add a RegexFile expression that matches
1021 all files. It does not match directory names, so all directories in
1022 /myfile will be backed up (the directory entry) and any *.Z and *.gz
1023 files contained in them. If you know that certain directories do
1024 not contain any *.Z or *.gz files and you do not want the directory
1025 entries backed up, you will need to explicitly exclude those directories.
1026 Backing up a directory entries is not very expensive.
1028 Bacula uses the system regex library and some of them are
1029 different on different OSes. The above has been reported not to work
1030 on FreeBSD. This can be tested by using the {\bf estimate job=job-name
1031 listing} command in the console and adapting the RegexFile expression
1032 appropriately. In a future version of Bacula, we will supply our own
1033 Regex code to avoid such system dependencies.
1035 Please be aware that allowing Bacula to traverse or change file systems can be
1036 {\bf very} dangerous. For example, with the following:
1041 Name = "Bad example"
1043 Options { onefs=no }
1050 you will be backing up an NFS mounted partition ({\bf /mnt/matou}), and since
1051 {\bf onefs} is set to {\bf no}, Bacula will traverse file systems. Now if {\bf
1052 /mnt/matou} has the current machine's file systems mounted, as is often the
1053 case, you will get yourself into a recursive loop and the backup will never
1056 \subsubsection*{Backing up Raw Partitions}
1057 \index[general]{Backing up!Partitions }
1058 \index[general]{Backing up Raw Partitions }
1059 \addcontentsline{toc}{subsection}{Backing up Raw Partitions}
1061 The following FileSet definition will backup a raw partition:
1066 Name = "RawPartition"
1068 Options { sparse=yes }
1075 While backing up and restoring a raw partition, you should ensure that no
1076 other process including the system is writing to that partition. As a
1077 precaution, you are strongly urged to ensure that the raw partition is not
1078 mounted or is mounted read-only. If necessary, this can be done using the {\bf
1079 RunBeforeJob} directive.
1082 \subsubsection*{Excluding Files and Directories}
1083 \index[general]{Directories!Excluding Files and }
1084 \index[general]{Excluding Files and Directories }
1085 \addcontentsline{toc}{subsubsection}{Excluding Files and Directories}
1087 You may also include full filenames or directory names in addition to using
1088 wild-cards and {\bf Exclude=yes} in the Options resource as specified above by
1089 simply including the files to be excluded in an Exclude resource within the
1090 FileSet. For example:
1095 Name = Exclusion_example
1118 \subsubsection*{Windows FileSets}
1119 \index[general]{Windows FileSets }
1120 \index[general]{FileSets!Windows }
1121 \addcontentsline{toc}{subsection}{Windows FileSets}
1122 If you are entering Windows file names, the directory path may be preceded by
1123 the drive and a colon (as in c:). However, the path separators must be
1124 specified in Unix convention (i.e. forward slash (/)). If you wish to include
1125 a quote in a file name, precede the quote with a backslash
1126 (\textbackslash{}). For example you might use the following
1127 for a Windows machine to backup the "My Documents" directory:
1132 Name = "Windows Set"
1139 File = "c:/My Documents"
1145 For exclude lists to work correctly on Windows, you must observe the following
1149 \item Filenames are case sensitive, so you must use the correct case.
1150 \item To exclude a directory, you must not have a trailing slash on the
1152 \item If you have spaces in your filename, you must enclose the entire name
1153 in double-quote characters ("). Trying to use a backslash before the space
1155 \item If you are using the old Exclude syntax (noted below), you may not
1156 specify a drive letter in the exclude. The new syntax noted above should work
1157 fine including driver letters.
1160 Thanks to Thiago Lima for summarizing the above items for us. If you are
1161 having difficulties getting includes or excludes to work, you might want to
1162 try using the {\bf estimate job=xxx listing} command documented in the
1163 \ilink{Console chapter}{estimate} of this manual.
1165 On Win32 systems, if you move a directory or file or rename a file into the
1166 set of files being backed up, and a Full backup has already been made, Bacula
1167 will not know there are new files to be saved during an Incremental or
1168 Differential backup (blame Microsoft, not me). To avoid this problem, please
1169 {\bf copy} any new directory or files into the backup area. If you do not have
1170 enough disk to copy the directory or files, move them, but then initiate a
1174 \paragraph*{A Windows Example FileSet}
1175 \index[general]{FileSet!Windows Example }
1176 \index[general]{Windows Example FileSet }
1177 \addcontentsline{toc}{paragraph}{Windows Example FileSet}
1179 The following example was contributed by Russell Howe. Please note that
1180 for presentation purposes, the lines beginning with Data and Internet
1181 have been wrapped and should included on the previous line with one
1186 This is my Windows 2000 fileset:
1188 Name = "Windows 2000"
1194 # Exclude Mozilla-based programs' file caches
1195 WildDir = "[A-Z]:/Documents and Settings/*/Application
1196 Data/*/Profiles/*/*/Cache"
1197 WildDir = "[A-Z]:/Documents and Settings/*/Application
1198 Data/*/Profiles/*/*/Cache.Trash"
1199 WildDir = "[A-Z]:/Documents and Settings/*/Application
1200 Data/*/Profiles/*/*/ImapMail"
1202 # Exclude user's registry files - they're always in use anyway.
1203 WildFile = "[A-Z]:/Documents and Settings/*/Local Settings/Application
1204 Data/Microsoft/Windows/usrclass.*"
1205 WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
1207 # Exclude directories full of lots and lots of useless little files
1208 WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
1209 WildDir = "[A-Z]:/Documents and Settings/*/Recent"
1210 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
1211 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
1212 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary
1215 # These are always open and unable to be backed up
1216 WildFile = "[A-Z]:/Documents and Settings/All Users/Application
1217 Data/Microsoft/Network/Downloader/qmgr[01].dat"
1219 # Some random bits of Windows we want to ignore
1220 WildFile = "[A-Z]:/WINNT/security/logs/scepol.log"
1221 WildDir = "[A-Z]:/WINNT/system32/config"
1222 WildDir = "[A-Z]:/WINNT/msdownld.tmp"
1223 WildDir = "[A-Z]:/WINNT/Internet Logs"
1224 WildDir = "[A-Z]:/WINNT/$Nt*Uninstall*"
1225 WildDir = "[A-Z]:/WINNT/sysvol"
1226 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB"
1227 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB.LOG"
1228 WildFile = "[A-Z]:/WINNT/NTDS/edb.log"
1229 WildFile = "[A-Z]:/WINNT/NTDS/ntds.dit"
1230 WildFile = "[A-Z]:/WINNT/NTDS/temp.edb"
1231 WildFile = "[A-Z]:/WINNT/ntfrs/jet/log/edb.log"
1232 WildFile = "[A-Z]:/WINNT/ntfrs/jet/ntfrs.jdb"
1233 WildFile = "[A-Z]:/WINNT/ntfrs/jet/temp/tmp.edb"
1234 WildFile = "[A-Z]:/WINNT/system32/CPL.CFG"
1235 WildFile = "[A-Z]:/WINNT/system32/dhcp/dhcp.mdb"
1236 WildFile = "[A-Z]:/WINNT/system32/dhcp/j50.log"
1237 WildFile = "[A-Z]:/WINNT/system32/dhcp/tmp.edb"
1238 WildFile = "[A-Z]:/WINNT/system32/LServer/edb.log"
1239 WildFile = "[A-Z]:/WINNT/system32/LServer/TLSLic.edb"
1240 WildFile = "[A-Z]:/WINNT/system32/LServer/tmp.edb"
1241 WildFile = "[A-Z]:/WINNT/system32/wins/j50.log"
1242 WildFile = "[A-Z]:/WINNT/system32/wins/wins.mdb"
1243 WildFile = "[A-Z]:/WINNT/system32/wins/winstmp.mdb"
1245 # Temporary directories & files
1246 WildDir = "[A-Z]:/WINNT/Temp"
1247 WildDir = "[A-Z]:/temp"
1249 WildDir = "[A-Z]:/tmp"
1250 WildDir = "[A-Z]:/var/tmp"
1253 WildDir = "[A-Z]:/RECYCLER"
1256 WildFile = "[A-Z]:/pagefile.sys"
1258 # These are programs and are easier to reinstall than restore from
1260 WildDir = "[A-Z]:/cygwin"
1261 WildDir = "[A-Z]:/Program Files/Grisoft"
1262 WildDir = "[A-Z]:/Program Files/Java"
1263 WildDir = "[A-Z]:/Program Files/Java Web Start"
1264 WildDir = "[A-Z]:/Program Files/JavaSoft"
1265 WildDir = "[A-Z]:/Program Files/Microsoft Office"
1266 WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
1267 WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
1268 WildDir = "[A-Z]:/Program Files/mozilla.org"
1269 WildDir = "[A-Z]:/Program Files/OpenOffice*"
1272 # Our Win2k boxen all have C: and D: as the main hard drives.
1280 Note, the three line of the above Exclude were split to fit on the document
1281 page, they should be written on a single line in real use.
1283 \paragraph*{Windows NTFS Naming Considerations}
1284 \index[general]{Windows NTFS Naming Considerations }
1285 \index[general]{Considerations!Windows NTFS Naming }
1286 \addcontentsline{toc}{paragraph}{Windows NTFS Naming Considerations}
1288 NTFS filenames containing Unicode characters should now be supported
1289 as of version 1.37.30 or later.
1291 \subsubsection*{Testing Your FileSet}
1292 \index[general]{FileSet!Testing Your }
1293 \index[general]{Testing Your FileSet }
1294 \addcontentsline{toc}{subsection}{Testing Your FileSet}
1296 If you wish to get an idea of what your FileSet will really backup or if your
1297 exclusion rules will work correctly, you can test it by using the {\bf
1298 estimate} command in the Console program. See the
1299 \ilink{estimate command}{estimate} in the Console chapter of this