4 \subsection*{The FileSet Resource}
5 \label{FileSetResource}
6 \index[general]{Resource!FileSet }
7 \index[general]{FileSet Resource }
8 \addcontentsline{toc}{subsection}{FileSet Resource}
10 The FileSet resource defines what files are to be included or excluded in a
11 backup job. A {\bf FileSet} resource is required for each backup Job. It
12 consists of a list of files or directories to be included, a list of files
13 or directories to be excluded and the various backup options such as
14 compression, encryption, and signatures that are to be applied to each
17 Any change to the list of the included files will cause Bacula to
18 automatically create a new FileSet (defined by the name and an MD5 checksum
19 of the Include/Exclude contents). Each time a new FileSet is created,
20 Bacula will ensure that the next backup is always a Full save.
26 Start of the FileSet resource. One {\bf FileSet} resource must be
27 defined for each Backup job.
29 \item [Name = \lt{}name\gt{}]
31 The name of the FileSet resource. This directive is required.
33 \item [Ignore FileSet Changes = \lt{}yes|no\gt{}]
34 \index[dir]{Ignore FileSet Changes}
35 If this directive is set to {\bf yes}, any changes you make to the FileSet
36 Include or Exclude lists will be ignored and not cause Bacula to immediately
37 perform a Full backup. The default is {\bf no}, in which case, if you change
38 the Include or Exclude, Bacula will force a Full backup to ensure that
39 everything is properly backed up. It is not recommended to set this directive
40 to yes. This directive is available in Bacula version 1.35.4 or later.
42 \item [Enable VSS = \lt{}yes|no\gt{}]
43 \index[dir]{Enable VSS}
44 If this directive is set to {\bf yes} the File daemon will be notified
45 that the user wants to use a Volume Shadow Copy Service (VSS) backup
46 for this job. The default is {\bf no}. This directive is effective
47 only for VSS enabled Win32 File daemons. It permits a consistent copy
48 of open files to be made for cooperating writer applications, and for
49 applications that are not VSS away, Bacula can at least copy open files.
50 For more information, please see the
51 \ilink{Windows}{VSS} chapter of this manual.
53 \item [Include \{ Options \{\lt{}file-options\gt{}\} ...;
54 \lt{}file-list\gt{} \} ]
55 \index[dir]{Include \{ [ Options \{\lt{}file-options\gt{}\} ...]
56 \lt{}file-list\gt{} \} }
58 \item [Options \{ \lt{}file-options\gt{} \} ]
59 \index[dir]{Options \{ \lt{}file-options\gt{} \} }
61 \item [Exclude \{ \lt{}file-list\gt{} \}]
62 \index[dir]{Exclude \{ \lt{}file-list\gt{} \} }
66 The Include resource must contain a list of directories and/or files to be
67 processed in the backup job. Normally, all files found in all
68 subdirectories of any directory in the Include File list will be backed up.
69 The Include resource may also contain one or more Options resources that
70 specify options such as compression to be applied to all or any subset of
71 the files found for backup.
73 There can be any number of {\bf Include} resources within the FileSet, each
74 having its own list of directories or files to be backed up and the backup
75 options defined by one or more Options resources. The {\bf file-list}
76 consists of one file or directory name per line. Directory names should be
77 specified without a trailing slash with Unix path notation.
79 You should always specify a full path for every directory and file that you
80 list in the FileSet. In addition, on Windows machines, you should {\bf
81 always} prefix the directory or filename with the drive specification in
82 lower case (e.g. {\bf c:/xxx}) using Unix directory name separators
85 Bacula's default for processing directories is to recursively descend in
86 the directory saving all files and subdirectories. Bacula will not by
87 default cross filesystems (or mount points in Unix parlance). This means
88 that if you specify the root partition (e.g. {\bf /}), Bacula will save
89 only the root partition and not any of the other mounted filesystems.
90 Similarly on Windows systems, you must explicitly specify each of the
91 drives you want saved (e.g.
92 {\bf c:/} and {\bf d:/} ...). In addition, at least for Windows systems, you
93 will most likely want to enclose each specification within double quotes
94 particularly if the directory (or file) name contains spaces. The {\bf df}
95 command on Unix systems will show you which mount points you must specify to
96 save everything. See below for an example.
98 Take special care not to include a directory twice or Bacula will backup
99 the same files two times wasting a lot of space on your archive device.
100 Including a directory twice is very easy to do. For example:
107 Options { compression=GZIP }
112 on a Unix system where /usr is a subdirectory (rather than a mounted
113 filesystem) will cause /usr to be backed up twice. In this case, on Bacula
114 versions prior to 1.32f-5-09Mar04 due to a bug, you will not be able to
115 restore hard linked files that were backed up twice.
117 If you have used Bacula prior to version 1.36.3, you will note three things in
118 the new FileSet syntax:
121 \item There is no equal sign (=) after the Include and before the opening
122 brace (\{). The same is true for the Exclude.
123 \item Each directory (or filename) to be included or excluded is preceded by a {\bf File
124 =}. Previously they were simply listed on separate lines.
125 \item The options that previously appeared on the Include line now must be
126 specified within their own Options resource.
127 \item The Exclude resource does not accept Options.
128 \item When using wild-cards or regular expressions, directory names are
129 always terminated with a slash (/) and filenames have no trailing slash.
132 The Options resource is optional, but when specified, it will contain a
133 list of {\bf keyword=value} options to be applied to the file-list.
134 Multiple Options resources may be specified one after another. As the
135 files are found in the specified directories, the Options will applied to
136 the filenames to determine if and how the file should be backed up. The
137 Options resources are applied in the order they are specified in the
138 FileSet until the first one that matches.
140 Once Bacula determines that the Options resource matches the file under
141 consideration, that file will be saved without looking at any other Options
142 resources that may be present. This means that any wild cards must appear
143 before an Options resource without wild cards.
145 If for some reason, Bacula applies all the Options resources to a file
146 under consideration for backup, but there are no matches (generally because
147 of wild cards that don't match), Bacula as a default will then backup the
148 file. This is quite logical if you consider the case of no Options, where
149 you want everything to be backed up.
151 However, one additional point is that
152 in the case that no match was found, Bacula will use the options found in
153 the last Options resource. As a consequence, if you want a particular set
154 of "default" options, you should put them in an Options resource after
157 It is a good idea to put all your wild-card and regex expressions inside
158 double quotes to prevent conf file scanning problems.
160 This is perhaps a bit overwhelming, so there are a number of examples included
161 below to illustrate how this works.
163 The directives within an Options resource may be one of the following:
167 \item [compression=GZIP]
168 \index[fd]{compression }
169 All files saved will be software compressed using the GNU ZIP compression
170 format. The compression is done on a file by file basis by the File daemon.
171 If there is a problem reading the tape in a single record of a file, it will
172 at most affect that file and none of the other files on the tape. Normally
173 this option is {\bf not} needed if you have a modern tape drive as the drive
174 will do its own compression. In fact, if you specify software compression at
175 the same time you have hardware compression turned on, your files may
176 actually take more space on the volume.
178 Software compression is very important if you are writing your Volumes to a
179 file, and it can also be helpful if you have a fast computer but a slow
180 network, otherwise it is generally better to rely your tape drive's hardware
181 compression. As noted above, it is not generally a good idea to do both software
182 and hardware compression.
184 Specifying {\bf GZIP} uses the default compression level six (i.e. {\bf GZIP}
185 is identical to {\bf GZIP6}). If you want a different compression level (1
186 through 9), you can specify it by appending the level number with no
187 intervening spaces to {\bf GZIP}. Thus {\bf compression=GZIP1} would give
188 minimum compression but the fastest algorithm, and {\bf compression=GZIP9}
189 would give the highest level of compression, but requires more computation.
190 According to the GZIP documentation, compression levels greater than 6
191 generally give very little extra compression and are rather CPU intensive.
193 \item [signature=SHA1]
194 \index[fd]{signature }
195 An SHA1 signature will be computed for all The SHA1 algorithm is purported to
196 be some what slower than the MD5 algorithm, but at the same time is
197 significantly better from a cryptographic point of view (i.e. much fewer
198 collisions, much lower probability of being hacked.) It adds four more bytes
199 than the MD5 signature. We strongly recommend that either this option or MD5
200 be specified as a default for all files. Note, only one of the two options
201 MD5 or SHA1 can be computed for any file.
203 \item [signature=MD5]
204 \index[fd]{signature }
205 An MD5 signature will be computed for all files saved. Adding this option
206 generates about 5\% extra overhead for each file saved. In addition to the
207 additional CPU time, the MD5 signature adds 16 more bytes per file to your
208 catalog. We strongly recommend that this option or the SHA1 option be
209 specified as a default for all files.
211 \item [verify=\lt{}options\gt{}]
213 The options letters specified are used when running a {\bf Verify
214 Level=Catalog} as well as the {\bf DiskToCatalog} level job. The options
215 letters may be any combination of the following:
223 compare the permission bits
226 compare the number of links
238 compare the access time
241 compare the modification time (st\_mtime)
244 compare the change time (st\_ctime)
247 report file size decreases
250 compare the MD5 signature
253 compare the SHA1 signature
256 A useful set of general options on the {\bf Level=Catalog} or {\bf
257 Level=DiskToCatalog} verify is {\bf pins5} i.e. compare permission bits,
258 inodes, number of links, size, and MD5 changes.
262 If set to {\bf yes} (the default), {\bf Bacula} will remain on a single file
263 system. That is it will not backup file systems that are mounted on a
264 subdirectory. If you wish to backup multiple filesystems, you can explicitly
265 list each file system you want saved. Otherwise, if you set the onefs option
266 to {\bf no}, Bacula will backup all mounted file systems (i.e. traverse mount
267 points) that are found within the {\bf FileSet}. Thus if you have NFS or
268 Samba file systems mounted on a directory listed in your FileSet, they will
269 also be backed up. Normally, it is preferable to set {\bf onefs=yes} and to
270 explicitly name each filesystem you want backed up. Explicitly naming the
271 filesystems you want backed up avoids the possibility of getting into a
272 infinite loop recursing filesystems. See the example below for more details.
276 \item [portable=yes|no]
277 \index[dir]{portable }
278 If set to {\bf yes} (default is {\bf no}), the Bacula File daemon will backup
279 Win32 files in a portable format, but not all Win32 file attributes will be
280 saved and restored. By default, this option is set to {\bf no}, which means
281 that on Win32 systems, the data will be backed up using Windows API calls and
282 on WinNT/2K/XP, all the security and ownership attributes will be properly
283 backed up (and restored). However this format is not portable to other
284 systems -- e.g. Unix, Win95/98/Me. When backing up Unix systems, this option
285 is ignored, and unless you have a specific need to have portable backups, we
286 recommend accept the default ({\bf no}) so that the maximum information
287 concerning your files is saved.
289 \item [recurse=yes|no]
291 If set to {\bf yes} (the default), Bacula will recurse (or descend) into all
292 subdirectories found unless the directory is explicitly excluded using an
293 {\bf exclude} definition. If you set {\bf recurse=no}, Bacula will save the
294 subdirectory entries, but not descend into the subdirectories, and thus will
295 not save the files or directories contained in the subdirectories. Normally,
296 you will want the default ({\bf yes}).
298 \item [sparse=yes|no]
300 Enable special code that checks for sparse files such as created by
301 ndbm. The default is {\bf no}, so no checks are made for sparse files.
302 You may specify {\bf sparse=yes} even on files that are not sparse file.
303 No harm will be done, but there will be a small additional overhead to
304 check for buffers of all zero, and a small additional amount of space on
305 the output archive will be used to save the seek address of each
306 non-zero record read.
308 {\bf Restrictions:} Bacula reads files in 32K buffers. If the whole
309 buffer is zero, it will be treated as a sparse block and not written to
310 tape. However, if any part of the buffer is non-zero, the whole buffer
311 will be written to tape, possibly including some disk sectors (generally
312 4098 bytes) that are all zero. As a consequence, Bacula's detection of
313 sparse blocks is in 32K increments rather than the system block size.
314 If anyone considers this to be a real problem, please send in a request
315 for change with the reason.
317 If you are not familiar with sparse files, an example is say a file
318 where you wrote 512 bytes at address zero, then 512 bytes at address 1
319 million. The operating system will allocate only two blocks, and the
320 empty space or hole will have nothing allocated. However, when you read
321 the sparse file and read the addresses where nothing was written, the OS
322 will return all zeros as if the space were allocated, and if you backup
323 such a file, a lot of space will be used to write zeros to the volume.
324 Worse yet, when you restore the file, all the previously empty space
325 will now be allocated using much more disk space. By turning on the
326 {\bf sparse} option, Bacula will specifically look for empty space in
327 the file, and any empty space will not be written to the Volume, nor
328 will it be restored. The price to pay for this is that Bacula must
329 search each block it reads before writing it. On a slow system, this
330 may be important. If you suspect you have sparse files, you should
331 benchmark the difference or set sparse for only those files that are
336 \item [readfifo=yes|no]
337 \index[fd]{readfifo }
338 If enabled, tells the Client to read the data on a backup and write the
339 data on a restore to any FIFO (pipe) that is explicitly mentioned in the
340 FileSet. In this case, you must have a program already running that
341 writes into the FIFO for a backup or reads from the FIFO on a restore.
342 This can be accomplished with the {\bf RunBeforeJob} directive. If this
343 is not the case, Bacula will hang indefinitely on reading/writing the
344 FIFO. When this is not enabled (default), the Client simply saves the
345 directory entry for the FIFO.
347 \item [mtimeonly=yes|no]
348 \index[dir]{mtimeonly }
349 If enabled, tells the Client that the selection of files during
350 Incremental and Differential backups should based only on the st\_mtime
351 value in the stat() packet. The default is {\bf no} which means that
352 the selection of files to be backed up will be based on both the
353 st\_mtime and the st\_ctime values. In general, it is not recommended
356 \item [keepatime=yes|no]
357 \index[dir]{keepatime }
358 The default is {\bf no}. When enabled, Bacula will reset the st\_atime
359 (access time) field of files that it backs up to their value prior to
360 the backup. This option is not generally recommended as there are very
361 few programs that use st\_atime, and the backup overhead is increased
362 because of the additional system call necessary to reset the times.
363 However, for some files, such as mailboxes, when Bacula backs up the
364 file, the user will notice that someone (Bacula) has accessed the
365 file. In this, case keepatime can be useful.
366 (I'm not sure this works on Win32).
368 Note, if you use this feature, when Bacula resets the access time, the
369 change time (st\_ctime) will automatically be modified by the system,
370 so on the next incremental job, the file will be backed up even if
371 it has not changed. As a consequence, you will probably also want
372 to use {\bf mtimeonly = yes} as well as keepatime (thanks to
373 Rudolf Cejka for this tip).
375 \item [hardlinks=yes|no]
376 \index[dir]{hardlinks}
377 When enabled (default), this directive will cause hard inks to be
378 backed up. However, the File daemon keeps track of hard linked files and
379 will backup the data only once. The process of keeping track of the
380 hard links can be quite expensive if you have lots of them (tens of
381 thousands or more). This doesn't occur on normal Unix systems, but if
382 you use a program like BackupPC, it can create hundreds of thousands, or
383 even millions of hard links. Backups become very long and the File daemon
384 will consume a lot of CPU power checking hard links. In such a case,
385 set {\bf hardlinks=no} and hard links will not be backed up. Note, using
386 this option will most likely backup more data and on a restore the file
387 system will not be restored identically to the original.
389 \item [wild=\lt{}string\gt{}]
391 Specifies a wild-card string to be applied to the filenames and
392 directory names. Note, if {\bf Exclude} is not enabled, the wild-card
393 will select which files are to be included. If {\bf Exclude=yes} is
394 specified, the wild-card will select which files are to be excluded.
395 Multiple wild-card directives may be specified, and they will be applied
396 in turn until the first one that matches. Note, if you exclude a
397 directory, no files or directories below it will be matched.
398 It is recommended to enclose the string in double quotes.
400 \item [wildfile=\lt{}string\gt{}]
401 \index[dir]{wildfile }
402 Specifies a wild-card string to be applied to filenames only. No
403 directories will be matched by this directive. Note, if {\bf Exclude}
404 is not enabled, the wild-card will select which files are to be
405 included. If {\bf Exclude=yes} is specified, the wild-card will select
406 which files are to be excluded. Multiple wild-card directives may be
407 specified, and they will be applied in turn until the first one that
409 It is recommended to enclose the string in double quotes.
411 \item [wilddir=\lt{}string\gt{}]
412 \index[dir]{wilddir }
413 Specifies a wild-card string to be applied to directory names only. No
414 filenames will be matched by this directive. Note, if {\bf Exclude} is
415 not enabled, the wild-card will select directories files are to be
416 included. If {\bf Exclude=yes} is specified, the wild-card will select
417 which files are to be excluded. Multiple wild-card directives may be
418 specified, and they will be applied in turn until the first one that
419 matches. Note, if you exclude a directory, no files or directories
420 below it will be matched.
421 It is recommended to enclose the string in double quotes.
424 \item [regex=\lt{}string\gt{}]
426 Specifies a POSIX extended regular expression to be applied to the
427 filenames and directory names.
428 This directive is available in version 1.35 and later. If {\bf
429 Exclude} is not enabled, the regex will select which files are to be
430 included. If {\bf Exclude=yes} is specified, the regex will select
431 which files are to be excluded. Multiple regex directives may be
432 specified within an Options resource, and they will be applied in turn
433 until the first one that matches. Note, if you exclude a
434 directory, no files or directories below it will be matched.
435 It is recommended to enclose the string in double quotes.
437 \item [regexfile=\lt{}string\gt{}]
438 \index[dir]{regexfile }
439 Specifies a POSIX extended regular expression to be applied to filenames
440 only. No directories will be matched by this directive. Note, if {\bf
441 Exclude} is not enabled, the regex will select which files are to be
442 included. If {\bf Exclude=yes} is specified, the regex will select
443 which files are to be excluded. Multiple regex directives may be
444 specified, and they will be applied in turn until the first one that
446 It is recommended to enclose the string in double quotes.
448 \item [regexdir=\lt{}string\gt{}]
449 \index[dir]{regexdir }
450 Specifies a POSIX extended regular expression to be applied to directory
451 names only. No filenames will be matched by this directive. Note, if
452 {\bf Exclude} is not enabled, the regex will select directories
453 files are to be included. If {\bf Exclude=yes} is specified, the
454 regex will select which files are to be excluded. Multiple
455 regex directives may be specified, and they will be applied in turn
456 until the first one that matches. Note, if you exclude a directory, no
457 files or directories below it will be matched.
458 It is recommended to enclose the string in double quotes.
460 \item [exclude=yes|no]
461 \index[dir]{exclude }
462 The default is {\bf no}. When enabled, any files matched within the Options
463 will be excluded from the backup.
467 \item [aclsupport=yes|no]
468 \index[dir]{aclsupport }
469 The default is {\bf no}. If this option is set to yes, and you have the
470 POSIX {\bf libacl} installed on your system, Bacula will backup the file
471 and directory UNIX Access Control Lists (ACL) as defined in IEEE Std
472 1003.1e draft 17 and "POSIX.1e" (abandoned). This feature is
473 available on UNIX only and depends on the ACL library. Bacula is
474 automatically compiled with ACL support if the {\bf libacl} library is
475 installed on your system (shown in config.out). While restoring the
476 files Bacula will try to restore the ACLs, if there is no ACL support
477 available on the system, Bacula restores the files and directories but
478 not the ACL information. Please note, if you backup an EXT3 or XFS
479 filesystem with ACLs, then you restore them to a different filesystem
480 (perhaps reiserfs) that does not have ACLs, the ACLs will be ignored.
482 \item [ignore case=yes|no]
483 \index[dir]{ignore case }
484 The default is {\bf no}, except on Windows systems where the default
485 is {\bf yes}. When this directive is set to {\bf yes} all the case
486 of character will be ignored in wild-card and regex comparisons.
487 That is an uppercase A will match a lowercase a.
489 \item [fstype=filesystem-type]
491 This option allows you to select files and directories by the
492 filesystem type. The permitted filesystem-type names are:
494 ext2, jfs, ntfs, proc, reiserfs, xfs, usbdevfs, sysfs, smbfs,
495 iso9660. For ext3 systems, use ext2.
497 You may have multiple Fstype directives, and thus permit matching
498 of multiple filesystem types within a single Options resource. If
499 the type specified on the fstype directive does not match the
500 filesystem for a particular directive, that directory will not be
501 backed up. This directive can be used to prevent backing up
502 non-local filesystems.
504 This option is not implemented in Win32 systems.
507 \item [hfsplussupport=yes|no]
508 \index[dir]{hfsplussupport }
509 This option allows you to turn on support for Mac OSX HFS plus
514 {\bf \lt{}file-list\gt{}} is a list of directory and/or filename names
515 specified with a {\bf File =} directive. To include names containing spaces,
516 enclose the name between double-quotes.
518 There are a number of special cases when specifying directories and files in a
519 {\bf file-list}. They are:
522 \item Any name preceded by an at-sign (@) is assumed to be the name of a
523 file, which contains a list of files each preceded by a "File =". The
524 named file is read once when the configuration file is parsed during the
525 Director startup. Note, that the file is read on the Director's machine
526 and not on the Client's. In fact, the @filename can appear anywhere
527 within the conf file where a token would be read, and the contents of
528 the named file will be logically inserted in the place of the @filename.
529 What must be in the file depends on the location the @filename is
530 specified in the conf file. For example:
535 Options { compression=GZIP }
536 @/home/files/my-files
541 \item Any name beginning with a vertical bar (|) is assumed to be the name of
542 a program. This program will be executed on the Director's machine at
543 the time the Job starts (not when the Director reads the configuration
544 file), and any output from that program will be assumed to be a list of
545 files or directories, one per line, to be included.
548 have a job that, for example, includes all the local partitions even if
549 you change the partitioning by adding a disk. The examples
550 below show you how to do this. However, please note two
551 things: 1. if you want the local filesystems, you probably
552 should be using the new {\bf fstype} directive, which was
553 added in version 1.36.3. 2. the exact syntax of the command
554 needed in the examples below is very system dependent. For
555 example, on recent Linux systems, you may need to add the -P
556 option, on FreeBSD systems, the options will be different as
559 In general, you will need
560 to prefix your command or commands with a {\bf sh -c} so that they are
561 invoked by a shell. This will not be the case if you are invoking a
562 script as in the second example below. Also, you must take care to
563 escape (precede with a \textbackslash{}) wild-cards, shell character,
564 and to ensure that any spaces in your command are escaped as well. If
565 you use a single quotes (') within a double quote ("), Bacula will
566 treat everything between the single quotes as one field so it will not
567 be necessary to escape the spaces. In general, getting all the quotes
568 and escapes correct is a real pain as you can see by the next example.
569 As a consequence, it is often easier to put everything in a file and
570 simply use the file name within Bacula. In that case the {\bf sh -c}
571 will not be necessary providing the first line of the file is {\bf
580 Options { signature = SHA1 }
581 File = "|sh -c 'df -l | grep \"^/dev/hd[ab]\" | grep -v \".*/tmp\" \
582 | awk \"{print \\$6}\"'"
587 will produce a list of all the local partitions on a RedHat Linux system.
588 Note, the above line was split, but should normally be written on one line.
589 Quoting is a real problem because you must quote for Bacula which consists of
590 preceding every \textbackslash{} and every " with a \textbackslash{}, and
591 you must also quote for the shell command. In the end, it is probably easier
592 just to execute a small file with:
600 File = "|my_partitions"
605 where my\_partitions has:
610 df -l | grep "^/dev/hd[ab]" | grep -v ".*/tmp" \
615 If the vertical bar (|) in front of my\_partitions is preceded by a
616 backslash as in \textbackslash{}|, the program will be executed on the
617 Client's machine instead of on the Director's machine.
618 Please note that if the filename is given within quotes, you
619 will need to use two slashes. An example, provided by John Donagher,
620 that backs up all the local UFS partitions on a remote system is:
625 Name = "All local partitions"
627 Options { signature=SHA1; onefs=yes; }
628 File = "\\|bash -c \"df -klF ufs | tail +2 | awk '{print \$6}'\""
634 The above requires two backslash characters after the double quote (one
635 preserves the next one). If you are a Linux user, just change the {\bf ufs}
636 to {\bf ext3} (or your preferred filesystem type), and you will be in
639 \item Any file-list item preceded by a less-than sign (\lt{}) will be taken
640 to be a file. This file will be read on the Director's machine at the time
641 the Job starts, and the data will be assumed to be a list of directories or
642 files, one per line, to be included. The names should start in column 1 and
643 should not be quoted even if they contain spaces. This feature allows you to
644 modify the external file and change what will be saved without stopping and
645 restarting Bacula as would be necessary if using the @ modifier noted above.
651 Options { signature = SHA1 }
652 File = "</home/files/local-filelist"
657 If you precede the less-than sign (\lt{}) with a backslash as in
658 \textbackslash{}\lt{}, the file-list will be read on the Client machine
659 instead of on the Director's machine. Please note that if the filename
660 is given within quotes, you will need to use two slashes.
665 Options { signature = SHA1 }
666 File = "\\</home/xxx/filelist-on-client"
671 \item If you explicitly specify a block device such as {\bf /dev/hda1}, then
672 Bacula (starting with version 1.28) will assume that this is a raw partition
673 to be backed up. In this case, you are strongly urged to specify a {\bf
674 sparse=yes} include option, otherwise, you will save the whole partition
675 rather than just the actual data that the partition contains. For example:
680 Options { signature=MD5; sparse=yes }
686 will backup the data in device /dev/hd6.
688 Ludovic Strappazon has pointed out that this feature can be used to backup a
689 full Microsoft Windows disk. Simply boot into the system using a Linux Rescue
690 disk, then load a statically linked Bacula as described in the
691 \ilink{ Disaster Recovery Using Bacula}{_ChapterStart38} chapter of
692 this manual. Then save the whole disk partition. In the case of a disaster,
693 you can then restore the desired partition by again booting with the rescue
694 disk and doing a restore of the partition.
695 \item If you explicitly specify a FIFO device name (created with mkfifo), and
696 you add the option {\bf readfifo=yes} as an option, Bacula will read the FIFO
697 and back its data up to the Volume. For example:
706 File = /home/abc/fifo
711 if {\bf /home/abc/fifo} is a fifo device, Bacula will open the fifo, read it,
712 and store all data thus obtained on the Volume. Please note, you must have a
713 process on the system that is writing into the fifo, or Bacula will hang,
714 and after one minute of waiting, Bacula will give up and go on to the next
715 file. The data read can be anything since Bacula treats it as a stream.
717 This feature can be an excellent way to do a "hot" backup of a very large
718 database. You can use the {\bf RunBeforeJob} to create the fifo and to start
719 a program that dynamically reads your database and writes it to the fifo.
720 Bacula will then write it to the Volume.
722 During the restore operation, the inverse is true, after Bacula creates the
723 fifo if there was any data stored with it (no need to explicitly list it or
724 add any options), that data will be written back to the fifo. As a
725 consequence, if any such FIFOs exist in the fileset to be restored, you must
726 ensure that there is a reader program or Bacula will block, and after one
727 minute, Bacula will time out the write to the fifo and move on to the next
731 \subsubsection*{FileSet Examples}
732 \index[general]{Examples!FileSet }
733 \index[general]{FileSet Examples}
734 \addcontentsline{toc}{subsection}{FileSet Examples}
736 The following is an example of a valid FileSet resource definition. Note, the
737 first Include pulls in the contents of the file {\bf /etc/backup.list} when
738 Bacula is started (i.e. the @), and that file must have each filename to be
739 backed up preceded by a {\bf File =} and on a separate line.
760 File = /usr/lib/another_file
766 In the above example, all the files contained in /etc/backup.list will
767 be compressed with GZIP compression, an SHA1 signature will be computed on the
768 file's contents (its data), and sparse file handling will apply.
770 The two directories /root/myfile and /usr/lib/another\_file will also be saved
771 without any options, but all files in those directories with the extensions
772 {\bf .o} and {\bf .exe} will be excluded.
774 Let's say that you now want to exclude the directory /tmp. The simplest way
775 to do so is to add an exclude directive that lists /tmp. The example
776 above would then become:
797 File = /usr/lib/another_file
807 You can add wild-cards to the File directives listed in the Exclude
808 directory, but you need to take care because if you exclude a directory,
809 it and all files and directories below it will also be excluded.
811 Now lets take a slight variation on the above and suppose
812 you want to save all your whole filesystem except {\bf /tmp}.
813 The problem that comes up is that Bacula will not normally
814 cross from one filesystem to another.
815 Doing a {\bf df} command, you get the following output:
820 Filesystem 1k-blocks Used Available Use% Mounted on
821 /dev/hda5 5044156 439232 4348692 10% /
822 /dev/hda1 62193 4935 54047 9% /boot
823 /dev/hda9 20161172 5524660 13612372 29% /home
824 /dev/hda2 62217 6843 52161 12% /rescue
825 /dev/hda8 5044156 42548 4745376 1% /tmp
826 /dev/hda6 5044156 2613132 2174792 55% /usr
827 none 127708 0 127708 0% /dev/shm
828 //minimatou/c$ 14099200 9895424 4203776 71% /mnt/mmatou
829 lmatou:/ 1554264 215884 1258056 15% /mnt/matou
830 lmatou:/home 2478140 1589952 760072 68% /mnt/matou/home
831 lmatou:/usr 1981000 1199960 678628 64% /mnt/matou/usr
832 lpmatou:/ 995116 484112 459596 52% /mnt/pmatou
833 lpmatou:/home 19222656 2787880 15458228 16% /mnt/pmatou/home
834 lpmatou:/usr 2478140 2038764 311260 87% /mnt/pmatou/usr
835 deuter:/ 4806936 97684 4465064 3% /mnt/deuter
836 deuter:/home 4806904 280100 4282620 7% /mnt/deuter/home
837 deuter:/files 44133352 27652876 14238608 67% /mnt/deuter/files
841 And we see that there are a number of separate filesystems (/ /boot
842 /home /rescue /tmp and /usr not to mention mounted systems).
843 If you specify only {\bf /} in your Include list, Bacula will only save the
844 Filesystem {\bf /dev/hda5}. To save all filesystems except {\bf /tmp} with
845 out including any of the Samba or NFS mounted systems, and explicitly
846 excluding a /tmp, /proc, .journal, and .autofsck, which you will not want to
847 be saved and restored, you can use the following:
852 Name = Include_example
857 wildfile = ".journal"
858 wildfile = ".autofsck"
871 Since /tmp is on its own filesystem and it was not explicitly named in the
872 Include list, it is not really needed in the exclude list. It is better to
873 list it in the Exclude list for clarity, and in case the disks are changed so
874 that it is no longer in its own partition.
876 Now, lets assume you only want to backup .Z and .gz files and nothing
877 else. This is a bit trickier because Bacula by default will select
878 everything to backup, so we must exclude everything but .Z and .gz files.
879 If we take the first example above and make the obvious modifications
880 to it, we might come up with a FileSet that looks like this:
886 Include { !!!!!!!!!!!!
888 wildfile = "*.Z" example
889 wildfile = "*.gz" doesn't
898 The *.Z and *.gz files will indeed be backed up, but all other files
899 that are not matched by the Options directives will automatically
900 be backed up too (i.e. that is the default rule).
902 To accomplish what we want, we must explicitly exclude all other files.
903 We do this with the fillowing:
925 The "trick" here was to add a RegexFile expression that matches
926 all files. It does not match directory names, so all directories in
927 /myfile will be backed up (the directory entry) and any *.Z and *.gz
928 files contained in them. If you know that certain directories do
929 not contain any *.Z or *.gz files and you do not want the directory
930 entries backed up, you will need to explicitly exclude those directories.
931 Backing up a directory entries is not very expensive.
933 Bacula uses the system regex library and some of them are
934 different on different OSes. The above has been reported not to work
935 on FreeBSD. This can be tested by using the {\bf estimate job=job-name
936 listing} command in the console and adapting the RegexFile expression
937 appropriately. In a future version of Bacula, we will supply our own
938 Regex code to avoid such system dependencies.
940 Please be aware that allowing Bacula to traverse or change file systems can be
941 {\bf very} dangerous. For example, with the following:
955 you will be backing up an NFS mounted partition ({\bf /mnt/matou}), and since
956 {\bf onefs} is set to {\bf no}, Bacula will traverse file systems. Now if {\bf
957 /mnt/matou} has the current machine's file systems mounted, as is often the
958 case, you will get yourself into a recursive loop and the backup will never
961 \subsubsection*{Backing up Raw Partitions}
962 \index[general]{Backing up!Partitions }
963 \index[general]{Backing up Raw Partitions }
964 \addcontentsline{toc}{subsection}{Backing up Raw Partitions}
966 The following FileSet definition will backup a raw partition:
971 Name = "RawPartition"
973 Options { sparse=yes }
980 While backing up and restoring a raw partition, you should ensure that no
981 other process including the system is writing to that partition. As a
982 precaution, you are strongly urged to ensure that the raw partition is not
983 mounted or is mounted read-only. If necessary, this can be done using the {\bf
984 RunBeforeJob} directive.
987 \subsubsection*{Excluding Files and Directories}
988 \index[general]{Directories!Excluding Files and }
989 \index[general]{Excluding Files and Directories }
990 \addcontentsline{toc}{subsubsection}{Excluding Files and Directories}
992 You may also include full filenames or directory names in addition to using
993 wild-cards and {\bf Exclude=yes} in the Options resource as specified above by
994 simply including the files to be excluded in an Exclude resource within the
995 FileSet. For example:
1000 Name = Exclusion_example
1023 \subsubsection*{Windows FileSets}
1024 \index[general]{Windows FileSets }
1025 \index[general]{FileSets!Windows }
1026 \addcontentsline{toc}{subsection}{Windows FileSets}
1027 If you are entering Windows file names, the directory path may be preceded by
1028 the drive and a colon (as in c:). However, the path separators must be
1029 specified in Unix convention (i.e. forward slash (/)). If you wish to include
1030 a quote in a file name, precede the quote with a backslash
1031 (\textbackslash{}). For example you might use the following
1032 for a Windows machine to backup the "My Documents" directory:
1037 Name = "Windows Set"
1044 File = "c:/My Documents"
1050 For exclude lists to work correctly on Windows, you must observe the following
1054 \item Filenames are case sensitive, so you must use the correct case.
1055 \item To exclude a directory, you must not have a trailing slash on the
1057 \item If you have spaces in your filename, you must enclose the entire name
1058 in double-quote characters ("). Trying to use a backslash before the space
1060 \item If you are using the old Exclude syntax (noted below), you may not
1061 specify a drive letter in the exclude. The new syntax noted above should work
1062 fine including driver letters.
1065 Thanks to Thiago Lima for summarizing the above items for us. If you are
1066 having difficulties getting includes or excludes to work, you might want to
1067 try using the {\bf estimate job=xxx listing} command documented in the
1068 \ilink{Console chapter}{estimate} of this manual.
1070 On Win32 systems, if you move a directory or file or rename a file into the
1071 set of files being backed up, and a Full backup has already been made, Bacula
1072 will not know there are new files to be saved during an Incremental or
1073 Differential backup (blame Microsoft, not me). To avoid this problem, please
1074 {\bf copy} any new directory or files into the backup area. If you do not have
1075 enough disk to copy the directory or files, move them, but then initiate a
1079 \paragraph*{A Windows Example FileSet}
1080 \index[general]{FileSet!Windows Example }
1081 \index[general]{Windows Example FileSet }
1082 \addcontentsline{toc}{paragraph}{Windows Example FileSet}
1084 The following example was contributed by Russell Howe. Please note that
1085 for presentation purposes, the lines beginning with Data and Internet
1086 have been wrapped and should included on the previous line with one
1091 This is my Windows 2000 fileset:
1093 Name = "Windows 2000"
1099 # Exclude Mozilla-based programs' file caches
1100 WildDir = "[A-Z]:/Documents and Settings/*/Application
1101 Data/*/Profiles/*/*/Cache"
1102 WildDir = "[A-Z]:/Documents and Settings/*/Application
1103 Data/*/Profiles/*/*/Cache.Trash"
1104 WildDir = "[A-Z]:/Documents and Settings/*/Application
1105 Data/*/Profiles/*/*/ImapMail"
1107 # Exclude user's registry files - they're always in use anyway.
1108 WildFile = "[A-Z]:/Documents and Settings/*/Local Settings/Application
1109 Data/Microsoft/Windows/usrclass.*"
1110 WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
1112 # Exclude directories full of lots and lots of useless little files
1113 WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
1114 WildDir = "[A-Z]:/Documents and Settings/*/Recent"
1115 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
1116 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
1117 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary
1120 # These are always open and unable to be backed up
1121 WildFile = "[A-Z]:/Documents and Settings/All Users/Application
1122 Data/Microsoft/Network/Downloader/qmgr[01].dat"
1124 # Some random bits of Windows we want to ignore
1125 WildFile = "[A-Z]:/WINNT/security/logs/scepol.log"
1126 WildDir = "[A-Z]:/WINNT/system32/config"
1127 WildDir = "[A-Z]:/WINNT/msdownld.tmp"
1128 WildDir = "[A-Z]:/WINNT/Internet Logs"
1129 WildDir = "[A-Z]:/WINNT/$Nt*Uninstall*"
1130 WildDir = "[A-Z]:/WINNT/sysvol"
1131 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB"
1132 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB.LOG"
1133 WildFile = "[A-Z]:/WINNT/NTDS/edb.log"
1134 WildFile = "[A-Z]:/WINNT/NTDS/ntds.dit"
1135 WildFile = "[A-Z]:/WINNT/NTDS/temp.edb"
1136 WildFile = "[A-Z]:/WINNT/ntfrs/jet/log/edb.log"
1137 WildFile = "[A-Z]:/WINNT/ntfrs/jet/ntfrs.jdb"
1138 WildFile = "[A-Z]:/WINNT/ntfrs/jet/temp/tmp.edb"
1139 WildFile = "[A-Z]:/WINNT/system32/CPL.CFG"
1140 WildFile = "[A-Z]:/WINNT/system32/dhcp/dhcp.mdb"
1141 WildFile = "[A-Z]:/WINNT/system32/dhcp/j50.log"
1142 WildFile = "[A-Z]:/WINNT/system32/dhcp/tmp.edb"
1143 WildFile = "[A-Z]:/WINNT/system32/LServer/edb.log"
1144 WildFile = "[A-Z]:/WINNT/system32/LServer/TLSLic.edb"
1145 WildFile = "[A-Z]:/WINNT/system32/LServer/tmp.edb"
1146 WildFile = "[A-Z]:/WINNT/system32/wins/j50.log"
1147 WildFile = "[A-Z]:/WINNT/system32/wins/wins.mdb"
1148 WildFile = "[A-Z]:/WINNT/system32/wins/winstmp.mdb"
1150 # Temporary directories & files
1151 WildDir = "[A-Z]:/WINNT/Temp"
1152 WildDir = "[A-Z]:/temp"
1154 WildDir = "[A-Z]:/tmp"
1155 WildDir = "[A-Z]:/var/tmp"
1158 WildDir = "[A-Z]:/RECYCLER"
1161 WildFile = "[A-Z]:/pagefile.sys"
1163 # These are programs and are easier to reinstall than restore from
1165 WildDir = "[A-Z]:/cygwin"
1166 WildDir = "[A-Z]:/Program Files/Grisoft"
1167 WildDir = "[A-Z]:/Program Files/Java"
1168 WildDir = "[A-Z]:/Program Files/Java Web Start"
1169 WildDir = "[A-Z]:/Program Files/JavaSoft"
1170 WildDir = "[A-Z]:/Program Files/Microsoft Office"
1171 WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
1172 WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
1173 WildDir = "[A-Z]:/Program Files/mozilla.org"
1174 WildDir = "[A-Z]:/Program Files/OpenOffice*"
1177 # Our Win2k boxen all have C: and D: as the main hard drives.
1185 Note, the three line of the above Exclude were split to fit on the document
1186 page, they should be written on a single line in real use.
1188 \paragraph*{Windows NTFS Naming Considerations}
1189 \index[general]{Windows NTFS Naming Considerations }
1190 \index[general]{Considerations!Windows NTFS Naming }
1191 \addcontentsline{toc}{paragraph}{Windows NTFS Naming Considerations}
1193 NTFS filenames containing Unicode characters should now be supported
1194 as of version 1.37.30 or later.
1196 \subsubsection*{Testing Your FileSet}
1197 \index[general]{FileSet!Testing Your }
1198 \index[general]{Testing Your FileSet }
1199 \addcontentsline{toc}{subsection}{Testing Your FileSet}
1201 If you wish to get an idea of what your FileSet will really backup or if your
1202 exclusion rules will work correctly, you can test it by using the {\bf
1203 estimate} command in the Console program. See the
1204 \ilink{estimate command}{estimate} in the Console chapter of this