4 \subsection*{The FileSet Resource}
5 \label{FileSetResource}
6 \index[general]{Resource!FileSet }
7 \index[general]{FileSet Resource }
8 \addcontentsline{toc}{subsection}{FileSet Resource}
10 The FileSet resource defines what files are to be included or excluded in a
11 backup job. A {\bf FileSet} resource is required for each backup Job. It
12 consists of a list of files or directories to be included, a list of files
13 or directories to be excluded and the various backup options such as
14 compression, encryption, and signatures that are to be applied to each
17 Any change to the list of the included files will cause Bacula to
18 automatically create a new FileSet (defined by the name and an MD5 checksum
19 of the Include/Exclude contents). Each time a new FileSet is created,
20 Bacula will ensure that the next backup is always a Full save.
26 Start of the FileSet resource. One {\bf FileSet} resource must be
27 defined for each Backup job.
29 \item [Name = \lt{}name\gt{}]
31 The name of the FileSet resource. This directive is required.
33 \item [Ignore FileSet Changes = \lt{}yes|no\gt{}]
34 \index[dir]{Ignore FileSet Changes }
35 If this directive is set to {\bf yes}, any changes you make to the FileSet
36 Include or Exclude lists will be ignored and not cause Bacula to immediately
37 perform a Full backup. The default is {\bf no}, in which case, if you change
38 the Include or Exclude, Bacula will force a Full backup to ensure that
39 everything is properly backed up. It is not recommended to set this directive
40 to yes. This directive is available in Bacula version 1.35.4 or later.
42 \item [Include \{ Options \{\lt{}file-options\gt{}\} ...;
43 \lt{}file-list\gt{} \} ]
44 \index[dir]{Include \ \{ [ Options \{\lt{}file-options\gt{}\} ...]
45 \lt{}file-list\gt{} \} }
47 \item [Options \ \{ \lt{}file-options\gt{} \} ]
48 \index[dir]{Options \ \{ \lt{}file-options\gt{} \} }
50 \item [Exclude \ \{ \lt{}file-list\gt{} \}]
51 \index[dir]{Exclude \ \{ \lt{}file-list\gt{} \} }
55 The Include resource must contain a list of directories and/or files to be
56 processed in the backup job. Normally, all files found in all
57 subdirectories of any directory in the Include File list will be backed up.
58 The Include resource may also contain one or more Options resources that
59 specify options such as compression to be applied to all or any subset of
60 the files found for backup.
62 There can be any number of {\bf Include} resources within the FileSet, each
63 having its own list of directories or files to be backed up and the backup
64 options defined by one or more Options resources. The {\bf file-list}
65 consists of one file or directory name per line. Directory names should be
66 specified without a trailing slash with Unix path notation.
68 You should always specify a full path for every directory and file that you
69 list in the FileSet. In addition, on Windows machines, you should {\bf
70 always} prefix the directory or filename with the drive specification in
71 lower case (e.g. {\bf c:/xxx}) using Unix directory name separators
74 Bacula's default for processing directories is to recursively descend in
75 the directory saving all files and subdirectories. Bacula will not by
76 default cross filesystems (or mount points in Unix parlance). This means
77 that if you specify the root partition (e.g. {\bf /}), Bacula will save
78 only the root partition and not any of the other mounted filesystems.
79 Similarly on Windows systems, you must explicitly specify each of the
80 drives you want saved (e.g.
81 {\bf c:/} and {\bf d:/} ...). In addition, at least for Windows systems, you
82 will most likely want to enclose each specification within double quotes
83 particularly if the directory (or file) name contains spaces. The {\bf df}
84 command on Unix systems will show you which mount points you must specify to
85 save everything. See below for an example.
87 Take special care not to include a directory twice or Bacula will backup
88 the same files two times wasting a lot of space on your archive device.
89 Including a directory twice is very easy to do. For example:
96 Options { compression=GZIP }
101 on a Unix system where /usr is a subdirectory (rather than a mounted
102 filesystem) will cause /usr to be backed up twice. In this case, on Bacula
103 versions prior to 1.32f-5-09Mar04 due to a bug, you will not be able to
104 restore hard linked files that were backed up twice.
106 If you have used Bacula prior to version 1.34.3, you will note three things in
107 the new FileSet syntax:
110 \item There is no equal sign (=) after the Include and before the opening
111 brace (\{). The same is true for the Exclude.
112 \item Each directory (or filename) to be included or excluded is preceded by a {\bf File
113 =}. Previously they were simply listed on separate lines.
114 \item The options that previously appeared on the Include line now must be
115 specified within their own Options resource.
116 \item The Exclude resource does not accept Options.
117 \item When using wild-cards or regular expressions, directory names are
118 always terminated with a slash (/) and filenames have no trailing slash.
121 The Options resource is optional, but when specified, it will contain a
122 list of {\bf keyword=value} options to be applied to the file-list.
123 Multiple Options resources may be specified one after another. As the
124 files are found in the specified directories, the Options will applied to
125 the filenames to determine if and how the file should be backed up. The
126 Options resources are applied in the order they are specified in the
127 FileSet until the first one that matches.
129 Once Bacula determines that the Options resource matches the file under
130 consideration, that file will be saved without looking at any other Options
131 resources that may be present. This means that any wild cards must appear
132 before an Options resource without wild cards.
134 If for some reason, Bacula applies all the Options resources to a file
135 under consideration for backup, but there are no matches (generally because
136 of wild cards that don't match), Bacula as a default will then backup the
137 file. This is quite logical if you consider the case of no Options, where
138 you want everything to be backed up.
140 However, one additional point is that
141 in the case that no match was found, Bacula will use the options found in
142 the last Options resource. As a consequence, if you want a particular set
143 of ``default'' options, you should put them in an Options resource after
146 This is perhaps a bit overwhelming, so there are a number of examples included
147 below to illustrate how this works.
149 The directives within an Options resource may be one of the following:
153 \item [compression=GZIP]
154 \index[fd]{compression }
155 All files saved will be software compressed using the GNU ZIP compression
156 format. The compression is done on a file by file basis by the File daemon.
157 If there is a problem reading the tape in a single record of a file, it will
158 at most affect that file and none of the other files on the tape. Normally
159 this option is {\bf not} needed if you have a modern tape drive as the drive
160 will do its own compression. In fact, if you specify software compression at
161 the same time you have hardware compression turned on, your files may
162 actually take more space on the volume.
164 Software compression is very important if you are writing your Volumes to a
165 file, and it can also be helpful if you have a fast computer but a slow
166 network, otherwise it is generally better to rely your tape drive's hardware
167 compression. As noted above, it is not generally a good idea to do both software
168 and hardware compression.
170 Specifying {\bf GZIP} uses the default compression level six (i.e. {\bf GZIP}
171 is identical to {\bf GZIP6}). If you want a different compression level (1
172 through 9), you can specify it by appending the level number with no
173 intervening spaces to {\bf GZIP}. Thus {\bf compression=GZIP1} would give
174 minimum compression but the fastest algorithm, and {\bf compression=GZIP9}
175 would give the highest level of compression, but requires more computation.
176 According to the GZIP documentation, compression levels greater than 6
177 generally give very little extra compression and are rather CPU intensive.
179 \item [signature=SHA1]
180 \index[fd]{signature }
181 An SHA1 signature will be computed for all The SHA1 algorithm is purported to
182 be some what slower than the MD5 algorithm, but at the same time is
183 significantly better from a cryptographic point of view (i.e. much fewer
184 collisions, much lower probability of being hacked.) It adds four more bytes
185 than the MD5 signature. We strongly recommend that either this option or MD5
186 be specified as a default for all files. Note, only one of the two options
187 MD5 or SHA1 can be computed for any file.
189 \item [signature=MD5]
190 \index[fd]{signature }
191 An MD5 signature will be computed for all files saved. Adding this option
192 generates about 5\% extra overhead for each file saved. In addition to the
193 additional CPU time, the MD5 signature adds 16 more bytes per file to your
194 catalog. We strongly recommend that this option or the SHA1 option be
195 specified as a default for all files.
197 \item [verify=\lt{}options\gt{}]
199 The options letters specified are used when running a {\bf Verify
200 Level=Catalog} as well as the {\bf DiskToCatalog} level job. The options
201 letters may be any combination of the following:
209 compare the permission bits
212 compare the number of links
224 compare the access time
227 compare the modification time (st\_mtime)
230 compare the change time (st\_ctime)
233 report file size decreases
236 compare the MD5 signature
239 compare the SHA1 signature
242 A useful set of general options on the {\bf Level=Catalog} or {\bf
243 Level=DiskToCatalog} verify is {\bf pins5} i.e. compare permission bits,
244 inodes, number of links, size, and MD5 changes.
248 If set to {\bf yes} (the default), {\bf Bacula} will remain on a single file
249 system. That is it will not backup file systems that are mounted on a
250 subdirectory. If you wish to backup multiple filesystems, you can explicitly
251 list each file system you want saved. Otherwise, if you set the onefs option
252 to {\bf no}, Bacula will backup all mounted file systems (i.e. traverse mount
253 points) that are found within the {\bf FileSet}. Thus if you have NFS or
254 Samba file systems mounted on a directory listed in your FileSet, they will
255 also be backed up. Normally, it is preferable to set {\bf onefs=yes} and to
256 explicitly name each filesystem you want backed up. Explicitly naming the
257 filesystems you want backed up avoids the possibility of getting into a
258 infinite loop recursing filesystems. See the example below for more details.
262 \item [portable=yes|no]
263 \index[dir]{portable }
264 If set to {\bf yes} (default is {\bf no}), the Bacula File daemon will backup
265 Win32 files in a portable format, but not all Win32 file attributes will be
266 saved and restored. By default, this option is set to {\bf no}, which means
267 that on Win32 systems, the data will be backed up using Windows API calls and
268 on WinNT/2K/XP, all the security and ownership attributes will be properly
269 backed up (and restored). However this format is not portable to other
270 systems -- e.g. Unix, Win95/98/Me. When backing up Unix systems, this option
271 is ignored, and unless you have a specific need to have portable backups, we
272 recommend accept the default ({\bf no}) so that the maximum information
273 concerning your files is saved.
275 \item [recurse=yes|no]
277 If set to {\bf yes} (the default), Bacula will recurse (or descend) into all
278 subdirectories found unless the directory is explicitly excluded using an
279 {\bf exclude} definition. If you set {\bf recurse=no}, Bacula will save the
280 subdirectory entries, but not descend into the subdirectories, and thus will
281 not save the files or directories contained in the subdirectories. Normally,
282 you will want the default ({\bf yes}).
284 \item [sparse=yes|no]
286 Enable special code that checks for sparse files such as created by
287 ndbm. The default is {\bf no}, so no checks are made for sparse files.
288 You may specify {\bf sparse=yes} even on files that are not sparse file.
289 No harm will be done, but there will be a small additional overhead to
290 check for buffers of all zero, and a small additional amount of space on
291 the output archive will be used to save the seek address of each
292 non-zero record read.
294 {\bf Restrictions:} Bacula reads files in 32K buffers. If the whole
295 buffer is zero, it will be treated as a sparse block and not written to
296 tape. However, if any part of the buffer is non-zero, the whole buffer
297 will be written to tape, possibly including some disk sectors (generally
298 4098 bytes) that are all zero. As a consequence, Bacula's detection of
299 sparse blocks is in 32K increments rather than the system block size.
300 If anyone considers this to be a real problem, please send in a request
301 for change with the reason.
303 If you are not familiar with sparse files, an example is say a file
304 where you wrote 512 bytes at address zero, then 512 bytes at address 1
305 million. The operating system will allocate only two blocks, and the
306 empty space or hole will have nothing allocated. However, when you read
307 the sparse file and read the addresses where nothing was written, the OS
308 will return all zeros as if the space were allocated, and if you backup
309 such a file, a lot of space will be used to write zeros to the volume.
310 Worse yet, when you restore the file, all the previously empty space
311 will now be allocated using much more disk space. By turning on the
312 {\bf sparse} option, Bacula will specifically look for empty space in
313 the file, and any empty space will not be written to the Volume, nor
314 will it be restored. The price to pay for this is that Bacula must
315 search each block it reads before writing it. On a slow system, this
316 may be important. If you suspect you have sparse files, you should
317 benchmark the difference or set sparse for only those files that are
322 \item [readfifo=yes|no]
323 \index[fd]{readfifo }
324 If enabled, tells the Client to read the data on a backup and write the
325 data on a restore to any FIFO (pipe) that is explicitly mentioned in the
326 FileSet. In this case, you must have a program already running that
327 writes into the FIFO for a backup or reads from the FIFO on a restore.
328 This can be accomplished with the {\bf RunBeforeJob} directive. If this
329 is not the case, Bacula will hang indefinitely on reading/writing the
330 FIFO. When this is not enabled (default), the Client simply saves the
331 directory entry for the FIFO.
333 \item [mtimeonly=yes|no]
334 \index[dir]{mtimeonly }
335 If enabled, tells the Client that the selection of files during
336 Incremental and Differential backups should based only on the st\_mtime
337 value in the stat() packet. The default is {\bf no} which means that
338 the selection of files to be backed up will be based on both the
339 st\_mtime and the st\_ctime values. In general, it is not recommended
342 \item [keepatime=yes|no]
343 \index[dir]{keepatime }
344 The default is {\bf no}. When enabled, Bacula will reset the st\_atime
345 (access time) field of files that it backs up to their value prior to
346 the backup. This option is not generally recommended as there are very
347 few programs that use st\_atime, and the backup overhead is increased
348 because of the additional system call necessary to reset the times.
349 (I'm not sure this works on Win32).
351 \item [wild=\lt{}string\gt{}]
353 Specifies a wild-card string to be applied to the filenames and
354 directory names. Note, if {\bf Exclude} is not enabled, the wild-card
355 will select which files are to be included. If {\bf Exclude=yes} is
356 specified, the wild-card will select which files are to be excluded.
357 Multiple wild-card directives may be specified, and they will be applied
358 in turn until the first one that matches. Note, if you exclude a
359 directory, no files or directories below it will be matched.
361 \item [wildfile=\lt{}string\gt{}]
362 \index[dir]{wildfile }
363 Specifies a wild-card string to be applied to filenames only. No
364 directories will be matched by this directive. Note, if {\bf Exclude}
365 is not enabled, the wild-card will select which files are to be
366 included. If {\bf Exclude=yes} is specified, the wild-card will select
367 which files are to be excluded. Multiple wild-card directives may be
368 specified, and they will be applied in turn until the first one that
371 \item [wilddir=\lt{}string\gt{}]
372 \index[dir]{wilddir }
373 Specifies a wild-card string to be applied to directory names only. No
374 filenames will be matched by this directive. Note, if {\bf Exclude} is
375 not enabled, the wild-card will select directories files are to be
376 included. If {\bf Exclude=yes} is specified, the wild-card will select
377 which files are to be excluded. Multiple wild-card directives may be
378 specified, and they will be applied in turn until the first one that
379 matches. Note, if you exclude a directory, no files or directories
380 below it will be matched.
383 \item [regex=\lt{}string\gt{}]
385 Specifies a POSIX extended regular expression to be applied to the
386 filenames and directory names.
387 This directive is available in version 1.35 and later. If {\bf
388 Exclude} is not enabled, the regex will select which files are to be
389 included. If {\bf Exclude=yes} is specified, the regex will select
390 which files are to be excluded. Multiple regex directives may be
391 specified within an Options resource, and they will be applied in turn
392 until the first one that matches. Note, if you exculde a
393 directory, no files or directories below it will be matched.
395 \item [regexfile=\lt{}string\gt{}]
396 \index[dir]{regexfile }
397 Specifies a POSIX extended regular expression to be applied to filenames
398 only. No directories will be matched by this directive. Note, if {\bf
399 Exclude} is not enabled, the regex will select which files are to be
400 included. If {\bf Exclude=yes} is specified, the regex will select
401 which files are to be excluded. Multiple regex directives may be
402 specified, and they will be applied in turn until the first one that
405 \item [regexdir=\lt{}string\gt{}]
406 \index[dir]{regexdir }
407 Specifies a POSIX extended regular expression to be applied to directory
408 names only. No filenames will be matched by this directive. Note, if
409 {\bf Exclude} is not enabled, the regex will select directories
410 files are to be included. If {\bf Exclude=yes} is specified, the
411 regex will select which files are to be excluded. Multiple
412 regex directives may be specified, and they will be applied in turn
413 until the first one that matches. Note, if you exclude a directory, no
414 files or directories below it will be matched.
416 \item [exclude=yes|no]
417 \index[dir]{exclude }
418 The default is {\bf no}. When enabled, any files matched within the Options
419 will be excluded from the backup.
423 \item [aclsupport=yes|no]
424 \index[dir]{aclsupport }
425 The default is {\bf no}. If this option is set to yes, and you have the
426 POSIX {\bf libacl} installed on your system, Bacula will backup the file
427 and directory UNIX Access Control Lists (ACL) as defined in IEEE Std
428 1003.1e draft 17 and ``POSIX.1e'' (abandoned). This feature is
429 available on UNIX only and depends on the ACL library. Bacula is
430 automatically compiled with ACL support if the {\bf libacl} library is
431 installed on your system (shown in config.out). While restoring the
432 files Bacula will try to restore the ACLs, if there is no ACL support
433 available on the system, Bacula restores the files and directories but
434 not the ACL information. Please note, if you backup an EXT3 or XFS
435 filesystem with ACLs, then you restore them to a different filesystem
436 (perhaps reiserfs) that does not have ACLs, the ACLs will be ignored.
438 \item [ignore case=yes|no]
439 \index[dir]{ignore case }
440 The default is {\bf no}, except on Windows systems where the default
441 is {\bf yes}. When this directive is set to {\bf yes} all the case
442 of character will be ignored in wild-card and regex comparisons.
443 That is an uppercase A will match a lowercase a.
445 \item [fstype=filesystem-type]
447 This option allows you to select files and directories by the
448 filesystem type. The permitted filesystem-type names are:
450 ext2, jfs, ntfs, proc, reiserfs, xfs, usbdevfs, sysfs, smbfs,
451 iso9660. For ext3 systems, use ext2.
453 You may have multiple Fstype directives, and thus permit matching
454 of multiple filesystem types within a single Options resource. If
455 the type specified on the fstype directive does not match the
456 filesystem for a particular directive, that directory will not be
457 backed up. This directive can be used to prevent backing up
458 non-local filesystems.
460 This option is not implemented in Win32 systems.
463 \item [hfsplussupport=yes|no]
464 \index[dir]{hfsplussupport }
465 This option allows you to turn on support for Mac OSX HFS plus
470 {\bf \lt{}file-list\gt{}} is a list of directory and/or filename names
471 specified with a {\bf File =} directive. To include names containing spaces,
472 enclose the name between double-quotes.
474 There are a number of special cases when specifying directories and files in a
475 {\bf file-list}. They are:
478 \item Any name preceded by an at-sign (@) is assumed to be the name of a
479 file, which contains a list of files each preceded by a ``File =''. The
480 named file is read once when the configuration file is parsed during the
481 Director startup. Note, that the file is read on the Director's machine
482 and not on the Client's. In fact, the @filename can appear anywhere
483 within the conf file where a token would be read, and the contents of
484 the named file will be logically inserted in the place of the @filename.
485 What must be in the file depends on the location the @filename is
486 specified in the conf file. For example:
491 Options { compression=GZIP }
492 @/home/files/my-files
497 \item Any name beginning with a vertical bar (|) is assumed to be the name of
498 a program. This program will be executed on the Director's machine at
499 the time the Job starts (not when the Director reads the configuration
500 file), and any output from that program will be assumed to be a list of
501 files or directories, one per line, to be included. This allows you to
502 have a job that for example includes all the local partitions even if
503 you change the partitioning by adding a disk. In general, you will need
504 to prefix your command or commands with a {\bf sh -c} so that they are
505 invoked by a shell. This will not be the case if you are invoking a
506 script as in the second example below. Also, you must take care to
507 escape (precede with a \textbackslash{}) wild-cards, shell character,
508 and to ensure that any spaces in your command are escaped as well. If
509 you use a single quotes (') within a double quote (``), Bacula will
510 treat everything between the single quotes as one field so it will not
511 be necessary to escape the spaces. In general, getting all the quotes
512 and escapes correct is a real pain as you can see by the next example.
513 As a consequence, it is often easier to put everything in a file and
514 simply use the file name within Bacula. In that case the {\bf sh -c}
515 will not be necessary providing the first line of the file is {\bf
524 Options { signature = SHA1 }
525 File = "|sh -c 'df -l | grep \"^/dev/hd[ab]\" | grep -v \".*/tmp\" \
526 | awk \"{print \\$6}\"'"
531 will produce a list of all the local partitions on a RedHat Linux system.
532 Note, the above line was split, but should normally be written on one line.
533 Quoting is a real problem because you must quote for Bacula which consists of
534 preceding every \textbackslash{} and every '' with a \textbackslash{}, and
535 you must also quote for the shell command. In the end, it is probably easier
536 just to execute a small file with:
544 File = "|my_partitions"
549 where my\_partitions has:
554 df -l | grep "^/dev/hd[ab]" | grep -v ".*/tmp" \
559 If the vertical bar (|) in front of my\_partitions is preceded by a
560 backslash as in \textbackslash{}|, the program will be executed on the
561 Client's machine instead of on the Director's machine.
562 Please note that if the filename is given within quotes, you
563 will need to use two slashes. An example, provided by John Donagher,
564 that backs up all the local UFS partitions on a remote system is:
569 Name = "All local partitions"
571 Options { signature=SHA1; onefs=yes; }
572 File = "\\|bash -c \"df -klF ufs | tail +2 | awk '{print \$6}'\""
578 The above requires two backslash characters after the double quote (one
579 preserves the next one). If you are a Linux user, just change the {\bf ufs}
580 to {\bf ext3} (or your preferred filesystem type), and you will be in
583 \item Any file-list item preceded by a less-than sign (\lt{}) will be taken
584 to be a file. This file will be read on the Director's machine at the time
585 the Job starts, and the data will be assumed to be a list of directories or
586 files, one per line, to be included. The names should start in column 1 and
587 should not be quoted even if they contain spaces. This feature allows you to
588 modify the external file and change what will be saved without stopping and
589 restarting Bacula as would be necessary if using the @ modifier noted above.
595 Options { signature = SHA1 }
596 File = "</home/files/local-filelist"
601 If you precede the less-than sign (\lt{}) with a backslash as in
602 \textbackslash{}\lt{}, the file-list will be read on the Client machine
603 instead of on the Director's machine. Please note that if the filename
604 is given within quotes, you will need to use two slashes.
609 Options { signature = SHA1 }
610 File = "\\</home/xxx/filelist-on-client"
615 \item If you explicitly specify a block device such as {\bf /dev/hda1}, then
616 Bacula (starting with version 1.28) will assume that this is a raw partition
617 to be backed up. In this case, you are strongly urged to specify a {\bf
618 sparse=yes} include option, otherwise, you will save the whole partition
619 rather than just the actual data that the partition contains. For example:
624 Options { signature=MD5; sparse=yes }
630 will backup the data in device /dev/hd6.
632 Ludovic Strappazon has pointed out that this feature can be used to backup a
633 full Microsoft Windows disk. Simply boot into the system using a Linux Rescue
634 disk, then load a statically linked Bacula as described in the
635 \ilink{ Disaster Recovery Using Bacula}{_ChapterStart38} chapter of
636 this manual. Then save the whole disk partition. In the case of a disaster,
637 you can then restore the desired partition by again booting with the rescue
638 disk and doing a restore of the partition.
639 \item If you explicitly specify a FIFO device name (created with mkfifo), and
640 you add the option {\bf readfifo=yes} as an option, Bacula will read the FIFO
641 and back its data up to the Volume. For example:
650 File = /home/abc/fifo
655 if {\bf /home/abc/fifo} is a fifo device, Bacula will open the fifo, read it,
656 and store all data thus obtained on the Volume. Please note, you must have a
657 process on the system that is writing into the fifo, or Bacula will hang,
658 and after one minute of waiting, Bacula will give up and go on to the next
659 file. The data read can be anything since Bacula treats it as a stream.
661 This feature can be an excellent way to do a ``hot'' backup of a very large
662 database. You can use the {\bf RunBeforeJob} to create the fifo and to start
663 a program that dynamically reads your database and writes it to the fifo.
664 Bacula will then write it to the Volume.
666 During the restore operation, the inverse is true, after Bacula creates the
667 fifo if there was any data stored with it (no need to explicitly list it or
668 add any options), that data will be written back to the fifo. As a
669 consequence, if any such FIFOs exist in the fileset to be restored, you must
670 ensure that there is a reader program or Bacula will block, and after one
671 minute, Bacula will time out the write to the fifo and move on to the next
675 \subsubsection*{FileSet Examples}
676 \index[general]{Examples!FileSet }
677 \index[general]{FileSet Examples}
678 \addcontentsline{toc}{subsection}{FileSet Examples}
680 The following is an example of a valid FileSet resource definition. Note, the
681 first Include pulls in the contents of the file {\bf /etc/backup.list} when
682 Bacula is started (i.e. the @).
694 File = @/etc/backup.list
703 File = /usr/lib/another_file
709 In the above example, all the files contained in /etc/backup.list will
710 be compressed with GZIP compression, an SHA1 signature will be computed on the
711 file's contents (its data), and sparse file handling will apply.
713 The two directories /root/myfile and /usr/lib/another\_file will also be saved
714 without any options, but all files in those directories with the extensions
715 {\bf .o} and {\bf .exe} will be excluded.
717 Let's say that you now want to exclude the directory /tmp. The simplest way
718 to do so is to add an exclude directive that lists /tmp. The example
719 above would then become:
731 File = @/etc/backup.list
740 File = /usr/lib/another_file
750 You can add wild-cards to the File directives listed in the Exclude
751 directory, but you need to take care because if you exclude a directory,
752 it and all files and directories below it will also be excluded.
754 Now lets take a slight variation on the above and suppose
755 you want to save all your whole filesystem except {\bf /tmp}.
756 The problem that comes up is that Bacula will not normally
757 cross from one filesystem to another.
758 Doing a {\bf df} command, you get the following output:
763 Filesystem 1k-blocks Used Available Use% Mounted on
764 /dev/hda5 5044156 439232 4348692 10% /
765 /dev/hda1 62193 4935 54047 9% /boot
766 /dev/hda9 20161172 5524660 13612372 29% /home
767 /dev/hda2 62217 6843 52161 12% /rescue
768 /dev/hda8 5044156 42548 4745376 1% /tmp
769 /dev/hda6 5044156 2613132 2174792 55% /usr
770 none 127708 0 127708 0% /dev/shm
771 //minimatou/c$ 14099200 9895424 4203776 71% /mnt/mmatou
772 lmatou:/ 1554264 215884 1258056 15% /mnt/matou
773 lmatou:/home 2478140 1589952 760072 68% /mnt/matou/home
774 lmatou:/usr 1981000 1199960 678628 64% /mnt/matou/usr
775 lpmatou:/ 995116 484112 459596 52% /mnt/pmatou
776 lpmatou:/home 19222656 2787880 15458228 16% /mnt/pmatou/home
777 lpmatou:/usr 2478140 2038764 311260 87% /mnt/pmatou/usr
778 deuter:/ 4806936 97684 4465064 3% /mnt/deuter
779 deuter:/home 4806904 280100 4282620 7% /mnt/deuter/home
780 deuter:/files 44133352 27652876 14238608 67% /mnt/deuter/files
784 And we see that there are a number of separate filesystems (/ /boot
785 /home /rescue /tmp and /usr not to mention mounted systems).
786 If you specify only {\bf /} in your Include list, Bacula will only save the
787 Filesystem {\bf /dev/hda5}. To save all filesystems except {\bf /tmp} with
788 out including any of the Samba or NFS mounted systems, and explicitly
789 excluding a /tmp, /proc, .journal, and .autofsck, which you will not want to
790 be saved and restored, you can use the following:
795 Name = Include_example
801 wildfile = \.autofsck
814 Since /tmp is on its own filesystem and it was not explicitly named in the
815 Include list, it is not really needed in the exclude list. It is better to
816 list it in the Exclude list for clarity, and in case the disks are changed so
817 that it is no longer in its own partition.
819 Now, lets assume you only want to backup .Z and .gz files and nothing
820 else. This is a bit trickier because Bacula by default will select
821 everything to backup, so we must exclude everything but .Z and .gz files.
822 If we take the first example above and make the obvious modifications
823 to it, we might come up with a FileSet that looks like this:
829 Include { !!!!!!!!!!!!
831 wildfile = *.Z example
832 wildfile = *.gz doesn't
841 The *.Z and *.gz files will indeed be backed up, but all other files
842 that are not matched by the Options directives will automatically
843 be backed up too (i.e. that is the default rule).
845 To accomplish what we want, we must explicitly exclude all other files.
846 We do this with the fillowing:
868 The ``trick'' here was to add a RegexFile expression that matches
869 all files. It does not match directory names, so all directories in
870 /myfile will be backed up (the directory entry) and any *.Z and *.gz
871 files contained in them. If you know that certain directories do
872 not contain any *.Z or *.gz files and you do not want the directory
873 entries backed up, you will need to explicitly exclude those directories.
874 Backing up a directory entries is not very expensive.
876 Bacula uses the system regex library and some of them are
877 different on different OSes. The above has been reported not to work
878 on FreeBSD. This can be tested by using the {\bf estimate job=job-name
879 listing} command in the console and adapting the RegexFile expression
880 appropriately. In a future version of Bacula, we will supply our own
881 Regex code to avoid such system dependencies.
883 Please be aware that allowing Bacula to traverse or change file systems can be
884 {\bf very} dangerous. For example, with the following:
898 you will be backing up an NFS mounted partition ({\bf /mnt/matou}), and since
899 {\bf onefs} is set to {\bf no}, Bacula will traverse file systems. Now if {\bf
900 /mnt/matou} has the current machine's file systems mounted, as is often the
901 case, you will get yourself into a recursive loop and the backup will never
904 \subsubsection*{Backing up Raw Partitions}
905 \index[general]{Backing up!Partitions }
906 \index[general]{Backing up Raw Partitions }
907 \addcontentsline{toc}{subsection}{Backing up Raw Partitions}
909 The following FileSet definition will backup a raw partition:
914 Name = "RawPartition"
916 Options { sparse=yes }
923 While backing up and restoring a raw partition, you should ensure that no
924 other process including the system is writing to that partition. As a
925 precaution, you are strongly urged to ensure that the raw partition is not
926 mounted or is mounted read-only. If necessary, this can be done using the {\bf
927 RunBeforeJob} directive.
930 \subsubsection*{Excluding Files and Directories}
931 \index[general]{Directories!Excluding Files and }
932 \index[general]{Excluding Files and Directories }
933 \addcontentsline{toc}{subsubsection}{Excluding Files and Directories}
935 You may also include full filenames or directory names in addition to using
936 wild-cards and {\bf Exclude=yes} in the Options resource as specified above by
937 simply including the files to be excluded in an Exclude resource within the
938 FileSet. For example:
943 Name = Exclusion_example
966 \subsubsection*{Windows FileSets}
967 \index[general]{Windows FileSets }
968 \index[general]{FileSets!Windows }
969 \addcontentsline{toc}{subsection}{Windows FileSets}
970 If you are entering Windows file names, the directory path may be preceded by
971 the drive and a colon (as in c:). However, the path separators must be
972 specified in Unix convention (i.e. forward slash (/)). If you wish to include
973 a quote in a file name, precede the quote with a backslash
974 (\textbackslash{}). For example you might use the following
975 for a Windows machine to backup the ``My Documents'' directory:
987 File = "c:/My Documents"
993 For exclude lists to work correctly on Windows, you must observe the following
997 \item Filenames are case sensitive, so you must use the correct case.
998 \item To exclude a directory, you must not have a trailing slash on the
1000 \item If you have spaces in your filename, you must enclose the entire name
1001 in double-quote characters (``). Trying to use a backslash before the space
1003 \item If you are using the old Exclude syntax (noted below), you may not
1004 specify a drive letter in the exclude. The new syntax noted above should work
1005 fine including driver letters.
1008 Thanks to Thiago Lima for summarizing the above items for us. If you are
1009 having difficulties getting includes or excludes to work, you might want to
1010 try using the {\bf estimate job=xxx listing} command documented in the
1011 \ilink{Console chapter}{estimate} of this manual.
1013 On Win32 systems, if you move a directory or file or rename a file into the
1014 set of files being backed up, and a Full backup has already been made, Bacula
1015 will not know there are new files to be saved during an Incremental or
1016 Differential backup (blame Microsoft, not me). To avoid this problem, please
1017 {\bf copy} any new directory or files into the backup area. If you do not have
1018 enough disk to copy the directory or files, move them, but then initiate a
1022 \paragraph*{A Windows Example FileSet}
1023 \index[general]{FileSet!Windows Example }
1024 \index[general]{Windows Example FileSet }
1025 \addcontentsline{toc}{paragraph}{Windows Example FileSet}
1027 The following example was contributed by Russell Howe. Please note that
1028 for presentation purposes, the lines beginning with Data and Internet
1029 have been wrapped and should included on the previous line with one
1034 This is my Windows 2000 fileset:
1036 Name = "Windows 2000"
1042 # Exclude Mozilla-based programs' file caches
1043 WildDir = "[A-Z]:/Documents and Settings/*/Application
1044 Data/*/Profiles/*/*/Cache"
1045 WildDir = "[A-Z]:/Documents and Settings/*/Application
1046 Data/*/Profiles/*/*/Cache.Trash"
1047 WildDir = "[A-Z]:/Documents and Settings/*/Application
1048 Data/*/Profiles/*/*/ImapMail"
1050 # Exclude user's registry files - they're always in use anyway.
1051 WildFile = "[A-Z]:/Documents and Settings/*/Local Settings/Application
1052 Data/Microsoft/Windows/usrclass.*"
1053 WildFile = "[A-Z]:/Documents and Settings/*/ntuser.*"
1055 # Exclude directories full of lots and lots of useless little files
1056 WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
1057 WildDir = "[A-Z]:/Documents and Settings/*/Recent"
1058 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
1059 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
1060 WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary
1063 # These are always open and unable to be backed up
1064 WildFile = "[A-Z]:/Documents and Settings/All Users/Application
1065 Data/Microsoft/Network/Downloader/qmgr[01].dat"
1067 # Some random bits of Windows we want to ignore
1068 WildFile = "[A-Z]:/WINNT/security/logs/scepol.log"
1069 WildDir = "[A-Z]:/WINNT/system32/config"
1070 WildDir = "[A-Z]:/WINNT/msdownld.tmp"
1071 WildDir = "[A-Z]:/WINNT/Internet Logs"
1072 WildDir = "[A-Z]:/WINNT/$Nt*Uninstall*"
1073 WildDir = "[A-Z]:/WINNT/sysvol"
1074 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB"
1075 WildFile = "[A-Z]:/WINNT/cluster/CLUSDB.LOG"
1076 WildFile = "[A-Z]:/WINNT/NTDS/edb.log"
1077 WildFile = "[A-Z]:/WINNT/NTDS/ntds.dit"
1078 WildFile = "[A-Z]:/WINNT/NTDS/temp.edb"
1079 WildFile = "[A-Z]:/WINNT/ntfrs/jet/log/edb.log"
1080 WildFile = "[A-Z]:/WINNT/ntfrs/jet/ntfrs.jdb"
1081 WildFile = "[A-Z]:/WINNT/ntfrs/jet/temp/tmp.edb"
1082 WildFile = "[A-Z]:/WINNT/system32/CPL.CFG"
1083 WildFile = "[A-Z]:/WINNT/system32/dhcp/dhcp.mdb"
1084 WildFile = "[A-Z]:/WINNT/system32/dhcp/j50.log"
1085 WildFile = "[A-Z]:/WINNT/system32/dhcp/tmp.edb"
1086 WildFile = "[A-Z]:/WINNT/system32/LServer/edb.log"
1087 WildFile = "[A-Z]:/WINNT/system32/LServer/TLSLic.edb"
1088 WildFile = "[A-Z]:/WINNT/system32/LServer/tmp.edb"
1089 WildFile = "[A-Z]:/WINNT/system32/wins/j50.log"
1090 WildFile = "[A-Z]:/WINNT/system32/wins/wins.mdb"
1091 WildFile = "[A-Z]:/WINNT/system32/wins/winstmp.mdb"
1093 # Temporary directories & files
1094 WildDir = "[A-Z]:/WINNT/Temp"
1095 WildDir = "[A-Z]:/temp"
1097 WildDir = "[A-Z]:/tmp"
1098 WildDir = "[A-Z]:/var/tmp"
1101 WildDir = "[A-Z]:/RECYCLER"
1104 WildFile = "[A-Z]:/pagefile.sys"
1106 # These are programs and are easier to reinstall than restore from
1108 WildDir = "[A-Z]:/cygwin"
1109 WildDir = "[A-Z]:/Program Files/Grisoft"
1110 WildDir = "[A-Z]:/Program Files/Java"
1111 WildDir = "[A-Z]:/Program Files/Java Web Start"
1112 WildDir = "[A-Z]:/Program Files/JavaSoft"
1113 WildDir = "[A-Z]:/Program Files/Microsoft Office"
1114 WildDir = "[A-Z]:/Program Files/Mozilla Firefox"
1115 WildDir = "[A-Z]:/Program Files/Mozilla Thunderbird"
1116 WildDir = "[A-Z]:/Program Files/mozilla.org"
1117 WildDir = "[A-Z]:/Program Files/OpenOffice*"
1120 # Our Win2k boxen all have C: and D: as the main hard drives.
1128 Note, the three line of the above Exclude were split to fit on the document
1129 page, they should be written on a single line in real use.
1131 \paragraph*{Windows NTFS Naming Considerations}
1132 \index[general]{Windows NTFS Naming Considerations }
1133 \index[general]{Considerations!Windows NTFS Naming }
1134 \addcontentsline{toc}{paragraph}{Windows NTFS Naming Considerations}
1136 NTFS filenames containing Unicode characters (i.e. \gt{} 0xFF) cannot be
1137 explicitly named at the moment. You must include such names by naming a higher
1138 level directory or a drive letter that does not contain Unicode characters.
1140 \subsubsection*{Testing Your FileSet}
1141 \index[general]{FileSet!Testing Your }
1142 \index[general]{Testing Your FileSet }
1143 \addcontentsline{toc}{subsection}{Testing Your FileSet}
1145 If you wish to get an idea of what your FileSet will really backup or if your
1146 exclusion rules will work correctly, you can test it by using the {\bf
1147 estimate} command in the Console program. See the
1148 \ilink{estimate command}{estimate} in the Console chapter of this
1151 \subsubsection*{The Old FileSet Resource}
1152 \index[general]{Resource!Old FileSet }
1153 \index[general]{Old FileSet Resource }
1154 \addcontentsline{toc}{subsection}{Old FileSet Resource}
1156 The old pre-version 1.34.3 FileSet Resource has been deprecated but may still
1157 work. You are encouraged to convert to using the new form since the old code
1158 will be removed sometime during 1.37 development.