5 \label{NewFeaturesChapter}
6 \index[general]{New Features}
8 This chapter presents the new features added to the development 2.5.x
9 versions to be released as Bacula version 3.0.0 near the end of 2008.
11 \section{Accurate Backup}
12 \index[general]{Accurate Backup}
14 As with most other backup programs, by default Bacula decides what files to backup for
15 Incremental and Differental backup by comparing the change (st\_ctime) and
16 modification (st\_mtime) times of the file to the time the last backup
17 completed. If one of those two times is later than the last backup time,
18 then the file will be backed up. This does not, however, permit tracking what
19 files have been deleted and will miss any file with an old time that may have
20 been restored to or moved onto the client filesystem.
22 \subsection{Accurate = \lt{}yes|no\gt{}}
23 If the {\bf Accurate = \lt{}yes|no\gt{}} directive is enabled (default no) in
24 the Job resource, the job will be run as an Accurate Job. For a {\bf Full}
25 backup, there is no difference, but for {\bf Differential} and {\bf
26 Incremental} backups, the Director will send a list of all previous files
27 backed up, and the File daemon will use that list to determine if any new files
28 have been added or or moved and if any files have been deleted. This allows
29 Bacula to make an accurate backup of your system to that point in time so that
30 if you do a restore, it will restore your system exactly. One note of caution
31 about using Accurate backup is that it requires more resources (CPU and memory)
32 on both the Director and the Client machines to create the list of previous
33 files backed up, to send that list to the File daemon, for the File daemon to
34 keep the list (possibly very big) in memory, and for the File daemon to do
35 comparisons between every file in the FileSet and the list.
39 \index[general]{Copy Jobs}
40 A new {\bf Copy} job type has been implemented. It is essentially
41 identical to the existing Migration feature with the exception that
42 the Job that is copied is left unchanged. This essentially creates
43 two identical copies of the same backup. The Copy Job runs without
44 using the File daemon by copying the data from the old backup Volume to
45 a different Volume in a different Pool. See the Migration documentation
46 for additional details. For copy Jobs there is a new selection criterium
47 named PoolUncopiedJobs which copies all jobs from a pool to an other
48 pool which were not copied before. Next to that the client, volume, job
49 or sql query are possible ways of selecting jobs which should be copied.
50 Selection types like smallestvolume, oldestvolume, pooloccupancy and
51 pooltime are probably more suited for migration jobs only. But we could
52 imagine some people have a valid use for those kind of copy jobs too.
54 A nice solution which can be build with the new copy jobs is what is
55 called the disk-to-disk-to-tape backup (DTDTT). A sample config could
56 look somethings like the one below:
60 Name = FullBackupsVirtualPool
62 Purge Oldest Volume = Yes
64 NextPool = FullBackupsTapePool
68 Name = FullBackupsTapePool
72 Volume Retention = 365 days
77 # Fake fileset for copy jobs
89 # Fake client for copy jobs
99 # Default template for a CopyDiskToTape Job
102 Name = CopyDiskToTape
104 Messages = StandardCopy
107 Selection Type = PoolUncopiedJobs
108 Maximum Concurrent Jobs = 10
110 Allow Duplicate Jobs = Yes
111 Allow Higher Duplicates = No
112 Cancel Queued Duplicates = No
113 Cancel Running Duplicates = No
118 Name = DaySchedule7:00
119 Run = Level=Full daily at 7:00
123 Name = CopyDiskToTapeFullBackups
125 Schedule = DaySchedule7:00
126 Pool = FullBackupsVirtualPool
127 JobDefs = CopyDiskToTape
131 The example above had 2 pool which are copied using the PoolUncopiedJobs selection
132 criteria. Normal Full backups go to the Virtual pool and are copied to the Tape pool
135 \section{ACL Updates}
136 The whole ACL code had been overhauled and in this version each platforms has different
137 streams for each type of acl available on such an platform. As acls between platforms
138 tend to be not that portable (most implement POSIX acls but some use an other draft or
139 a completely different format) we currently only allow certain platform specific ACL
140 streams to be decoded and restored on the same platform that they were created on.
141 The old code allowed to restore ACL cross platform but the comments already mention
142 that not being to wise. For backward compatability the new code will accept the 2 old
143 ACL streams and handle those with the platform specific handler. But for all new backups
144 it will save the ACLs using the new streams.
146 Currently the following platforms support ACLs:
150 \item {\bf Darwin/OSX}
159 Currently we support the following ACL types (these ACL streams use a reserved part
160 of the stream numbers):
163 \item {\bf STREAM\_ACL\_AIX\_TEXT} 1000 AIX specific string representation from acl\_get
164 \item {\bf STREAM\_ACL\_DARWIN\_ACCESS\_ACL\_T} 1001 Darwin (OSX) specific acl\_t string
165 representation from acl\_to\_text (POSIX acl)
166 \item {\bf STREAM\_ACL\_FREEBSD\_DEFAULT\_ACL\_T} 1002 FreeBSD specific acl\_t
167 string representation from acl\_to\_text (POSIX acl) for default acls.
168 \item {\bf STREAM\_ACL\_FREEBSD\_ACCESS\_ACL\_T} 1003 FreeBSD specific acl\_t
169 string representation from acl\_to\_text (POSIX acl) for access acls.
170 \item {\bf STREAM\_ACL\_HPUX\_ACL\_ENTRY} 1004 HPUX specific acl\_entry string representation
171 from acltostr (POSIX acl)
172 \item {\bf STREAM\_ACL\_IRIX\_DEFAULT\_ACL\_T} 1005 IRIX specific acl\_t string representation
173 from acl\_to\_text (POSIX acl) for default acls.
174 \item {\bf STREAM\_ACL\_IRIX\_ACCESS\_ACL\_T} 1006 IRIX specific acl\_t string representation
175 from acl\_to\_text (POSIX acl) for access acls.
176 \item {\bf STREAM\_ACL\_LINUX\_DEFAULT\_ACL\_T} 1007 Linux specific acl\_t string representation
177 from acl\_to\_text (POSIX acl) for default acls.
178 \item {\bf STREAM\_ACL\_LINUX\_ACCESS\_ACL\_T} 1008 Linux specific acl\_t string representation
179 from acl\_to\_text (POSIX acl) for access acls.
180 \item {\bf STREAM\_ACL\_TRU64\_DEFAULT\_ACL\_T} 1009 Tru64 specific acl\_t string representation
181 from acl\_to\_text (POSIX acl) for default acls.
182 \item {\bf STREAM\_ACL\_TRU64\_DEFAULT\_DIR\_ACL\_T} 1010 Tru64 specific acl\_t string representation
183 from acl\_to\_text (POSIX acl) for default acls.
184 \item {\bf STREAM\_ACL\_TRU64\_ACCESS\_ACL\_T} 1011 Tru64 specific acl\_t string representation
185 from acl\_to\_text (POSIX acl) for access acls.
186 \item {\bf STREAM\_ACL\_SOLARIS\_ACLENT\_T} 1012 Solaris specific aclent\_t string representation
187 from acltotext or acl\_totext (POSIX acl)
188 \item {\bf STREAM\_ACL\_SOLARIS\_ACE\_T} 1013 Solaris specific ace\_t string representation from
189 from acl\_totext (NFSv4 or ZFS acl)
192 In future versions we might support conversion functions from one type of acl into an other
193 for types that are either the same or easily convertable. For now the streams are seperate
194 and restoring them on a platform that doesn't recognize them will give you a warning.
196 \section{Extended Attributes}
197 Something that was on the project list for some time is now implemented for platforms
198 that support a similar kind of interface. Its the support for backup and restore of
199 so called extended attributes. As extended attributes are so platform specific these
200 attributes are saved in seperate streams for each platform. Restores can only be performed
201 on the same platform the backup was done. There is support for all types of extended
202 attributes, but restoring from one type of filesystem onto an other type of filesystem
203 on the same platform may lead to supprises. As extended attributes can contain any
204 type of data they are stored as a series of so called value-pairs. This data must be
205 seen as mostly binary and is stored as such. As security labels from selinux are
206 also extended attributes this option also stores those labels and no specific
207 code is enabled for handling selinux security labels.
209 Currently the following platforms support extended attributes:
211 \item {\bf Darwin/OSX}
217 On linux acls are also extended attributes, as such when you enable ACLs on a Linux
218 platform it will NOT save the same data twice e.g. it will save the ACLs and not
219 the same exteneded attribute.
221 To enable the backup of extended attributes please add the following to your fileset
236 \section{Shared objects}
237 A default build of Bacula will now create the libraries as shared objects
238 (.so) rather than static libraries as was previously the case.
239 The shared libraries are built using {\bf libtool} so it should be quite
242 An important advantage of using shared objects is that on a machine with the
243 Directory, File daemon, the Storage daemon, and a console, you will have only one copy
244 of the code in memory rather than four copies. Also the total size of the
245 binary release is smaller since the library code appears only once rather than
246 once for every program that uses it; this results in significant reduction in
247 the size of the binaries particularly for the utility tools.
249 In order for the system loader to find the shared objects when loading
250 the Bacula binaries, the Bacula shared objects must either be in a shared object
251 directory known to the loader (typically /usr/lib) or they must be in the directory
252 that may be specified on the {\bf ./configure} line using the
253 {\bf {-}{-}libdir} option as:
256 ./configure --libdir=/full-path/dir
259 the default is /usr/lib. If {-}{-}libdir is specified, there should be
260 no need to modify your loader configuration provided that
261 the shared objects are installed in that directory (Bacula
262 does this with the make install command). The shared objects
263 that Bacula references are:
272 These files are symbolically linked to the real shared object file,
273 which has a version number to permit running multiple versions of
274 the libraries if desired (not normally the case).
276 If you have problems with libtool or you wish to use the old
277 way of building static libraries, you can do so by disabling
278 libtool on the configure command line with:
281 ./configure --disable-libtool
285 \section{Virtual Backup (Vbackup)}
286 \index[general]{Virtual Backup}
287 \index[general]{Vbackup}
289 Bacula's virtual backup feature is often called Synthetic Backup or
290 Consolidation in other backup products. It permits you to consolidate
291 the previous Full backup plus the most recent Differential backup and any
292 subsequent Incremental backups into a new Full backup. This is accomplished
293 without contacting the client by reading the previous backup data and
294 writing it to a volume in a different pool.
296 In some respects the Vbackup feature works similar to a Migration job, in
297 that Bacula normally reads the data from the pool specified in the
298 Job resource, and writes it to the {\bf Next Pool} specified in the
299 Job resource. The input Storage resource and the Output Storage resource
302 The Vbackup is enabled on a Job by Job in the Job resource by specifying
303 a level of {\bf VirtualFull}.
305 A typical Job resource definition might look like the following:
319 # Default pool definition
323 Recycle = yes # Automatically recycle Volumes
324 AutoPrune = yes # Prune expired volumes
325 Volume Retention = 365d # one year
333 Recycle = yes # Automatically recycle Volumes
334 AutoPrune = yes # Prune expired volumes
335 Volume Retention = 365d # one year
336 Storage = DiskChanger
339 # Definition of file storage device
346 Maximum Concurrent Jobs = 5
349 # Definition of DDS Virtual tape disk storage device
352 Address = localhost # N.B. Use a fully qualified name here
355 Media Type = DiskChangerMedia
356 Maximum Concurrent Jobs = 4
361 Then in bconsole or via a Run schedule, you would run the job as:
364 run job=MyBackup level=Full
365 run job=MyBackup level=Incremental
366 run job=MyBackup level=Differential
367 run job=MyBackup level=Incremental
368 run job=MyBackup level=Incremental
371 So providing there were changes between each of those jobs, you would end up
372 with a Full backup, a Differential, which includes the first Incremental
373 backup, then two Incremental backups. All the above jobs would be written to
374 the {\bf Default} pool.
376 To consolidate those backups into a new Full backup, you would run the
380 run job=MyBackup level=VirtualFull
383 And it would produce a new Full backup without using the client, and the output
384 would be written to the {\bf Full} Pool which uses the Diskchanger Storage.
386 If the Virtual Full is run, and there are no prior Jobs, the Virtual Full will
389 \section{Duplicate Job Control}
390 \index[general]{Duplicate Jobs}
391 The new version of Bacula provides four new directives that
392 give additional control over what Bacula does if duplicate jobs
393 are started. A duplicate job in the sense we use it here means
394 a second or subsequent job with the same name starts. This
395 happens most frequently when the first job runs longer than expected because no
398 The four directives each take as an argument a {\bf yes} or {\bf no} value and
399 are specified in the Job resource.
403 \subsection{Allow Duplicate Jobs = \lt{}yes|no\gt{}}
404 If this directive is enabled duplicate jobs will be run. If
405 the directive is set to {\bf no} (default) then only one job of a given name
406 may run at one time, and the action that Bacula takes to ensure only
407 one job runs is determined by the other directives (see below).
409 \subsection{Allow Higher Duplicates = \lt{}yes|no\gt{}}
410 If this directive is set to {\bf yes} (default) the job with a higher
411 priority (lower priority number) will be permitted to run. If the
412 priorities of the two jobs are the same, the outcome is determined by
413 other directives (see below).
415 \subsection{Cancel Queued Duplicates = \lt{}yes|no\gt{}}
416 If this directive is set to {\bf yes} (default) any job that is
417 already queued to run but not yet running will be canceled.
419 \subsection{Cancel Running Duplicates = \lt{}yes|no\gt{}}
420 If this directive is set to {\bf yes} any job that is already running
421 will be canceled. The default is {\bf no}.
424 \section{TLS Authentication}
425 \index[general]{TLS Authentication}
426 In Bacula version 2.5.x and later, in addition to the normal Bacula
427 CRAM-MD5 authentication that is used to authenticate each Bacula
428 connection, you can specify that you want TLS Authentication as well,
429 which will provide more secure authentication.
431 This new feature uses Bacula's existing TLS code (normally used for
432 communications encryption) to do authentication. To use it, you must
433 specify all the TLS directives normally used to enable communications
434 encryption (TLS Enable, TLS Verify Peer, TLS Certificate, ...) and
437 \subsection{TLS Authenticate = yes}
439 TLS Authenticate = yes
442 in the main daemon configuration resource (Director for the Director,
443 Client for the File daemon, and Storage for the Storage daemon).
445 When {\bf TLS Authenticate} is enabled, after doing the CRAM-MD5
446 authentication, Bacula will do the normal TLS authentication, then TLS
447 encryption will be turned off.
449 If you want to encrypt communications data, do not turn on {\bf TLS
452 \section{bextract non-portable Win32 data}
453 \index[general]{bextract handles Win32 non-portable data}
454 {\bf bextract} has been enhanced to be able to restore
455 non-portable Win32 data to any OS. Previous versions were
456 unable to restore non-portable Win32 data to machines that
457 did not have the Win32 BackupRead and BackupWrite API calls.
459 \section{State File updated at Job Termination}
460 \index[general]{State File}
461 In previous versions of Bacula, the state file, which provides a
462 summary of previous jobs run in the {\bf status} command output was
463 updated only when Bacula terminated, thus if the daemon crashed, the
464 state file might not contain all the run data. This version of
465 the Bacula daemons updates the state file on each job termination.
467 \section{MaxFullInterval = \lt{}time-interval\gt{}}
468 \index[general]{MaxFullInterval}
469 The new Job resource directive {\bf Max Full Interval = \lt{}time-interval\gt{}}
470 can be used to specify the maximum time interval between {\bf Full} backup
471 jobs. When a job starts, if the time since the last Full backup is
472 greater than the specified interval, and the job would normally be an
473 {\bf Incremental} or {\bf Differential}, it will be automatically
474 upgraded to a {\bf Full} backup.
476 \section{MaxDiffInterval = \lt{}time-interval\gt{}}
477 \index[general]{MaxDiffInterval}
478 The new Job resource directive {\bf Max Diff Interval = \lt{}time-interval\gt{}}
479 can be used to specify the maximum time interval between {\bf Differential} backup
480 jobs. When a job starts, if the time since the last Differential backup is
481 greater than the specified interval, and the job would normally be an
482 {\bf Incremental}, it will be automatically
483 upgraded to a {\bf Differential} backup.
485 \section{Honor No Dump Flag = \lt{}yes|no\gt{}}
486 \index[general]{MaxDiffInterval}
487 On FreeBSD systems, each file has a {\bf no dump flag} that can be set
488 by the user, and when it is set it is an indication to backup programs
489 to not backup that particular file. This version of Bacula contains a
490 new Options directive within a FileSet resource, which instructs Bacula to
491 obey this flag. The new directive is:
494 Honor No Dump Flag = yes|no
497 The default value is {\bf no}.
500 \section{Exclude Dirs Containing = \lt{}filename-string\gt{}}
501 \index[general]{IgnoreDir}
502 The {\bf ExcludeDirsContaining = \lt{}filename\gt{}} is a new directive that can be added to the Include
503 section of the FileSet resource. If the specified
504 filename is found on the Client in any directory to be backed up,
505 the whole directory will be ignored (not backed up).
509 # List of files to be backed up
517 Exclude Dirs Containing = .excludeme
522 But in /home, there may be hundreds of directories of users and some
523 people want to indicate that they don't want to have certain
524 directories backed up. For example, with the above FileSet, if
525 the user or sysadmin creates a file named {\bf .excludeme} in
526 specific directories, such as
529 /home/user/www/cache/.excludeme
530 /home/user/temp/.excludeme
533 then Bacula will not backup the two directories named:
540 NOTE: subdirectories will not be backed up. That is, the directive
541 applies to the two directories in question and any children (be they
542 files, directories, etc).
546 \section{Bacula Plugins}
547 \index[general]{Plugin}
548 Support for shared object plugins has been implemented in the Linux
549 (and Unix) File daemon. The API will be documented separately in
550 the Developer's Guide or in a new document. For the moment, there is
551 a single plugin named {\bf bpipe} that allows an external program to
552 get control to backup and restore a file.
554 Plugins are also planned (partially implemented) in the Director and the
555 Storage daemon. The code is also implemented to work on Win32 machines,
556 but it has not yet been tested.
558 \subsection{Plugin Directory}
559 Each daemon (DIR, FD, SD) has a new {\bf Plugin Directory} directive that may
560 be added to the daemon definition resource. The directory takes a quoted
561 string argument, which is the name of the directory in which the daemon can
562 find the Bacula plugins. If this directive is not specified, Bacula will not
563 load any plugins. Since each plugin has a distinctive name, all the daemons
564 can share the same plugin directory.
568 \subsection{Plugin Options}
569 The {\bf Plugin Options} directive takes a quoted string
570 arguement (after the equal sign) and may be specified in the
571 Job resource. The options specified will be passed to the plugin
572 when it is run. The value defined in the Job resource can be modified
573 by the user when he runs a Job via the {\bf bconsole} command line
576 Note: this directive may be specified, but it is not yet passed to
577 the plugin (i.e. not fully implemented).
579 \subsection{Plugin Options ACL}
580 The {\bf Plugin Options ACL} directive may be specified in the
581 Director's Console resource. It functions as all the other ACL commands
582 do by permitting users running restricted consoles to specify a
583 {\bf Plugin Options} that overrides the one specified in the Job
584 definition. Without this directive restricted consoles may not modify
587 \subsection{Plugin = \lt{}plugin-command-string\gt{}}
588 The {\bf Plugin} directive is specified in the Include section of
589 a FileSet resource where you put your {\bf File = xxx} directives.
605 In the above example, when the File daemon is processing the directives
606 in the Include section, it will first backup all the files in {\bf /home}
607 then it will load the plugin named {\bf bpipe} (actually bpipe-dir.so) from
608 the Plugin Directory. The syntax and semantics of the Plugin directive
609 require the first part of the string up to the colon (:) to be the name
610 of the plugin. Everything after the first colon is ignored by the File daemon but
611 is passed to the plugin. Thus the plugin writer may define the meaning of the
612 rest of the string as he wishes.
614 Please see the next section for information about the {\bf bpipe} Bacula
617 \section{The bpipe Plugin}
618 The {\bf bpipe} plugin is provided in the directory src/plugins/fd/bpipe-fd.c of
619 the Bacula source distribution. When the plugin is compiled and linking into
620 the resulting dynamic shared object (DSO), it will have the name {\bf bpipe-fd.so}.
622 The purpose of the plugin is to provide an interface to any system program for
623 backup and restore. As specified above the {\bf bpipe} plugin is specified in
624 the Include section of your Job's FileSet resource. The full syntax of the
625 plugin directive as interpreted by the {\bf bpipe} plugin (each plugin is free
626 to specify the sytax as it wishes) is:
629 Plugin = "<field1>:<field2>:<field3>:<field4>"
634 \item {\bf field1} is the name of the plugin with the trailing {\bf -fd.so}
635 stripped off, so in this case, we would put {\bf bpipe} in this field.
637 \item {\bf field2} specifies the namespace, which for {\bf bpipe} is the
638 pseudo path and filename under which the backup will be saved. This pseudo
639 path and filename will be seen by the user in the restore file tree.
640 For example, if the value is {\bf /MYSQL/regress.sql}, the data
641 backed up by the plugin will be put under that "pseudo" path and filename.
642 You must be careful to choose a naming convention that is unique to avoid
643 a conflict with a path and filename that actually exists on your system.
645 \item {\bf field3} for the {\bf bpipe} plugin
646 specifies the "reader" program that is called by the plugin during
647 backup to read the data. {\bf bpipe} will call this program by doing a
650 \item {\bf field4} for the {\bf bpipe} plugin
651 specifies the "writer" program that is called by the plugin during
652 restore to write the data back to the filesystem.
655 Putting it all together, the full plugin directive line might look
659 Plugin = "bpipe:/MYSQL/regress.sql:mysqldump -f
660 --opt --databases bacula:mysql"
663 The directive has been split into two lines, but within the {\bf bacula-dir.conf} file
664 would be written on a single line.
666 This causes the File daemon to call the {\bf bpipe} plugin, which will write
667 its data into the "pseudo" file {\bf /MYSQL/regress.sql} by calling the
668 program {\bf mysqldump -f --opt --database bacula} to read the data during
669 backup. The mysqldump command outputs all the data for the database named
670 {\bf bacula}, which will be read by the plugin and stored in the backup.
671 During restore, the data that was backed up will be sent to the program
672 specified in the last field, which in this case is {\bf mysql}. When
673 {\bf mysql} is called, it will read the data sent to it by the plugn
674 then write it back to the same database from which it came ({\bf bacula}
677 The {\bf bpipe} plugin is a generic pipe program, that simply transmits
678 the data from a specified program to Bacula for backup, and then from Bacula to
679 a specified program for restore.
681 By using different command lines to {\bf bpipe},
682 you can backup any kind of data (ASCII or binary) depending
683 on the program called.
685 \section{Microsoft Exchange Server 2003/2007 Plugin}
687 \subsection{Concepts}
689 Although it is possible to backup Exchange using Bacula VSS the Exchange
690 plugin adds a good deal of functionality, because while Bacula VSS
691 completes a full backup (snapshot) of Exchange, it does
692 not support Incremental or Differential backups, restoring is more
693 complicated, and a single database restore is not possible.
695 Microsoft Exchange organises its storage into Storage Groups with
696 Databases inside them. A default installation of Exchange will have a
697 single Storage Group called 'First Storage Group', with two Databases
698 inside it, "Mailbox Store (SERVER NAME)" and
699 "Public Folder Store (SERVER NAME)",
700 which hold user email and public folders respectively.
702 In the default configuration, Exchange logs everything that happens to
703 log files, such that if you have a backup, and all the log files since,
704 you can restore to the present time. Each Storage Group has its own set
705 of log files and operates independently of any other Storage Groups. At
706 the Storage Group level, the logging can be turned off by enabling a
707 function called "Enable circular logging". At this time the Exchange
708 plugin will not function if this option is enabled.
710 The plugin allows backing up of entire storage groups, and the restoring
711 of entire storage groups or individual databases. Backing up and
712 restoring at the individual mailbox or email item is not supported but
713 can be simulated by use of the "Recovery" Storage Group (see below).
715 \subsection{Installing}
717 The Exchange plugin requires a DLL that is shipped with Microsoft
718 Exchanger Server called {\bf esebcli2.dll}. Assuming Exchange is installed
719 correctly the Exchange plugin should find this automatically and run
720 without any additional installation.
722 If the DLL can not be found automatically it will need to be copied into
723 the Bacula installation
724 directory (eg C:\verb+\+Program Files\verb+\+Bacula\verb+\+bin). The Exchange API DLL is
725 named esebcli2.dll and is found in C:\verb+\+Program Files\verb+\+Exchsrvr\verb+\+bin on a
726 default Exchange installation.
728 \subsection{Backup up}
730 To back up an Exchange server the Fileset definition must contain at
731 least {\bf Plugin = "exchange:/@EXCHANGE/Microsoft Information Store"} for
732 the backup to work correctly. The 'exchange:' bit tells Bacula to look
733 for the exchange plugin, the '@EXCHANGE' bit makes sure all the backed
734 up files are prefixed with something that isn't going to share a name
735 with something outside the plugin, and the 'Microsoft Information Store'
736 bit is required also. It is also possible to add the name of a storage
737 group to the "Plugin =" line, eg \\
738 {\bf Plugin = "exchange:/@EXCHANGE/Microsoft Information Store/First Storage Group"} \\
739 if you want only a single storage group backed up.
741 Additionally, you can suffix the 'Plugin =' directive with
742 ":notrunconfull" which will tell the plugin not to truncate the Exchange
743 database at the end of a full backup.
745 An Incremental or Differential backup will backup only the database logs
746 for each Storage Group by inspecting the "modified date" on each
747 physical log file. Because of the way the Exchange API works, the last
748 logfile backed up on each backup will always be backed up by the next
749 Incremental or Differential backup too. This adds 5MB to each
750 Incremental or Differential backup size but otherwise does not cause any
753 By default, a normal VSS fileset containing all the drive letters will
754 also back up the Exchange databases using VSS. This will interfere with
755 the plugin and Exchange's shared ideas of when the last full backup was
756 done, and may also truncate log files incorrectly. It is important,
757 therefore, that the Exchange database files be excluded from the backup,
758 although the folders the files are in should be included, or they will
759 have to be recreated manually if a baremetal restore is done.
764 File = C:/Program Files/Exchsrvr/mdbdata
765 Plugin = "exchange:..."
768 File = C:/Program Files/Exchsrvr/mdbdata/E00.chk
769 File = C:/Program Files/Exchsrvr/mdbdata/E00.log
770 File = C:/Program Files/Exchsrvr/mdbdata/E000000F.log
771 File = C:/Program Files/Exchsrvr/mdbdata/E0000010.log
772 File = C:/Program Files/Exchsrvr/mdbdata/E0000011.log
773 File = C:/Program Files/Exchsrvr/mdbdata/E00tmp.log
774 File = C:/Program Files/Exchsrvr/mdbdata/priv1.edb
779 The advantage of excluding the above files is that you can significantly
780 reduce the size of your backup since all the important Exchange files
781 will be properly saved by the Plugin.
784 \subsection{Restoring}
786 The restore operation is much the same as a normal Bacula restore, with
787 the following provisos:
790 \item The {\bf Where} restore option must not be specified
791 \item Each Database directory must be marked as a whole. You cannot just
792 select (say) the .edb file and not the others.
793 \item If a Storage Group is restored, the directory of the Storage Group
795 \item It is possible to restore only a subset of the available log files,
796 but they {\bf must} be contiguous. Exchange will fail to restore correctly
797 if a log file is missing from the sequence of log files
798 \item Each database to be restored must be dismounted and marked as "Can be
799 overwritten by restore"
800 \item If an entire Storage Group is to be restored (eg all databases and
801 logs in the Storage Group), then it is best to manually delete the
802 database files from the server (eg C:\verb+\+Program Files\verb+\+Exchsrvr\verb+\+mdbdata\verb+\+*)
803 as Exchange can get confused by stray log files lying around.
806 \subsection{Restoring to the Recovery Storage Group}
808 The concept of the Recovery Storage Group is well documented by
810 \elink{http://support.microsoft.com/kb/824126}{http://support.microsoft.com/kb/824126},
811 but to briefly summarize...
813 Microsoft Exchange allows the creation of an additional Storage Group
814 called the Recovery Storage Group, which is used to restore an older
815 copy of a database (e.g. before a mailbox was deleted) into without
816 messing with the current live data. This is required as the Standard and
817 Small Business Server versions of Exchange can not ordinarily have more
818 than one Storage Group.
820 To create the Recovery Storage Group, drill down to the Server in
821 Exchange System Manager, right click, and select
822 {\bf "New -> Recovery Storage Group..."}. Accept or change the file locations and click OK. On
823 the Recovery Storage Group, right click and select
824 {\bf "Add Database to Recover..."} and select the database you will be restoring.
826 In Bacula, select the Database and the log files, making sure to mark
827 the Storage Group directory itself too. Once you have selected the files
828 to back up, use the RegexWhere clause to remove the prefix of
829 "/@EXCHANGE/Microsoft Information Store/\lt{}storage group name\gt{}/" and
830 replace it with "/@EXCHANGE/Microsoft Information Store/Recovery Storage Group/".
831 Then run the restore.
835 This plugin is still being developed, so you should consider it
836 currently in BETA test, and thus use in a production environment
837 should be done only after very careful testing.
839 The "Enable Circular Logging" option cannot be enabled or the plugin
842 Exchange insists that a successful Full backup must have taken place if
843 an Incremental or Differential backup is desired, and the plugin will
844 fail if this is not the case. If a restore is done, Exchange will
845 require that a Full backup be done before an Incremental or Differential
848 The plugin will most likely not work well if another backup application
849 (eg NTBACKUP) is backing up the Exchange database, especially if the
850 other backup application is truncating the log files.
852 The Exchange plugin has not been tested with the {\bf Accurate} option, so
853 we recommend either carefully testing or that you avoid this option for
856 The Exchange plugin is not called during processing the bconsole {\bf estimate} command,
857 and so anything that would be backed up by the plugin will not be added
858 to the estimate total that is displayed.
861 \section{libdbi Framework}
862 As a general guideline, Bacula has support for a few catalog database drivers
863 (MySQL, PostgreSQL, SQLite)
864 coded natively by the Bacula team. With the libdbi implementation, which is a
865 Bacula driver that uses libdbi to access the catalog, we have an open field to
866 use many different kinds database engines following the needs of users.
868 The according to libdbi (http://libdbi.sourceforge.net/) project: libdbi
869 implements a database-independent abstraction layer in C, similar to the
870 DBI/DBD layer in Perl. Writing one generic set of code, programmers can
871 leverage the power of multiple databases and multiple simultaneous database
872 connections by using this framework.
874 Currently the libdbi driver in Bacula project only supports the same drivers
875 natively coded in Bacula. However the libdbi project has support for many
876 others database engines. You can view the list at
877 http://libdbi-drivers.sourceforge.net/. In the future all those drivers can be
878 supported by Bacula, however, they must be tested properly by the Bacula team.
880 Some of benefits of using libdbi are:
882 \item The possibility to use proprietary databases engines in which your
883 proprietary licenses prevent the Bacula team from developing the driver.
884 \item The possibility to use the drivers written for the libdbi project.
885 \item The possibility to use other database engines without recompiling Bacula
886 to use them. Just change one line in bacula-dir.conf
887 \item Abstract Database access, this is, unique point to code and profiling
888 catalog database access.
891 The following drivers have been tested:
893 \item PostgreSQL, with and without batch insert
894 \item Mysql, with and without batch insert
899 In the future, we will test and approve to use others databases engines
900 (proprietary or not) like DB2, Oracle, Microsoft SQL.
902 To compile Bacula to support libdbi we need to configure the code with the
903 --with-dbi and --with-dbi-driver=[database] ./configure options, where
904 [database] is the database engine to be used with Bacula (of course we can
905 change the driver in file bacula-dir.conf, see below). We must configure the
906 access port of the database engine with the option --with-db-port, because the
907 libdbi framework doesn't know the default access port of each database.
909 The next phase is checking (or configuring) the bacula-dir.conf, example:
913 dbdriver = dbi:mysql; dbaddress = 127.0.0.1; dbport = 3306
914 dbname = regress; user = regress; password = ""
918 The parameter {\bf dbdriver} indicates that we will use the driver dbi with a
919 mysql database. Currently the drivers supported by Bacula are: postgresql,
920 mysql, sqlite, sqlite3; these are the names that may be added to string "dbi:".
922 The following limitations apply when Bacula is set to use the libdbi framework:
923 - Not tested on the Win32 platform
924 - A little performance is lost if comparing with native database driver.
925 The reason is bound with the database driver provided by libdbi and the
926 simple fact that one more layer of code was added.
928 It is important to remember, when compiling Bacula with libdbi, the
929 following packages are needed:
931 \item libdbi version 1.0.0, http://libdbi.sourceforge.net/
932 \item libdbi-drivers 1.0.0, http://libdbi-drivers.sourceforge.net/
935 You can download them and compile them on your system or install the packages
936 from your OS distribution.
939 \section{Display Autochanger Content}
940 \index[general]{StatusSlots}
942 The {\bf status slots storage=\lt{}storage-name\gt{}} command displays
947 Slot | Volume Name | Status | Media Type | Pool |
948 ------+---------------+----------+-------------------+------------|
949 1 | 00001 | Append | DiskChangerMedia | Default |
950 2 | 00002 | Append | DiskChangerMedia | Default |
951 3*| 00003 | Append | DiskChangerMedia | Scratch |
956 If you an asterisk ({\bf *}) appears after the slot number, you must run an
957 {\bf update slots} command to synchronize autochanger content with your
960 \section{Miscellaneous}
961 \index[general]{Misc New Features}
963 \subsection{Allow Mixed Priority = \lt{}yes|no\gt{}}
964 This directive is only implemented in version 2.5 and later. When
965 set to {\bf yes} (default {\bf no}), this job may run even if lower
966 priority jobs are already running. This means a high priority job
967 will not have to wait for other jobs to finish before starting.
968 The scheduler will only mix priorities when all running jobs have
971 Note that only higher priority jobs will start early. Suppose the
972 director will allow two concurrent jobs, and that two jobs with
973 priority 10 are running, with two more in the queue. If a job with
974 priority 5 is added to the queue, it will be run as soon as one of
975 the running jobs finishes. However, new priority 10 jobs will not
976 be run until the priority 5 job has finished.
978 \subsection{Bootstrap File Directive -- FileRegex}
979 {\bf FileRegex} is a new command that can be added to the bootstrap
980 (.bsr) file. The value is a regular expression. When specified, only
981 matching filenames will be restored.
983 During a restore, if all File records are pruned from the catalog
984 for a Job, normally Bacula can restore only all files saved. That
985 is there is no way using the catalog to select individual files.
986 With this new command, Bacula will ask if you want to specify a Regex
987 expression for extracting only a part of the full backup.
989 \subsection{Solaris ZFS/NFSv4 ACLs}
990 This is an upgrade of the previous Solaris ACL backup code
991 to the new library format, which will backup both the old
992 POSIX(UFS) ACLs as well as the ZFS ACLs.
994 The new code can also restore POSIX(UFS) ACLs to a ZFS filesystem
995 (it will translate the POSIX(UFS)) ACL into a ZFS/NFSv4 one) it can also
996 be used to transfer from UFS to ZFS filesystems.
999 \subsection{Virtual Tape Emulation}
1000 We now have a Virtual Tape emulator that allows us to run though 99.9\% of
1001 the tape code but actually reading and writing to a disk file. Used with the
1002 \textbf{disk-changer} script, you can now emulate an autochanger with 10 drives
1003 and 700 slots. This feature is most useful in testing. It is enabled
1004 by using {\bf Device Type = vtape} in the Storage daemon's Device
1005 directive. This feature is only implemented on Linux machines.
1007 \subsection{Bat Enhancements}
1008 Bat (the Bacula Administration Tool) GUI program has been significantly
1009 enhanced and stabilized. In particular, there are new table based status
1010 commands; it can now be easily localized using Qt4 Linguist.
1012 The Bat communications protocol has been significantly enhanced to improve
1015 \subsection{RunScript Enhancements}
1016 The {\bf RunScript} resource has been enhanced to permit multiple
1017 commands per RunScript. Simply specify multiple {\bf Command} directives
1024 Command = "/bin/echo test"
1025 Command = "/bin/echo an other test"
1026 Command = "/bin/echo 3 commands in the same runscript"
1033 A new Client RunScript {\bf RunsWhen} keyword of {\bf AfterVSS} has been
1034 implemented, which runs the command after the Volume Shadow Copy has been made.
1036 Console commands can be specified within a RunScript by using:
1037 {\bf Console = \lt{}command\gt{}}, however, this command has not been
1038 carefully tested and debugged and is known to easily crash the Director.
1039 We would appreciate feedback. Due to the recursive nature of this command, we
1040 may remove it before the final release.
1042 \subsection{Status Enhancements}
1043 The bconsole {\bf status dir} output has been enhanced to indicate
1044 Storage daemon job spooling and despooling activity.
1046 \subsection{Connect Timeout}
1047 The default connect timeout to the File
1048 daemon has been set to 3 minutes. Previously it was 30 minutes.
1050 \subsection{ftruncate for NFS Volumes}
1051 If you write to a Volume mounted by NFS (say on a local file server),
1052 in previous Bacula versions, when the Volume was recycled, it was not
1053 properly truncated because NFS does not implement ftruncate (file
1054 truncate). This is now corrected in the new version because we have
1055 written code (actually a kind user) that deletes and recreates the Volume,
1056 thus accomplishing the same thing as a truncate.
1058 \subsection{Support for Ubuntu}
1059 The new version of Bacula now recognizes the Ubuntu (and Kubuntu)
1060 version of Linux, and thus now provides correct autostart routines.
1061 Since Ubuntu officially supports Bacula, you can also obtain any
1062 recent release of Bacula from the Ubuntu repositories.
1064 \subsection{Recycle Pool = \lt{}pool-name\gt{}}
1065 The new \textbf{RecyclePool} directive defines to which pool the Volume will
1066 be placed (moved) when it is recycled. Without this directive, a Volume will
1067 remain in the same pool when it is recycled. With this directive, it can be
1068 moved automatically to any existing pool during a recycle. This directive is
1069 probably most useful when defined in the Scratch pool, so that volumes will
1070 be recycled back into the Scratch pool.
1072 \subsection{FD Version}
1073 The File daemon to Director protocol now includes a version
1074 number, which although there is no visible change for users,
1075 will help us in future versions automatically determine
1076 if a File daemon is not compatible.
1078 \subsection{Max Run Sched Time = \lt{}time-period-in-seconds\gt{}}
1079 The time specifies the maximum allowed time that a job may run, counted from
1080 when the job was scheduled. This can be useful to prevent jobs from running
1081 during working hours. We can see it like \texttt{Max Start Delay + Max Run
1084 \subsection{Max Wait Time = \lt{}time-period-in-seconds\gt{}}
1086 Previous \textbf{MaxWaitTime} directives aren't working as expected, instead
1087 of checking the maximum allowed time that a job may block for a resource,
1088 those directives worked like \textbf{MaxRunTime}. Some users are reporting to
1089 use \textbf{Incr/Diff/Full Max Wait Time} to control the maximum run time of
1090 their job depending on the level. Now, they have to use
1091 \textbf{Incr/Diff/Full Max Run Time}. \textbf{Incr/Diff/Full Max Wait Time}
1092 directives are now deprecated.
1094 \subsection{Incremental|Differential Max Wait Time = \lt{}time-period-in-seconds\gt{}}
1095 Theses directives have been deprecated in favor of
1096 \texttt{Incremental|Differential Max Run Time}.
1098 \subsection{Max Run Time directives}
1099 Using \textbf{Full/Diff/Incr Max Run Time}, it's now possible to specify the
1100 maximum allowed time that a job can run depending on the level.
1102 \addcontentsline{lof}{figure}{Job time control directives}
1103 \includegraphics{\idir different_time.eps}
1105 \subsection{Statistics Enhancements}
1106 If you (or probably your boss) want to have statistics on your backups to
1107 provide some \textit{Service Level Agreement} indicators, you could use a few
1108 SQL queries on the Job table to report how many:
1112 \item jobs have been successful
1113 \item files have been backed up
1117 However, these statistics are accurate only if your job retention is greater
1118 than your statistics period. Ie, if jobs are purged from the catalog, you won't
1119 be able to use them.
1121 Now, you can use the \textbf{update stats [days=num]} console command to fill
1122 the JobHistory table with new Job records. If you want to be sure to take in
1123 account only \textbf{good jobs}, ie if one of your important job has failed but
1124 you have fixed the problem and restarted it on time, you probably want to
1125 delete the first \textit{bad} job record and keep only the successful one. For
1126 that simply let your staff do the job, and update JobHistory table after two or
1127 three days depending on your organization using the \textbf{[days=num]} option.
1129 These statistics records aren't used for restoring, but mainly for
1130 capacity planning, billings, etc.
1132 The Bweb interface provides a statistics module that can use this feature. You
1133 can also use tools like Talend or extract information by yourself.
1135 The {\textbf Statistics Retention = \lt{}time\gt{}} director directive defines
1136 the length of time that Bacula will keep statistics job records in the Catalog
1137 database after the Job End time. (In \texttt{JobHistory} table) When this time
1138 period expires, and if user runs \texttt{prune stats} command, Bacula will
1139 prune (remove) Job records that are older than the specified period.
1141 You can use the following Job resource in your nightly \textbf{BackupCatalog}
1142 job to maintain statistics.
1145 Name = BackupCatalog
1148 Console = "update stats days=3"
1149 Console = "prune stats yes"
1156 \subsection{SpoolSize = \lt{}size-specification-in-bytes\gt{}}
1157 A new job directive permits to specify the spool size per job. This is used
1158 in advanced job tunning. {\bf SpoolSize={\it bytes}}
1160 \subsection{MaxConsoleConnections = \lt{}number\gt{}}
1161 A new director directive permits to specify the maximum number of Console
1162 Connections that could run concurrently. The default is set to 20, but you may
1163 set it to a larger number.
1165 \section{Building Bacula Plugins}
1166 There is currently one sample program {\bf example-plugin-fd.c} and
1167 one working plugin {\bf bpipe-fd.c} that can be found in the Bacula
1168 {\bf src/plugins/fd} directory. Both are built with the following:
1172 ./configure <your-options>
1180 After building Bacula and changing into the src/plugins/fd directory,
1181 the {\bf make} command will build the {\bf bpipe-fd.so} plugin, which
1182 is a very useful and working program.
1184 The {\bf make test} command will build the {\bf example-plugin-fd.so}
1185 plugin and a binary named {\bf main}, which is build from the source
1186 code located in {\bf src/filed/fd\_plugins.c}.
1188 If you execute {\bf ./main}, it will load and run the example-plugin-fd
1189 plugin simulating a small number of the calling sequences that Bacula uses
1190 in calling a real plugin. This allows you to do initial testing of
1191 your plugin prior to trying it with Bacula.
1193 You can get a good idea of how to write your own plugin by first
1194 studying the example-plugin-fd, and actually running it. Then
1195 it can also be instructive to read the bpipe-fd.c code as it is
1196 a real plugin, which is still rather simple and small.
1198 When actually writing your own plugin, you may use the example-plugin-fd.c
1199 code as a template for your code.
1205 \chapter{Bacula FD Plugin API}
1206 To write a Bacula plugin, you create a dynamic shared object
1207 program (or dll on Win32) with a particular name and two
1208 exported entry points, place it in the {\bf Plugins Directory}, which is defined in the
1209 {\bf bacula-fd.conf} file in the {\bf Client} resource, and when the FD
1210 starts, it will load all the plugins that end with {\bf -fd.so} (or {\bf -fd.dll}
1211 on Win32) found in that directory.
1213 \section{Normal vs Command Plugins}
1214 In general, there are two ways that plugins are called. The first way,
1215 is when a particular event is detected in Bacula, it will transfer control
1216 to each plugin that is loaded in turn informing the plugin of the event.
1217 This is very similar to how a {\bf RunScript} works, and the events are very similar.
1218 Once the plugin gets control, it can interact with Bacula by getting and
1219 setting Bacula variables. In this way, it behaves much like a RunScript.
1220 Currently very few Bacula variables are defined, but they will be implemented
1221 as the need arrises, and it is very extensible.
1223 We plan to have plugins register to receive events that they normally would
1224 not receive, such as an event for each file examined for backup or restore.
1225 This feature is not yet implemented.
1227 The second type of plugin, which is more useful and fully implemented
1228 in the current version is what we call a command plugin. As with all
1229 plugins, it gets notified of important events as noted above (details described below),
1230 but in addition, this kind of plugin can accept a command line, which
1234 Plugin = <command-string>
1237 directive that is placed in the Include section of a FileSet and is very
1238 similar to the "File = " directive. When this Plugin directive is encountered
1239 by Bacula during backup, it passes the "command" part of the Plugin directive
1240 only to the plugin that is explicitly named in the first field of that command string.
1241 This allows that plugin to backup any file or files on the system that it wants. It can
1242 even create "virtual files" in the catalog that contain data to be restored but do
1243 not necessarily correspond to actual files on the filesystem.
1245 The important features of the command plugin entry points are:
1247 \item It is triggered by a "Plugin =" directive in the FileSet
1248 \item Only a single plugin is called that is named on the "Plugin =" directive.
1249 \item The full command string after the "Plugin =" is passed to the plugin
1250 so that it can be told what to backup/restore.
1254 \section{Loading Plugins}
1255 Once the File daemon loads the plugins, it asks the OS for the
1256 two entry points (loadPlugin and unloadPlugin) then calls the
1257 {\bf loadPlugin} entry point (see below).
1259 Bacula passes information to the plugin through this call and it gets
1260 back information that it needs to use the plugin. Later, Bacula
1261 will call particular functions that are defined by the
1262 {\bf loadPlugin} interface.
1264 When Bacula is finished with the plugin
1265 (when Bacula is going to exit), it will call the {\bf unloadPlugin}
1268 The two entry points are:
1271 bRC loadPlugin(bInfo *lbinfo, bFuncs *lbfuncs, pInfo **pinfo, pFuncs **pfuncs)
1278 both these external entry points to the shared object are defined as C entry points
1279 to avoid name mangling complications with C++. However, the shared object
1280 can actually be written in any language (preferrably C or C++) providing that it
1281 follows C language calling conventions.
1283 The definitions for {\bf bRC} and the arguments are {\bf
1284 src/filed/fd-plugins.h} and so this header file needs to be included in
1285 your plugin. It along with {\bf src/lib/plugins.h} define basically the whole
1286 plugin interface. Within this header file, it includes the following
1290 #include <sys/types.h>
1292 #include "bc_types.h"
1293 #include "lib/plugins.h"
1294 #include <sys/stat.h>
1297 Aside from the {\bf bc\_types.h} and {\bf confit.h} headers, the plugin definition uses the
1298 minimum code from Bacula. The bc\_types.h file is required to ensure that
1299 the data type defintions in arguments correspond to the Bacula core code.
1301 The return codes are defined as:
1304 bRC_OK = 0, /* OK */
1305 bRC_Stop = 1, /* Stop calling other plugins */
1306 bRC_Error = 2, /* Some kind of error */
1307 bRC_More = 3, /* More files to backup */
1312 At a future point in time, we hope to make the Bacula libbac.a into a
1313 shared object so that the plugin can use much more of Bacula's
1314 infrastructure, but for this first cut, we have tried to minimize the
1315 dependence on Bacula.
1317 \section{loadPlugin}
1318 As previously mentioned, the {\bf loadPlugin} entry point in the plugin
1319 is called immediately after Bacula loads the plugin when the File daemon
1320 itself is first starting. This entry point is only called once during the
1321 execution of the File daemon. In calling the
1322 plugin, the first two arguments are information from Bacula that
1323 is passed to the plugin, and the last two arguments are information
1324 about the plugin that the plugin must return to Bacula. The call is:
1327 bRC loadPlugin(bInfo *lbinfo, bFuncs *lbfuncs, pInfo **pinfo, pFuncs **pfuncs)
1330 and the arguments are:
1334 This is information about Bacula in general. Currently, the only value
1335 defined in the bInfo structure is the version, which is the Bacula plugin
1336 interface version, currently defined as 1. The {\bf size} is set to the
1337 byte size of the structure. The exact definition of the bInfo structure
1338 as of this writing is:
1341 typedef struct s_baculaInfo {
1348 The bFuncs structure defines the callback entry points within Bacula
1349 that the plugin can use register events, get Bacula values, set
1350 Bacula values, and send messages to the Job output or debug output.
1352 The exact definition as of this writing is:
1354 typedef struct s_baculaFuncs {
1357 bRC (*registerBaculaEvents)(bpContext *ctx, ...);
1358 bRC (*getBaculaValue)(bpContext *ctx, bVariable var, void *value);
1359 bRC (*setBaculaValue)(bpContext *ctx, bVariable var, void *value);
1360 bRC (*JobMessage)(bpContext *ctx, const char *file, int line,
1361 int type, time_t mtime, const char *fmt, ...);
1362 bRC (*DebugMessage)(bpContext *ctx, const char *file, int line,
1363 int level, const char *fmt, ...);
1364 void *(*baculaMalloc)(bpContext *ctx, const char *file, int line,
1366 void (*baculaFree)(bpContext *ctx, const char *file, int line, void *mem);
1370 We will discuss these entry points and how to use them a bit later when
1371 describing the plugin code.
1375 When the loadPlugin entry point is called, the plugin must initialize
1376 an information structure about the plugin and return a pointer to
1377 this structure to Bacula.
1379 The exact definition as of this writing is:
1382 typedef struct s_pluginInfo {
1385 const char *plugin_magic;
1386 const char *plugin_license;
1387 const char *plugin_author;
1388 const char *plugin_date;
1389 const char *plugin_version;
1390 const char *plugin_description;
1396 \item [version] is the current Bacula defined plugin interface version, currently
1397 set to 1. If the interface version differs from the current version of
1398 Bacula, the plugin will not be run (not yet implemented).
1399 \item [plugin\_magic] is a pointer to the text string "*FDPluginData*", a
1400 sort of sanity check. If this value is not specified, the plugin
1401 will not be run (not yet implemented).
1402 \item [plugin\_license] is a pointer to a text string that describes the
1403 plugin license. Bacula will only accept compatible licenses (not yet
1405 \item [plugin\_author] is a pointer to the text name of the author of the program.
1406 This string can be anything but is generally the author's name.
1407 \item [plugin\_date] is the pointer text string containing the date of the plugin.
1408 This string can be anything but is generally some human readable form of
1410 \item [plugin\_version] is a pointer to a text string containing the version of
1411 the plugin. The contents are determined by the plugin writer.
1412 \item [plugin\_description] is a pointer to a string describing what the
1413 plugin does. The contents are determined by the plugin writer.
1416 The pInfo structure must be defined in static memory because Bacula does not
1417 copy it and may refer to the values at any time while the plugin is
1418 loaded. All values must be supplied or the plugin will not run (not yet
1419 implemented). All text strings must be either ASCII or UTF-8 strings that
1420 are terminated with a zero byte.
1423 When the loadPlugin entry point is called, the plugin must initialize
1424 an entry point structure about the plugin and return a pointer to
1425 this structure to Bacula. This structure contains pointer to each
1426 of the entry points that the plugin must provide for Bacula. When
1427 Bacula is actually running the plugin, it will call the defined
1428 entry points at particular times. All entry points must be defined.
1430 The pFuncs structure must be defined in static memory because Bacula does not
1431 copy it and may refer to the values at any time while the plugin is
1434 The exact definition as of this writing is:
1437 typedef struct s_pluginFuncs {
1440 bRC (*newPlugin)(bpContext *ctx);
1441 bRC (*freePlugin)(bpContext *ctx);
1442 bRC (*getPluginValue)(bpContext *ctx, pVariable var, void *value);
1443 bRC (*setPluginValue)(bpContext *ctx, pVariable var, void *value);
1444 bRC (*handlePluginEvent)(bpContext *ctx, bEvent *event, void *value);
1445 bRC (*startBackupFile)(bpContext *ctx, struct save_pkt *sp);
1446 bRC (*endBackupFile)(bpContext *ctx);
1447 bRC (*startRestoreFile)(bpContext *ctx, const char *cmd);
1448 bRC (*endRestoreFile)(bpContext *ctx);
1449 bRC (*pluginIO)(bpContext *ctx, struct io_pkt *io);
1450 bRC (*createFile)(bpContext *ctx, struct restore_pkt *rp);
1451 bRC (*setFileAttributes)(bpContext *ctx, struct restore_pkt *rp);
1455 The details of the entry points will be presented in
1456 separate sections below.
1460 \item [size] is the byte size of the structure.
1461 \item [version] is the plugin interface version currently set to 1.
1464 Sample code for loadPlugin:
1466 bfuncs = lbfuncs; /* set Bacula funct pointers */
1468 *pinfo = &pluginInfo; /* return pointer to our info */
1469 *pfuncs = &pluginFuncs; /* return pointer to our functions */
1474 where pluginInfo and pluginFuncs are statically defined structures.
1475 See bpipe-fd.c for details.
1481 \section{Plugin Entry Points}
1482 This section will describe each of the entry points (subroutines) within
1483 the plugin that the plugin must provide for Bacula, when they are called
1484 and their arguments. As noted above, pointers to these subroutines are
1485 passed back to Bacula in the pFuncs structure when Bacula calls the
1486 loadPlugin() externally defined entry point.
1488 \subsection{newPlugin(bpContext *ctx)}
1489 This is the entry point that Bacula will call
1490 when a new "instance" of the plugin is created. This typically
1491 happens at the beginning of a Job. If 10 Jobs are running
1492 simultaneously, there will be at least 10 instances of the
1495 The bpContext structure will be passed to the plugin, and
1496 during this call, if the plugin needs to have its private
1497 working storage that is associated with the particular
1498 instance of the plugin, it should create it from the heap
1499 (malloc the memory) and store a pointer to
1500 its private working storage in the {\bf pContext} variable.
1501 Note: since Bacula is a multi-threaded program, you must not
1502 keep any variable data in your plugin unless it is truely meant
1503 to apply globally to the whole plugin. In addition, you must
1504 be aware that except the first and last call to the plugin
1505 (loadPlugin and unloadPlugin) all the other calls will be
1506 made by threads that correspond to a Bacula job. The
1507 bpContext that will be passed for each thread will remain the
1508 same throughout the Job thus you can keep your privat Job specific
1509 data in it ({\bf bContext}).
1512 typedef struct s_bpContext {
1513 void *pContext; /* Plugin private context */
1514 void *bContext; /* Bacula private context */
1519 This context pointer will be passed as the first argument to all
1520 the entry points that Bacula calls within the plugin. Needless
1521 to say, the plugin should not change the bContext variable, which
1522 is Bacula's private context pointer for this instance (Job) of this
1525 \subsection{freePlugin(bpContext *ctx)}
1526 This entry point is called when the
1527 this instance of the plugin is no longer needed (the Job is
1528 ending), and the plugin should release all memory it may
1529 have allocated for this particular instance (Job) i.e. the pContext.
1530 This is not the final termination
1531 of the plugin signaled by a call to {\bf unloadPlugin}.
1532 Any other instances (Job) will
1533 continue to run, and the entry point {\bf newPlugin} may be called
1534 again if other jobs start.
1536 \subsection{getPluginValue(bpContext *ctx, pVariable var, void *value)}
1537 Bacula will call this entry point to get
1538 a value from the plugin. This entry point is currently not called.
1540 \subsection{setPluginValue(bpContext *ctx, pVariable var, void *value)}
1541 Bacula will call this entry point to set
1542 a value in the plugin. This entry point is currently not called.
1544 \subsection{handlePluginEvent(bpContext *ctx, bEvent *event, void *value)}
1545 This entry point is called when Bacula
1546 encounters certain events (discussed below). This is, in fact, the
1547 main way that most plugins get control when a Job runs and how
1548 they know what is happening in the job. It can be likened to the
1549 {\bf RunScript} feature that calls external programs and scripts,
1550 and is very similar to the Bacula Python interface.
1551 When the plugin is called, Bacula passes it the pointer to an event
1552 structure (bEvent), which currently has one item, the eventType:
1555 typedef struct s_bEvent {
1560 which defines what event has been triggered, and for each event,
1561 Bacula will pass a pointer to a value associated with that event.
1562 If no value is associated with a particular event, Bacula will
1563 pass a NULL pointer, so the plugin must be careful to always check
1564 value pointer prior to dereferencing it.
1566 The current list of events are:
1572 bEventStartBackupJob = 3,
1573 bEventEndBackupJob = 4,
1574 bEventStartRestoreJob = 5,
1575 bEventEndRestoreJob = 6,
1576 bEventStartVerifyJob = 7,
1577 bEventEndVerifyJob = 8,
1578 bEventBackupCommand = 9,
1579 bEventRestoreCommand = 10,
1586 Most of the above are self-explanatory.
1589 \item [bEventJobStart] is called whenever a Job starts. The value
1590 passed is a pointer to a string that contains: "Jobid=nnn
1591 Job=job-name". Where nnn will be replaced by the JobId and job-name
1592 will be replaced by the Job name. The variable is temporary so if you
1593 need the values, you must copy them.
1595 \item [bEventJobEnd] is called whenever a Job ends. No value is passed.
1597 \item [bEventStartBackupJob] is called when a Backup Job begins. No value
1600 \item [bEventEndBackupJob] is called when a Backup Job ends. No value is
1603 \item [bEventStartRestoreJob] is called when a Restore Job starts. No value
1606 \item [bEventEndRestoreJob] is called when a Restore Job ends. No value is
1609 \item [bEventStartVerifyJob] is called when a Verify Job starts. No value
1612 \item [bEventEndVerifyJob] is called when a Verify Job ends. No value
1615 \item [bEventBackupCommand] is called prior to the bEventStartBackupJob and
1616 the plugin is passed the command string (everything after the equal sign
1617 in "Plugin =" as the value.
1619 Note, if you intend to backup a file, this is an important first point to
1620 write code that copies the command string passed into your pContext area
1621 so that you will know that a backup is being performed and you will know
1622 the full contents of the "Plugin =" command (i.e. what to backup and
1623 what virtual filename the user wants to call it.
1625 \item [bEventRestoreCommand] is called prior to the bEventStartRestoreJob and
1626 the plugin is passed the command string (everything after the equal sign
1627 in "Plugin =" as the value.
1629 See the notes above concerning backup and the command string. This is the
1630 point at which Bacula passes you the original command string that was
1631 specified during the backup, so you will want to save it in your pContext
1632 area for later use when Bacula calls the plugin again.
1634 \item [bEventLevel] is called when the level is set for a new Job. The value
1635 is a 32 bit integer stored in the void*, which represents the Job Level code.
1637 \item [bEventSince] is called when the since time is set for a new Job. The
1638 value is a time\_t time at which the last job was run.
1641 During each of the above calls, the plugin receives either no specific value or
1642 only one value, which in some cases may not be sufficient. However, knowing the
1643 context of the event, the plugin can call back to the Bacula entry points it
1644 was passed during the {\bf loadPlugin} call and get to a number of Bacula variables.
1645 (at the current time few Bacula variables are implemented, but it easily extended
1646 at a future time and as needs require).
1648 \subsection{startBackupFile(bpContext *ctx, struct save\_pkt *sp)}
1649 This entry point is called only if your plugin is a command plugin, and
1650 it is called when Bacula encounters the "Plugin = " directive in
1651 the Include section of the FileSet.
1652 Called when beginning the backup of a file. Here Bacula provides you
1653 with a pointer to the {\bf save\_pkt} structure and you must fill in
1654 this packet with the "attribute" data of the file.
1658 int32_t pkt_size; /* size of this packet */
1659 char *fname; /* Full path and filename */
1660 char *link; /* Link name if any */
1661 struct stat statp; /* System stat() packet for file */
1662 int32_t type; /* FT_xx for this file */
1663 uint32_t flags; /* Bacula internal flags */
1664 bool portable; /* set if data format is portable */
1665 char *cmd; /* command */
1666 int32_t pkt_end; /* end packet sentinel */
1670 The second argument is a pointer to the {\bf save\_pkt} structure for the file
1671 to be backed up. The plugin is responsible for filling in all the fields
1672 of the {\bf save\_pkt}. If you are backing up
1673 a real file, then generally, the statp structure can be filled in by doing
1674 a {\bf stat} system call on the file.
1676 If you are backing up a database or
1677 something that is more complex, you might want to create a virtual file.
1678 That is a file that does not actually exist on the filesystem, but represents
1679 say an object that you are backing up. In that case, you need to ensure
1680 that the {\bf fname} string that you pass back is unique so that it
1681 does not conflict with a real file on the system, and you need to
1682 artifically create values in the statp packet.
1684 Example programs such as {\bf bpipe-fd.c} show how to set these fields.
1685 You must take care not to store pointers the stack in the pointer fields such
1686 as fname and link, because when you return from your function, your stack entries
1687 will be destroyed. The solution in that case is to malloc() and return the pointer
1688 to it. In order to not have memory leaks, you should store a pointer to all memory
1689 allocated in your pContext structure so that in subsequent calls or at termination,
1690 you can release it back to the system.
1692 Once the backup has begun, Bacula will call your plugin at the {\bf pluginIO}
1693 entry point to "read" the data to be backed up. Please see the {\bf bpipe-fd.c}
1694 plugin for how to do I/O.
1696 Example of filling in the save\_pkt as used in bpipe-fd.c:
1699 struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext;
1700 time_t now = time(NULL);
1701 sp->fname = p_ctx->fname;
1702 sp->statp.st_mode = 0700 | S_IFREG;
1703 sp->statp.st_ctime = now;
1704 sp->statp.st_mtime = now;
1705 sp->statp.st_atime = now;
1706 sp->statp.st_size = -1;
1707 sp->statp.st_blksize = 4096;
1708 sp->statp.st_blocks = 1;
1709 p_ctx->backup = true;
1713 Note: the filename to be created has already been created from the
1714 command string previously sent to the plugin and is in the plugin
1715 context (p\_ctx->fname) and is a malloc()ed string. This example
1716 creates a regular file (S\_IFREG), with various fields being created.
1718 In general, the sequence of commands issued from Bacula to the plugin
1719 to do a backup while processing the "Plugin = " directive are:
1722 \item generate a bEventBackupCommand event to the specified plugin
1723 and pass it the command string.
1724 \item make a startPluginBackup call to the plugin, which
1725 fills in the data needed in save\_pkt to save as the file
1726 attributes and to put on the Volume and in the catalog.
1727 \item call Bacula's internal save\_file() subroutine to save the specified
1728 file. The plugin will then be called at pluginIO() to "open"
1729 the file, and then to read the file data.
1730 Note, if you are dealing with a virtual file, the "open" operation
1731 is something the plugin does internally and it doesn't necessarily
1732 mean opening a file on the filesystem. For example in the case of
1733 the bpipe-fd.c program, it initiates a pipe to the requested program.
1734 Finally when the plugin signals to Bacula that all the data was read,
1735 Bacula will call the plugin with the "close" pluginIO() function.
1739 \subsection{endBackupFile(bpContext *ctx)}
1740 Called at the end of backing up a file for a command plugin. If the plugin's work
1741 is done, it should return bRC\_OK. If the plugin wishes to create another
1742 file and back it up, then it must return bRC\_More (not yet implemented).
1743 This is probably a good time to release any malloc()ed memory you used to
1744 pass back filenames.
1746 \subsection{startRestoreFile(bpContext *ctx, const char *cmd)}
1747 Called when the first record is read from the Volume that was
1748 previously written by the command plugin.
1750 \subsection{createFile(bpContext *ctx, struct restore\_pkt *rp)}
1751 Called for a command plugin to create a file during a Restore job before
1753 This entry point is called before any I/O is done on the file. After
1754 this call, Bacula will call pluginIO() to open the file for write.
1757 restore\_pkt is passed to the plugin and is based on the data that was
1758 originally given by the plugin during the backup and the current user
1759 restore settings (e.g. where, RegexWhere, replace). This allows the
1760 plugin to first create a file (if necessary) so that the data can
1761 be transmitted to it. The next call to the plugin will be a
1762 pluginIO command with a request to open the file write-only.
1764 This call must return one of the following values:
1768 CF_SKIP = 1, /* skip file (not newer or something) */
1769 CF_ERROR, /* error creating file */
1770 CF_EXTRACT, /* file created, data to extract */
1771 CF_CREATED /* file created, no data to extract */
1775 in the restore\_pkt value {\bf create\_status}. For a normal file,
1776 unless there is an error, you must return {\bf CF\_EXTRACT}.
1780 struct restore_pkt {
1781 int32_t pkt_size; /* size of this packet */
1782 int32_t stream; /* attribute stream id */
1783 int32_t data_stream; /* id of data stream to follow */
1784 int32_t type; /* file type FT */
1785 int32_t file_index; /* file index */
1786 int32_t LinkFI; /* file index to data if hard link */
1787 uid_t uid; /* userid */
1788 struct stat statp; /* decoded stat packet */
1789 const char *attrEx; /* extended attributes if any */
1790 const char *ofname; /* output filename */
1791 const char *olname; /* output link name */
1792 const char *where; /* where */
1793 const char *RegexWhere; /* regex where */
1794 int replace; /* replace flag */
1795 int create_status; /* status from createFile() */
1796 int32_t pkt_end; /* end packet sentinel */
1801 Typical code to create a regular file would be the following:
1804 struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext;
1805 time_t now = time(NULL);
1806 sp->fname = p_ctx->fname; /* set the full path/filename I want to create */
1808 sp->statp.st_mode = 0700 | S_IFREG;
1809 sp->statp.st_ctime = now;
1810 sp->statp.st_mtime = now;
1811 sp->statp.st_atime = now;
1812 sp->statp.st_size = -1;
1813 sp->statp.st_blksize = 4096;
1814 sp->statp.st_blocks = 1;
1818 This will create a virtual file. If you are creating a file that actually
1819 exists, you will most likely want to fill the statp packet using the
1822 Creating a directory is similar, but requires a few extra steps:
1825 struct plugin_ctx *p_ctx = (struct plugin_ctx *)ctx->pContext;
1826 time_t now = time(NULL);
1827 sp->fname = p_ctx->fname; /* set the full path I want to create */
1828 sp->link = xxx; where xxx is p_ctx->fname with a trailing forward slash
1829 sp->type = FT_DIREND
1830 sp->statp.st_mode = 0700 | S_IFDIR;
1831 sp->statp.st_ctime = now;
1832 sp->statp.st_mtime = now;
1833 sp->statp.st_atime = now;
1834 sp->statp.st_size = -1;
1835 sp->statp.st_blksize = 4096;
1836 sp->statp.st_blocks = 1;
1840 The link field must be set with the full cononical path name, which always
1841 ends with a forward slash. If you do not terminate it with a forward slash,
1842 you will surely have problems later.
1844 As with the example that creates a file, if you are backing up a real
1845 directory, you will want to do an stat() on the directory.
1847 Note, if you want the directory permissions and times to be correctly
1848 restored, you must create the directory {\bf after} all the file directories
1849 have been sent to Bacula. That allows the restore process to restore all the
1850 files in a directory using default directory options, then at the end, restore
1851 the directory permissions. If you do it the other way around, each time you
1852 restore a file, the OS will modify the time values for the directory entry.
1854 \subsection{setFileAttributes(bpContext *ctx, struct restore\_pkt *rp)}
1855 This is call not yet implemented. Called for a command plugin.
1857 See the definition of {\bf restre\_pkt} in the above section.
1859 \subsection{endRestoreFile(bpContext *ctx)}
1860 Called when a command plugin is done restoring a file.
1862 \subsection{pluginIO(bpContext *ctx, struct io\_pkt *io)}
1863 Called to do the input (backup) or output (restore) of data from or to a
1864 file for a command plugin. These routines simulate the Unix read(), write(), open(), close(),
1865 and lseek() I/O calls, and the arguments are passed in the packet and
1866 the return values are also placed in the packet. In addition for Win32
1867 systems the plugin must return two additional values (described below).
1879 int32_t pkt_size; /* Size of this packet */
1880 int32_t func; /* Function code */
1881 int32_t count; /* read/write count */
1882 mode_t mode; /* permissions for created files */
1883 int32_t flags; /* Open flags */
1884 char *buf; /* read/write buffer */
1885 const char *fname; /* open filename */
1886 int32_t status; /* return status */
1887 int32_t io_errno; /* errno code */
1888 int32_t lerror; /* Win32 error code */
1889 int32_t whence; /* lseek argument */
1890 boffset_t offset; /* lseek argument */
1891 bool win32; /* Win32 GetLastError returned */
1892 int32_t pkt_end; /* end packet sentinel */
1896 The particular Unix function being simulated is indicated by the {\bf func},
1897 which will have one of the IO\_OPEN, IO\_READ, ... codes listed above.
1898 The status code that would be returned from a Unix call is returned in
1899 {\bf status} for IO\_OPEN, IO\_CLOSE, IO\_READ, and IO\_WRITE. The return value for
1900 IO\_SEEK is returned in {\bf offset} which in general is a 64 bit value.
1902 When there is an error on Unix systems, you must always set io\_error, and
1903 on a Win32 system, you must always set win32, and the returned value from
1904 the OS call GetLastError() in lerror.
1906 For all except IO\_SEEK, {\bf status} is the return result. In general it is
1907 a positive integer unless there is an error in which case it is -1.
1909 The following describes each call and what you get and what you
1914 You will be passed fname, mode, and flags.
1915 You must set on return: status, and if there is a Unix error
1916 io\_errno must be set to the errno value, and if there is a
1917 Win32 error win32 and lerror.
1920 You will be passed: count, and buf (buffer of size count).
1921 You must set on return: status to the number of bytes
1922 read into the buffer (buf) or -1 on an error,
1923 and if there is a Unix error
1924 io\_errno must be set to the errno value, and if there is a
1925 Win32 error, win32 and lerror must be set.
1928 You will be passed: count, and buf (buffer of size count).
1929 You must set on return: status to the number of bytes
1930 written from the buffer (buf) or -1 on an error,
1931 and if there is a Unix error
1932 io\_errno must be set to the errno value, and if there is a
1933 Win32 error, win32 and lerror must be set.
1936 Nothing will be passed to you. On return you must set
1937 status to 0 on success and -1 on failure. If there is a Unix error
1938 io\_errno must be set to the errno value, and if there is a
1939 Win32 error, win32 and lerror must be set.
1942 You will be passed: offset, and whence. offset is a 64 bit value
1943 and is the position to seek to relative to whence. whence is one
1944 of the following SEEK\_SET, SEEK\_CUR, or SEEK\_END indicating to
1945 either to seek to an absolute possition, relative to the current
1946 position or relative to the end of the file.
1947 You must pass back in offset the absolute location to which you
1948 seeked. If there is an error, offset should be set to -1.
1949 If there is a Unix error
1950 io\_errno must be set to the errno value, and if there is a
1951 Win32 error, win32 and lerror must be set.
1953 Note: Bacula will call IO\_SEEK only when writing a sparse file.
1957 \section{Bacula Plugin Entrypoints}
1958 When Bacula calls one of your plugin entrypoints, you can call back to
1959 the entrypoints in Bacula that were supplied during the xxx plugin call
1960 to get or set information within Bacula.
1962 \subsection{bRC registerBaculaEvents(bpContext *ctx, ...)}
1963 This Bacula entrypoint will allow you to register to receive events
1964 that are not autmatically passed to your plugin by default. This
1965 entrypoint currently is unimplemented.
1967 \subsection{bRC getBaculaValue(bpContext *ctx, bVariable var, void *value)}
1968 Calling this entrypoint, you can obtain specific values that are available
1971 \subsection{bRC setBaculaValue(bpContext *ctx, bVariable var, void *value)}
1972 Calling this entrypoint allows you to set particular values in
1975 \subsection{bRC JobMessage(bpContext *ctx, const char *file, int line,
1976 int type, time\_t mtime, const char *fmt, ...)}
1977 This call permits you to put a message in the Job Report.
1980 \subsection{bRC DebugMessage(bpContext *ctx, const char *file, int line,
1981 int level, const char *fmt, ...)}
1982 This call permits you to print a debug message.
1985 \subsection{void baculaMalloc(bpContext *ctx, const char *file, int line,
1987 This call permits you to obtain memory from Bacula's memory allocator.
1990 \subsection{void baculaFree(bpContext *ctx, const char *file, int line, void *mem)}
1991 This call permits you to free memory obtained from Bacula's memory allocator.