1 \chapter{Python Scripting}
3 \index[general]{Python Scripting}
4 \index[general]{Scripting!Python}
6 You may be asking what Python is and why a scripting language is
7 needed in Bacula. The answer to the first question is that Python
8 is an Object Oriented scripting language with features similar
9 to those found in Perl, but the syntax of the language is much
10 cleaner and simpler. The answer to why have scripting in Bacula is to
11 give the user more control over the whole backup process. Probably
12 the simplest example is when Bacula needs a new Volume name, with
13 a scripting language such as Python, you can generate any name
14 you want, based on the current state of Bacula.
16 \section{Python Configuration}
17 \index[general]{Python Configuration}
18 \index[general]{Configuration!Python}
20 Python must be enabled during the configuration process by adding
21 a \verb:--:with-python, and possibly specifying an alternate
22 directory if your Python is not installed in a standard system
23 location. If you are using RPMs you will need the python-devel package
26 When Python is configured, it becomes an integral part of Bacula and
27 runs in Bacula's address space, so even though it is an interpreted
28 language, it is very efficient.
30 When the Director starts, it looks to see if you have a {\bf
31 Scripts Directory} Directive defined (normal default {\bf
32 /etc/bacula/scripts}, if so, it looks in that directory for a file named
33 {\bf DirStartUp.py}. If it is found, Bacula will pass this file to Python
34 for execution. The {\bf Scripts Directory} is a new directive that you add
35 to the Director resource of your bacula-dir.conf file.
37 Note: Bacula does not install Python scripts by default because these
38 scripts are for you to program. This means that with a default
39 installation with Python enabled, Bacula will print the following error
43 09-Jun 15:14 bacula-dir: ERROR in pythonlib.c:131 Could not import
44 Python script /etc/bacula/scripts/DirStartUp. Python disabled.
47 The source code directory {\bf examples/python} contains sample scripts
48 for DirStartUp.py, SDStartUp.py, and FDStartUp.py that you might want
49 to use as a starting point. Normally, your scripts directory (at least
50 where you store the Python scripts) should be writable by Bacula, because
51 Python will attempt to write a compiled version of the scripts (e.g.
52 DirStartUp.pyc) back to that directory.
54 When starting with the sample scripts, you can delete any part that
55 you will not need, but you should keep all the Bacula Event and Job Event
56 definitions. If you do not want a particular event, simply replace the
57 existing code with a {\bf noop = 1}.
59 \section{Bacula Events}
60 \index[general]{Bacula Events}
61 \index[general]{Events}
62 A Bacula event is a point in the Bacula code where Bacula
63 will call a subroutine (actually a method) that you have
64 defined in the Python StartUp script. Events correspond
65 to some significant event such as a Job Start, a Job End,
66 Bacula needs a new Volume Name, ... When your script is
67 called, it will have access to all the Bacula variables
68 specific to the Job (attributes of the Job Object), and
69 it can even call some of the Job methods (subroutines)
70 or set new values in the Job attributes, such as the
71 Priority. You will see below how the events are used.
73 \section{Python Objects}
74 \index[general]{Python Objects}
75 \index[general]{Objects!Python}
77 There are four Python objects that you will need to work with:
79 \item [The Bacula Object]
80 The Bacula object is created by the Bacula daemon (the Director
81 in the present case) when the daemon starts. It is available to
82 the Python startup script, {\bf DirStartup.py}, by importing the
83 Bacula definitions with {\bf import bacula}. The methods
84 available with this object are described below.
86 \item [The Bacula Events Class]
87 You create this class in the startup script, and you pass
88 it to the Bacula Object's {\bf set\_events} method. The
89 purpose of the Bacula Events Class is to define what global
90 or daemon events you want to monitor. When one of those events
91 occurs, your Bacula Events Class will be called at the method
92 corresponding to the event. There are currently three events,
93 JobStart, JobEnd, and Exit, which are described in detail below.
95 \item [The Job Object]
96 When a Job starts, and assuming you have defined a JobStart method
97 in your Bacula Events Class, Bacula will create a Job Object. This
98 object will be passed to the JobStart event. The Job Object has a
99 has good number of read-only members or attributes providing many
100 details of the Job, and it also has a number of writable attributes
101 that allow you to pass information into the Job. These attributes
104 \item [The Job Events Class]
105 You create this class in the JobStart method of your Bacula Events
106 class, and it allows you to define which of the possible Job Object
107 events you want to see. You must pass an instance of your Job Events
108 class to the Job Object set\_events() method.
109 Normally, you will probably only have one
110 Job Events Class, which will be instantiated for each Job. However,
111 if you wish to see different events in different Jobs, you may have
112 as many Job Events classes as you wish.
116 The first thing the startup script must do is to define what global Bacula
117 events (daemon events), it wants to see. This is done by creating a
118 Bacula Events class, instantiating it, then passing it to the
119 {\bf set\_events} method. There are three possible
124 \index[general]{JobStart}
125 This Python method, if defined, will be called each time a Job is started.
126 The method is passed the class instantiation object as the first argument,
127 and the Bacula Job object as the second argument. The Bacula Job object
128 has several built-in methods, and you can define which ones you
129 want called. If you do not define this method, you will not be able
130 to interact with Bacula jobs.
133 This Python method, if defined, will be called each time a Job terminates.
134 The method is passed the class instantiation object as the first argument,
135 and the Bacula Job object as the second argument.
138 This Python method, if defined, will be called when the Director terminates.
139 The method is passed the class instantiation object as the first argument.
142 Access to the Bacula variables and methods is done with:
146 The following are the read-only attributes provided by the bacula object.
151 \item [Version] string consisting of "Version Build-date"
155 A simple definition of the Bacula Events Class might be the following:
160 def JobStart(self, job):
164 Then to instantiate the class and pass it to Bacula, you
168 bacula.set_events(BaculaEvents()) # register Bacula Events wanted
171 And at that point, each time a Job is started, your BaculaEvents JobStart
172 method will be called.
174 Now to actually do anything with a Job, you must define which Job events
175 you want to see, and this is done by defining a JobEvents class containing
176 the methods you want called. Each method name corresponds to one of the
177 Job Events that Bacula will generate.
179 A simple Job Events class might look like the following:
183 def NewVolume(self, job):
187 Here, your JobEvents class method NewVolume will be called each time
188 the Job needs a new Volume name. To actually register the events defined
189 in your class with the Job, you must instantiate the JobEvents class and
190 set it in the Job {\bf set\_events} variable. Note, this is a bit different
191 from how you registered the Bacula events. The registration process must
192 be done in the Bacula JobStart event (your method). So, you would modify
193 Bacula Events (not the Job events) as follows:
198 def JobStart(self, job):
199 events = JobEvents() # create instance of Job class
200 job.set_events(events) # register Job events desired
204 When a job event is triggered, the appropriate event definition is
205 called in the JobEvents class. This is the means by which your Python
206 script or code gets control. Once it has control, it may read job
207 attributes, or set them. See below for a list of read-only attributes,
208 and those that are writable.
210 In addition, the Bacula {\bf job} object in the Director has
211 a number of methods (subroutines) that can be called. They
214 \item [set\_events] The set\_events method takes a single
215 argument, which is the instantiation of the Job Events class
216 that contains the methods that you want called. The method
217 names that will be called must correspond to the Bacula
218 defined events. You may define additional methods but Bacula
220 \item [run] The run method takes a single string
221 argument, which is the run command (same as in the Console)
222 that you want to submit to start a new Job. The value
223 returned by the run method is the JobId of the job that
224 started, or -1 if there was an error.
225 \item [write] The write method is used to be able to send
226 print output to the Job Report. This will be described later.
227 \item[cancel] The cancel method takes a single integer argument,
228 which is a JobId. If JobId is found, it will be canceled.
229 \item [DoesVolumeExist] The DoesVolumeExist method takes a single
230 string argument, which is the Volume name, and returns
231 1 if the volume exists in the Catalog and 0 if the volume
235 The following attributes are read/write within the Director
236 for the {\bf job} object.
239 \item [Priority] Read or set the Job priority.
240 Note, that setting a Job Priority is effective only before
241 the Job actually starts.
242 \item [Level] This attribute contains a string representing the Job
243 level, e.g. Full, Differential, Incremental, ... if read.
244 The level can also be set.
247 The following read-only attributes are available within the Director
248 for the {\bf job} object.
251 \item [Type] This attribute contains a string representing the Job
252 type, e.g. Backup, Restore, Verify, ...
253 \item [JobId] This attribute contains an integer representing the
255 \item [Client] This attribute contains a string with the name of the
257 \item [NumVols] This attribute contains an integer with the number of
258 Volumes in the Pool being used by the Job.
259 \item [Pool] This attribute contains a string with the name of the Pool
260 being used by the Job.
261 \item [Storage] This attribute contains a string with the name of the
262 Storage resource being used by the Job.
263 \item [Catalog] This attribute contains a string with the name of the
264 Catalog resource being used by the Job.
265 \item [MediaType] This attribute contains a string with the name of the
266 Media Type associated with the Storage resource being used by the Job.
267 \item [Job] This attribute contains a string containing the name of the
268 Job resource used by this job (not unique).
269 \item [JobName] This attribute contains a string representing the full
271 \item [JobStatus] This attribute contains a single character string
272 representing the current Job status. The status may change
273 during execution of the job. It may take on the following
276 \item [C] Created, not yet running
279 \item [T] Completed successfully
280 \item [E] Terminated with errors
281 \item [e] Non-fatal error
282 \item [f] Fatal error
283 \item [D] Verify found differences
284 \item [A] Canceled by user
285 \item [F] Waiting for Client
286 \item [S] Waiting for Storage daemon
287 \item [m] Waiting for new media
288 \item [M] Waiting for media mount
289 \item [s] Waiting for storage resource
290 \item [j] Waiting for job resource
291 \item [c] Waiting for client resource
292 \item [d] Waiting on maximum jobs
293 \item [t] Waiting on start time
294 \item [p] Waiting on higher priority jobs
297 \item [Priority] This attribute contains an integer with the priority
299 \item [CatalogRes] tuple consisting of (DBName, Address, User,
300 Password, Socket, Port, Database Vendor) taken from the Catalog resource
301 for the Job with the exception of Database Vendor, which is
302 one of the following: MySQL, PostgreSQL, SQLite, Internal,
303 depending on what database you configured.
305 After a Volume has been purged, this attribute will contain the
306 name of that Volume. At other times, this value may have no meaning.
309 The following write-only attributes are available within the
313 \item [JobReport] Send line to the Job Report.
314 \item [VolumeName] Set a new Volume name. Valid only during the
318 \section{Python Console Command}
319 \index[general]{Python Console Command}
320 \index[general]{Console Command!Python}
322 There is a new Console command named {\bf python}. It takes
323 a single argument {\bf restart}. Example:
328 This command restarts the Python interpreter in the Director.
329 This can be useful when you are modifying the DirStartUp script,
330 because normally Python will cache it, and thus the
331 script will be read one time.
333 \section{Debugging Python Scripts}
334 \index[general]{Debugging Python Scripts}
335 In general, you debug your Python scripts by using print statements.
336 You can also develop your script or important parts of it as a
337 separate file using the Python interpreter to run it. Once you
338 have it working correctly, you can then call the script from
339 within the Bacula Python script (DirStartUp.py).
341 If you are having problems loading DirStartUp.py, you will probably
342 not get any error messages because Bacula can only print Python
343 error messages after the Python interpreter is started. However, you
344 may be able to see the error messages by starting Bacula in
345 a shell window with the {\bf -d1} option on the command line. That
346 should cause the Python error messages to be printed in the shell
349 If you are getting error messages such as the following when
350 loading DirStartUp.py:
353 Traceback (most recent call last):
354 File "/etc/bacula/scripts/DirStartUp.py", line 6, in ?
355 import time, sys, bacula
356 ImportError: /usr/lib/python2.3/lib-dynload/timemodule.so: undefined
357 symbol: PyInt_FromLong
358 bacula-dir: pythonlib.c:134 Python Import error.
361 It is because the DirStartUp script is calling a dynamically loaded
362 module (timemodule.so in the above case) that then tries to use
363 Python functions exported from the Python interpreter (in this case
364 PyInt\_FromLong). The way Bacula is currently linked with Python does
365 not permit this. The solution to the problem is to put such functions
366 (in this case the import of time into a separate Python script, which
367 will do your calculations and return the values you want. Then call
368 (not import) this script from the Bacula DirStartUp.py script, and
369 it all should work as you expect.
375 \section{Python Example}
376 \index[general]{Python Example}
377 \index[general]{Example!Python}
379 An example script for the Director startup file is provided in
380 {\bf examples/python/DirStartup.py} as follows:
384 # Bacula Python interface script for the Director
387 # You must import both sys and bacula
390 # This is the list of Bacula daemon events that you
392 class BaculaEvents(object):
394 # Called here when a new Bacula Events class is
395 # is created. Normally not used
398 def JobStart(self, job):
400 Called here when a new job is started. If you want
401 to do anything with the Job, you must register
402 events you want to receive.
404 events = JobEvents() # create instance of Job class
405 events.job = job # save Bacula's job pointer
406 job.set_events(events) # register events desired
407 sys.stderr = events # send error output to Bacula
408 sys.stdout = events # send stdout to Bacula
409 jobid = job.JobId; client = job.Client
410 numvols = job.NumVols
411 job.JobReport="Python Dir JobStart: JobId=%d Client=%s NumVols=%d\n" % (jobid,client,numvols)
413 # Bacula Job is going to terminate
414 def JobEnd(self, job):
417 job.JobReport="Python Dir JobEnd output: JobId=%d Client=%s.\n" % (jobid, client)
419 # Called here when the Bacula daemon is going to exit
421 print "Daemon exiting."
423 bacula.set_events(BaculaEvents()) # register daemon events desired
426 These are the Job events that you can receive.
428 class JobEvents(object):
430 # Called here when you instantiate the Job. Not
434 def JobInit(self, job):
435 # Called when the job is first scheduled
438 def JobRun(self, job):
439 # Called just before running the job after initializing
440 # This is the point to change most Job parameters.
441 # It is equivalent to the JobRunBefore point.
444 def NewVolume(self, job):
445 # Called when Bacula wants a new Volume name. The Volume
446 # name returned, if any, must be stored in job.VolumeName
449 numvol = job.NumVols;
451 job.JobReport = "JobId=%d Client=%s NumVols=%d" % (jobid, client, numvol)
452 job.JobReport="Python before New Volume set for Job.\n"
453 Vol = "TestA-%d" % numvol
454 job.JobReport = "Exists=%d TestA-%d" % (job.DoesVolumeExist(Vol), numvol)
455 job.VolumeName="TestA-%d" % numvol
456 job.JobReport="Python after New Volume set for Job.\n"
459 def VolumePurged(self, job):
460 # Called when a Volume is purged. The Volume name can be referenced
461 # with job.VolumeName