4 \chapter{Python Scripting}
6 \index[general]{Python Scripting}
7 \index[general]{Scripting!Python}
9 You may be asking what Python is and why a scripting language is
10 needed in Bacula. The answer to the first question is that Python
11 is an Object Oriented scripting language with features similar
12 to those found in Perl, but the syntax of the language is much
13 cleaner and simpler. The answer to why have scripting in Bacula is to
14 give the user more control over the whole backup process. Probably
15 the simplest example is when Bacula needs a new Volume name, with
16 a scripting language such as Python, you can generate any name
17 you want, based on the current state of Bacula.
19 \section{Python Configuration}
20 \index[general]{Python Configuration}
21 \index[general]{Configuration!Python}
23 Python must be enabled during the configuration process by adding
24 a \verb:--:with-python, and possibly specifying an alternate
25 directory if your Python is not installed in a standard system
26 location. If you are using RPMs you will need the python-devel package
29 When Python is configured, it becomes an integral part of Bacula and
30 runs in Bacula's address space, so even though it is an interpreted
31 language, it is very efficient.
33 When the Director starts, it looks to see if you have a {\bf
34 Scripts Directory} Directive defined (normal default {\bf
35 /etc/bacula/scripts}, if so, it looks in that directory for a file named
36 {\bf DirStartUp.py}. If it is found, Bacula will pass this file to Python
37 for execution. The {\bf Scripts Directory} is a new directive that you add
38 to the Director resource of your bacula-dir.conf file.
40 Note: Bacula does not install Python scripts by default because these
41 scripts are for you to program. This means that with a default
42 installation with Python enabled, Bacula will print the following error
46 09-Jun 15:14 bacula-dir: ERROR in pythonlib.c:131 Could not import
47 Python script /etc/bacula/scripts/DirStartUp. Python disabled.
50 The source code directory {\bf examples/python} contains sample scripts
51 for DirStartUp.py, SDStartUp.py, and FDStartUp.py that you might want
52 to use as a starting point. Normally, your scripts directory (at least
53 where you store the Python scripts) should be writable by Bacula, because
54 Python will attempt to write a compiled version of the scripts (e.g.
55 DirStartUp.pyc) back to that directory.
57 When starting with the sample scripts, you can delete any part that
58 you will not need, but you should keep all the Bacula Event and Job Event
59 definitions. If you do not want a particular event, simply replace the
60 existing code with a {\bf noop = 1}.
62 \section{Bacula Events}
63 \index[general]{Bacula Events}
64 \index[general]{Events}
65 A Bacula event is a point in the Bacula code where Bacula
66 will call a subroutine (actually a method) that you have
67 defined in the Python StartUp script. Events correspond
68 to some significant event such as a Job Start, a Job End,
69 Bacula needs a new Volume Name, ... When your script is
70 called, it will have access to all the Bacula variables
71 specific to the Job (attributes of the Job Object), and
72 it can even call some of the Job methods (subroutines)
73 or set new values in the Job attributes, such as the
74 Priority. You will see below how the events are used.
76 \section{Python Objects}
77 \index[general]{Python Objects}
78 \index[general]{Objects!Python}
80 There are four Python objects that you will need to work with:
82 \item [The Bacula Object]
83 The Bacula object is created by the Bacula daemon (the Director
84 in the present case) when the daemon starts. It is available to
85 the Python startup script, {\bf DirStartup.py}, by importing the
86 Bacula definitions with {\bf import bacula}. The methods
87 available with this object are described below.
89 \item [The Bacula Events Class]
90 You create this class in the startup script, and you pass
91 it to the Bacula Object's {\bf set\_events} method. The
92 purpose of the Bacula Events Class is to define what global
93 or daemon events you want to monitor. When one of those events
94 occurs, your Bacula Events Class will be called at the method
95 corresponding to the event. There are currently three events,
96 JobStart, JobEnd, and Exit, which are described in detail below.
98 \item [The Job Object]
99 When a Job starts, and assuming you have defined a JobStart method
100 in your Bacula Events Class, Bacula will create a Job Object. This
101 object will be passed to the JobStart event. The Job Object has a
102 has good number of read-only members or attributes providing many
103 details of the Job, and it also has a number of writable attributes
104 that allow you to pass information into the Job. These attributes
107 \item [The Job Events Class]
108 You create this class in the JobStart method of your Bacula Events
109 class, and it allows you to define which of the possible Job Object
110 events you want to see. You must pass an instance of your Job Events
111 class to the Job Object set\_events() method.
112 Normally, you will probably only have one
113 Job Events Class, which will be instantiated for each Job. However,
114 if you wish to see different events in different Jobs, you may have
115 as many Job Events classes as you wish.
119 The first thing the startup script must do is to define what global Bacula
120 events (daemon events), it wants to see. This is done by creating a
121 Bacula Events class, instantiating it, then passing it to the
122 {\bf set\_events} method. There are three possible
127 \index[dir]{JobStart}
128 This Python method, if defined, will be called each time a Job is started.
129 The method is passed the class instantiation object as the first argument,
130 and the Bacula Job object as the second argument. The Bacula Job object
131 has several built-in methods, and you can define which ones you
132 want called. If you do not define this method, you will not be able
133 to interact with Bacula jobs.
136 This Python method, if defined, will be called each time a Job terminates.
137 The method is passed the class instantiation object as the first argument,
138 and the Bacula Job object as the second argument.
141 This Python method, if defined, will be called when the Director terminates.
142 The method is passed the class instantiation object as the first argument.
145 Access to the Bacula variables and methods is done with:
149 The following are the read-only attributes provided by the bacula object.
154 \item [Version] string consisting of "Version Build-date"
158 A simple definition of the Bacula Events Class might be the following:
164 def JobStart(self, job):
169 Then to instantiate the class and pass it to Bacula, you
174 bacula.set_events(BaculaEvents()) # register Bacula Events wanted
178 And at that point, each time a Job is started, your BaculaEvents JobStart
179 method will be called.
181 Now to actually do anything with a Job, you must define which Job events
182 you want to see, and this is done by defining a JobEvents class containing
183 the methods you want called. Each method name corresponds to one of the
184 Job Events that Bacula will generate.
186 A simple Job Events class might look like the following:
191 def NewVolume(self, job):
196 Here, your JobEvents class method NewVolume will be called each time
197 the Job needs a new Volume name. To actually register the events defined
198 in your class with the Job, you must instantiate the JobEvents class and
199 set it in the Job {\bf set\_events} variable. Note, this is a bit different
200 from how you registered the Bacula events. The registration process must
201 be done in the Bacula JobStart event (your method). So, you would modify
202 Bacula Events (not the Job events) as follows:
208 def JobStart(self, job):
209 events = JobEvents() # create instance of Job class
210 job.set_events(events) # register Job events desired
215 When a job event is triggered, the appropriate event definition is
216 called in the JobEvents class. This is the means by which your Python
217 script or code gets control. Once it has control, it may read job
218 attributes, or set them. See below for a list of read-only attributes,
219 and those that are writable.
221 In addition, the Bacula {\bf job} object in the Director has
222 a number of methods (subroutines) that can be called. They
225 \item [set\_events] The set\_events method takes a single
226 argument, which is the instantiation of the Job Events class
227 that contains the methods that you want called. The method
228 names that will be called must correspond to the Bacula
229 defined events. You may define additional methods but Bacula
231 \item [run] The run method takes a single string
232 argument, which is the run command (same as in the Console)
233 that you want to submit to start a new Job. The value
234 returned by the run method is the JobId of the job that
235 started, or -1 if there was an error.
236 \item [write] The write method is used to be able to send
237 print output to the Job Report. This will be described later.
238 \item[cancel] The cancel method takes a single integer argument,
239 which is a JobId. If JobId is found, it will be canceled.
240 \item [DoesVolumeExist] The DoesVolumeExist method takes a single
241 string argument, which is the Volume name, and returns
242 1 if the volume exists in the Catalog and 0 if the volume
246 The following attributes are read/write within the Director
247 for the {\bf job} object.
250 \item [Priority] Read or set the Job priority.
251 Note, that setting a Job Priority is effective only before
252 the Job actually starts.
253 \item [Level] This attribute contains a string representing the Job
254 level, e.g. Full, Differential, Incremental, ... if read.
255 The level can also be set.
258 The following read-only attributes are available within the Director
259 for the {\bf job} object.
262 \item [Type] This attribute contains a string representing the Job
263 type, e.g. Backup, Restore, Verify, ...
264 \item [JobId] This attribute contains an integer representing the
266 \item [Client] This attribute contains a string with the name of the
268 \item [NumVols] This attribute contains an integer with the number of
269 Volumes in the Pool being used by the Job.
270 \item [Pool] This attribute contains a string with the name of the Pool
271 being used by the Job.
272 \item [Storage] This attribute contains a string with the name of the
273 Storage resource being used by the Job.
274 \item [Catalog] This attribute contains a string with the name of the
275 Catalog resource being used by the Job.
276 \item [MediaType] This attribute contains a string with the name of the
277 Media Type associated with the Storage resource being used by the Job.
278 \item [Job] This attribute contains a string containing the name of the
279 Job resource used by this job (not unique).
280 \item [JobName] This attribute contains a string representing the full
282 \item [JobStatus] This attribute contains a single character string
283 representing the current Job status. The status may change
284 during execution of the job. It may take on the following
287 \item [C] Created, not yet running
290 \item [T] Completed successfully
291 \item [E] Terminated with errors
292 \item [e] Non-fatal error
293 \item [f] Fatal error
294 \item [D] Verify found differences
295 \item [A] Canceled by user
296 \item [F] Waiting for Client
297 \item [S] Waiting for Storage daemon
298 \item [m] Waiting for new media
299 \item [M] Waiting for media mount
300 \item [s] Waiting for storage resource
301 \item [j] Waiting for job resource
302 \item [c] Waiting for client resource
303 \item [d] Waiting on maximum jobs
304 \item [t] Waiting on start time
305 \item [p] Waiting on higher priority jobs
308 \item [Priority] This attribute contains an integer with the priority
310 \item [CatalogRes] tuple consisting of (DBName, Address, User,
311 Password, Socket, Port, Database Vendor) taken from the Catalog resource
312 for the Job with the exception of Database Vendor, which is
313 one of the following: MySQL, PostgreSQL, SQLite, Internal,
314 depending on what database you configured.
316 After a Volume has been purged, this attribute will contain the
317 name of that Volume. At other times, this value may have no meaning.
320 The following write-only attributes are available within the
324 \item [JobReport] Send line to the Job Report.
325 \item [VolumeName] Set a new Volume name. Valid only during the
329 \section{Python Console Command}
330 \index[general]{Python Console Command}
331 \index[general]{Console Command!Python}
333 There is a new Console command named {\bf python}. It takes
334 a single argument {\bf restart}. Example:
339 This command restarts the Python interpreter in the Director.
340 This can be useful when you are modifying the DirStartUp script,
341 because normally Python will cache it, and thus the
342 script will be read one time.
344 \section{Debugging Python Scripts}
345 \index[general]{Debugging Python Scripts}
346 In general, you debug your Python scripts by using print statements.
347 You can also develop your script or important parts of it as a
348 separate file using the Python interpreter to run it. Once you
349 have it working correctly, you can then call the script from
350 within the Bacula Python script (DirStartUp.py).
352 If you are having problems loading DirStartUp.py, you will probably
353 not get any error messages because Bacula can only print Python
354 error messages after the Python interpreter is started. However, you
355 may be able to see the error messages by starting Bacula in
356 a shell window with the {\bf -d1} option on the command line. That
357 should cause the Python error messages to be printed in the shell
360 If you are getting error messages such as the following when
361 loading DirStartUp.py:
364 Traceback (most recent call last):
365 File "/etc/bacula/scripts/DirStartUp.py", line 6, in ?
366 import time, sys, bacula
367 ImportError: /usr/lib/python2.3/lib-dynload/timemodule.so: undefined
368 symbol: PyInt_FromLong
369 bacula-dir: pythonlib.c:134 Python Import error.
372 It is because the DirStartUp script is calling a dynamically loaded
373 module (timemodule.so in the above case) that then tries to use
374 Python functions exported from the Python interpreter (in this case
375 PyInt\_FromLong). The way Bacula is currently linked with Python does
376 not permit this. The solution to the problem is to put such functions
377 (in this case the import of time into a separate Python script, which
378 will do your calculations and return the values you want. Then call
379 (not import) this script from the Bacula DirStartUp.py script, and
380 it all should work as you expect.
386 \section{Python Example}
387 \index[general]{Python Example}
388 \index[general]{Example!Python}
390 An example script for the Director startup file is provided in
391 {\bf examples/python/DirStartup.py} as follows:
396 # Bacula Python interface script for the Director
399 # You must import both sys and bacula
402 # This is the list of Bacula daemon events that you
404 class BaculaEvents(object):
406 # Called here when a new Bacula Events class is
407 # is created. Normally not used
410 def JobStart(self, job):
412 Called here when a new job is started. If you want
413 to do anything with the Job, you must register
414 events you want to receive.
416 events = JobEvents() # create instance of Job class
417 events.job = job # save Bacula's job pointer
418 job.set_events(events) # register events desired
419 sys.stderr = events # send error output to Bacula
420 sys.stdout = events # send stdout to Bacula
421 jobid = job.JobId; client = job.Client
422 numvols = job.NumVols
423 job.JobReport="Python Dir JobStart: JobId=%d Client=%s NumVols=%d\n" % (jobid,client,numvols)
425 # Bacula Job is going to terminate
426 def JobEnd(self, job):
429 job.JobReport="Python Dir JobEnd output: JobId=%d Client=%s.\n" % (jobid, client)
431 # Called here when the Bacula daemon is going to exit
433 print "Daemon exiting."
435 bacula.set_events(BaculaEvents()) # register daemon events desired
438 These are the Job events that you can receive.
440 class JobEvents(object):
442 # Called here when you instantiate the Job. Not
446 def JobInit(self, job):
447 # Called when the job is first scheduled
450 def JobRun(self, job):
451 # Called just before running the job after initializing
452 # This is the point to change most Job parameters.
453 # It is equivalent to the JobRunBefore point.
456 def NewVolume(self, job):
457 # Called when Bacula wants a new Volume name. The Volume
458 # name returned, if any, must be stored in job.VolumeName
461 numvol = job.NumVols;
463 job.JobReport = "JobId=%d Client=%s NumVols=%d" % (jobid, client, numvol)
464 job.JobReport="Python before New Volume set for Job.\n"
465 Vol = "TestA-%d" % numvol
466 job.JobReport = "Exists=%d TestA-%d" % (job.DoesVolumeExist(Vol), numvol)
467 job.VolumeName="TestA-%d" % numvol
468 job.JobReport="Python after New Volume set for Job.\n"
471 def VolumePurged(self, job):
472 # Called when a Volume is purged. The Volume name can be referenced
473 # with job.VolumeName