4 \chapter{Bacula Developer Notes}
5 \label{_ChapterStart10}
6 \index{Bacula Developer Notes}
7 \index{Notes!Bacula Developer}
8 \addcontentsline{toc}{section}{Bacula Developer Notes}
10 This document is intended mostly for developers and describes how you can
11 contribute to the Bacula project and the the general
12 framework of making Bacula source changes.
14 \subsection{Contributions}
16 \addcontentsline{toc}{subsubsection}{Contributions}
18 Contributions to the Bacula project come in many forms: ideas,
19 participation in helping people on the bacula-users email list,
20 packaging Bacula binaries for the community, helping improve
21 the documentation, and submitting code.
23 Contributions in the form of submissions for inclusion in the project are
24 broken into two groups. The first are contributions that are aids and not
25 essential to Bacula. In general, these will be scripts or will go into and
26 examples or contributions directory. For these kinds of non-essential
27 contributions there is no obligation to do a copyright assignment as
28 described below. However, a copyright assignment would still be
31 The second class of contributions are those which will be integrated with
32 Bacula and become an essential part (code, scripts, documentation, ...)
33 Within this class of contributions, there are two hurdles to surmount. One
34 is getting your patch accepted, and two is dealing with copyright issues.
35 The following text describes some of the requirements for such code.
39 \addcontentsline{toc}{subsubsection}{Patches}
41 Subject to the copyright assignment described below, your patches should be
42 sent in {\bf diff -u} format relative to the current contents of the Source
43 Forge SVN. The diff -u format is the easiest for us to understand and integrate.
44 Please be sure to use the Bacula indenting standard (see below) for source code.
45 If you have checked out the source with SVN, you can get a diff using:
49 svn diff > change.patch
52 If you plan on doing significant development work over a period of time,
53 after having your first patch reviewed and approved, you will be eligible
54 for having developer SVN write access so that you can commit your changes
55 directly to the SVN repository. To do so, you will need a userid on Source
58 \subsection{Copyrights}
60 \addcontentsline{toc}{subsubsection}{Copyrights}
62 To avoid future problems concerning changing licensing or
63 copyrights, all code contributions more than a hand full of lines
64 must be in the Public Domain or have the copyright transferred to
65 the Free Software Foundation Europe e.V. with a Fiduciary License
66 Agreement (FLA) as in the current code. Note, prior to
67 November 2004, the code was copyrighted by Kern Sibbald and John
68 Walker. After November 2004, the code was copyrighted by Kern
69 Sibbald, then on the 15th of November 2006, the copyright was
70 transferred to the Free Software Foundation Europe e.V.
71 In signing the FLA and transferring the copyright, you retain the
72 right to use the code you have submitted as you want.
74 Your name should be clearly indicated as the author of the code, and you
75 must be extremely careful not to violate any copyrights or patents or use other
76 people's code without acknowledging it. The purpose of this requirement is
77 to avoid future copyright, patent, or intellectual property problems.
78 Please read the LICENSE agreement in the main Bacula source code
79 directory. When you sign the Fiduciary License Agreement (FLA)
80 and send it in, you are argeeing to the terms of that LICENSE
83 If you don't understand what we mean by future problems, please
84 examine the difficulties Mozilla was having finding
85 previous contributors at \elink{
86 http://www.mozilla.org/MPL/missing.html}
87 {http://www.mozilla.org/MPL/missing.html}. The other important issue is to
88 avoid copyright, patent, or intellectual property violations as was
89 (May 2003) claimed by SCO against IBM.
91 Although the copyright will be held by the Free Software
92 Foundation Europe e.V., each developer is expected to indicate
93 that he wrote and/or modified a particular module (or file) and
94 any other sources. The copyright assignment may seem a bit
95 unusual, but in reality, it is not. Most large projects require
98 If you have any doubts about this, please don't hesitate to ask. The
99 objective is to assure the long term servival of the Bacula project.
101 Items not needing a copyright assignment are: most small changes,
102 enhancements, or bug fixes of 5-10 lines of code, which amount to
103 less than 20% of any particular file.
105 \subsection{Copyright Assignment -- Fiduciary License Agreement}
106 \index{Copyright Assignment}
107 \index{Assignment!Copyright}
108 \addcontentsline{toc}{subsubsection}{Copyright Assignment -- Fiduciary License Agreement}
110 Since this is not a commercial enterprise, and we prefer to believe in
111 everyone's good faith, previously developers could assign the copyright by
112 explicitly acknowledging that they do so in their first submission. This
113 was sufficient if the developer is independent, or an employee of a
114 not-for-profit organization or a university. However, in an effort to
115 ensure that the Bacula code is really clean, beginning in August 2006, all
116 previous and future developers with SVN write access will be asked to submit a
117 copyright assignment (or Fiduciary License Agreement -- FLA),
118 which means you agree to the LICENSE in the main source
119 directory. It also means that you receive back the right to use
120 the code that you have submitted.
122 Any developer who wants to contribute and is employed by a company should
123 either list the employer as the owner of the code, or get explicit
124 permission from him to sign the copyright assignment. This is because in
125 many countries, all work that an employee does whether on company time or
126 in the employee's free time is considered to be Intellectual Property of
127 the company. Obtaining official approval or an FLA from the company will
128 avoid misunderstandings between the employee, the company, and the Bacula
129 project. A good number of companies have already followed this procedure.
131 The Fiduciary License Agreement is posted on the Bacula web site at:
132 \elink{http://www.bacula.org/en/FLA-bacula.en.pdf}{http://www.bacula.org/en/FLA-bacula.en.pdf}
134 The instructions for filling out this agreement are also at:
135 \elink{http://www.bacula.org/?page=fsfe}{http://www.bacula.org/?page=fsfe}
137 It should be filled out, then sent to:
141 Cotes-de-Montmoiret 9
146 Please note that the above address is different from the officially
147 registered office mentioned in the document. When you send in such a
148 complete document, please notify me: kern at sibbald dot com, and
149 please add your email address to the FLA so that I can contact you
150 to confirm reception of the signed FLA.
153 \section{The Development Cycle}
154 \index{Developement Cycle}
155 \index{Cycle!Developement}
156 \addcontentsline{toc}{subsubsection}{Development Cycle}
158 As I noted in the 1.38 ReleaseNotes, version 1.38 was different from prior
159 versions because it had a lot more contributions. We expect that this trend
160 will continue. As a consequence, we have modified how we do
161 development, and instead of making a list of all the features that we will
162 implement in the next version, each developer will sign up for one (maybe
163 two) projects at a time, and when they are complete, we will release a new
166 The difference is that we will have more time to review the new code that is
167 being contributed, and will be able to devote more time to a smaller number
168 of projects (1.38 had too many new features for us to handle correctly).
170 Future release schedules will be much the same, and the
171 number of new features will also be much the same providing that the
172 contributions continue to come -- and they show no signs of let up :-)
174 \index{Feature Requests}
175 {\bf Feature Requests:} \\
176 In addition, we would like to "formalize" the feature requests a bit.
178 Instead of me maintaining an informal list of everything I run into
179 (kernstodo), we would like to maintain a "formal" list of projects. This
180 means that all new feature requests, including those recently discussed on
181 the email lists, must be formally submitted and approved.
183 Formal submission of feature requests will take two forms: \\
184 1. non-mandatory, but highly recommended is to discuss proposed new features
185 on the mailing list.\\
186 2. Formal submission of an Feature Request in a special format.
187 We'll give an example of this below, but you can also find it on the web
188 site under "Support -\gt{} Feature Requests". Since it takes a bit of time to
189 properly fill out a Feature Request form, you probably should check on the email list
192 Once the Feature Request is received by the keeper of the projects list, it
193 will be sent to the Bacula project manager (Kern), and he will either
194 accept it, send it back asking for clarification, send it to the email list
195 asking for opinions, or reject it.
197 If it is accepted, it will go in the "projects" file (a simple ASCII file)
198 maintained in the main Bacula source directory.
200 {\bf Implementation of Feature Requests:}\\
201 Any qualified developer can sign up for a project. The project must have
202 an entry in the projects file, and the developer's name will appear in the
205 {\bf How Feature Requests are accepted:}\\
206 Acceptance of Feature Requests depends on several things: \\
207 1. feedback from users. If it is negative, the Feature Request will probably not be
209 2. the difficulty of the project. A project that is so
210 difficult that we cannot imagine finding someone to implement probably won't
211 be accepted. Obviously if you know how to implement it, don't hesitate
212 to put it in your Feature Request \\
213 3. whether or not the Feature Request fits within the
214 current stategy of Bacula (for example an Feature Request that requests changing the
215 tape to tar format probably would not be accepted, ...).
217 {\bf How Feature Requests are prioritized:}\\
218 Once an Feature Request is accepted, it needs to be implemented. If you
219 can find a developer for it, or one signs up for implementing it, then the
220 Feature Request becomes top priority (at least for that developer).
222 Between releases of Bacula, we will generally solicit Feature Request input
223 for the next version, and by way of this email, we suggest that you send
224 discuss and send in your Feature Requests for the next release. Please
225 verify that the Feature Request is not in the current list (attached to this email).
227 Once users have had several weeks to submit Feature Requests, the keeper of
228 the projects list will organize them, and request users to vote on them.
229 This will allow fixing prioritizing the Feature Requests. Having a
230 priority is one thing, but getting it implement is another thing -- we are
231 hoping that the Bacula community will take more responsibility for assuring
232 the implementation of accepted Feature Requests.
234 Feature Request format:
236 ============= Empty Feature Request form ===========
237 Item n: One line summary ...
239 Origin: Name and email of originator.
242 What: More detailed explanation ...
244 Why: Why it is important ...
246 Notes: Additional notes or features (omit if not used)
247 ============== End Feature Request form ==============
251 ============= Example Completed Feature Request form ===========
252 Item 1: Implement a Migration job type that will move the job
253 data from one device to another.
254 Origin: Sponsored by Riege Sofware International GmbH. Contact:
255 Daniel Holtkamp <holtkamp at riege dot com>
256 Date: 28 October 2005
257 Status: Partially coded in 1.37 -- much more to do. Assigned to
260 What: The ability to copy, move, or archive data that is on a
261 device to another device is very important.
263 Why: An ISP might want to backup to disk, but after 30 days
264 migrate the data to tape backup and delete it from
265 disk. Bacula should be able to handle this
266 automatically. It needs to know what was put where,
267 and when, and what to migrate -- it is a bit like
268 retention periods. Doing so would allow space to be
269 freed up for current backups while maintaining older
272 Notes: Migration could be triggered by:
276 Highwater size (keep total size)
278 =================================================
282 \section{Bacula Code Submissions and Projects}
283 \index{Submissions and Projects}
284 \addcontentsline{toc}{subsection}{Code Submissions and Projects}
286 Getting code implemented in Bacula works roughly as follows:
290 \item Kern is the project manager, but prefers not to be a "gate keeper".
291 This means that the developers are expected to be self-motivated,
292 and once they have experience submit directly to the SVN. However,
293 it is a good idea to have your patches reviewed prior to submitting,
294 and it is a bad idea to submit monster patches because no one will
295 be able to properly review them. See below for more details on this.
297 \item There are growing numbers of contributions (very good).
299 \item Some contributions come in the form of relatively small patches,
300 which Kern reviews, integrates, documents, tests, and maintains.
302 \item All Bacula developers take full
303 responsibility for writing the code, posting as patches so that we can
304 review it as time permits, integrating it at an appropriate time,
305 responding to our requests for tweaking it (name changes, ...),
306 document it in the code, document it in the manual (even though
307 their mother tongue is not English), test it, develop and commit
308 regression scripts, and answer in a timely fashion all bug reports --
309 even occassionally accepting additional bugs :-)
311 This is a sustainable way of going forward with Bacula, and the
312 direction that the project will be taking more and more. For
313 example, in the past, we have had some very dedicated programmers
314 who did major projects. However, some of these
315 programmers due to outside obligations (job responsibilities change of
316 job, school duties, ...) could not continue to maintain the code. In
317 those cases, the code suffers from lack of maintenance, sometimes we
318 patch it, sometimes not. In the end, the code gets dropped from the
319 project (there are two such contributions that are heading in that
320 direction). When ever possible, we would like to avoid this, and
321 ensure a continuation of the code and a sharing of the development,
322 debugging, documentation, and maintenance responsibilities.
325 \section{Patches for Released Versions}
326 \index{Patches for Released Versions}
327 \addcontentsline{toc}{subsection}{Patches for Released Versions}
328 If you fix a bug in a released version, you should, unless it is
329 an absolutely trivial bug, create and release a patch file for the
330 bug. The procedure is as follows:
332 Fix the bug in the branch and in the trunk.
334 Make a patch file for the branch and add the branch patch to
335 the patches directory in both the branch and the trunk.
336 The name should be 2.2.4-xxx.patch where xxx is unique, in this case it can
337 be "restore", e.g. 2.2.4-restore.patch. Add to the top of the
338 file a brief description and instructions for applying it -- see for example
339 2.2.4-poll-mount.patch. The best way to create the patch file is as
343 (edit) 2.2.4-restore.patch
347 svn diff >>2.2.4-restore.patch
350 check to make sure no extra junk got put into the patch file (i.e.
351 it should have the patch for that bug only).
353 If there is not a bug report on the problem, create one, then add the
354 patch to the bug report.
356 Uthen upload it to the 2.2.x release of bacula-patches.
358 So, end the end, the patch file is:
360 \item Attached to the bug report
362 \item In Branch-2.2/bacula/patches/...
366 \item Loaded on Source Forge bacula-patches 2.2.x release. When
367 you add it, click on the check box to send an Email so that all the
368 users that are monitoring SF patches get notified.
375 \addcontentsline{toc}{subsection}{SVN Usage}
377 Please note that if you are familar with CVS, SVN is very
378 similar (and better), but there can be a few surprising
381 The *entire* Bacula SourceForge.net Subversion repository can be
382 checked out through SVN with the following command:
385 svn checkout https://bacula.svn.sourceforge.net/svnroot/bacula bacula
388 With the above command, you will get everything, which is a very large
410 Note, you should NEVER commit code to any checkout that you have
411 done of a tag. All tags (e.g. Release-1.1, ... Release-2.0.2)
412 should be considered read-only.
414 You may commit code to the most recent item in
415 branches (in the above the most recent one is Branch-2.0). If
416 you want to commit code to an older branch, then please contact
419 You may create your own tags and/or branches, but they should
420 have a name clearly distinctive from Branch-, Release-, or Beta-,
421 which are official names used by the project. If you create a
422 tag, then you should NEVER commit code to it, for the same
423 reason noted above -- it should serve as a marker for something
424 you released. If you create a branch, then you are free to
425 commit to it as you wish.
427 You may, of course, commit to the trunk.
439 are reserved names to be created only by the project manager (or
440 with his OK), where the nnn is any sequence of numbers and
441 periods (e.g. 2.0, 2.0.1, ...).
443 In addition all tags even those that you create are read-only
444 forever. Typically tags represent release points either in the
445 trunc or in a branch.
448 Coming back to getting source code.
449 If you only want the current Bacula source code, you could use:
452 svn checkout https://bacula.svn.sourceforge.net/svnroot/bacula/trunk/bacula bacula
455 To view what is in the SVN, point your browser at the following URL:
456 http://bacula.svn.sourceforge.net/viewvc/bacula/
458 Many of the Subversion (svn) commands are almost identical to those that
459 you have used for cvs, but some (such as a checkout) can have surprising
460 results, so you should take a careful look at the documentation.
462 The following documentation on the new
463 svn repository will help you know how to use it:
465 Here is the list of branches:
482 Release-1.1 Release-1.19 Release-1.19a Release-1.19b
483 Release-1.20 Release-1.21 Release-1.22 Release-1.23
484 Release-1.23a Release-1.24 Release-1.25 Release-1.25a
485 Release-1.26 Release-1.27 Release-1.27a Release-1.27b
486 Release-1.27c Release-1.28 Release-1.29 Release-1.30
487 Release-1.31 Release-1.31a Release-1.32 Release-1.32a
488 Release-1.32b Release-1.32c Release-1.32d Release-1.32e
489 Release-1.32f Release-1.32f-2 Release-1.32f-3 Release-1.32f-4
490 Release-1.32f-5 Release-1.34.0 Release-1.34.1 Release-1.34.3
491 Release-1.34.4 Release-1.34.5 Release-1.34.6 Release-1.35.1
492 Release-1.35.2 Release-1.35.3 Release-1.35.6 Release-1.35.7
493 Release-1.35.8 Release-1.36.0 Release-1.36.1 Release-1.36.2
494 Release-1.36.3 Release-1.38.0 Release-1.38.1 Release-1.38.10
495 Release-1.38.11 Release-1.38.2 Release-1.38.3 Release-1.38.4
496 Release-1.38.5 Release-1.38.6 Release-1.38.7 Release-1.38.8
497 Release-1.38.9 Release-1.8.1 Release-1.8.2 Release-1.8.3
498 Release-1.8.4 Release-1.8.5 Release-1.8.6 Release-2.0.0
499 Release-2.0.1 Release-2.0.2
502 Here is a list of commands to get you started. The recommended book is
503 "Version Control with Subversion", by Ben Collins-Sussmann,
504 Brian W. Fitzpatrick, and Michael Pilato, O'Reilly. The book is
505 Open Source, so it is also available on line at:
508 http://svnbook.red-bean.com
511 Get a list of commands
517 Get a help with a command
523 Checkout the HEAD revision of all modules from the project into the
527 svn co https://bacula.svn.sourceforge.net/svnroot/bacula/trunk bacula.new
530 Checkout the HEAD revision of the bacula module into the bacula subdirectory
533 svn checkout https://bacula.svn.sourceforge.net/svnroot/bacula/trunk/bacula
536 See which files have changed in the working copy
542 See which files are out of date
548 Add a new file file.c
554 Create a new directory
560 Delete an obsolete file
569 svn move file.c newfile.c
572 Move a file to a new location
575 svn move file.c ../newdir/file.c
578 Copy a file retaining the original history in the new file
581 svn copy file.c newfile.c
584 Update the working copy with the outstanding changes
590 Compare working copy with the repository
596 Commit the changes in the local working copy
602 Specify which files are ignored in the current directory
605 svn propedit svn:ignore .
608 Mark a file to be executable
611 svn propset svn:executable '*' prog.sh
614 Unmark a file as executable
617 svn propdel svn:executable prog.sh
620 List a file's properties
626 Create a branch for a new version
629 svn copy https://bacula.svn.sourceforge.net/svnroot/bacula/trunk \
630 https://bacula.svn.sourceforge.net/svnroot/bacula/branches/Branch-2.1
633 Tag a version for a new release
636 svn copy https://bacula.svn.sourceforge.net/svnroot/bacula/branches/Branch-2.1 \
637 https://bacula.svn.sourceforge.net/svnroot/bacula/branches/Release-2.1
641 Let's say you are working in the directory scripts. You would then do:
648 when you are happy with your changes, you can do the following:
651 cd bacula (to your top level directory)
652 svn diff my-changes.patch
655 When the command is done, you can look in the file my-changes.patch
656 and you will see all the changes you have made to your copy of the
657 repository. Make sure that you understand all the changes that
658 it reports before proceeding. If you modified files that you do
659 do not want to commit to the main repository, you can simply delete
660 them from your local directory, and they will be restored from the
661 repository with the "svn update" that is shown below. Normally, you
662 should not find changes to files that you do not want to commit, and
663 if you find yourself in that position a lot, you are probably doing
666 Let's assume that now you want to commit your changes to the main
676 When you do this, it will pull any changes made by other developers into
677 your local copy of the repository, and it will check for conflicts. If there
678 are any, it will tell you, and you will need to resolve them. The problems
679 of resolving conflicts are a bit more than this document can cover, but
680 you can examine the files it claims have conflicts and look for \lt{}\lt{}\lt{}\lt{}
681 or look in the .rej files that it creates. If you have problems, just ask
682 on the developer's list.
684 Note, doing the above "svn update" is not absolutely necessary. There are
685 times when you may be working on code and you want to commit it, but you
686 explicitly do not want to move up to the latest version of the code in
687 the SVN. If that is the case, you can simply skip the "svn update" and
688 do the commit shown below. If the commit fails because of a conflict, it
689 will tell you, and you must resolve the conflict before it will permit
690 you to do the commit.
692 Once your local copy of the repository has been updated, you can now
696 svn commit -m "Some comment about what you changed"
699 or if you really only want to commit a single file, you can
703 svn commit -m "comment" scripts/file-I-edited
706 Note, if you have done a build in your directory, or you have added
707 other new files, the commit will update only the files that are
708 actually in the repository. For example, none of the object files
709 are stored in the repository, so when you do a commit, those object
710 files will simply be ignored.
712 If you want to add new files or remove files from the main SVN
713 repository, and you are not experienced with SVN, please ask Kern
714 to do it. If you follow the simple steps above, it is unlikely that
715 you will do any damage to the repository, and if you do, it is always
716 possible for us to recover, but it can be painful.
718 If you are only working in one subdirectory of say the bacula project,
719 for example, the scripts directory, you can do your commit from
720 that subdirectory, and only the changes in that directory and all its
721 subdirectories will be committed. This can be helpful for translators.
722 If you are doing a French translation, you will be working in
723 docs/manual-fr, and if you are always cd'ed into that directory when
724 doing your commits, your commit will effect only that directory. As
725 long as you are careful only to change files that you want changed,
726 you have little to worry about.
728 \section{Subversion Resources}
729 \index{Subversion (svn) Resources}
730 \addcontentsline{toc}{subsection}{Subversion Resources}
732 Main Subversion Web Page
733 \elink{http://subversion.tigris.org}{http://subversion.tigris.org}
736 \elink{http://svnbook.red-bean.com}{http://svnbook.red-bean.com}
739 \elink{http://subversion.tigris.org/project\_packages.html}{http://subversion.tigris.org/project\_packages.html}
741 (For Windows users the TortoiseSVN package is awesome)
744 \elink{http://rapidsvn.tigris.org/}{http://rapidsvn.tigris.org/}
746 A nice KDE GUI client:
751 \section{Developing Bacula}
752 \index{Developing Bacula}
753 \index{Bacula!Developing}
754 \addcontentsline{toc}{subsubsection}{Developing Bacula}
756 Typically the simplest way to develop Bacula is to open one xterm window
757 pointing to the source directory you wish to update; a second xterm window at
758 the top source directory level, and a third xterm window at the bacula
759 directory \lt{}top\gt{}/src/bacula. After making source changes in one of the
760 directories, in the top source directory xterm, build the source, and start
761 the daemons by entering:
765 ./startit then in the enter:
769 ./gnome-console to start the Console program. Enter any commands for testing.
770 For example: run kernsverify full.
772 Note, the instructions here to use {\bf ./startit} are different from using a
773 production system where the administrator starts Bacula by entering {\bf
774 ./bacula start}. This difference allows a development version of {\bf Bacula}
775 to be run on a computer at the same time that a production system is running.
776 The {\bf ./startit} strip starts {\bf Bacula} using a different set of
777 configuration files, and thus permits avoiding conflicts with any production
780 To make additional source changes, exit from the Console program, and in the
781 top source directory, stop the daemons by entering:
783 ./stopit then repeat the process.
785 \subsection{Debugging}
787 \addcontentsline{toc}{subsubsection}{Debugging}
789 Probably the first thing to do is to turn on debug output.
791 A good place to start is with a debug level of 20 as in {\bf ./startit -d20}.
792 The startit command starts all the daemons with the same debug level.
793 Alternatively, you can start the appropriate daemon with the debug level you
794 want. If you really need more info, a debug level of 60 is not bad, and for
795 just about everything a level of 200.
797 \subsection{Using a Debugger}
798 \index{Using a Debugger}
799 \index{Debugger!Using a}
800 \addcontentsline{toc}{subsubsection}{Using a Debugger}
802 If you have a serious problem such as a segmentation fault, it can usually be
803 found quickly using a good multiple thread debugger such as {\bf gdb}. For
804 example, suppose you get a segmentation violation in {\bf bacula-dir}. You
805 might use the following to find the problem:
807 \lt{}start the Storage and File daemons\gt{}
810 run -f -s -c ./dird.conf
811 \lt{}it dies with a segmentation fault\gt{}
813 The {\bf -f} option is specified on the {\bf run} command to inhibit {\bf
814 dird} from going into the background. You may also want to add the {\bf -s}
815 option to the run command to disable signals which can potentially interfere
818 As an alternative to using the debugger, each {\bf Bacula} daemon has a built
819 in back trace feature when a serious error is encountered. It calls the
820 debugger on itself, produces a back trace, and emails the report to the
821 developer. For more details on this, please see the chapter in the main Bacula
822 manual entitled ``What To Do When Bacula Crashes (Kaboom)''.
824 \subsection{Memory Leaks}
827 \addcontentsline{toc}{subsubsection}{Memory Leaks}
829 Because Bacula runs routinely and unattended on client and server machines, it
830 may run for a long time. As a consequence, from the very beginning, Bacula
831 uses SmartAlloc to ensure that there are no memory leaks. To make detection of
832 memory leaks effective, all Bacula code that dynamically allocates memory MUST
833 have a way to release it. In general when the memory is no longer needed, it
834 should be immediately released, but in some cases, the memory will be held
835 during the entire time that Bacula is executing. In that case, there MUST be a
836 routine that can be called at termination time that releases the memory. In
837 this way, we will be able to detect memory leaks. Be sure to immediately
838 correct any and all memory leaks that are printed at the termination of the
841 \subsection{Special Files}
842 \index{Files!Special}
843 \index{Special Files}
844 \addcontentsline{toc}{subsubsection}{Special Files}
846 Kern uses files named 1, 2, ... 9 with any extension as scratch files. Thus
847 any files with these names are subject to being rudely deleted at any time.
849 \subsection{When Implementing Incomplete Code}
850 \index{Code!When Implementing Incomplete}
851 \index{When Implementing Incomplete Code}
852 \addcontentsline{toc}{subsubsection}{When Implementing Incomplete Code}
854 Please identify all incomplete code with a comment that contains
860 where there are three asterisks (*) before and after the word
861 FIXME (in capitals) and no intervening spaces. This is important as it allows
862 new programmers to easily recognize where things are partially implemented.
864 \subsection{Bacula Source File Structure}
865 \index{Structure!Bacula Source File}
866 \index{Bacula Source File Structure}
867 \addcontentsline{toc}{subsubsection}{Bacula Source File Structure}
869 The distribution generally comes as a tar file of the form {\bf
870 bacula.x.y.z.tar.gz} where x, y, and z are the version, release, and update
871 numbers respectively.
873 Once you detar this file, you will have a directory structure as follows:
880 |- mtx (autochanger control program + tape drive info)
881 |- sqlite (SQLite database program)
885 |- pthreads (Native win32 pthreads library -- dll)
886 |- zlib (Native win32 zlib library)
887 |- wx (wxWidgets source code)
890 |- bacula (main source directory containing configuration
891 | and installation files)
892 |- autoconf (automatic configuration files, not normally used
894 |- intl (programs used to translate)
895 |- platforms (OS specific installation files)
896 |- redhat (Red Hat installation)
897 |- solaris (Sun installation)
898 |- freebsd (FreeBSD installation)
899 |- irix (Irix installation -- not tested)
900 |- unknown (Default if system not identified)
901 |- po (translations of source strings)
902 |- src (source directory; contains global header files)
903 |- cats (SQL catalog database interface directory)
904 |- console (bacula user agent directory)
905 |- dird (Director daemon)
906 |- filed (Unix File daemon)
907 |- win32 (Win32 files to make bacula-fd be a service)
908 |- findlib (Unix file find library for File daemon)
909 |- gnome-console (GNOME version of console program)
910 |- lib (General Bacula library)
911 |- stored (Storage daemon)
912 |- tconsole (Tcl/tk console program -- not yet working)
913 |- testprogs (test programs -- normally only in Kern's tree)
914 |- tools (Various tool programs)
915 |- win32 (Native Win32 File daemon)
916 |- baculafd (Visual Studio project file)
917 |- compat (compatibility interface library)
918 |- filed (links to src/filed)
919 |- findlib (links to src/findlib)
920 |- lib (links to src/lib)
921 |- console (beginning of native console program)
922 |- wx-console (wxWidget console Win32 specific parts)
923 |- wx-console (wxWidgets console main source program)
926 |- regress (Regression scripts)
927 |- bin (temporary directory to hold Bacula installed binaries)
928 |- build (temporary directory to hold Bacula source)
929 |- scripts (scripts and .conf files)
930 |- tests (test scripts)
931 |- tmp (temporary directory for temp files)
932 |- working (temporary working directory for Bacula daemons)
935 |- docs (documentation directory)
936 |- developers (Developer's guide)
937 |- home-page (Bacula's home page source)
938 |- manual (html document directory)
939 |- manual-fr (French translation)
940 |- manual-de (German translation)
941 |- techlogs (Technical development notes);
944 |- rescue (Bacula rescue CDROM)
945 |- linux (Linux rescue CDROM)
946 |- cdrom (Linux rescue CDROM code)
948 |- solaris (Solaris rescue -- incomplete)
949 |- freebsd (FreeBSD rescue -- incomplete)
952 |- gui (Bacula GUI projects)
953 |- bacula-web (Bacula web php management code)
954 |- bimagemgr (Web application for burning CDROMs)
960 \subsection{Header Files}
963 \addcontentsline{toc}{subsubsection}{Header Files}
965 Please carefully follow the scheme defined below as it permits in general only
966 two header file includes per C file, and thus vastly simplifies programming.
967 With a large complex project like Bacula, it isn't always easy to ensure that
968 the right headers are invoked in the right order (there are a few kludges to
969 make this happen -- i.e. in a few include files because of the chicken and egg
970 problem, certain references to typedefs had to be replaced with {\bf void} ).
972 Every file should include {\bf bacula.h}. It pulls in just about everything,
973 with very few exceptions. If you have system dependent ifdefing, please do it
974 in {\bf baconfig.h}. The version number and date are kept in {\bf version.h}.
976 Each of the subdirectories (console, cats, dird, filed, findlib, lib, stored,
977 ...) contains a single directory dependent include file generally the name of
978 the directory, which should be included just after the include of {\bf
979 bacula.h}. This file (for example, for the dird directory, it is {\bf dird.h})
980 contains either definitions of things generally needed in this directory, or
981 it includes the appropriate header files. It always includes {\bf protos.h}.
984 Each subdirectory contains a header file named {\bf protos.h}, which contains
985 the prototypes for subroutines exported by files in that directory. {\bf
986 protos.h} is always included by the main directory dependent include file.
988 \subsection{Programming Standards}
989 \index{Standards!Programming}
990 \index{Programming Standards}
991 \addcontentsline{toc}{subsubsection}{Programming Standards}
993 For the most part, all code should be written in C unless there is a burning
994 reason to use C++, and then only the simplest C++ constructs will be used.
995 Note, Bacula is slowly evolving to use more and more C++.
997 Code should have some documentation -- not a lot, but enough so that I can
998 understand it. Look at the current code, and you will see that I document more
999 than most, but am definitely not a fanatic.
1001 We prefer simple linear code where possible. Gotos are strongly discouraged
1002 except for handling an error to either bail out or to retry some code, and
1003 such use of gotos can vastly simplify the program.
1005 Remember this is a C program that is migrating to a {\bf tiny} subset of C++,
1006 so be conservative in your use of C++ features.
1008 \subsection{Do Not Use}
1011 \addcontentsline{toc}{subsubsection}{Do Not Use}
1014 \item STL -- it is totally incomprehensible.
1017 \subsection{Avoid if Possible}
1018 \index{Possible!Avoid if}
1019 \index{Avoid if Possible}
1020 \addcontentsline{toc}{subsubsection}{Avoid if Possible}
1023 \item Using {\bf void *} because this generally means that one must
1024 using casting, and in C++ casting is rather ugly. It is OK to use
1025 void * to pass structure address where the structure is not known
1026 to the routines accepting the packet (typically callback routines).
1027 However, declaring "void *buf" is a bad idea. Please use the
1028 correct types whenever possible.
1030 \item Using undefined storage specifications such as (short, int, long,
1031 long long, size\_t ...). The problem with all these is that the number of bytes
1032 they allocate depends on the compiler and the system. Instead use
1033 Bacula's types (int8\_t, uint8\_t, int32\_t, uint32\_t, int64\_t, and
1034 uint64\_t). This guarantees that the variables are given exactly the
1035 size you want. Please try at all possible to avoid using size\_t ssize\_t
1036 and the such. They are very system dependent. However, some system
1037 routines may need them, so their use is often unavoidable.
1039 \item Returning a malloc'ed buffer from a subroutine -- someone will forget
1042 \item Heap allocation (malloc) unless needed -- it is expensive. Use
1045 \item Templates -- they can create portability problems.
1047 \item Fancy or tricky C or C++ code, unless you give a good explanation of
1050 \item Too much inheritance -- it can complicate the code, and make reading it
1051 difficult (unless you are in love with colons)
1055 \subsection{Do Use Whenever Possible}
1056 \index{Possible!Do Use Whenever}
1057 \index{Do Use Whenever Possible}
1058 \addcontentsline{toc}{subsubsection}{Do Use Whenever Possible}
1061 \item Locking and unlocking within a single subroutine.
1063 \item A single point of exit from all subroutines. A goto is
1064 perfectly OK to use to get out early, but only to a label
1065 named bail\_out, and possibly an ok\_out. See current code
1068 \item Malloc and free within a single subroutine.
1070 \item Comments and global explanations on what your code or algorithm does.
1074 \subsection{Indenting Standards}
1075 \index{Standards!Indenting}
1076 \index{Indenting Standards}
1077 \addcontentsline{toc}{subsubsection}{Indenting Standards}
1079 We find it very hard to read code indented 8 columns at a time.
1080 Even 4 at a time uses a lot of space, so we have adopted indenting
1081 3 spaces at every level. Note, indention is the visual appearance of the
1082 source on the page, while tabbing is replacing a series of up to 8 spaces from
1085 The closest set of parameters for the Linux {\bf indent} program that will
1086 produce reasonably indented code are:
1090 -nbad -bap -bbo -nbc -br -brs -c36 -cd36 -ncdb -ce -ci3 -cli0
1091 -cp36 -d0 -di1 -ndj -nfc1 -nfca -hnl -i3 -ip0 -l85 -lp -npcs
1092 -nprs -npsl -saf -sai -saw -nsob -nss -nbc -ncs -nbfda
1096 You can put the above in your .indent.pro file, and then just invoke indent on
1097 your file. However, be warned. This does not produce perfect indenting, and it
1098 will mess up C++ class statements pretty badly.
1100 Braces are required in all if statements (missing in some very old code). To
1101 avoid generating too many lines, the first brace appears on the first line
1102 (e.g. of an if), and the closing brace is on a line by itself. E.g.
1112 Just follow the convention in the code. For example we I prefer non-indented cases.
1129 Avoid using // style comments except for temporary code or turning off debug
1130 code. Standard C comments are preferred (this also keeps the code closer to
1133 Attempt to keep all lines less than 85 characters long so that the whole line
1134 of code is readable at one time. This is not a rigid requirement.
1136 Always put a brief description at the top of any new file created describing
1137 what it does and including your name and the date it was first written. Please
1138 don't forget any Copyrights and acknowledgments if it isn't 100\% your code.
1139 Also, include the Bacula copyright notice that is in {\bf src/c}.
1141 In general you should have two includes at the top of the an include for the
1142 particular directory the code is in, for includes are needed, but this should
1145 In general (except for self-contained packages), prototypes should all be put
1146 in {\bf protos.h} in each directory.
1148 Always put space around assignment and comparison operators.
1159 but your can compress things in a {\bf for} statement:
1163 for (i=0; i < del.num_ids; i++) {
1168 Don't overuse the inline if (?:). A full {\bf if} is preferred, except in a
1169 print statement, e.g.:
1173 if (ua->verbose \&& del.num_del != 0) {
1174 bsendmsg(ua, _("Pruned %d %s on Volume %s from catalog.\n"), del.num_del,
1175 del.num_del == 1 ? "Job" : "Jobs", mr->VolumeName);
1180 Leave a certain amount of debug code (Dmsg) in code you submit, so that future
1181 problems can be identified. This is particularly true for complicated code
1182 likely to break. However, try to keep the debug code to a minimum to avoid
1183 bloating the program and above all to keep the code readable.
1185 Please keep the same style in all new code you develop. If you include code
1186 previously written, you have the option of leaving it with the old indenting
1187 or re-indenting it. If the old code is indented with 8 spaces, then please
1188 re-indent it to Bacula standards.
1190 If you are using {\bf vim}, simply set your tabstop to 8 and your shiftwidth
1193 \subsection{Tabbing}
1195 \addcontentsline{toc}{subsubsection}{Tabbing}
1197 Tabbing (inserting the tab character in place of spaces) is as normal on all
1198 Unix systems -- a tab is converted space up to the next column multiple of 8.
1199 My editor converts strings of spaces to tabs automatically -- this results in
1200 significant compression of the files. Thus, you can remove tabs by replacing
1201 them with spaces if you wish. Please don't confuse tabbing (use of tab
1202 characters) with indenting (visual alignment of the code).
1206 \addcontentsline{toc}{subsubsection}{Don'ts}
1221 They are system dependent and un-safe. These should be replaced by the Bacula
1226 char *bstrncpy(char *dest, char *source, int dest_size);
1227 char *bstrncat(char *dest, char *source, int dest_size);
1228 int bsnprintf(char *buf, int32_t buf_len, const char *fmt, ...);
1229 int bvsnprintf(char *str, int32_t size, const char *format, va_list ap);
1233 See src/lib/bsys.c for more details on these routines.
1235 Don't use the {\bf \%lld} or the {\bf \%q} printf format editing types to edit
1236 64 bit integers -- they are not portable. Instead, use {\bf \%s} with {\bf
1237 edit\_uint64()}. For example:
1242 uint64_t num = something;
1244 bsnprintf(buf, sizeof(buf), "Num=%s\n", edit_uint64(num, ed1));
1248 Note: {\bf \%lld} is now permitted in Bacula code -- we have our
1249 own printf routines which handle it correctly. The edit\_uint64() subroutine
1250 can still be used if you wish, but over time, most of that old style will
1253 The edit buffer {\bf ed1} must be at least 27 bytes long to avoid overflow.
1254 See src/lib/edit.c for more details. If you look at the code, don't start
1255 screaming that I use {\bf lld}. I actually use subtle trick taught to me by
1256 John Walker. The {\bf lld} that appears in the editing routine is actually
1257 {\bf \#define} to a what is needed on your OS (usually ``lld'' or ``q'') and
1258 is defined in autoconf/configure.in for each OS. C string concatenation causes
1259 the appropriate string to be concatenated to the ``\%''.
1261 Also please don't use the STL or Templates or any complicated C++ code.
1263 \subsection{Message Classes}
1264 \index{Classes!Message}
1265 \index{Message Classes}
1266 \addcontentsline{toc}{subsubsection}{Message Classes}
1268 Currently, there are five classes of messages: Debug, Error, Job, Memory,
1271 \subsection{Debug Messages}
1272 \index{Messages!Debug}
1273 \index{Debug Messages}
1274 \addcontentsline{toc}{subsubsection}{Debug Messages}
1276 Debug messages are designed to be turned on at a specified debug level and are
1277 always sent to STDOUT. There are designed to only be used in the development
1278 debug process. They are coded as:
1280 DmsgN(level, message, arg1, ...) where the N is a number indicating how many
1281 arguments are to be substituted into the message (i.e. it is a count of the
1282 number arguments you have in your message -- generally the number of percent
1283 signs (\%)). {\bf level} is the debug level at which you wish the message to
1284 be printed. message is the debug message to be printed, and arg1, ... are the
1285 arguments to be substituted. Since not all compilers support \#defines with
1286 varargs, you must explicitly specify how many arguments you have.
1288 When the debug message is printed, it will automatically be prefixed by the
1289 name of the daemon which is running, the filename where the Dmsg is, and the
1290 line number within the file.
1292 Some actual examples are:
1294 Dmsg2(20, ``MD5len=\%d MD5=\%s\textbackslash{}n'', strlen(buf), buf);
1296 Dmsg1(9, ``Created client \%s record\textbackslash{}n'', client->hdr.name);
1298 \subsection{Error Messages}
1299 \index{Messages!Error}
1300 \index{Error Messages}
1301 \addcontentsline{toc}{subsubsection}{Error Messages}
1303 Error messages are messages that are related to the daemon as a whole rather
1304 than a particular job. For example, an out of memory condition my generate an
1305 error message. They should be very rarely needed. In general, you should be
1306 using Job and Job Queued messages (Jmsg and Qmsg). They are coded as:
1308 EmsgN(error-code, level, message, arg1, ...) As with debug messages, you must
1309 explicitly code the of arguments to be substituted in the message. error-code
1310 indicates the severity or class of error, and it may be one of the following:
1312 \addcontentsline{lot}{table}{Message Error Code Classes}
1313 \begin{longtable}{lp{3in}}
1314 {{\bf M\_ABORT} } & {Causes the daemon to immediately abort. This should be
1315 used only in extreme cases. It attempts to produce a traceback. } \\
1316 {{\bf M\_ERROR\_TERM} } & {Causes the daemon to immediately terminate. This
1317 should be used only in extreme cases. It does not produce a traceback. } \\
1318 {{\bf M\_FATAL} } & {Causes the daemon to terminate the current job, but the
1319 daemon keeps running } \\
1320 {{\bf M\_ERROR} } & {Reports the error. The daemon and the job continue
1322 {{\bf M\_WARNING} } & {Reports an warning message. The daemon and the job
1323 continue running } \\
1324 {{\bf M\_INFO} } & {Reports an informational message.}
1328 There are other error message classes, but they are in a state of being
1329 redesigned or deprecated, so please do not use them. Some actual examples are:
1332 Emsg1(M\_ABORT, 0, ``Cannot create message thread: \%s\textbackslash{}n'',
1335 Emsg3(M\_WARNING, 0, ``Connect to File daemon \%s at \%s:\%d failed. Retrying
1336 ...\textbackslash{}n'', client-\gt{}hdr.name, client-\gt{}address,
1339 Emsg3(M\_FATAL, 0, ``bdird\lt{}filed: bad response from Filed to \%s command:
1340 \%d \%s\textbackslash{}n'', cmd, n, strerror(errno));
1342 \subsection{Job Messages}
1343 \index{Job Messages}
1344 \index{Messages!Job}
1345 \addcontentsline{toc}{subsubsection}{Job Messages}
1347 Job messages are messages that pertain to a particular job such as a file that
1348 could not be saved, or the number of files and bytes that were saved. They
1351 Jmsg(jcr, M\_FATAL, 0, "Text of message");
1353 A Jmsg with M\_FATAL will fail the job. The Jmsg() takes varargs so can
1354 have any number of arguments for substituted in a printf like format.
1355 Output from the Jmsg() will go to the Job report.
1357 If the Jmsg is followed with a number such as Jmsg1(...), the number
1358 indicates the number of arguments to be substituted (varargs is not
1359 standard for \#defines), and what is more important is that the file and
1360 line number will be prefixed to the message. This permits a sort of debug
1363 \subsection{Queued Job Messages}
1364 \index{Queued Job Messages}
1365 \index{Messages!Job}
1366 \addcontentsline{toc}{subsubsection}{Queued Job Messages}
1367 Queued Job messages are similar to Jmsg()s except that the message is
1368 Queued rather than immediately dispatched. This is necessary within the
1369 network subroutines and in the message editing routines. This is to prevent
1370 recursive loops, and to ensure that messages can be delivered even in the
1371 event of a network error.
1374 \subsection{Memory Messages}
1375 \index{Messages!Memory}
1376 \index{Memory Messages}
1377 \addcontentsline{toc}{subsubsection}{Memory Messages}
1379 Memory messages are messages that are edited into a memory buffer. Generally
1380 they are used in low level routines such as the low level device file dev.c in
1381 the Storage daemon or in the low level Catalog routines. These routines do not
1382 generally have access to the Job Control Record and so they return error
1383 essages reformatted in a memory buffer. Mmsg() is the way to do this.
1385 \subsection{Bugs Database}
1386 \index{Database!Bugs}
1387 \index{Bugs Database}
1388 \addcontentsline{toc}{subsubsection}{Bugs Database}
1389 We have a bugs database which is at:
1390 \elink{http://bugs.bacula.org}{http://bugs.bacula.org}, and as
1391 a developer you will need to respond to bugs, perhaps bugs in general
1392 if you have time, otherwise just bugs that correspond to code that
1395 If you need to answer bugs, please be sure to ask the Project Manager
1396 (currently Kern) to give you Developer access to the bugs database. This
1397 allows you to modify statuses and close bugs.
1399 The first thing is if you want to take over a bug, rather than just make a
1400 note, you should assign the bug to yourself. This helps other developers
1401 know that you are the principal person to deal with the bug. You can do so
1402 by going into the bug and clicking on the {\bf Update Issue} button. Then
1403 you simply go to the {\bf Assigned To} box and select your name from the
1404 drop down box. To actually update it you must click on the {\bf Update
1405 Information} button a bit further down on the screen, but if you have other
1406 things to do such as add a Note, you might wait before clicking on the {\bf
1407 Update Information} button.
1409 Generally, we set the {\bf Status} field to either acknowledged, confirmed,
1410 or feedback when we first start working on the bug. Feedback is set when
1411 we expect that the user should give us more information.
1413 Normally, once you are reasonably sure that the bug is fixed, and a patch
1414 is made and attached to the bug report, and/or in the SVN, you can close
1415 the bug. If you want the user to test the patch, then leave the bug open,
1416 otherwise close it and set {\bf Resolution} to {\bf Fixed}. We generally
1417 close bug reports rather quickly, even without confirmation, especially if
1418 we have run tests and can see that for us the problem is fixed. However,
1419 in doing so, it avoids misunderstandings if you leave a note while you are
1420 closing the bug that says something to the following effect:
1421 We are closing this bug because ... If for some reason, it does not fix
1422 your problem, please feel free to reopen it, or to open a new bug report
1423 describing the problem".
1425 We do not recommend that you attempt to edit any of the bug notes that have
1426 been submitted, nor to delete them or make them private. In fact, if
1427 someone accidentally makes a bug note private, you should ask the reason
1428 and if at all possible (with his agreement) make the bug note public.
1430 If the user has not properly filled in most of the important fields
1431 (platorm, OS, Product Version, ...) please do not hesitate to politely ask
1432 him. Also, if the bug report is a request for a new feature, please
1433 politely send the user to the Feature Request menu item on www.bacula.org.
1434 The same applies to a support request (we answer only bugs), you might give
1435 the user a tip, but please politely refer him to the manual and the
1436 Getting Support page of www.bacula.org.