[bacula/docs] / docs / manuals / de / developers / generaldevel.tex

%%
%%

\chapter{Bacula Developer Notes}
\label{_ChapterStart10}
\index{Bacula Developer Notes}
\index{Notes!Bacula Developer}
\addcontentsline{toc}{section}{Bacula Developer Notes}

This document is intended mostly for developers and describes how you can
contribute to the Bacula project and the the general framework of making
Bacula source changes.

\subsection{Contributions}
\index{Contributions}
\addcontentsline{toc}{subsubsection}{Contributions}

Contributions to the Bacula project come in many forms: ideas,
participation in helping people on the bacula-users email list, packaging
Bacula binaries for the community, helping improve the documentation, and
submitting code.

Contributions in the form of submissions for inclusion in the project are
broken into two groups.  The first are contributions that are aids and not
essential to Bacula.  In general, these will be scripts or will go into the
{\bf bacula/examples} directory.  For these kinds of non-essential
contributions there is no obligation to do a copyright assignment as
described below.  However, a copyright assignment would still be
appreciated.

The second class of contributions are those which will be integrated with
Bacula and become an essential part (code, scripts, documentation, ...)
Within this class of contributions, there are two hurdles to surmount.  One
is getting your patch accepted, and two is dealing with copyright issues.
The following text describes some of the requirements for such code.

\subsection{Patches}
\index{Patches}
\addcontentsline{toc}{subsubsection}{Patches}

Subject to the copyright assignment described below, your patches should be
sent in {\bf git format-patch} format relative to the current contents of the 
master branch of the Source Forge Git repository.  Please attach the
output file or files generated by the {\bf git format-patch} to the email
rather than include them directory to avoid wrapping of the lines
in the patch.  Please be sure to use the Bacula
indenting standard (see below) for source code.  If you have checked out
the source with Git, you can get a diff using.

\begin{verbatim}
git pull
git format-patch -M
\end{verbatim}

If you plan on doing significant development work over a period of time,
after having your first patch reviewed and approved, you will be eligible
for having developer Git write access so that you can commit your changes
directly to the Git repository.  To do so, you will need a userid on Source
Forge.

\subsection{Copyrights}
\index{Copyrights}
\addcontentsline{toc}{subsubsection}{Copyrights}

To avoid future problems concerning changing licensing or
copyrights, all code contributions more than a hand full of lines
must be in the Public Domain or have the copyright transferred to
the Free Software Foundation Europe e.V. with a Fiduciary License
Agreement (FLA) as the case for all the current code.  

Prior to November 2004, all the code was copyrighted by Kern Sibbald and
John Walker.  After November 2004, the code was copyrighted by Kern
Sibbald, then on the 15th of November 2006, Kern transferred the copyright
to the Free Software Foundation Europe e.V. In signing the FLA and
transferring the copyright, you retain the right to use the code you have
submitted as you want, and you ensure that Bacula will always remain Free
and Open Source.

Your name should be clearly indicated as the author of the code, and you
must be extremely careful not to violate any copyrights or patents or use
other people's code without acknowledging it.  The purpose of this
requirement is to avoid future copyright, patent, or intellectual property
problems.  Please read the LICENSE agreement in the main Bacula source code
directory.  When you sign the Fiduciary License Agreement (FLA) and send it
in, you are agreeing to the terms of that LICENSE file.

If you don't understand what we mean by future problems, please
examine the difficulties Mozilla was having finding
previous contributors at \elink{
http://www.mozilla.org/MPL/missing.html}
{http://www.mozilla.org/MPL/missing.html}. The other important issue is to
avoid copyright, patent, or intellectual property violations as was
(May 2003) claimed by SCO against IBM. 

Although the copyright will be held by the Free Software
Foundation Europe e.V., each developer is expected to indicate
that he wrote and/or modified a particular module (or file) and
any other sources.  The copyright assignment may seem a bit
unusual, but in reality, it is not.  Most large projects require
this.

If you have any doubts about this, please don't hesitate to ask.  The
objective is to assure the long term survival of the Bacula project. 

Items not needing a copyright assignment are: most small changes,
enhancements, or bug fixes of 5-10 lines of code, which amount to    
less than 20% of any particular file.

\subsection{Copyright Assignment -- Fiduciary License Agreement}
\index{Copyright Assignment}
\index{Assignment!Copyright}
\addcontentsline{toc}{subsubsection}{Copyright Assignment -- Fiduciary License Agreement}

Since this is not a commercial enterprise, and we prefer to believe in
everyone's good faith, previously developers could assign the copyright by
explicitly acknowledging that they do so in their first submission.  This
was sufficient if the developer is independent, or an employee of a
not-for-profit organization or a university.  However, in an effort to
ensure that the Bacula code is really clean, beginning in August 2006, all
previous and future developers with SVN write access will be asked to submit a
copyright assignment (or Fiduciary License Agreement -- FLA),
which means you agree to the LICENSE in the main source
directory. It also means that you receive back the right to use
the code that you have submitted.

Any developer who wants to contribute and is employed by a company should
either list the employer as the owner of the code, or get explicit
permission from him to sign the copyright assignment.  This is because in
many countries, all work that an employee does whether on company time or
in the employee's free time is considered to be Intellectual Property of
the company.  Obtaining official approval or an FLA from the company will
avoid misunderstandings between the employee, the company, and the Bacula
project.  A good number of companies have already followed this procedure.

The Fiduciary License Agreement is posted on the Bacula web site at:
\elink{http://www.bacula.org/en/FLA-bacula.en.pdf}{http://www.bacula.org/en/FLA-bacula.en.pdf}

The instructions for filling out this agreement are also at:
\elink{http://www.bacula.org/?page=fsfe}{http://www.bacula.org/?page=fsfe}

It should be filled out, then sent to:

\begin{verbatim}
     Kern Sibbald
     Cotes-de-Montmoiret 9
     1012 Lausanne
     Switzerland
\end{verbatim}

Please note that the above address is different from the officially
registered office mentioned in the document.  When you send in such a
complete document, please notify me: kern at sibbald dot com, and 
please add your email address to the FLA so that I can contact you
to confirm reception of the signed FLA.


\section{The Development Cycle}
\index{Developement Cycle}
\index{Cycle!Developement}
\addcontentsline{toc}{subsubsection}{Development Cycle}

As discussed on the email lists, the number of contributions are
increasing significantly.  We expect this positive trend
will continue.  As a consequence, we have modified how we do
development, and instead of making a list of all the features that we will
implement in the next version, each developer signs up for one (maybe
two) projects at a time, and when they are complete, and the code
is stable, we will release a new version.  The release cycle will probably
be roughly six months.

The difference is that with a shorter release cycle and fewer released
feature, we will have more time to review the new code that is being
contributed, and will be able to devote more time to a smaller number of
projects (some prior versions had too many new features for us to handle
correctly).

Future release schedules will be much the same, and the
number of new features will also be much the same providing that the
contributions continue to come -- and they show no signs of let up :-)

\index{Feature Requests}
{\bf Feature Requests:} \\
In addition, we have "formalizee" the feature requests a bit.

Instead of me maintaining an informal list of everything I run into 
(kernstodo), we now maintain a "formal" list of projects.  This 
means that all new feature requests, including those recently discussed on 
the email lists, must be formally submitted and approved. 

Formal submission of feature requests will take two forms: \\
1. non-mandatory, but highly recommended is to discuss proposed new features
on the mailing list.\\
2.  Formal submission of an Feature Request in a special format.  We'll
give an example of this below, but you can also find it on the web site
under "Support -\gt{} Feature Requests".  Since it takes a bit of time to
properly fill out a Feature Request form, you probably should check on the
email list first.

Once the Feature Request is received by the keeper of the projects list, it
will be sent to the Bacula project manager (Kern), and he will either
accept it (90% of the time), send it back asking for clarification (10% of
the time), send it to the email list asking for opinions, or reject it
(very few cases).

If it is accepted, it will go in the "projects" file (a simple ASCII file) 
maintained in the main Bacula source directory.

{\bf Implementation of Feature Requests:}\\
Any qualified developer can sign up for a project.  The project must have
an entry in the projects file, and the developer's name will appear in the
Status field.

{\bf How Feature Requests are accepted:}\\
Acceptance of Feature Requests depends on several things: \\
1.  feedback from users.  If it is negative, the Feature Request will probably not be
accepted.  \\
2.  the difficulty of the project.  A project that is so
difficult that we cannot imagine finding someone to implement probably won't
be accepted. Obviously if you know how to implement it, don't hesitate
to put it in your Feature Request  \\
 3.  whether or not the Feature Request fits within the current strategy of
Bacula (for example an Feature Request that requests changing the tape to
tar format probably would not be accepted, ...).

{\bf How Feature Requests are prioritized:}\\
Once an Feature Request is accepted, it needs to be implemented.  If you
can find a developer for it, or one signs up for implementing it, then the
Feature Request becomes top priority (at least for that developer).

Between releases of Bacula, we will generally solicit Feature Request input
for the next version, and by way of this email, we suggest that you send
discuss and send in your Feature Requests for the next release.  Please
verify that the Feature Request is not in the current list (attached to this email).

Once users have had several weeks to submit Feature Requests, the keeper of
the projects list will organize them, and request users to vote on them.
This will allow fixing prioritizing the Feature Requests.  Having a
priority is one thing, but getting it implement is another thing -- we are
hoping that the Bacula community will take more responsibility for assuring
the implementation of accepted Feature Requests.

Feature Request format:
\begin{verbatim}
============= Empty Feature Request form ===========
Item n:   One line summary ...
  Date:   Date submitted
  Origin: Name and email of originator.
  Status:

  What:   More detailed explanation ...

  Why:    Why it is important ...

  Notes:  Additional notes or features (omit if not used)
============== End Feature Request form ==============
\end{verbatim}

\begin{verbatim}
============= Example Completed  Feature Request form ===========
Item 1:   Implement a Migration job type that will move the job
          data from one device to another.
  Origin: Sponsored by Riege Sofware International GmbH. Contact:
          Daniel Holtkamp <holtkamp at riege dot com>
  Date:   28 October 2005
  Status: Partially coded in 1.37 -- much more to do. Assigned to
          Kern.

  What:   The ability to copy, move, or archive data that is on a
          device to another device is very important.

  Why:    An ISP might want to backup to disk, but after 30 days
          migrate the data to tape backup and delete it from
          disk.  Bacula should be able to handle this
          automatically.  It needs to know what was put where,
          and when, and what to migrate -- it is a bit like
          retention periods.  Doing so would allow space to be
          freed up for current backups while maintaining older
          data on tape drives.

  Notes:  Migration could be triggered by:
           Number of Jobs
           Number of Volumes
           Age of Jobs
           Highwater size (keep total size)
           Lowwater mark
=================================================
\end{verbatim}


\section{Bacula Code Submissions and Projects}
\index{Submissions and Projects}
\addcontentsline{toc}{subsection}{Code Submissions and Projects}

Getting code implemented in Bacula works roughly as follows:

\begin{itemize}

\item Kern is the project manager, but prefers not to be a "gate keeper".
      This means that the developers are expected to be self-motivated,
      and once they have experience submit directly to the Git
      repositories. However,
      it is a good idea to have your patches reviewed prior to submitting,
      and it is a bad idea to submit monster patches because no one will
      be able to properly review them. See below for more details on this.

\item  There are growing numbers of contributions (very good).

\item Some contributions come in the form of relatively small patches,
     which Kern reviews, integrates, documents, tests, and maintains.

\item All Bacula developers take full
   responsibility for writing the code, posting as patches so that we can
   review it as time permits, integrating it at an appropriate time,
   responding to our requests for tweaking it (name changes, ...),
   document it in the code, document it in the manual (even though
   their mother tongue is not English), test it, develop and commit
   regression scripts, and answer in a timely fashion all bug reports --
   even occasionally accepting additional bugs :-)

   This is a sustainable way of going forward with Bacula, and the
   direction that the project will be taking more and more.  For
   example, in the past, we have had some very dedicated programmers
   who did major projects. However, some of these
   programmers due to outside obligations (job responsibilities change of
   job, school duties, ...) could not continue to maintain the code.  In
   those cases, the code suffers from lack of maintenance, sometimes we
   patch it, sometimes not.  In the end, if the code is not maintained, the
   code gets dropped from the project (there are two such contributions
   that are heading in that direction).  When ever possible, we would like
   to avoid this, and ensure a continuation of the code and a sharing of
   the development, debugging, documentation, and maintenance
   responsibilities.
\end{itemize}

\section{Patches for Released Versions}
\index{Patches for Released Versions}
\addcontentsline{toc}{subsection}{Patches for Released Versions}
If you fix a bug in a released version, you should, unless it is
an absolutely trivial bug, create and release a patch file for the
bug.  The procedure is as follows:

Fix the bug in the released branch and in the develpment master branch.

Make a patch file for the branch and add the branch patch to 
the patches directory in both the branch and the trunk.
The name should be 2.2.4-xxx.patch  where xxx is unique, in this case it can 
be "restore", e.g. 2.2.4-restore.patch.  Add to the top of the
file  a brief description and instructions for applying it -- see for example 
2.2.4-poll-mount.patch.  The best way to create the patch file is as
follows:

\begin{verbatim}
  (edit) 2.2.4-restore.patch
  (input description)
  (end edit)

   git format-patch -M
   mv 0001-xxx 2.2.4-restore.patch
\end{verbatim}

check to make sure no extra junk got put into the patch file (i.e.
it should have the patch for that bug only).

If there is not a bug report on the problem, create one, then add the
patch to the bug report. 

Then upload it to the 2.2.x release of bacula-patches.

So, end the end, the patch file is:
\begin{itemize}
\item Attached to the bug report

\item In Branch-2.2/bacula/patches/...

\item In the trunk

\item Loaded on Source Forge bacula-patches 2.2.x release. When
   you add it, click on the check box to send an Email so that all the
   users that are monitoring SF patches get notified.
\end{itemize}


\section{Bacula Git repositories}
\index{Git}
\addcontentsline{toc}{subsection}{Git repositories}
As of September 2009, the Bacula source code has been split into
three Git repositories.  One is a repository that holds the
main Bacula source code with directories {\bf bacula}, {\bf gui},
and {\bf regress}.  The second repository contains
the directories {\bf docs} directory, and the third repository
contains the {\bf rescue} directory. All three repositories are 
hosted on Source Forge.

Previously everything was in a single SVN repository.
We have split the SVN repository into three because Git
offers significant advantages for ease of managing and integrating
developer's changes.  However, one of the disadvantages of Git is that you
must work with the full repository, while SVN allows you to checkout
individual directories.  If we put everything into a single Git
repository it would be far bigger than most developers would want
to checkout, so we have separted the docs and rescue into their own
repositories, and moved only the parts that are most actively 
worked on by the developers (bacula, gui, and regress) to a the
Git Bacula repository.  

Bacula developers must now have a certain knowledege
of Git.

\section{Git Usage}
\index{Git Usage}
\addcontentsline{toc}{subsection}{Git Usage}

Please note that if you are familiar with SVN, Git is similar,
(and better), but there can be a few surprising differences that
can lead to damaging the history of the repository (repo) if
you attempt to force pushing data into the Git repo.

The Bacula Git repo contains the subdirectories {\bf bacula}, {\bf gui},
and {\bf regress}. With Git it is not possible to pull only a
single directory, because of the hash code nature of Git, you
must take all or nothing.

For developers, the most important thing to remember about Git and
the Source Forge repository is not to "force" a {\bf push} to the
repository, and not to use the {\bf rebase} command on the {\bf
master} branch of the repository.  Doing so, will possibly rewrite
the Git repository history and cause a lot of problems for the 
project.

You may and should use {\bf rebase} on your own branches that you
want to synchronize with the {\bf master} branch, but please
do not use {\bf rebase} on the {\bf master} branch.  The proper
way of merging changes will be discussed below.

You can get a full copy of the Source Forge Bacula Git repository with the
following command:

\begin{verbatim}
git clone git://bacula.git.sourceforge.net/gitroot/bacula/bacula trunk
\end{verbatim}

This will put a read-only copy into the directory {\bf trunk} 
in your current directory, and {\bf trunk} will contain
the subdirectories: {\bf bacula}, {\bf gui}, and {\bf regress}.

If you have write permission, you can get a copy of the Git
repo with:

\begin{verbatim}
git clone ssh://<userid>@bacula.git.sourceforge.net/gitroot/bacula/bacula trunk
\end{verbatim}

where you replace \verb+<userid>+ with your Source Forge login 
userid, and you must have previously uploaded your public ssh key
to Source Forge.

The above command needs to be done only once. Thereafter, you can:

\begin{verbatim}
cd trunk
git pull
\end{verbatim}

As of August 2009, the size of the repository ({\bf trunk} in the above
example) will be approximately 55 Megabytes.  However, if you build
from source in this directory and do a lot of updates and regression
testing, the directory could become several hundred megabytes.

\subsection{Learning Git}
\index{Learning Git}
If you want to learn more about Git, we recommend that you visit:\\
\elink{http://book.git-scm.com/}{http://book.git-scm.com/}.

Some of the differences between Git and SVN are:
\begin{itemize}
\item Your main Git directory is a full Git repository to which you can
  and must commit.
\item The Git database is kept in the directory {\bf .git} at the 
  top level of the directory.
\item all the important Git configuration information is kept in the
  file {\bf .git/config} in ASCII format that is easy to manually edit.
\item When you do a {\bf commit} the changes are put in {\bf .git}
  rather than in the external repository.
\item You can upload your changes to the external repository using
  the command {\bf git push}.
\item You can download all the current changes in the external repository
  and merge them into your {\bf master} branch using the command
  {\bf gGit pull}.
\item The command {\bf git add} is used to add a new file to the
  repository AND to tell Git that you want a file that has changed
  to be in the next commit.  This has lots of advantages, because
  a {\bf git commit} only commits those files that have been 
  explicitly added.
\item You can add and commit all files modifed in one command
  using {\bf git commit -a}.
\item This extra use of {\bf add} allows you to make a number
  of changes then add only a few of the files and commit them,
  then add more files and commit them until you have committed
  everything. This has the advantage of allowing you to more
  easily group small changes and commit them.
\item If you {\bf git pull} from the main repository and make
  some changes, and before you do a {\bf git push}, someone
  else pushes changes to the Git repository, you will probably
  get an error message such as:

\begin{verbatim}
 git push
 To git@github.com:bacula/bacula.git
  ! [rejected]        master -> master (non-fast forward)
  error: failed to push some refs to 'git@github.com:bacula/bacula.git'
\end{verbatim}

 which is Git's way of telling you that the main repository has changed
 and that if you push your changes, they will not be integrated properly.
 As we have noted, you should never ask Git to force the push.
 See below for an explanation of why.
\item To integrate (merge) your changes properly, you should always do 
 a {\bf git pull} just prior to doing a {\bf git push}.
\item If Git is unable to merge your changes or finds a conflict it
  will tell you and you must do conflict resolution, which is much
  easier in Git than in SVN.
\item Resolving conflicts is described below in the {\bf github} section.
\end{itemize}

If you want to understand why it is not a good idea to force a 
push to the repository, look at the following picture:

\includegraphics[width=0.85\textwidth]{\idir git-edit-commit.eps}

The above graphic has three lines of circles. Each circle represents
a commit, and time runs from the left to the right.  The top line
shows the repository just before you are going to do a push. Note the
point at which you pulled is the circle on the left, your changes are
represented by the circle labeled {\bf Your mods}. It is shown below
to indicate that the changes are only in your local repository.  Finally,
there are pushes A and B that came after the time at which you pulled.

If you were to force your changes into the repository, Git would place them
immediately after the point at which you pulled them, so they would
go before the pushes A and B.  However, doing so would rewrite the history
of the repository and make it very difficult for other users to synchronize
since they would have to somehow wedge their changes at some point before the
current HEAD of the repository.  This situation is shown by the second line of
pushes.

What you really want to do is to put your changes after Push B (the current HEAD).
This is shown in the third line of pushes.  The best way to accomplish this is to
work in a branch, pull the repository so you have your master equal to HEAD (in first
line), then to rebase your branch on the current master and then commit it.  The 
exact commands to accomplish this are shown in the next couple of sections.

\subsection{Publishing your changes}
\index{Publishing}
Since Git is more complex than SVN, it takes a bit of time to learn how
to use it properly, and if you are not careful, you can potentially create
a new history in the repository. In addition, since Git is a distributed 
version control system, we prefer to receive a full branch submission rather
than simply a patch.  To accomplish this, you must create your changes in
a branch, then {\bf push} them to some public repository -- it can be your
own repository that you publish or another.  To simplify this phase for you, we
have created a publich Bacula Git repository on {\bf github} where you can
push your branch containing changes you would like integrated into the Bacula
source code.

Once you have pushed your branch to {\bf github} or told us where we can pull
from your public repository, one of the senior Bacula devlopers will fetch your 
changes, examine them, possibly make comments for changes they would like to
see, and as the final step, the senior developer will commit it to the
Bacula Source Forge Git repository.

\subsection{github}
\index{github}
If you are going to submit code, you create a login on
the Github website:\\
\elink{http://github.com/}{http://github.com/}\\
before you clone the repository.
You must also upload your public ssh key. Please see the instructions for
doing so at the above link.  Then you notify one of the senior Bacula developers,
who will authorize your Github user name as a committer to the Bacula repository. Finally,
you clone the Bacula repository with:

\begin{verbatim}
 git clone git@github.com:bacula/bacula.git <xxx>
\end{verbatim}

where you replace \verb+<xxx>+ with the name
of a directory that you want Git to create to hold your local Bacula Git
repository.

Normally, you will work by creating a branch of the master branch of your
repository, make your modifications, then make sure it is up to date, and finally
push it to Github. Assuming you call the Bacula repository {\bf bacula}, you might
use the following commands:

\begin{verbatim}
cd bacula
git checkout master
git pull 
git branch <your-name>/newbranch
git checkout <your-name>/newbranch
(edit, ...)
git add <file-edited>
git commit -m "<comment about commit>"
...
\end{verbatim}

Note, we request you to create the branch name ({\bf \verb+<your-name>+/newbranch} with your Github
login name. This guarantees that the branch name will be unique and
easily identified as well.

When you have completed working on your branch, you will do:

\begin{verbatim}
cd bacula
git checkout <your-name>/newbranch
git pull
git rebase master
\end{verbatim}

If you have completed your edits before anyone has modified the repository,
the {\bf git rebase master} will report that there was nothing to do. Otherwise,
it will merge the changes that were made in the repository before your changes.
If there are any conflicts, Git will tell you. Typically resolving conflicts with
Git is relatively easy.  You simply make a diff:

\begin{verbatim}
git diff
\end{verbatim}

Then edit each file that was listed in the {\bf git diff} to remove the
conflict, which will be indicated by lines of:

\begin{verbatim}
<<<<<<< HEAD
text
>>>>>>>>
other text
=====
\end{verbatim}

where {\bf text} is what is in the Bacula repository, and {\bf other text}
is what you have changed.

Once you have eliminated the conflict, the {\bf git diff} will show nothing,
and you must do a:

\begin{verbatim}
git add <file-with-conflicts-fixed>
\end{verbatim}

Once you have fixed all the files with conflicts in the above manner, you enter:

\begin{verbatim}
git rebase --continue
\end{verbatim}

and your rebase will be complete.

If for some reason, before doing the --continue, you want to abort the rebase and return to what you had, you enter:

\begin{verbatim}
git rebase --abort
\end{verbatim}

Finally to upload your branch, you do:

\begin{verbatim}
git push origin <your-name>/newbranch
\end{verbatim}

If you wish to delete it later, you can use:

\begin{verbatim}
git push origin :<your-name>/newbranch
\end{verbatim}


\section{Developing Bacula}
\index{Developing Bacula}
\index{Bacula!Developing}
\addcontentsline{toc}{subsubsection}{Developing Bacula}

Typically the simplest way to develop Bacula is to open one xterm window
pointing to the source directory you wish to update; a second xterm window at
the top source directory level, and a third xterm window at the bacula
directory \lt{}top\gt{}/src/bacula. After making source changes in one of the
directories, in the top source directory xterm, build the source, and start
the daemons by entering: 

make and 

./startit then in the enter: 

./console or 

./gnome-console to start the Console program. Enter any commands for testing.
For example: run kernsverify full. 

Note, the instructions here to use {\bf ./startit} are different from using a
production system where the administrator starts Bacula by entering {\bf
./bacula start}. This difference allows a development version of {\bf Bacula}
to be run on a computer at the same time that a production system is running.
The {\bf ./startit} strip starts {\bf Bacula} using a different set of
configuration files, and thus permits avoiding conflicts with any production
system. 

To make additional source changes, exit from the Console program, and in the
top source directory, stop the daemons by entering: 

./stopit then repeat the process. 

\subsection{Debugging}
\index{Debugging}
\addcontentsline{toc}{subsubsection}{Debugging}

Probably the first thing to do is to turn on debug output. 

A good place to start is with a debug level of 20 as in {\bf ./startit -d20}.
The startit command starts all the daemons with the same debug level.
Alternatively, you can start the appropriate daemon with the debug level you
want. If you really need more info, a debug level of 60 is not bad, and for
just about everything a level of 200. 

\subsection{Using a Debugger}
\index{Using a Debugger}
\index{Debugger!Using a}
\addcontentsline{toc}{subsubsection}{Using a Debugger}

If you have a serious problem such as a segmentation fault, it can usually be
found quickly using a good multiple thread debugger such as {\bf gdb}. For
example, suppose you get a segmentation violation in {\bf bacula-dir}. You
might use the following to find the problem: 

\lt{}start the Storage and File daemons\gt{}
cd dird
gdb ./bacula-dir
run -f -s -c ./dird.conf
\lt{}it dies with a segmentation fault\gt{}
where
The {\bf -f} option is specified on the {\bf run} command to inhibit {\bf
dird} from going into the background. You may also want to add the {\bf -s}
option to the run command to disable signals which can potentially interfere
with the debugging. 

As an alternative to using the debugger, each {\bf Bacula} daemon has a built
in back trace feature when a serious error is encountered. It calls the
debugger on itself, produces a back trace, and emails the report to the
developer. For more details on this, please see the chapter in the main Bacula
manual entitled ``What To Do When Bacula Crashes (Kaboom)''. 

\subsection{Memory Leaks}
\index{Leaks!Memory}
\index{Memory Leaks}
\addcontentsline{toc}{subsubsection}{Memory Leaks}

Because Bacula runs routinely and unattended on client and server machines, it
may run for a long time. As a consequence, from the very beginning, Bacula
uses SmartAlloc to ensure that there are no memory leaks. To make detection of
memory leaks effective, all Bacula code that dynamically allocates memory MUST
have a way to release it. In general when the memory is no longer needed, it
should be immediately released, but in some cases, the memory will be held
during the entire time that Bacula is executing. In that case, there MUST be a
routine that can be called at termination time that releases the memory. In
this way, we will be able to detect memory leaks. Be sure to immediately
correct any and all memory leaks that are printed at the termination of the
daemons. 

\subsection{Special Files}
\index{Files!Special}
\index{Special Files}
\addcontentsline{toc}{subsubsection}{Special Files}

Kern uses files named 1, 2, ... 9 with any extension as scratch files. Thus
any files with these names are subject to being rudely deleted at any time. 

\subsection{When Implementing Incomplete Code}
\index{Code!When Implementing Incomplete}
\index{When Implementing Incomplete Code}
\addcontentsline{toc}{subsubsection}{When Implementing Incomplete Code}

Please identify all incomplete code with a comment that contains 

\begin{verbatim}
***FIXME***
\end{verbatim} 

where there are three asterisks (*) before and after the word
FIXME (in capitals) and no intervening spaces. This is important as it allows
new programmers to easily recognize where things are partially implemented. 

\subsection{Bacula Source File Structure}
\index{Structure!Bacula Source File}
\index{Bacula Source File Structure}
\addcontentsline{toc}{subsubsection}{Bacula Source File Structure}

The distribution generally comes as a tar file of the form {\bf
bacula.x.y.z.tar.gz} where x, y, and z are the version, release, and update
numbers respectively. 

Once you detar this file, you will have a directory structure as follows: 

\footnotesize
\begin{verbatim}
|
Tar file:
|- depkgs
   |- mtx              (autochanger control program + tape drive info)
   |- sqlite           (SQLite database program)

Tar file:
|- depkgs-win32
   |- pthreads         (Native win32 pthreads library -- dll)
   |- zlib             (Native win32 zlib library)
   |- wx               (wxWidgets source code)

Project bacula:
|- bacula              (main source directory containing configuration
   |                    and installation files)
   |- autoconf         (automatic configuration files, not normally used
   |                    by users)
   |- intl             (programs used to translate)
   |- platforms        (OS specific installation files)
      |- redhat        (Red Hat installation)
      |- solaris       (Sun installation)
      |- freebsd       (FreeBSD installation)
      |- irix          (Irix installation -- not tested)
      |- unknown       (Default if system not identified)
   |- po               (translations of source strings)
   |- src              (source directory; contains global header files)
      |- cats          (SQL catalog database interface directory)
      |- console       (bacula user agent directory)
      |- dird          (Director daemon)
      |- filed         (Unix File daemon)
         |- win32      (Win32 files to make bacula-fd be a service)
      |- findlib       (Unix file find library for File daemon)
      |- gnome-console (GNOME version of console program)
      |- lib           (General Bacula library)
      |- stored        (Storage daemon)
      |- tconsole      (Tcl/tk console program -- not yet working)
      |- testprogs     (test programs -- normally only in Kern's tree)
      |- tools         (Various tool programs)
      |- win32         (Native Win32 File daemon)
         |- baculafd   (Visual Studio project file)
         |- compat     (compatibility interface library)
         |- filed      (links to src/filed)
         |- findlib    (links to src/findlib)
         |- lib        (links to src/lib)
         |- console    (beginning of native console program)
         |- wx-console (wxWidget console Win32 specific parts)
     |- wx-console     (wxWidgets console main source program)

Project regress:
|- regress             (Regression scripts)
   |- bin              (temporary directory to hold Bacula installed binaries)
   |- build            (temporary directory to hold Bacula source)
   |- scripts          (scripts and .conf files)
   |- tests            (test scripts)
   |- tmp              (temporary directory for temp files)
   |- working          (temporary working directory for Bacula daemons)

Project docs:
|- docs                (documentation directory)
   |- developers       (Developer's guide)
   |- home-page        (Bacula's home page source)
   |- manual           (html document directory)
   |- manual-fr        (French translation) 
   |- manual-de        (German translation) 
   |- techlogs         (Technical development notes);

Project rescue:
|- rescue              (Bacula rescue CDROM)
   |- linux            (Linux rescue CDROM)
      |- cdrom         (Linux rescue CDROM code)
        ...
   |- solaris          (Solaris rescue -- incomplete) 
   |- freebsd          (FreeBSD rescue -- incomplete)

Project gui:
|- gui                 (Bacula GUI projects)
  |- bacula-web        (Bacula web php management code)
  |- bimagemgr         (Web application for burning CDROMs)


\end{verbatim}
\normalsize

\subsection{Header Files}
\index{Header Files}
\index{Files!Header}
\addcontentsline{toc}{subsubsection}{Header Files}

Please carefully follow the scheme defined below as it permits in general only
two header file includes per C file, and thus vastly simplifies programming.
With a large complex project like Bacula, it isn't always easy to ensure that
the right headers are invoked in the right order (there are a few kludges to
make this happen -- i.e. in a few include files because of the chicken and egg
problem, certain references to typedefs had to be replaced with {\bf void} ). 

Every file should include {\bf bacula.h}. It pulls in just about everything,
with very few exceptions. If you have system dependent ifdefing, please do it
in {\bf baconfig.h}. The version number and date are kept in {\bf version.h}. 

Each of the subdirectories (console, cats, dird, filed, findlib, lib, stored,
...) contains a single directory dependent include file generally the name of
the directory, which should be included just after the include of {\bf
bacula.h}. This file (for example, for the dird directory, it is {\bf dird.h})
contains either definitions of things generally needed in this directory, or
it includes the appropriate header files. It always includes {\bf protos.h}.
See below. 

Each subdirectory contains a header file named {\bf protos.h}, which contains
the prototypes for subroutines exported by files in that directory. {\bf
protos.h} is always included by the main directory dependent include file. 

\subsection{Programming Standards}
\index{Standards!Programming}
\index{Programming Standards}
\addcontentsline{toc}{subsubsection}{Programming Standards}

For the most part, all code should be written in C unless there is a burning
reason to use C++, and then only the simplest C++ constructs will be used.
Note, Bacula is slowly evolving to use more and more C++. 

Code should have some documentation -- not a lot, but enough so that I can
understand it. Look at the current code, and you will see that I document more
than most, but am definitely not a fanatic. 

We prefer simple linear code where possible. Gotos are strongly discouraged
except for handling an error to either bail out or to retry some code, and
such use of gotos can vastly simplify the program. 

Remember this is a C program that is migrating to a {\bf tiny} subset of C++,
so be conservative in your use of C++ features. 

\subsection{Do Not Use}
\index{Use!Do Not}
\index{Do Not Use}
\addcontentsline{toc}{subsubsection}{Do Not Use}

\begin{itemize}
 \item STL -- it is totally incomprehensible. 
\end{itemize}

\subsection{Avoid if Possible}
\index{Possible!Avoid if}
\index{Avoid if Possible}
\addcontentsline{toc}{subsubsection}{Avoid if Possible}

\begin{itemize}
\item Using {\bf void *} because this generally means that one must
   using casting, and in C++ casting is rather ugly.  It is OK to use
   void * to pass structure address where the structure is not known  
   to the routines accepting the packet (typically callback routines).
   However, declaring "void *buf" is a bad idea.  Please use the
   correct types whenever possible.

\item Using undefined storage specifications such as (short, int, long,
   long long, size\_t ...).  The problem with all these is that the number of bytes
   they allocate depends on the compiler and the system.  Instead use
   Bacula's types (int8\_t, uint8\_t, int32\_t, uint32\_t, int64\_t, and 
   uint64\_t).  This guarantees that the variables are given exactly the
   size you want. Please try at all possible to avoid using size\_t ssize\_t
   and the such. They are very system dependent.  However, some system
   routines may need them, so their use is often unavoidable.

\item Returning a malloc'ed buffer from a subroutine --  someone will forget
   to release it. 

\item Heap allocation (malloc) unless needed -- it is expensive. Use
      POOL\_MEM instead.

\item Templates -- they can create portability problems. 

\item Fancy or tricky C or C++ code, unless you give a  good explanation of
   why you used it. 

\item Too much inheritance -- it can complicate the code,  and make reading it
   difficult (unless you are in love  with colons) 

\end{itemize}

\subsection{Do Use Whenever Possible}
\index{Possible!Do Use Whenever}
\index{Do Use Whenever Possible}
\addcontentsline{toc}{subsubsection}{Do Use Whenever Possible}

\begin{itemize}
\item Locking and unlocking within a single subroutine.  

\item A single point of exit from all subroutines. A goto is 
     perfectly OK to use to get out early, but only to a label
     named bail\_out, and possibly an ok\_out.  See current code
     examples.

\item Malloc and free within a single subroutine.  

\item Comments and global explanations on what your code or  algorithm does. 

\end{itemize}

\subsection{Indenting Standards}
\index{Standards!Indenting}
\index{Indenting Standards}
\addcontentsline{toc}{subsubsection}{Indenting Standards}

We find it very hard to read code indented 8 columns at a time. 
Even 4 at a time uses a lot of space, so we have adopted indenting
3 spaces at every level. Note, indention is the visual appearance of the
source on the page, while tabbing is replacing a series of up to 8 spaces from
a tab character. 

The closest set of parameters for the Linux {\bf indent} program that will
produce reasonably indented code are: 

\footnotesize
\begin{verbatim}
-nbad -bap -bbo -nbc -br -brs -c36 -cd36 -ncdb -ce -ci3 -cli0
-cp36 -d0 -di1 -ndj -nfc1 -nfca -hnl -i3 -ip0 -l85 -lp -npcs
-nprs -npsl -saf -sai -saw -nsob -nss -nbc -ncs -nbfda
\end{verbatim}
\normalsize

You can put the above in your .indent.pro file, and then just invoke indent on
your file. However, be warned. This does not produce perfect indenting, and it
will mess up C++ class statements pretty badly. 

Braces are required in all if statements (missing in some very old code). To
avoid generating too many lines, the first brace appears on the first line
(e.g. of an if), and the closing brace is on a line by itself. E.g. 

\footnotesize
\begin{verbatim}
   if (abc) {
      some_code;
  }
\end{verbatim}
\normalsize

Just follow the convention in the code. For example we I prefer non-indented cases. 

\footnotesize
\begin{verbatim}
   switch (code) {
   case 'A':
      do something
      break;
   case 'B':
      again();
      break;
   default:
      break;
  }
\end{verbatim}
\normalsize

Avoid using // style comments except for temporary code or turning off debug
code. Standard C comments are preferred (this also keeps the code closer to
C). 

Attempt to keep all lines less than 85 characters long so that the whole line
of code is readable at one time. This is not a rigid requirement. 

Always put a brief description at the top of any new file created describing
what it does and including your name and the date it was first written. Please
don't forget any Copyrights and acknowledgments if it isn't 100\% your code.
Also, include the Bacula copyright notice that is in {\bf src/c}. 

In general you should have two includes at the top of the an include for the
particular directory the code is in, for includes are needed, but this should
be rare. 

In general (except for self-contained packages), prototypes should all be put
in {\bf protos.h} in each directory. 

Always put space around assignment and comparison operators. 

\footnotesize
\begin{verbatim}
   a = 1;
   if (b >= 2) {
     cleanup();
  }
\end{verbatim}
\normalsize

but your can compress things in a {\bf for} statement: 

\footnotesize
\begin{verbatim}
   for (i=0; i < del.num_ids; i++) {
    ...
\end{verbatim}
\normalsize

Don't overuse the inline if (?:). A full {\bf if} is preferred, except in a
print statement, e.g.: 

\footnotesize
\begin{verbatim}
   if (ua->verbose \&& del.num_del != 0) {
      bsendmsg(ua, _("Pruned %d %s on Volume %s from catalog.\n"), del.num_del,
         del.num_del == 1 ? "Job" : "Jobs", mr->VolumeName);
  }
\end{verbatim}
\normalsize

Leave a certain amount of debug code (Dmsg) in code you submit, so that future
problems can be identified. This is particularly true for complicated code
likely to break. However, try to keep the debug code to a minimum to avoid
bloating the program and above all to keep the code readable. 

Please keep the same style in all new code you develop. If you include code
previously written, you have the option of leaving it with the old indenting
or re-indenting it. If the old code is indented with 8 spaces, then please
re-indent it to Bacula standards. 

If you are using {\bf vim}, simply set your tabstop to 8 and your shiftwidth
to 3. 

\subsection{Tabbing}
\index{Tabbing}
\addcontentsline{toc}{subsubsection}{Tabbing}

Tabbing (inserting the tab character in place of spaces) is as normal on all
Unix systems -- a tab is converted space up to the next column multiple of 8.
My editor converts strings of spaces to tabs automatically -- this results in
significant compression of the files. Thus, you can remove tabs by replacing
them with spaces if you wish. Please don't confuse tabbing (use of tab
characters) with indenting (visual alignment of the code). 

\subsection{Don'ts}
\index{Don'ts}
\addcontentsline{toc}{subsubsection}{Don'ts}

Please don't use: 

\footnotesize
\begin{verbatim}
strcpy()
strcat()
strncpy()
strncat();
sprintf()
snprintf()
\end{verbatim}
\normalsize

They are system dependent and un-safe. These should be replaced by the Bacula
safe equivalents: 

\footnotesize
\begin{verbatim}
char *bstrncpy(char *dest, char *source, int dest_size);
char *bstrncat(char *dest, char *source, int dest_size);
int bsnprintf(char *buf, int32_t buf_len, const char *fmt, ...);
int bvsnprintf(char *str, int32_t size, const char  *format, va_list ap);
\end{verbatim}
\normalsize

See src/lib/bsys.c for more details on these routines. 

Don't use the {\bf \%lld} or the {\bf \%q} printf format editing types to edit
64 bit integers -- they are not portable. Instead, use {\bf \%s} with {\bf
edit\_uint64()}. For example: 

\footnotesize
\begin{verbatim}
   char buf[100];
   uint64_t num = something;
   char ed1[50];
   bsnprintf(buf, sizeof(buf), "Num=%s\n", edit_uint64(num, ed1));
\end{verbatim}
\normalsize

Note: {\bf \%lld} is now permitted in Bacula code -- we have our
own printf routines which handle it correctly. The edit\_uint64() subroutine
can still be used if you wish, but over time, most of that old style will
be removed.

The edit buffer {\bf ed1} must be at least 27 bytes long to avoid overflow.
See src/lib/edit.c for more details. If you look at the code, don't start
screaming that I use {\bf lld}. I actually use subtle trick taught to me by
John Walker. The {\bf lld} that appears in the editing routine is actually
{\bf \#define} to a what is needed on your OS (usually ``lld'' or ``q'') and
is defined in autoconf/configure.in for each OS. C string concatenation causes
the appropriate string to be concatenated to the ``\%''. 

Also please don't use the STL or Templates or any complicated C++ code. 

\subsection{Message Classes}
\index{Classes!Message}
\index{Message Classes}
\addcontentsline{toc}{subsubsection}{Message Classes}

Currently, there are five classes of messages: Debug, Error, Job, Memory, 
and Queued.

\subsection{Debug Messages}
\index{Messages!Debug}
\index{Debug Messages}
\addcontentsline{toc}{subsubsection}{Debug Messages}

Debug messages are designed to be turned on at a specified debug level and are
always sent to STDOUT. There are designed to only be used in the development
debug process. They are coded as: 

DmsgN(level, message, arg1, ...) where the N is a number indicating how many
arguments are to be substituted into the message (i.e. it is a count of the
number arguments you have in your message -- generally the number of percent
signs (\%)). {\bf level} is the debug level at which you wish the message to
be printed. message is the debug message to be printed, and arg1, ... are the
arguments to be substituted. Since not all compilers support \#defines with
varargs, you must explicitly specify how many arguments you have. 

When the debug message is printed, it will automatically be prefixed by the
name of the daemon which is running, the filename where the Dmsg is, and the
line number within the file. 

Some actual examples are: 

Dmsg2(20, ``MD5len=\%d MD5=\%s\textbackslash{}n'', strlen(buf), buf); 

Dmsg1(9, ``Created client \%s record\textbackslash{}n'', client->hdr.name); 

\subsection{Error Messages}
\index{Messages!Error}
\index{Error Messages}
\addcontentsline{toc}{subsubsection}{Error Messages}

Error messages are messages that are related to the daemon as a whole rather
than a particular job. For example, an out of memory condition my generate an
error message. They should be very rarely needed. In general, you should be
using Job and Job Queued messages (Jmsg and Qmsg). They are coded as: 

EmsgN(error-code, level, message, arg1, ...) As with debug messages, you must
explicitly code the of arguments to be substituted in the message. error-code
indicates the severity or class of error, and it may be one of the following: 

\addcontentsline{lot}{table}{Message Error Code Classes}
\begin{longtable}{lp{3in}}
{{\bf M\_ABORT} } & {Causes the daemon to immediately abort. This should be
used only  in extreme cases. It attempts to produce a  traceback. } \\
{{\bf M\_ERROR\_TERM} } & {Causes the daemon to immediately terminate. This
should be used only  in extreme cases. It does not produce a  traceback. } \\
{{\bf M\_FATAL} } & {Causes the daemon to terminate the current job, but the
daemon keeps running } \\
{{\bf M\_ERROR} } & {Reports the error. The daemon and the job continue
running } \\
{{\bf M\_WARNING} } & {Reports an warning message. The daemon and the job
continue running } \\
{{\bf M\_INFO} } & {Reports an informational message.}

\end{longtable}

There are other error message classes, but they are in a state of being
redesigned or deprecated, so please do not use them. Some actual examples are:


Emsg1(M\_ABORT, 0, ``Cannot create message thread: \%s\textbackslash{}n'',
strerror(status)); 

Emsg3(M\_WARNING, 0, ``Connect to File daemon \%s at \%s:\%d failed. Retrying
...\textbackslash{}n'',  client-\gt{}hdr.name, client-\gt{}address,
client-\gt{}port); 

Emsg3(M\_FATAL, 0, ``bdird\lt{}filed: bad response from Filed to \%s command:
\%d \%s\textbackslash{}n'',  cmd, n, strerror(errno)); 

\subsection{Job Messages}
\index{Job Messages}
\index{Messages!Job}
\addcontentsline{toc}{subsubsection}{Job Messages}

Job messages are messages that pertain to a particular job such as a file that
could not be saved, or the number of files and bytes that were saved. They
Are coded as:
\begin{verbatim}
Jmsg(jcr, M\_FATAL, 0, "Text of message");
\end{verbatim}
A Jmsg with M\_FATAL will fail the job. The Jmsg() takes varargs so can
have any number of arguments for substituted in a printf like format.
Output from the Jmsg() will go to the Job report.
<br>
If the Jmsg is followed with a number such as Jmsg1(...), the number
indicates the number of arguments to be substituted (varargs is not
standard for \#defines), and what is more important is that the file and
line number will be prefixed to the message. This permits a sort of debug
from user's output.

\subsection{Queued Job Messages}
\index{Queued Job Messages}
\index{Messages!Job}
\addcontentsline{toc}{subsubsection}{Queued Job Messages}
Queued Job messages are similar to Jmsg()s except that the message is
Queued rather than immediately dispatched. This is necessary within the
network subroutines and in the message editing routines. This is to prevent
recursive loops, and to ensure that messages can be delivered even in the
event of a network error.


\subsection{Memory Messages}
\index{Messages!Memory}
\index{Memory Messages}
\addcontentsline{toc}{subsubsection}{Memory Messages}

Memory messages are messages that are edited into a memory buffer. Generally
they are used in low level routines such as the low level device file dev.c in
the Storage daemon or in the low level Catalog routines. These routines do not
generally have access to the Job Control Record and so they return error
essages reformatted in a memory buffer. Mmsg() is the way to do this. 

\subsection{Bugs Database}
\index{Database!Bugs}
\index{Bugs Database}
\addcontentsline{toc}{subsubsection}{Bugs Database}
We have a bugs database which is at:
\elink{http://bugs.bacula.org}{http://bugs.bacula.org}, and as
a developer you will need to respond to bugs, perhaps bugs in general 
if you have time, otherwise just bugs that correspond to code that
you wrote.

If you need to answer bugs, please be sure to ask the Project Manager
(currently Kern) to give you Developer access to the bugs database. This
allows you to modify statuses and close bugs.

The first thing is if you want to take over a bug, rather than just make a
note, you should assign the bug to yourself. This helps other developers
know that you are the principal person to deal with the bug.  You can do so
by going into the bug and clicking on the {\bf Update Issue} button. Then
you simply go to the {\bf Assigned To} box and select your name from the
drop down box.  To actually update it you must click on the {\bf Update
Information} button a bit further down on the screen, but if you have other
things to do such as add a Note, you might wait before clicking on the {\bf
Update Information} button.

Generally, we set the {\bf Status} field to either acknowledged, confirmed,
or feedback when we first start working on the bug.  Feedback is set when
we expect that the user should give us more information.

Normally, once you are reasonably sure that the bug is fixed, and a patch
is made and attached to the bug report, and/or in the SVN, you can close
the bug.  If you want the user to test the patch, then leave the bug open,
otherwise close it and set {\bf Resolution} to {\bf Fixed}.  We generally
close bug reports rather quickly, even without confirmation, especially if
we have run tests and can see that for us the problem is fixed.  However,
in doing so, it avoids misunderstandings if you leave a note while you are
closing the bug that says something to the following effect:
We are closing this bug because ...   If for some reason, it does not fix
your problem, please feel free to reopen it, or to open a new bug report
describing the problem".

We do not recommend that you attempt to edit any of the bug notes that have
been submitted, nor to delete them or make them private.  In fact, if
someone accidentally makes a bug note private, you should ask the reason
and if at all possible (with his agreement) make the bug note public.  

If the user has not properly filled in most of the important fields
(platorm, OS, Product Version, ...) please do not hesitate to politely ask
him.  Also, if the bug report is a request for a new feature, please
politely send the user to the Feature Request menu item on www.bacula.org.
The same applies to a support request (we answer only bugs), you might give
the user a tip, but please politely refer him to the manual and the
Getting Support page of www.bacula.org.