Commit two patches from Philippe to cleanup the manuals

[bacula/docs] / docs / manuals / en / developers / git.tex

\chapter{Bacula Git Usage}
\label{_GitChapterStart}
\index{Git}
\index{Git!Repo}
\addcontentsline{toc}{section}{Bacula Bit Usage}

This chapter is intended to help you use the Git source code
repositories to obtain, modify, and submit Bacula source code.


\section{Bacula Git repositories}
\index{Git}
\addcontentsline{toc}{subsection}{Git repositories}
As of September 2009, the Bacula source code has been split into
three Git repositories.  One is a repository that holds the
main Bacula source code with directories {\bf bacula}, {\bf gui},
and {\bf regress}.  The second repository contains
the directories {\bf docs} directory, and the third repository
contains the {\bf rescue} directory. All three repositories are 
hosted by UKFast.

Previously everything was in a single SVN repository.
We have split the SVN repository into three because Git
offers significant advantages for ease of managing and integrating
developer's changes.  However, one of the disadvantages of Git is that you
must work with the full repository, while SVN allows you to checkout
individual directories.  If we put everything into a single Git
repository it would be far bigger than most developers would want
to checkout, so we have separted the docs and rescue into their own
repositories, and moved only the parts that are most actively 
worked on by the developers (bacula, gui, and regress) to a the
Git Bacula repository.  

Bacula developers must now have a certain knowledege of Git.

\section{Git Usage}
\index{Git Usage}
\addcontentsline{toc}{subsection}{Git Usage}

Please note that if you are familiar with SVN, Git is similar,
(and better), but there can be a few surprising differences that
can be very confusing (nothing worse than converting from CVS to SVN).

The main Bacula Git repo contains the subdirectories {\bf bacula}, {\bf gui},
and {\bf regress}. With Git it is not possible to pull only a
single directory, because of the hash code nature of Git, you
must take all or nothing.

For developers, the most important thing to remember about Git and
the Source Forge repository is not to "force" a {\bf push} to the
repository. Doing so, can possibly rewrite
the Git repository history and cause a lot of problems for the 
project.

You can get a full copy of the Source Forge Bacula Git repository with the
following command:

\begin{verbatim}
git clone http://git.bacula.org/bacula trunk
\end{verbatim}

This will put a read-only copy into the directory {\bf trunk} 
in your current directory, and {\bf trunk} will contain
the subdirectories: {\bf bacula}, {\bf gui}, and {\bf regress}.
Obviously you can use any name an not just {\bf trunk}.  In fact,
once you have the repository in say {\bf trunk}, you can copy the
whole directory to another place and have a fully functional
git repository.

The above command needs to be done only once. Thereafter, you can:

\begin{verbatim}
cd trunk
git pull     # refresh my repo with the latest code
\end{verbatim}

As of August 2009, the size of the repository ({\bf trunk} in the above
example) will be approximately 55 Megabytes.  However, if you build
from source in this directory and do a lot of updates and regression
testing, the directory could become several hundred megabytes.

\subsection{Learning Git}
\index{Learning Git}
If you want to learn more about Git, we recommend that you visit:\\
\elink{http://book.git-scm.com/}{http://book.git-scm.com/}.

Some of the differences between Git and SVN are:
\begin{itemize}
\item Your main Git directory is a full Git repository to which you can
  and must commit. In fact, we suggest you commit frequently.
\item When you commit, the commit goes into your local Git
  database.  You must use another command to write it to the
  master Source Forge repository (see below).
\item The local Git database is kept in the directory {\bf .git} at the 
  top level of the directory.
\item All the important Git configuration information is kept in the
  file {\bf .git/config} in ASCII format that is easy to manually edit.
\item When you do a {\bf commit} the changes are put in {\bf .git}
  rather but not in the main Source Forge repository.
\item You can push your changes to the external repository using
  the command {\bf git push} providing you have write permission
  on the repository.
\item We restrict developers just learning git to have read-only
  access until they feel comfortable with git before giving them
  write access.
\item You can download all the current changes in the external repository
  and merge them into your {\bf master} branch using the command
  {\bf git pull}.
\item The command {\bf git add} is used to add a new file to the
  repository AND to tell Git that you want a file that has changed
  to be in the next commit.  This has lots of advantages, because
  a {\bf git commit} only commits those files that have been 
  explicitly added.  Note with SVN {\bf add} is used only
  to add new files to the repo.
\item You can add and commit all files modifed in one command
  using {\bf git commit -a}.
\item This extra use of {\bf add} allows you to make a number
  of changes then add only a few of the files and commit them,
  then add more files and commit them until you have committed
  everything. This has the advantage of allowing you to more
  easily group small changes and do individaual commits on them.
  By keeping commits smaller, and separated into topics, it makes
  it much easier to later select certain commits for backporting.
\item If you {\bf git pull} from the main repository and make
  some changes, and before you do a {\bf git push} someone
  else pushes changes to the Git repository, your changes will
  apply to an older version of the repository you will probably
  get an error message such as:

\begin{verbatim}
 git push
 To git@github.com:bacula/bacula.git
  ! [rejected]        master -> master (non-fast forward)
  error: failed to push some refs to 'git@github.com:bacula/bacula.git'
\end{verbatim}

 which is Git's way of telling you that the main repository has changed
 and that if you push your changes, they will not be integrated properly.
 This is very similar to what happens when you do an "svn update" and
 get merge conflicts.
 As we have noted above, you should never ask Git to force the push.
 See below for an explanation of why.
\item To integrate (merge) your changes properly, you should always do 
 a {\bf git pull} just prior to doing a {\bf git push}.
\item If Git is unable to merge your changes or finds a conflict it
  will tell you and you must do conflict resolution, which is much
  easier in Git than in SVN.
\item Resolving conflicts is described below in the {\bf github} section.
\end{itemize}

\section{Step by Step Modifying Bacula Code}
Suppose you want to download Bacula source code, build it, make
a change, then submit your change to the Bacula developers.  What
would you do?

\begin{itemize}
\item Tell git who you are:\\
\begin{verbatim}
git config --global user.name "First-name Last-name"
git config --global user.email "email@address.com"
\end{verbatim}

Where you put your real name and your email address. Since
this is global, you only need to do it once on any given
machine regardless of how many git repos you work with.

\item Download the Source code:\\
\begin{verbatim}
git clone http://git.bacula.org/bacula trunk
\end{verbatim}

\item Configure and Build Bacula:\\
\begin{verbatim}
./configure (all-your-normal-options)
make
\end{verbatim}

\item Create a branch to work on:
\begin{verbatim}
cd trunk/bacula
git checkout -b bugfix master
\end{verbatim}

\item Edit, build, Test, ...\\
\begin{verbatim}
edit file jcr.h
make
test
\end{verbatim}

Note: if you forget to create a working branch prior to making
changes, and you make them on master, this is no problem providing
that you create the working branch before your first commit.
So assuming that you have edited master instead of your bugfix
branch, you can simply:

\begin{verbatim}
git checkout -b bugfix master
\end{verbatim}

and a new bugfix branch will be created and checked out.
You can then proceed to committing to your bugfix branch as
described in the next step.

\item commit your work:
\begin{verbatim}
git commit -am "Short comment on what I did"
\end{verbatim}

\item Possibly repeat the above two items

\item Switch back to the master branch:\\
\begin{verbatim}
git checkout master
\end{verbatim}

\item Pull the latest changes:\\
\begin{verbatim}
git pull
\end{verbatim}

\item Get back on your bugfix branch:\\
\begin{verbatim}
git checkout bugfix
\end{verbatim}

\item Merge your changes and correct any conflicts:\\
\begin{verbatim}
git rebase master bugfix
\end{verbatim}

\item Fix any conflicts:\\
You will be notified if there are conflicts. The first
thing to do is:

\begin{verbatim}
git diff
\end{verbatim}

This will produce a diff of only the files having a conflict.
Fix each file in turn. When it is fixed, the diff for that file
will go away. 

For each file fixed, you must do the same as SVN, inform git with:

\begin{verbatim}
git add (name-of-file-no-longer-in-conflict)
\end{verbatim}

\item When all files are fixed do:
\begin{verbatim}
git rebase --continue
\end{verbatim}

\item If you find that it is impossible to reconcile the two
      branches or you made a mistake in correcting and adding files,
before you enter the:
\begin{verbatim}
git rebase --continue
\end{verbatim}
you can instead enter:

\begin{verbatim}
git rebase --abort
\end{verbatim}

      which will essentially cancel the the original git rebase and reset
      everything to the beginning with no changes to your bugfix branch.

\item When you have completed the rebase and 
      are ready to send a patch, do the following:\\
\begin{verbatim}
git checkout bugfix
git format-patch -M master
\end{verbatim}
       Look at the files produced.  They should be numbered 0001-xxx.patch
       where there is one file for each commit you did, number sequentially,
       and the xxx is what you put in the commit comment.

\item If the patch files are good, send them by email to the developers
      as attachments.

\item Then you can continue working on your code if you want, or
      start another branch with a new project.

\item If you continue working on your bugfix branch, you should
      do a {\bf git rebase master} from time to time, and when
      your changes are committed to the repo, you will be automatically
      synchronized.  So that the next {\bf git format-patch} will produce
      only the changes you made since the last format-patch you sent
      to the developers.


\end{itemize}



\subsection{More Details}

Normally, you will work by creating a branch of the master branch of your
repository, make your modifications, then make sure it is up to date, and finally
create format-patch patches or push it to the Source Forge repo. Assuming
you call the Bacula repository {\bf trunk}, you might use the following
commands:

\begin{verbatim}
cd trunk
git checkout master
git pull 
git checkout -b newbranch master
(edit, ...)
git add <file-edited>
git commit -m "<comment about commit>"
...
\end{verbatim}

When you have completed working on your branch, you will do:

\begin{verbatim}
cd trunk
git checkout newbranch  # ensure I am on my branch
git pull                # get latest source code
git rebase master       # merge my code
\end{verbatim}

If you have completed your edits before anyone has modified the repository,
the {\bf git rebase master} will report that there was nothing to do. Otherwise,
it will merge the changes that were made in the repository before your changes.
If there are any conflicts, Git will tell you. Typically resolving conflicts with
Git is relatively easy.  You simply make a diff:

\begin{verbatim}
git diff
\end{verbatim}

Then edit each file that was listed in the {\bf git diff} to remove the
conflict, which will be indicated by lines of:

\begin{verbatim}
<<<<<<< HEAD
text
>>>>>>>>
other text
=====
\end{verbatim}

where {\bf text} is what is in the Bacula repository, and {\bf other text}
is what you have changed.

Once you have eliminated the conflict, the {\bf git diff} will show nothing,
and you must do a:

\begin{verbatim}
git add <file-with-conflicts-fixed>
\end{verbatim}

Once you have fixed all the files with conflicts in the above manner, you enter:

\begin{verbatim}
git rebase --continue
\end{verbatim}

and your rebase will be complete.

If for some reason, before doing the --continue, you want to abort the rebase and return to what you had, you enter:

\begin{verbatim}
git rebase --abort
\end{verbatim}

Finally to make a set of patch files

\begin{verbatim}
git format-patch -M master
\end{verbatim}

When you see your changes have been integrated and pushed to the
main repo, you can delete your branch with:

\begin{verbatim}
git checkout master
git branch -D newbranch
\end{verbatim}


\section{Forcing Changes}
If you want to understand why it is not a good idea to force a 
push to the repository, look at the following picture:

\begin{center}
\includegraphics[width=0.65\linewidth]{git-edit-commit}
\end{center}
The above graphic has three lines of circles. Each circle represents
a commit, and time runs from the left to the right.  The top line
shows the repository just before you are going to do a push. Note the
point at which you pulled is the circle on the left, your changes are
represented by the circle labeled {\bf Your mods}. It is shown below
to indicate that the changes are only in your local repository.  Finally,
there are pushes A and B that came after the time at which you pulled.

If you were to force your changes into the repository, Git would place them
immediately after the point at which you pulled them, so they would
go before the pushes A and B.  However, doing so would rewrite the history
of the repository and make it very difficult for other users to synchronize
since they would have to somehow wedge their changes at some point before the
current HEAD of the repository.  This situation is shown by the second line of
pushes.

What you really want to do is to put your changes after Push B (the current HEAD).
This is shown in the third line of pushes.  The best way to accomplish this is to
work in a branch, pull the repository so you have your master equal to HEAD (in first
line), then to rebase your branch on the current master and then commit it.  The 
exact commands to accomplish this are shown in the next couple of sections.