Updates

[bacula/docs] / docs / manual / restore.tex

%%
%%

\section*{The Bacula Console Restore Command}
\label{RestoreChapter}
\index[general]{Command!Bacula Console Restore }
\index[general]{Bacula Console Restore Command }

\section{General}
\index[general]{General }

Below, we will discuss restoring files with the Console {\bf restore} command,
which is the recommended way of doing it. However, there is a standalone
program named {\bf bextract}, which also permits restoring files. For more
information on this program, please see the 
\ilink{Bacula Utility Programs}{bextract} chapter of this manual.
You will also want to look at the {\bf bls} program in the same chapter, which
allows you to list the contents of your Volumes. Finally, if you have an old
Volume that is no longer in the catalog, you can restore the catalog entries
using the program named {\bf bscan}, documented in the same 
\ilink{Bacula Utility Programs}{bextract} chapter. 

In general, to restore a file or a set of files, you must run a {\bf restore}
job. That is a job with {\bf Type = Restore}. As a consequence, you will need
a predefined {\bf restore} job in your {\bf bacula-dir.conf} (Director's
config) file. The exact parameters (Client, FileSet, ...) that you define are
not important as you can either modify them manually before running the job or
if you use the {\bf restore} command, explained below, Bacula will  
automatically set them for you. In fact, you can no longer simply run a restore
job.  You must use the restore command.                                  

Since Bacula is a network backup program, you must be aware that when you
restore files, it is up to you to ensure that you or Bacula have selected the
correct Client and the correct hard disk location for restoring those files.
{\bf Bacula} will quite willingly backup client A, and restore it by sending
the files to a different directory on client B. Normally, you will want to
avoid this, but assuming the operating systems are not too different in their
file structures, this should work perfectly well, if so desired. 
By default, Bacula will restore data to the same Client that was backed
up, and those data will be restored not to the original places but to
{\bf /tmp/bacula-restores}.  You may modify any of these defaults when the
restore command prompts you to run the job by selecting the {\bf mod}
option.

\label{Example1}
\section{The Restore Command}
\index[general]{Command!Restore }
\index[general]{Restore Command }

Since Bacula maintains a catalog of your files and on which Volumes (disk or
tape), they are stored, it can do most of the bookkeeping work, allowing you
simply to specify what kind of restore you want (current, before a particular
date), and what files to restore. Bacula will then do the rest. 

This is accomplished using the {\bf restore} command in the Console. First you
select the kind of restore you want, then the JobIds are selected,
the File records for those Jobs are placed in an internal Bacula directory
tree, and the restore enters a file selection mode that allows you to
interactively walk up and down the file tree selecting individual files to be
restored. This mode is somewhat similar to the standard Unix {\bf restore}
program's interactive file selection mode. 

If a Job's file records have been pruned from the catalog, the {\bf
restore} command will be unable to find any files to restore.  See below
for more details on this.

Within the Console program, after entering the {\bf restore} command, you are
presented with the following selection prompt:  

\footnotesize
\begin{verbatim}
First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.
To select the JobIds, you have the following choices:
     1: List last 20 Jobs run
     2: List Jobs where a given File is saved
     3: Enter list of comma separated JobIds to select
     4: Enter SQL list command
     5: Select the most recent backup for a client
     6: Select backup for a client before a specified time
     7: Enter a list of files to restore
     8: Enter a list of files to restore before a specified time
     9: Find the JobIds of the most recent backup for a client
    10: Find the JobIds for a backup for a client before a specified time
    11: Enter a list of directories to restore for found JobIds
    12: Cancel
Select item:  (1-12):
\end{verbatim}
\normalsize

There are a lot of options, and as a point of reference, most people will
want to slect item 5 (the most recent backup for a client). The details
of the above options are:

\begin{itemize}
\item Item 1 will list the last 20 jobs run. If you find the Job you want,
   you can then select item 3 and enter its JobId(s).

\item Item 2 will list all the Jobs where a specified file is saved.  If you
   find the Job you want, you can then select item 3 and enter the JobId.

\item Item 3 allows you the enter a list of comma separated JobIds whose 
   files will be put into the directory tree. You may then select which
   files from those JobIds to restore. Normally, you would use this option
   if you have a particular version of a file that you want to restore and
   you know its JobId. The most common options (5 and 6) will not select
   a job that did not terminate normally, so if you know a file is 
   backed up by a Job that failed (possibly because of a system crash), you
   can access it through this option by specifying the JobId. 

\item Item 4 allows you to enter any arbitrary SQL command.  This is
   probably the most primitive way of finding the desired JobIds, but at
   the same time, the most flexible.  Once you have found the JobId(s), you
   can select item 3 and enter them.

\item Item 5 will automatically select the most recent Full backup and all
   subsequent incremental and differential backups for a specified Client.
   These are the Jobs and Files which, if reloaded, will restore your
   system to the most current saved state.  It automatically enters the
   JobIds found into the directory tree in an optimal way such that only
   the most recent copy of any particular file found in the set of Jobs
   will be restored.  This is probably the most convenient of all the above
   options to use if you wish to restore a selected Client to its most
   recent state.

   There are two important things to note. First, this automatic selection
   will never select a job that failed (terminated with an error status).
   If you have such a job and want to recover one or more files from it,
   you will need to explicitly enter the JobId in item 3, then choose the
   files to restore.

   If some of the Jobs that are needed to do the restore have had their 
   File records pruned, the restore will be incomplete. Bacula currently
   does not correctly detect this condition.  You can however, check for
   this by looking carefully at the list of Jobs that Bacula selects and
   prints. If you find Jobs with the JobFiles column set to zero, when
   files should have been backed up, then you should expect problems.

   If all the File records have been pruned, Bacula will realize that there
   are no file records in any of the JobIds chosen and will inform you. It
   will then propose doing a full restore (non-selective) of those JobIds.
   This is possible because Bacula still knows where the beginning of the
   Job data is on the Volumes, even if it does not know where particular
   files are located or what their names are.

\item Item 6 allows you to specify a date and time, after which Bacula  will
   automatically select the most recent Full backup and all  subsequent
   incremental and differential backups that started  before the specified date
   and time.  

\item Item 7 allows you to specify one or more filenames  (complete path
   required) to be restored. Each filename  is entered one at a time or if you
   prefix a filename  with the less-than symbol (\lt{}) Bacula will read that 
   file and assume it is a list of filenames to be restored.  If you
   prefix the filename with a question mark (?), then the filename will
   be interpreted as an SQL table name, and Bacula will include the rows
   of that table in the list to be restored. The table must contain the
   JobId in the first column and the FileIndex in the second column. 
   This table feature is intended for external programs that want to build
   their own list of files to be restored.
   The filename entry mode is terminated by entering a  blank line.

\item Item 8 allows you to specify a date and time before  entering the
   filenames. See Item 7 above for more  details.

\item Item 9 allows you find the JobIds of the most recent backup for
   a client. This is much like option 5 (it uses the same code), but
   those JobIds are retained internally as if you had entered them
   manually.  You may then select item 11 (see below) to restore one
   or more directories.

\item Item 10 is the same as item 9, except that it allows you to enter
   a before date (as with item 6). These JobIds will then be retained
   internally.

\index[general]{Restore Directories}
\item Item 11 allows you to enter a list of JobIds from which you can
   select directories to be restored. The list of JobIds can have been
   previously created by using either item 9 or 10 on the menu.  You
   may then enter a full path to a directory name or a filename preceded
   by a less than sign (\lt{}). The filename should contain a list
   of directories to be restored.  All files in those directories will
   be restored, but if the directory contains subdirectories, nothing
   will be restored in the subdirectory unless you explicitly enter its
   name.

\item Item 12 allows you to cancel the restore command.  
\end{itemize}

As an example, suppose that we select item 5 (restore to most recent state).
If you have not specified a client=xxx on the command line, it
it will then ask for the desired Client, which on my system, will print all
the Clients found in the database as follows:  

\footnotesize
\begin{verbatim}
Defined clients:
     1: Rufus
     2: Matou
     3: Polymatou
     4: Minimatou
     5: Minou
     6: MatouVerify
     7: PmatouVerify
     8: RufusVerify
     9: Watchdog
Select Client (File daemon) resource (1-9):
\end{verbatim}
\normalsize

You will probably have far fewer Clients than this example, and if you have
only one Client, it will be automatically selected.  In this case, I enter
{\bf Rufus} to select the Client. Then  Bacula needs to know what FileSet is
to be restored, so it prompts with:  

\footnotesize
\begin{verbatim}
The defined FileSet resources are:
     1: Full Set
     2: Other Files
Select FileSet resource (1-2):
\end{verbatim}
\normalsize

If you have only one FileSet defined for the Client, it will be selected
automatically.  I choose item 1, which is my full backup.  Normally, you
will only have a single FileSet for each Job, and if your machines are
similar (all Linux) you may only have one FileSet for all your Clients.

At this point, {\bf Bacula} has all the information it needs to find the most
recent set of backups. It will then query the database, which may take a bit
of time, and it will come up with something like the following. Note, some of
the columns are truncated here for presentation: 

\footnotesize
\begin{verbatim}
+-------+------+----------+-------------+-------------+------+-------+----------
--+
| JobId | Levl | JobFiles | StartTime   | VolumeName  | File | SesId |
VolSesTime |
+-------+------+----------+-------------+-------------+------+-------+----------
--+
| 1,792 | F    |  128,374 | 08-03 01:58 | DLT-19Jul02 |   67 |    18 |
1028042998 |
| 1,792 | F    |  128,374 | 08-03 01:58 | DLT-04Aug02 |    0 |    18 |
1028042998 |
| 1,797 | I    |      254 | 08-04 13:53 | DLT-04Aug02 |    5 |    23 |
1028042998 |
| 1,798 | I    |       15 | 08-05 01:05 | DLT-04Aug02 |    6 |    24 |
1028042998 |
+-------+------+----------+-------------+-------------+------+-------+----------
--+
You have selected the following JobId: 1792,1792,1797
Building directory tree for JobId 1792 ...
Building directory tree for JobId 1797 ...
Building directory tree for JobId 1798 ...
cwd is: /
$
\end{verbatim}
\normalsize

Depending on the number of {\bf JobFiles} for each JobId, the {\bf Building
directory tree ..."} can take a bit of time. If you notice ath all the
JobFiles are zero, your Files have probably been pruned and you will not be
able to select any individual files -- it will be restore everything or
nothing.

In our example, Bacula found four Jobs that comprise the most recent backup of
the specified Client and FileSet. Two of the Jobs have the same JobId because
that Job wrote on two different Volumes. The third Job was an incremental
backup to the previous Full backup, and it only saved 254 Files compared to
128,374 for the Full backup. The fourth Job was also an incremental backup
that saved 15 files. 

Next Bacula entered those Jobs into the directory tree, with no files marked
to be restored as a default, tells you how many files are in the tree, and
tells you that the current working directory ({\bf cwd}) is /. Finally, Bacula
prompts with the dollar sign (\$) to indicate that you may enter commands to
move around the directory tree and to select files. 
             
If you want all the files to automatically be marked when the directory
tree is built, you could have entered the command {\bf restore all}, or
at the \$ prompt, you can simply enter {\bf mark *}.

Instead of choosing item 5 on the first menu (Select the most recent backup
for a client), if we had chosen item 3 (Enter list of JobIds to select) and we
had entered the JobIds {\bf 1792,1797,1798} we would have arrived at the same
point.                 

One point to note, if you are manually entering JobIds, is that you must enter
them in the order they were run (generally in increasing JobId order). If you
enter them out of order and the same file was saved in two or more of the
Jobs, you may end up with an old version of that file (i.e. not the most
recent). 

Directly entering the JobIds can also permit you to recover data from
a Job that wrote files to tape but that terminated with an error status.

While in file selection mode, you can enter {\bf help} or a question mark (?)
to produce a summary of the available commands:  

\footnotesize
\begin{verbatim}
 Command    Description
  =======    ===========
  cd         change current directory
  count      count marked files in and below the cd
  dir        long list current directory, wildcards allowed
  done       leave file selection mode
  estimate   estimate restore size
  exit       same as done command
  find       find files, wildcards allowed
  help       print help
  ls         list current directory, wildcards allowed
  lsmark     list the marked files in and below the cd
  mark       mark dir/file to be restored recursively in dirs
  markdir    mark directory name to be restored (no files)
  pwd        print current working directory
  unmark     unmark dir/file to be restored recursively in dir
  unmarkdir  unmark directory name only no recursion
  quit       quit and do not do restore
  ?          print help
\end{verbatim}
\normalsize

As a default no files have been selected for restore (unless you
added {\bf all} to the command line. If you want to restore
everything, at this point, you should enter {\bf mark *}, and then {\bf done}
and {\bf Bacula} will write the bootstrap records to a file and request your
approval to start a restore job. 

If you do not enter the above mentioned {\bf mark *} command, you will start
with an empty slate. Now you can simply start looking at the tree and {\bf
mark} particular files or directories you want restored. It is easy to make
a mistake in specifying a file to mark or unmark, and Bacula's error handling
is not perfect, so please check your work by using the {\bf ls} or {\bf dir}
commands to see what files are actually selected. Any selected file has its
name preceded by an asterisk. 

To check what is marked or not marked, enter the {\bf count} command, which
displays:  

\footnotesize
\begin{verbatim}
128401 total files. 128401 marked to be restored.
     
\end{verbatim}
\normalsize

Each of the above commands will be described in more detail in the next
section. We continue with the above example, having accepted to restore all
files as Bacula set by default. On entering the {\bf done} command, Bacula
prints:  

\footnotesize
\begin{verbatim}
Bootstrap records written to /home/kern/bacula/working/restore.bsr
The job will require the following   
   Volume(s)                 Storage(s)                SD Device(s)
===========================================================================

   DLT-19Jul02               Tape                      DLT8000
   DLT-04Aug02               Tape                      DLT8000

128401 files selected to restore.
Run Restore job
JobName:    kernsrestore
Bootstrap:  /home/kern/bacula/working/restore.bsr
Where:      /tmp/bacula-restores
Replace:    always
FileSet:    Other Files
Client:     Rufus
Storage:    Tape
When:       2006-12-11 18:20:33
Catalog:    MyCatalog
Priority:   10
OK to run? (yes/mod/no):
    
\end{verbatim}
\normalsize

Please examine each of the items very carefully to make sure that they are
correct.  In particular, look at {\bf Where}, which tells you where in the
directory structure the files will be restored, and {\bf Client}, which
tells you which client will receive the files.  Note that by default the
Client which will receive the files is the Client that was backed up.
These items will not always be completed with the correct values depending
on which of the restore options you chose.  You can change any of these
default items by entering {\bf mod} and responding to the prompts.

The above assumes that you have defined a {\bf Restore} Job resource in your
Director's configuration file. Normally, you will only need one Restore Job
resource definition because by its nature, restoring is a manual operation,
and using the Console interface, you will be able to modify the Restore Job to
do what you want. 

An example Restore Job resource definition is given below. 

Returning to the above example, you should verify that the Client name is
correct before running the Job. However, you may want to modify some of the
parameters of the restore job. For example, in addition to checking the Client
it is wise to check that the Storage device chosen by Bacula is indeed
correct. Although the {\bf FileSet} is shown, it will be ignored in restore.
The restore will choose the files to be restored either by reading the {\bf
Bootstrap} file, or if not specified, it will restore all files associated
with the specified backup {\bf JobId} (i.e. the JobId of the Job that
originally backed up the files). 

Finally before running the job, please note that the default location for
restoring files is {\bf not} their original locations, but rather the directory
{\bf /tmp/bacula-restores}. You can change this default by modifying your {\bf
bacula-dir.conf} file, or you can modify it using the {\bf mod} option. If you
want to restore the files to their original location, you must have {\bf
Where} set to nothing or to the root, i.e. {\bf /}. 

If you now enter {\bf yes}, Bacula will run the restore Job. The Storage
daemon will first request Volume {\bf DLT-19Jul02} and after the appropriate
files have been restored from that volume, it will request Volume {\bf
DLT-04Aug02}. 

\section{Selecting Files by Filename}
\index[general]{Selecting Files by Filename }
\index[general]{Filename!Selecting Files by }

If you have a small number of files to restore, and you know the filenames,
you can either put the list of filenames in a file to be read by Bacula, or
you can enter the names one at a time. The filenames must include the full
path and filename. No wild cards are used. 

To enter the files, after the {\bf restore}, you select item number 7 from the
prompt list: 

\footnotesize
\begin{verbatim}
To select the JobIds, you have the following choices:
     1: List last 20 Jobs run
     2: List Jobs where a given File is saved
     3: Enter list of comma separated JobIds to select
     4: Enter SQL list command
     5: Select the most recent backup for a client
     6: Select backup for a client before a specified time
     7: Enter a list of files to restore
     8: Enter a list of files to restore before a specified time
     9: Find the JobIds of the most recent backup for a client
    10: Find the JobIds for a backup for a client before a specified time
    11: Enter a list of directories to restore for found JobIds
    12: Cancel
Select item:  (1-12):
\end{verbatim}
\normalsize

which then prompts you for the client name: 

\footnotesize
\begin{verbatim}
Defined Clients:
     1: Timmy
     2: Tibs
     3: Rufus
Select the Client (1-3): 3
\end{verbatim}
\normalsize

Of course, your client list will be different, and if you have only one
client, it will be automatically selected. And finally, Bacula requests you to
enter a filename: 

\footnotesize
\begin{verbatim}
Enter filename:
\end{verbatim}
\normalsize

At this point, you can enter the full path and filename 

\footnotesize
\begin{verbatim}
Enter filename: /home/kern/bacula/k/Makefile.in
Enter filename:
\end{verbatim}
\normalsize

as you can see, it took the filename. If Bacula cannot find a copy of the
file, it prints the following: 

\footnotesize
\begin{verbatim}
Enter filename: junk filename
No database record found for: junk filename
Enter filename:
\end{verbatim}
\normalsize

If you want Bacula to read the filenames from a file, you simply precede the
filename with a less-than symbol (\lt{}). When you have entered all the
filenames, you enter a blank line, and Bacula will write the bootstrap file,
tells you what tapes will be used, and proposes a Restore job to be run: 

\footnotesize
\begin{verbatim}
Enter filename:
Automatically selected Storage: DDS-4
Bootstrap records written to /home/kern/bacula/working/restore.bsr
The restore job will require the following Volumes:
   
   test1
1 file selected to restore.
Run Restore job
JobName:    kernsrestore
Bootstrap:  /home/kern/bacula/working/restore.bsr
Where:      /tmp/bacula-restores
Replace:    always
FileSet:    Other Files
Client:     Rufus
Storage:    DDS-4
When:       2003-09-11 10:20:53
Priority:   10
OK to run? (yes/mod/no):
\end{verbatim}
\normalsize

It is possible to automate the selection by file by putting your list of files
in say {\bf /tmp/file-list}, then using the following command: 

\footnotesize
\begin{verbatim}
restore client=Rufus file=</tmp/file-list
\end{verbatim}
\normalsize

If in modifying the parameters for the Run Restore job, you find that Bacula
asks you to enter a Job number, this is because you have not yet specified
either a Job number or a Bootstrap file. Simply entering zero will allow you
to continue and to select another option to be modified. 
\label{CommandArguments}

\section{Command Line Arguments}
\index[general]{Arguments!Command Line }
\index[general]{Command Line Arguments }

If all the above sounds complicated, you will probably agree that it really
isn't after trying it a few times. It is possible to do everything that was
shown above, with the exception of selecting the FileSet, by using command
line arguments with a single command by entering: 

\footnotesize
\begin{verbatim}
restore client=Rufus select current all done yes
\end{verbatim}
\normalsize

The {\bf client=Rufus} specification will automatically select Rufus as the
client, the {\bf current} tells Bacula that you want to restore the system to
the most current state possible, and the {\bf yes} suppresses the final {\bf
yes/mod/no} prompt and simply runs the restore. 

The full list of possible command line arguments are: 

\begin{itemize}
\item {\bf all} -- select all Files to be restored.  
\item {\bf select} -- use the tree selection method.  
\item {\bf done} -- do not prompt the user in tree mode.  
\item {\bf current} -- automatically select the most current set of  backups
   for the specified client.  
\item {\bf client=xxxx} -- select the specified client.  
\item {\bf jobid=nnn} -- specify a JobId or comma separated list of  JobIds to
   be restored.  
\item {\bf before=YYYY-MM-DD HH:MM:SS} -- specify a date and time to  which
   the system should be restored. Only Jobs started before  the specified
   date/time will be selected, and as is the case  for {\bf current} Bacula will
   automatically find the most  recent prior Full save and all Differential and
   Incremental  saves run before the date you specify. Note, this command is  not
   too user friendly in that you must specify the date/time  exactly as shown. 
\item {\bf file=filename} -- specify a filename to be restored. You  must
   specify the full path and filename. Prefixing the entry  with a less-than
   sign
   (\lt{}) will cause Bacula to assume that the  filename is on your system and
   contains a list of files to be  restored. Bacula will thus read the list from
   that file. Multiple  file=xxx specifications may be specified on the command
   line. 
\item {\bf jobid=nnn} -- specify a JobId to be restored. 
\item {\bf pool=pool-name} -- specify a Pool name to be used for selection  of
   Volumes when specifying options 5 and 6 (restore current system,  and restore
   current system before given date). This permits you to  have several Pools,
   possibly one offsite, and to select the Pool to  be used for restoring.  
\item {\bf yes} -- automatically run the restore without prompting  for
   modifications (most useful in batch scripts). 
\end{itemize}

\section{Restoring Directory Attributes}
\index[general]{Attributes!Restoring Directory }
\index[general]{Restoring Directory Attributes }

Depending how you do the restore, you may or may not get the directory entries
back to their original state. Here are a few of the problems you can
encounter, and for same machine restores, how to avoid them. 

\begin{itemize}
\item You backed up on one machine and are restoring to another that is 
   either a different OS or doesn't have the same users/groups defined.  Bacula
   does the best it can in these situations. Note, Bacula has saved the
   user/groups in numeric form, which means on a different machine, they
   may map to different user/group names.
\item You are restoring into a directory that is already created and has
   file creation restrictions.  Bacula tries to reset everything but
   without walking up the full chain of directories and modifying them all
   during the restore, which Bacula does and will not do, getting
   permissions back correctly in this situation depends to a large extent
   on your OS.
\item You are doing a recursive restore of a directory tree.  In this case
   Bacula will restore a file before restoring the file's parent directory
   entry.  In the process of restoring the file Bacula will create the
   parent directory with open permissions and ownership of the file being
   restored.  Then when Bacula tries to restore the parent directory Bacula
   sees that it already exists (Similar to the previous situation).  If you
   had set the Restore job's "Replace" property to "never" then Bacula will
   not change the directory's permissions and ownerships to match what it
   backed up, you should also notice that the actual number of files
   restored is less then the expected number.  If you had set the Restore
   job's "Replace" property to "always" then Bacula will change the
   Directory's ownership and permissions to match what it backed up, also
   the actual number of files restored should be equal to the expected
   number.
\item You selected one or more files in a directory, but did not select the
   directory entry to be restored.  In that case, if the directory is not
   on disk Bacula simply creates the directory with some default attributes
   which may not be the same as the original.  If you do not select a
   directory and all its contents to be restored, you can still select
   items within the directory to be restored by individually marking those
   files, but in that case, you should individually use the "markdir"
   command to select all higher level directory entries (one at a time) to
   be restored if you want the directory entries properly restored.
\end{itemize}

\label{Windows}
\section{Restoring on Windows}
\index[general]{Restoring on Windows }
\index[general]{Windows!Restoring on }

If you are restoring on WinNT/2K/XP systems, Bacula will restore the files
with the original ownerships and permissions as would be expected.  This is
also true if you are restoring those files to an alternate directory (using
the Where option in restore).  However, if the alternate directory does not
already exist, the Bacula File daemon (Client) will try to create it.  In
some cases, it may not create the directories, and if it does since the
File daemon runs under the SYSTEM account, the directory will be created
with SYSTEM ownership and permissions.  In this case, you may have problems
accessing the newly restored files.

To avoid this problem, you should create any alternate directory before
doing the restore.  Bacula will not change the ownership and permissions of
the directory if it is already created as long as it is not one of the
directories being restored (i.e.  written to tape).

The default restore location is {\bf /tmp/bacula-restores/} and if you are
restoring from drive {\bf E:}, the default will be 
{\bf /tmp/bacula-restores/e/}, so you should ensure that this directory
exists before doing the restore, or use the {\bf mod} option to
select a different {\bf where} directory that does exist.

Some users have experienced problems restoring files that participate in
the Active Directory. They also report that changing the userid under which
Bacula (bacula-fd.exe) runs, from SYSTEM to a Domain Admin userid, resolves
the problem.


\section{Restoring Files Can Be Slow}
\index[general]{Slow!Restoring Files Can Be }
\index[general]{Restoring Files Can Be Slow }

Restoring files is generally {\bf much} slower than backing them up for several
reasons. The first is that during a backup the tape is normally already
positioned and Bacula only needs to write. On the other hand, because restoring
files is done so rarely, Bacula keeps only the start file and block on the
tape for the whole job rather than on a file by file basis which would use
quite a lot of space in the catalog. 

Bacula will forward space to the correct file mark on the tape for the Job,
then forward space to the correct block, and finally sequentially read each
record until it gets to the correct one(s) for the file or files you want to
restore. Once the desired files are restored, Bacula will stop reading the
tape. 

Finally, instead of just reading a file for backup, during the restore, Bacula
must create the file, and the operating system must allocate disk space for
the file as Bacula is restoring it. 

For all the above reasons the restore process is generally much slower than
backing up (sometimes it takes three times as long).

\section{Problems Restoring Files}
\index[general]{Files!Problems Restoring }
\index[general]{Problems Restoring Files }

The most frequent problems users have restoring files are error messages such
as: 

\footnotesize
\begin{verbatim}
04-Jan 00:33 z217-sd: RestoreFiles.2005-01-04_00.31.04 Error:
block.c:868 Volume data error at 20:0! Short block of 512 bytes on
device /dev/tape discarded.
\end{verbatim}
\normalsize

or 

\footnotesize
\begin{verbatim}
04-Jan 00:33 z217-sd: RestoreFiles.2005-01-04_00.31.04 Error:
block.c:264 Volume data error at 20:0! Wanted ID: "BB02", got ".".
Buffer discarded.
\end{verbatim}
\normalsize

Both these kinds of messages indicate that you were probably running your tape
drive in fixed block mode rather than variable block mode. Fixed block mode
will work with any program that reads tapes sequentially such as tar, but
Bacula repositions the tape on a block basis when restoring files because this
will speed up the restore by orders of magnitude when only a few files are being
restored. There are several ways that you can attempt to recover from this
unfortunate situation. 

Try the following things, each separately, and reset your Device resource to
what it is now after each individual test: 

\begin{enumerate}
\item Set "Block Positioning = no" in your Device resource  and try the
   restore. This is a new directive and untested. 
\item Set "Minimum Block Size = 512" and "Maximum  Block Size = 512" and
   try the restore.  If you are able to determine the block size your drive
   was previously using, you should try that size if 512 does not work.
   This is a really horrible solution, and it is not at all recommended
   to continue backing up your data without correcting this condition.
   Please see the Tape Testing chapter for more on this.
\item Try editing the restore.bsr file at the Run xxx yes/mod/no prompt 
   before starting the restore job and remove all the VolBlock statements.
   These are what causes Bacula to reposition the tape, and where problems
   occur if you have a fixed block size set for your drive.  The VolFile
   commands also cause repositioning, but this will work regardless of the
   block size.
\item Use bextract to extract the files you want -- it reads the  Volume
   sequentially if you use the include list feature, or if you use a .bsr
   file, but remove all the VolBlock statements after the .bsr file is
   created (at the Run yes/mod/no) prompt but before you start the restore.
\end{enumerate}

\section{Restore Errors}
\index[general]{Errors!Restore}
\index[general]{Restore Errors}

There are a number of reasons why there may be restore errors or
warning messages. Some of the more common ones are:

\begin{description}

\item [file count mismatch]
  This can occur for the following reasons:
  \begin{itemize}
  \item You requested Bacula not to overwrite existing or newer
     files.
  \item A Bacula miscount of files/directories. This is an
     on-going problem due to the complications of directories,
     soft/hard link, and such.  Simply check that all the files you
     wanted were actually restored.
  \end{itemize}
\item [file size error]
   When Bacula restores files, it checks that the size of the
   restored file is the same as the file status data it saved 
   when starting the backup of the file. If the sizes do not
   agree, Bacula will print an error message. This size mismatch
   most often occurs because the file was being written as Bacula
   backed up the file. In this case, the size that Bacula
   restored will be greater than the status size.  This often
   happens with log files.

   If the restored size is smaller, then you should be concerned
   about a possible tape error and check the Bacula output as
   well as your system logs.
\end{description}



\section{Example Restore Job Resource}
\index[general]{Example Restore Job Resource }
\index[general]{Resource!Example Restore Job }

\footnotesize
\begin{verbatim}
Job {
  Name = "RestoreFiles"
  Type = Restore
  Client = Any-client
  FileSet = "Any-FileSet"
  Storage = Any-storage
  Where = /tmp/bacula-restores
  Messages = Standard
  Pool = Default
}
\end{verbatim}
\normalsize

If {\bf Where} is not specified, the default location for restoring files will
be their original locations. 
\label{Selection}

\section{File Selection Commands}
\index[general]{Commands!File Selection }
\index[general]{File Selection Commands }

After you have selected the Jobs to be restored and Bacula has created the
in-memory directory tree, you will enter file selection mode as indicated by
the dollar sign ({\bf \$}) prompt. While in this mode, you may use the
commands listed above. The basic idea is to move up and down the in memory
directory structure with the {\bf cd} command much as you normally do on the
system. Once you are in a directory, you may select the files that you want
restored. As a default no files are marked to be restored. If you wish to
start with all files, simply enter: {\bf cd /} and {\bf mark *}. Otherwise
proceed to select the files you wish to restore by marking them with the {\bf
mark} command. The available commands are: 

\begin{description}

\item [cd]
   The {\bf cd} command changes the current directory to  the argument
   specified.
   It operates much like the Unix {\bf cd} command.  Wildcard specifications are
   not permitted.  

   Note, on Windows systems, the various drives (c:, d:, ...) are treated  like
   a
   directory within the file tree while in the file  selection mode. As a
   consequence, you must do a {\bf cd c:} or  possibly in some cases a {\bf cd
   C:} (note upper case)  to get down to the first directory.  

\item [dir]
   \index[dir]{dir }
   The {\bf dir} command is similar to the {\bf ls} command,  except that it
   prints it in long format (all details). This command  can be a bit slower
   than
   the {\bf ls} command because it must access  the catalog database for the
   detailed information for each file.  

\item [estimate]
   \index[dir]{estimate }
   The {\bf estimate} command prints a summary of  the total files in the tree,
   how many are marked to be restored, and  an estimate of the number of bytes
   to
   be restored. This can be useful  if you are short on disk space on the
   machine
   where the files will be  restored.  

\item [find]
   \index[dir]{find }
   The {\bf find} command accepts one or more arguments  and displays all files
   in the tree that match that argument. The argument  may have wildcards. It is
   somewhat similar to the Unix command  {\bf find / -name arg}.  

\item [ls]
   The {\bf ls} command produces a listing of all the files  contained in the
   current directory much like the Unix {\bf ls} command.  You may specify an
   argument containing wildcards, in which case only  those files will be
listed.
   Any file that is marked to be restored will  have its name preceded by an
   asterisk ({\bf *}). Directory names  will be terminated with a forward slash
   ({\bf /}) to distinguish them  from filenames.  

\item [lsmark]
   \index[fd]{lsmark }
   The {\bf lsmark} command is the same as the  {\bf ls} except that it will
   print only those files marked for  extraction. The other distinction is that
   it will recursively  descend into any directory selected. 

\item [mark]
   \index[dir]{mark }
   The {\bf mark} command allows you to mark files to be restored. It takes a
   single argument which is the filename  or directory name in the current
   directory to be marked for extraction.  The argument may be a wildcard
   specification, in which  case all files that match in the current directory
   are marked to be  restored. If the argument matches a directory rather than a
   file,  then the directory and all the files contained in that directory
   (recursively)  are marked to be restored. Any marked file will have its name 
   preceded with an asterisk ({\bf *}) in the output produced by the  {\bf ls}
or
   {\bf dir} commands. Note, supplying a full path on  the mark command does not
   work as expected to select a file or  directory in the current directory.
   Also, the {\bf mark} command works  on the current and lower directories but
   does not touch higher level  directories.  

   After executing the {\bf mark} command, it will print a brief summary:  

\footnotesize
\begin{verbatim}
    No files marked.
    
\end{verbatim}
\normalsize

   If no files were marked, or:  

\footnotesize
\begin{verbatim}
    nn files marked.
    
\end{verbatim}
\normalsize

   if some files are marked.  

\item [unmark]
   \index[dir]{unmark }
   The {\bf unmark} is identical to the {\bf mark}  command, except that it
   unmarks the specified file or files so that  they will not be restored. Note:
   the {\bf unmark} command works from  the current directory, so it does not
   unmark any files at a higher  level. First do a {\bf cd /} before the {\bf
   unmark *} command if  you want to unmark everything.  

\item [pwd]
   \index[dir]{pwd }
   The {\bf pwd} command prints the current working  directory. It accepts no
   arguments.  

\item [count]
   \index[dir]{count }
   The {\bf count} command prints the total files in the  directory tree and the
   number of files marked to be restored.  

\item [done]
   \index[dir]{done }
   This command terminates file selection mode.  

\item [exit]
   \index[fd]{exit }
   This command terminates file selection mode (the same as  done).  

\item [quit]
   \index[fd]{quit }
   This command terminates the file selection and does  not run the restore
job. 


\item [help]
   \index[fd]{help }
   This command prints a summary of the commands available.  

\item [?]
   This command is the same as the {\bf help} command.  
\end{description}

\label{database_restore}
\section{Restoring When Things Go Wrong}
\index[general]{Restoring When Things Go Wrong }
\index[general]{Restoring Your Database}
\index[general]{Database!Restoring}

This and the following sections will try to present a few of the kinds of
problems that can come up making restoring more difficult. I'll try to
provide a few ideas how to get out of these problem situations.
In addition to what is presented here, there is more specific information
on restoring a \ilink{Client}{restore_client} and your
\ilink{Server}{restore_server} in the \ilink{Disaster Recovery Using
Bacula}{RescueChapter} chapter of this manual.

\begin{description}
\item[Problem]
   My database is broken.
\item[Solution]
   For SQLite, use the vacuum command to try to fix the database. For either
   MySQL or PostgreSQL, see the vendor's documentation. They have specific tools
   that check and repair databases, see the \ilink{database
   repair}{DatabaseRepair} sections of this manual for links to vendor    
   information.

   Assuming the above does not resolve the problem, you will need to restore
   or rebuild your catalog.  Note, if it is a matter of some
   inconsistencies in the Bacula tables rather than a broken database, then
   running \ilink{dbcheck}{dbcheck} might help, but you will need to ensure
   that your database indexes are properly setup.  Please see
   the \ilink{Database Performance Issues}{DatabasePerformance} sections
   of this manual for more details.

\item[Problem]
   How do I restore my catalog?
\item[Solution with a Catalog backup]
   If you have backed up your database nightly (as you should) and you
   have made a bootstrap file, you can immediately load back your
   database (or the ASCII SQL output).  Make a copy of your current
   database, then re-initialize it, by running the following scripts:
\begin{verbatim}
   ./drop_bacula_tables
   ./make_bacula_tables
\end{verbatim}
   After re-initializing the database, you should be able to run 
   Bacula. If you now try to use the restore command, it will not 
   work because the database will be empty. However, you can manually
   run a restore job and specify your bootstrap file. You do so
   by entering the {bf run} command in the console and selecting the
   restore job.  If you are using the default bacula-dir.conf, this
   Job will be named {\bf RestoreFiles}. Most likely it will prompt
   you with something such as:
\footnotesize
\begin{verbatim}
Run Restore job
JobName:    RestoreFiles
Bootstrap:  /home/kern/bacula/working/restore.bsr
Where:      /tmp/bacula-restores
Replace:    always
FileSet:    Full Set
Client:     rufus-fd
Storage:    File
When:       2005-07-10 17:33:40
Catalog:    MyCatalog
Priority:   10
OK to run? (yes/mod/no): 
\end{verbatim}
\normalsize
   A number of the items will be different in your case.  What you want to
   do is: to use the mod option to change the Bootstrap to point to your
   saved bootstrap file; and to make sure all the other items such as
   Client, Storage, Catalog, and Where are correct.  The FileSet is not
   used when you specify a bootstrap file.  Once you have set all the
   correct values, run the Job and it will restore the backup of your
   database, which is most likely an ASCII dump. 

   You will then need to follow the instructions for your
   database type to recreate the database from the ASCII backup file.
   See the \ilink {Catalog Maintenance}{CatMaintenanceChapter} chapter of
   this manual for examples of the command needed to restore a 
   database from an ASCII dump (they are shown in the Compacting Your
   XXX Database sections).
    
   Also, please note that after you restore your database from an ASCII
   backup, you do NOT want to do a {\bf make\_bacula\_tables}  command, or
   you will probably erase your newly restored database tables.

      
\item[Solution with a Job listing]
   If you did save your database but did not make a bootstrap file, then
   recovering the database is more difficult.  You will probably need to
   use bextract to extract the backup copy.  First you should locate the
   listing of the job report from the last catalog backup.  It has
   important information that will allow you to quickly find your database
   file.  For example, in the job report for the CatalogBackup shown below,
   the critical items are the Volume name(s), the Volume Session Id and the
   Volume Session Time.  If you know those, you can easily restore your
   Catalog.
\footnotesize
\begin{verbatim}

22-Apr 10:22 HeadMan: Start Backup JobId 7510,
Job=CatalogBackup.2005-04-22_01.10.0
22-Apr 10:23 HeadMan: Bacula 1.37.14 (21Apr05): 22-Apr-2005 10:23:06
  JobId:                  7510
  Job:                    CatalogBackup.2005-04-22_01.10.00
  Backup Level:           Full
  Client:                 Polymatou
  FileSet:                "CatalogFile" 2003-04-10 01:24:01
  Pool:                   "Default"
  Storage:                "DLTDrive"
  Start time:             22-Apr-2005 10:21:00
  End time:               22-Apr-2005 10:23:06
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       210,739,395
  SD Bytes Written:       210,739,521
  Rate:                   1672.5 KB/s
  Software Compression:   None
  Volume name(s):         DLT-22Apr05
  Volume Session Id:      11
  Volume Session Time:    1114075126
  Last Volume Bytes:      1,428,240,465
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

\end{verbatim}
\normalsize

  From the above information, you can manually create a bootstrap file,
  and then follow the instructions given above for restoring your database.
  A reconstructed bootstrap file for the above backup Job would look
  like the following:

\footnotesize
\begin{verbatim}
Volume="DLT-22Apr05"
VolSessionId=11
VolSessionTime=1114075126
FileIndex=1-1
\end{verbatim}
\normalsize    

  Where we have inserted the Volume name, Volume Session Id, and Volume
  Session Time that correspond to the values in the job report.  We've also
  used a FileIndex of one, which will always be the case providing that
  there was only one file backed up in the job.
  
  The disadvantage of this bootstrap file compared to what is created when
  you ask for one to be written, is that there is no File and Block
  specified, so the restore code must search all data in the Volume to find
  the requested file.  A fully specified bootstrap file would have the File
  and Blocks specified as follows:

\footnotesize
\begin{verbatim}
Volume="DLT-22Apr05"
VolSessionId=11
VolSessionTime=1114075126
VolFile=118-118
VolBlock=0-4053
FileIndex=1-1
\end{verbatim}
\normalsize

   Once you have restored the ASCII dump of the database,
   you will then to follow the instructions for your
   database type to recreate the database from the ASCII backup file.
   See the \ilink {Catalog Maintenance}{CatMaintenanceChapter} chapter of
   this manual for examples of the command needed to restore a 
   database from an ASCII dump (they are shown in the Compacting Your
   XXX Database sections).
    
   Also, please note that after you restore your database from an ASCII
   backup, you do NOT want to do a {\bf make\_bacula\_tables}  command, or
   you will probably erase your newly restored database tables.

\item [Solution without a Job Listing]
   If you do not have a job listing, then it is a bit more difficult.
   Either you use the \ilink{bscan}{bscan} program to scan the contents
   of your tape into a database, which can be very time consuming 
   depending on the size of the tape, or you can use the \ilink{bls}{bls}
   program to list everything on the tape, and reconstruct a bootstrap 
   file from the bls listing for the file or files you want following
   the instructions given above.

   There is a specific example of how to use {\bf bls} below.

\item [Problem]
   I try to restore the last known good full backup by specifying
   item 3 on the restore menu then the JobId to restore.  Bacula 
   then reports:

\footnotesize
\begin{verbatim}
   1 Job 0 Files
\end{verbatim}
\normalsize
   and restores nothing.
\item[Solution]
   Most likely the File records were pruned from the database either due
   to the File Retention period expiring or by explicitly purging the
   Job. By using the "llist jobid=nn" command, you can obtain all the
   important information about the job:
\footnotesize
\begin{verbatim}
llist jobid=120
           JobId: 120
             Job: save.2005-12-05_18.27.33
        Job.Name: save
     PurgedFiles: 0
            Type: B
           Level: F
    Job.ClientId: 1
     Client.Name: Rufus
       JobStatus: T
       SchedTime: 2005-12-05 18:27:32
       StartTime: 2005-12-05 18:27:35
         EndTime: 2005-12-05 18:27:37
        JobTDate: 1133803657
    VolSessionId: 1
  VolSessionTime: 1133803624
        JobFiles: 236
       JobErrors: 0
 JobMissingFiles: 0
      Job.PoolId: 4
       Pool.Name: Full
   Job.FileSetId: 1
 FileSet.FileSet: BackupSet
\end{verbatim}
\normalsize

   Then you can find the Volume(s) used by doing:
\footnotesize
\begin{verbatim}
sql
select VolumeName from JobMedia,Media where JobId=1 and JobMedia.MediaId=Media.MediaId;
\end{verbatim}
\normalsize
   Finally, you can create a bootstrap file as described in the previous
   problem above using this information.

   If you are using Bacula version 1.38.0 or greater, when you select
   item 3 from the menu and enter the JobId, it will ask you if
   you would like to restore all the files in the job, and it will
   collect the above information and write the bootstrap file for
   you.

\item [Problem]
  You don't have a bootstrap file, and you don't have the Job report for
  the backup of your database, but you did backup the database, and you
  know the Volume to which it was backed up.
  
\item [Solution]
  Either bscan the tape, or use {\bf bls} to indicate where it is on the
  tape.  For example:

\footnotesize
\begin{verbatim}
./bls -j -V DLT-22Apr05 /dev/nst0
\end{verbatim}
\normalsize
  Might produce the following output:
\footnotesize
\begin{verbatim}
bls: butil.c:258 Using device: "/dev/nst0" for reading.
21-Jul 18:34 bls: Ready to read from volume "DLT-22Apr05" on device "DLTDrive"
(/dev/nst0).
Volume Record: File:blk=0:0 SessId=11 SessTime=1114075126 JobId=0 DataLen=164
...
Begin Job Session Record: File:blk=118:0 SessId=11 SessTime=1114075126
JobId=7510
   Job=CatalogBackup.2005-04-22_01.10.0 Date=22-Apr-2005 10:21:00 Level=F Type=B
End Job Session Record: File:blk=118:4053 SessId=11 SessTime=1114075126
JobId=7510
   Date=22-Apr-2005 10:23:06 Level=F Type=B Files=1 Bytes=210,739,395 Errors=0
Status=T
...
21-Jul 18:34 bls: End of Volume at file 201 on device "DLTDrive" (/dev/nst0),
Volume "DLT-22Apr05"
21-Jul 18:34 bls: End of all volumes.
\end{verbatim}
\normalsize
  Of course, there will be many more records printed, but we have indicated
  the essential lines of output. From the information on the Begin Job and End 
  Job Session Records, you can reconstruct a bootstrap file such as the one
  shown above.

\item[Problem]
  How can I find where a file is stored.
\item[Solution]
  Normally, it is not necessary, you just use the {\bf restore} command to
  restore the most recently saved version (menu option 5), or a version
  saved before a given date (menu option 8).  If you know the JobId of the
  job in which it was saved, you can use menu option 3 to enter that JobId.

  If you would like to know the JobId where a file was saved, select
  restore menu option 2.

  You can also use the {\bf query} command to find information such as: 
\footnotesize
\begin{verbatim}
*query
Available queries:
     1: List up to 20 places where a File is saved regardless of the
directory
     2: List where the most recent copies of a file are saved
     3: List last 20 Full Backups for a Client
     4: List all backups for a Client after a specified time
     5: List all backups for a Client
     6: List Volume Attributes for a selected Volume
     7: List Volumes used by selected JobId
     8: List Volumes to Restore All Files
     9: List Pool Attributes for a selected Pool
    10: List total files/bytes by Job
    11: List total files/bytes by Volume
    12: List Files for a selected JobId
    13: List Jobs stored on a selected MediaId
    14: List Jobs stored for a given Volume name
    15: List Volumes Bacula thinks are in changer
    16: List Volumes likely to need replacement from age or errors
Choose a query (1-16):
\end{verbatim}
\normalsize


\end{description}