\addcontentsline{toc}{section}{Catalog Maintenance}
Without proper setup and maintenance, your Catalog may continue to grow
-indefinitely as you run Jobs and backup Files. How fast the size of your
+indefinitely as you run Jobs and backup Files, and/or it may become
+very inefficient and slow. How fast the size of your
Catalog grows depends on the number of Jobs you run and how many files they
backup. By deleting records within the database, you can make space available
for the new records that will be added during the next Job. By constantly
As a consequence, I suggest you monitor the size of your database and from
time to time (once every 6 months or year), compress it.
-\label{RepairingMySQL}
+\label{DatabaseRepair}
+\label{RepairingMySQL}
\subsection*{Repairing Your MySQL Database}
\index[general]{Database!Repairing Your MySQL }
\index[general]{Repairing Your MySQL Database }
running dbcheck before (or possibly after) using the MySQL database repair
program. It can clean up many of the orphaned record problems, and certain
other inconsistencies in the Bacula database.
-\label{RepairingPSQL}
+If you are running into the error {\bf The table 'File' is full ...},
+it is probably because on version 4.x MySQL, the table is limited by
+default to a maximum size of 4 GB and you have probably run into
+the limit. The solution can be found at:
+\elink{http://dev.mysql.com/doc/refman/5.0/en/full-table.html}
+{http://dev.mysql.com/doc/refman/5.0/en/full-table.html}
+
+You can display the maximum length of your table with:
+
+\footnotesize
+\begin{verbatim}
+mysql bacula
+SHOW TABLE STATUS FROM bacula like "File";
+\end{verbatim}
+\normalsize
+
+If the column labeld "Max_data_length" is around 4Gb, this is likely
+to be the source of your problem, and you can modify it with:
+
+\footnotesize
+\begin{verbatim}
+mysql bacula
+ALTER TABLE File MAX_ROWS=281474976710656;
+\end{verbatim}
+\normalsize
+
+Alternatively you can modify your /etc/my.conf file and in the
+[mysqld] section set:
+
+\footnotesize
+\begin{verbatim}
+set-variable = myisam_data_pointer_size=6
+\end{verbatim}
+\normalsize
+
+The above row and point size changes should already be the default on MySQL
+version 5.x, so these changes should only be necessary on MySQL 4.x
+depending on the size of your catalog database.
+
+
+\label{RepairingPSQL}
\subsection*{Repairing Your PostgreSQL Database}
\index[general]{Database!Repairing Your PostgreSQL }
\index[general]{Repairing Your PostgreSQL Database }
consult the PostgreSQL documents for how to repair the database, and also
consider using Bacula's dbcheck program if the conditions are reasonable for
using (see above).
-\label{CompactingPostgres}
-\subsection*{Performance Issues}
+\label{DatabasePerformance}
+\subsection*{Database Performance Issues}
\index[general]{Database Performance Issues}
\index[general]{Performance!Database}
\addcontentsline{toc}{subsection}{Database Performance Issues}
corruption if your system crashes.
For PostgreSQL, you might want to consider turning fsync off. Of course
-doing so can cause corrupted databases in the even of a machine crash.
+doing so can cause corrupted databases in the event of a machine crash.
There are many different ways that you can tune PostgreSQL, the
following document discusses a few of them:
\elink{
\end{verbatim}
\normalsize
-\subsubsection*{SQLit Indexes}
+Though normally not a problem, you should ensure that the indexes
+defined for Filename and Path are both set to 255 characters. Some users
+reported performance problems when their indexes were set to 50 characters.
+To check, do:
+
+\footnotesize
+\begin{verbatim}
+mysql bacula
+show index from Filename;
+show index from Path;
+\end{verbatim}
+\normalsize
+
+and what is important is that for Filename, you have an index with
+Key_name "Name" and Sub_part "255". For Pth, you should have a Key_name
+"Path" and Sub_part "255". If one or the other does not exist or the
+Sub_part is less that 255, you can drop and recreate the appropriate
+index with:
+
+\footnotesize
+\begin{verbatim}
+mysql bacula
+DROP INDEX Path on Path;
+CREATE INDEX Path on Path (Path(255);
+
+DROP INDEX Name on Filename;
+CREATE INDEX Name on Filename (Name(255));
+\end{verbatim}
+\normalsize
+
+
+\subsubsection*{SQLite Indexes}
On SQLite, you can check if you have the proper indexes by:
\footnotesize
\begin{verbatim}
mysql bacula
CREATE INDEX file_jobid_idx on File (JobId);
-CREATE INDEX file_jpf_idx on File (Job, FilenameId, PathId);
+CREATE INDEX file_jfp_idx on File (Job, FilenameId, PathId);
\end{verbatim}
\normalsize
+\label{CompactingPostgres}
\subsection*{Compacting Your PostgreSQL Database}
\index[general]{Database!Compacting Your PostgreSQL }
\index[general]{Compacting Your PostgreSQL Database }
\footnotesize
\begin{verbatim}
-pg_dump bacula > bacula.sql
+pg_dump -c bacula > bacula.sql
cat bacula.sql | psql bacula
rm -f bacula.sql
\end{verbatim}
the Bacula database (typically /usr/local/pgsql/data or possible
/var/lib/pgsql/data) and check the size.
+There are certain PostgreSQL users who do not recommend the above
+procedure. They have the following to say:
+PostgreSQL does not
+need to be dumped/restored to keep the database efficient. A normal
+process of vacuuming will prevent the database from every getting too
+large. If you want to fine-tweak the database storage, commands such
+as VACUUM FULL, REINDEX, and CLUSTER exist specifically to keep you
+from having to do a dump/restore.
+
+Finally, you might want to look at the PostgreSQL documentation on
+this subject at
+\elink{http://www.postgresql.org/docs/8.1/interactive/maintenance.html}
+{http://www.postgresql.org/docs/8.1/interactive/maintenance.html}.
+
\subsection*{Compacting Your SQLite Database}
\index[general]{Compacting Your SQLite Database }
\index[general]{Database!Compacting Your SQLite }
\index[fd]{Directive!Name}
The name of the Director that may contact this Client. This name must be the
same as the name specified on the Director resource in the Director's
-configuration file. This record is required.
+configuration file. Note, the case (upper/lower) of the characters in
+the name are significant (i.e. S is not the same as s). This directive
+is required.
\item [Password = \lt{}password\gt{}]
\index[fd]{Password}
\index[fd]{Directive!Password}
Specifies the password that must be supplied for a Director to be authorized.
This password must be the same as the password specified in the Client
-resource in the Director's configuration file. This record is required.
+resource in the Director's configuration file. This directive is required.
\item [Monitor = \lt{}yes|no\gt{}]
\index[fd]{Monitor}
\section*{Automated Disk Backup}
\label{_ChapterStart11}
-\index[general]{Volumes!Using Pools to Manage }
+\index[general]{Volumes!Using Pools to Manage}
\index[general]{Disk!Automated Backup}
-\index[general]{Using Pools to Manage Volumes }
+\index[general]{Using Pools to Manage Volumes}
\index[general]{Automated Disk Backup}
\addcontentsline{toc}{section}{Using Pools to Manage Volumes}
\addcontentsline{toc}{section}{Automated Disk Backup}
\label{TheProblem}
\subsection*{The Problem}
-\index[general]{Problem }
+\index[general]{Problem}
\addcontentsline{toc}{subsection}{Problem}
A site that I administer (a charitable organization) had a tape DDS-3 tape
\label{TheSolution}
\subsection*{The Solution}
-\index[general]{Solution }
+\index[general]{Solution}
\addcontentsline{toc}{subsection}{Solution}
They want to maintain 6 months of backup data, and be able to access the old
The rest of this chapter will explain how to setup Bacula so that it would
automatically manage a set of disk files with the minimum intervention on my
-part. The system has been running since 22 January 2004 until today (08 April
-2004) with no intervention. Since we have not yet crossed the six month
-boundary, we still lack some data to be sure the system performs as desired.
-\label{OverallDesign}
+part. The system has been running since 22 January 2004 until today (17
+September 2006) with no intervention, with the exception that I had to
+add a second 120GB hard disk after a year because their needs grew
+over that time to more than the 120GB (168GB to be exact). The only other
+intervention I have made is a periodic (about once a year) Bacula upgrade.
+\label{OverallDesign}
\subsection*{Overall Design}
-\index[general]{Overall Design }
-\index[general]{Design!Overall }
+\index[general]{Overall Design}
+\index[general]{Design!Overall}
\addcontentsline{toc}{subsection}{Overall Design}
Getting Bacula to write to disk rather than tape in the simplest case is
\label{FullPool}
\subsubsection*{Full Pool}
-\index[general]{Pool!Full }
-\index[general]{Full Pool }
+\index[general]{Pool!Full}
+\index[general]{Full Pool}
\addcontentsline{toc}{subsubsection}{Full Pool}
Putting a single Full backup on each Volume, will require six Full save
\label{DiffPool}
\subsubsection*{Differential Pool}
-\index[general]{Pool!Differential }
-\index[general]{Differential Pool }
+\index[general]{Pool!Differential}
+\index[general]{Differential Pool}
\addcontentsline{toc}{subsubsection}{Differential Pool}
For the Differential backup Pool, we choose a retention period of a bit longer
\label{IncPool}
\subsubsection*{Incremental Pool}
-\index[general]{Incremental Pool }
-\index[general]{Pool!Incremental }
+\index[general]{Incremental Pool}
+\index[general]{Pool!Incremental}
\addcontentsline{toc}{subsubsection}{Incremental Pool}
Finally, here is the resource for the Incremental Pool:
\label{Example}
\subsection*{The Actual Conf Files}
-\index[general]{Files!Actual Conf }
-\index[general]{Actual Conf Files }
+\index[general]{Files!Actual Conf}
+\index[general]{Actual Conf Files}
\addcontentsline{toc}{subsection}{Actual Conf Files}
The following example shows you the actual files used, with only a few minor
WorkingDirectory = "/home/bacula/working"
PidDirectory = "/home/bacula/working"
Maximum Concurrent Jobs = 1
- Password = " "
+ Password = " *** CHANGE ME ***"
Messages = Standard
}
# By default, this job will back up to disk in /tmp
Write Bootstrap = "/home/bacula/working/client.bsr"
Priority = 10
}
+
+# Backup the catalog database (after the nightly save)
+Job {
+ Name = "BackupCatalog"
+ Type = Backup
+ Client = client-fd
+ FileSet="Catalog"
+ Schedule = "WeeklyCycleAfterBackup"
+ Storage = File
+ Messages = Standard
+ Pool = Default
+ # This creates an ASCII copy of the catalog
+ RunBeforeJob = "/home/bacula/bin/make_catalog_backup bacula bacula"
+ # This deletes the copy of the catalog
+ RunAfterJob = "/home/bacula/bin/delete_catalog_backup"
+ Write Bootstrap = "/home/bacula/working/BackupCatalog.bsr"
+ Priority = 11 # run after main backup
+}
+
+# Standard Restore template, to be changed by Console program
+Job {
+ Name = "RestoreFiles"
+ Type = Restore
+ Client = havana-fd
+ FileSet="Full Set"
+ Storage = File
+ Messages = Standard
+ Pool = Default
+ Where = /tmp/bacula-restores
+}
+
+
+
# List of files to be backed up
FileSet {
Name = "Full Set"
- Include = signature=SHA1 compression=GZIP9 {
- /
- /usr
- /home
+ Include = { Options { signature=SHA1; compression=GZIP9 }
+ File = /
+ File = /usr
+ File = /home
+ File = /boot
+ File = /var
+ File = /opt
}
Exclude = {
- /proc /tmp /.journal /.fsck
+ File = /proc
+ File = /tmp
+ File = /.journal
+ File = /.fsck
+ ...
}
}
Schedule {
Name = "WeeklyCycle"
- Run = Full 1st sun at 1:05
- Run = Differential 2nd-5th sun at 1:05
- Run = Incremental mon-sat at 1:05
+ Run = Full 1st sun at 2:05
+ Run = Differential 2nd-5th sun at 2:05
+ Run = Incremental mon-sat at 2:05
}
+
+# This schedule does the catalog. It starts after the WeeklyCycle
+Schedule {
+ Name = "WeeklyCycleAfterBackup"
+ Run = Full sun-sat at 2:10
+}
+
+# This is the backup of the catalog
+FileSet {
+ Name = "Catalog"
+ Include { Options { signature=MD5 }
+ File = /home/bacula/working/bacula.sql
+ }
+}
+
Client {
Name = client-fd
Address = client
FDPort = 9102
Catalog = MyCatalog
- Password = " "
+ Password = " *** CHANGE ME ***"
AutoPrune = yes # Prune expired Jobs/Files
Job Retention = 6 months
File Retention = 60 days
}
+
Storage {
Name = File
Address = localhost
SDPort = 9103
- Password = " "
+ Password = " *** CHANGE ME ***"
Device = FileStorage
Media Type = File
}
+
Catalog {
Name = MyCatalog
dbname = bacula; user = bacula; password = ""
}
+
Pool {
Name = Full-Pool
Pool Type = Backup
Volume Retention = 6 months
Maximum Volume Jobs = 1
Label Format = Full-
- Maximum Volumes = 6
+ Maximum Volumes = 8
}
+
Pool {
Name = Inc-Pool
Pool Type = Backup
Recycle = yes # automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
- Volume Retention = 20 days
+ Volume Retention = 14 days
Maximum Volume Jobs = 6
Label Format = Inc-
- Maximum Volumes = 5
+ Maximum Volumes = 6
}
+
Pool {
Name = Diff-Pool
Pool Type = Backup
Recycle = yes
AutoPrune = yes
- Volume Retention = 40 days
+ Volume Retention = 35 days
Maximum Volume Jobs = 1
Label Format = Diff-
- Maximum Volumes = 6
+ Maximum Volumes = 10
}
+
Messages {
Name = Standard
mailcommand = "bsmtp -h mail.domain.com -f \"\(Bacula\) %r\"
}
Director {
Name = bacula-dir
- Password = " "
+ Password = " *** CHANGE ME ***"
}
Device {
Name = FileStorage
\index[general]{Dbcheck}
\index[general]{program!dbcheck}
\addcontentsline{toc}{subsection}{dbcheck}
-
{\bf dbcheck} is a simple program that will search for logical
inconsistencies in the Bacula tables in your database, and optionally fix them.
It is a database maintenance routine, in the sense that it can
associated with a Job by adding {\bf Prune Files = yes} to the Job resource.
\label{Recycling}
+\label{RecyclingAlgorithm}
\subsection*{Recycling Algorithm}
\index[general]{Algorithm!Recycling }
\index[general]{Recycling Algorithm }
\item[Solution]
For SQLite, use the vacuum command to try to fix the database. For either
MySQL or PostgreSQL, see the vendor's documentation. They have specific tools
- that check and repair databases.
+ that check and repair databases, see the \ilink{database
+ repair}{DatabaseRepair} sections of this manual for links to vendor
+ information.
Assuming the above does not resolve the problem, you will need to restore
- or rebuild your catalog.
+ or rebuild your catalog. Note, if it is a matter of some
+ inconsistencies in the Bacula tables rather than a broken database, then
+ running \ilink{dbcheck}{dbcheck} might help, but you will need to ensure
+ that your database indexes are properly setup. Please see
+ the \ilink{Database Performance Issues}{DatabasePerformance} sections
+ of this manual for more details.
+
\item[Problem]
How do I restore my catalog?
-\item[Solution]
+\item[Solution with a Catalog backup]
If you have backed up your database nightly (as you should) and you
have made a bootstrap file, you can immediately load back your
database (or the ASCII SQL output). Make a copy of your current
you will probably erase your newly restored database tables.
-\item[Solution]
+\item[Solution with a Job listing]
If you did save your database but did not make a bootstrap file, then
recovering the database is more difficult. You will probably need to
use bextract to extract the backup copy. First you should locate the
backup, you do NOT want to do a {\bf make_bacula_tables} command, or
you will probably erase your newly restored database tables.
+\item [Solution withou a Job Listing]
+ If you do not have a job listing, then it is a bit more difficult.
+ Either you use the \ilink{bscan}{bscan} program to scan the contents
+ of your tape into a database, which can be very time consuming
+ depending on the size of the tape, or you can use the \ilink{bls}{bls}
+ program to list everything on the tape, and reconstruct a bootstrap
+ file from the bls listing for the file or files you want following
+ the instructions given above.
+
+ There is a specific example of how to use {\bf bls} below.
\item [Problem]
I try to restore the last known good full backup by specifying
know the Volume to which it was backed up.
\item [Solution]
- Use {\bf bls} to indicate where it is on the tape. For example:
+ Either bscan the tape, or use {\bf bls} to indicate where it is on the
+ tape. For example:
\footnotesize
\begin{verbatim}
-1.39.23 (10 September 2006)
+1.39.23 (16 September 2006)
projects / bacula / docs / commitdiff