\index[general]{Migration}
\addcontentsline{toc}{section}{Migration}
-The term Migration as used in the context of Bacula means moving data from one
-Volume to another, and in particular it is
-a Job (similar to a backup job) that reads data that was previously
-backed up to a Volume and writes it to another Volume purging the
-catalog records associated with the first backup job. In other words,
-Migration moves Bacula Job data from one Volume to another. Although we
-mention Volumes, in reality, Migration is based on moving individual Jobs from
-one Pool to another.
+The term Migration as used in the context of Bacula means moving data from
+one Volume to another, and in particular it is a Job (similar to a backup
+job) that reads data that was previously backed up to a Volume and writes
+it to another Volume purging the catalog records associated with the first
+backup job. In other words, Migration moves Bacula Job data from one
+Volume to another. Although we mention Volumes to simplify the subject,
+in reality, Migration is based on moving individual Jobs from one Pool to another.
Migrations can be based on quite a number of different criteria such as:
a single previous Job; a Volume; a Client; a regular expression matching
and low water marks (usage or occupation) of a Pool; Volume size, ...
The details of these selection criteria will be defined below.
-
To run a Migration job, you must have defined a Job resource very similar
to a Backup Job but with {\bf Type = Migrate} instead of {\bf Type =
-Backup}. The migration job then starts much like a backup job starts and
-searches for a previous backup Job or Jobs that matches the parameters you have
-specified in the migration Job resource (detailed a bit later). Then
-for each previous backup Job found, the Migration Job will run a new
-Job which copies the old Job data from the previous Volume to a new
-Volume defined in the Migration Pool. It is possible that no prior
-Jobs are found for migration, in which case, the Migration job will
-simply terminate having done nothing, but normally at a minimum, three jobs
-are involved during a migration:
+Backup}. One of the key points to remember is that the Pool that is
+specified for the migration job is the only pool from which jobs will
+be migrated, with one exception noted below.
+
+The migration job normally is either manually started or starts
+from a Schedule much like a backup job. It searches
+for a previous backup Job or Jobs that match the parameters you have
+specified in the migration Job resource, primiarily a {\bf Selection Type}
+(detailed a bit later). Then for
+each previous backup JobId found, the Migration Job will run a new Job which
+copies the old Job data from the previous Volume to a new Volume in
+the Migration Pool. It is possible that no prior Jobs are found for
+migration, in which case, the Migration job will simply terminate having
+done nothing, but normally at a minimum, three jobs are involved during a
+migration:
\begin{itemize}
-\item The currently running Migration Job
-\item The previous Backup Job
-\item A new Backup Job that writes the previous data to
- the new Volume.
+\item The currently running Migration control Job
+\item The previous Backup Job (already run)
+\item A new Migration Backup Job that moves the data from the
+ previous Backup job to the new Volume.
\end{itemize}
-If the Migration is for a particular Volume, multiple new Backup Jobs
-may be started, one for each Job on the Volume.
-
-
+If the Migration control job finds a number of JobIds to migrate (e.g.
+it is asked to migrate one or more Volumes), it will start one new
+migration backup job for each JobId found.
\subsection*{Migration Job Resource Directives}
\addcontentsline{toc}{section}{Migration Job Resource Directives}
are used to define a Migration job.
\begin{description}
+\item [Pool = \lt{}Pool-name\gt{}] The Pool specified in the Migration
+control Job is not a new directive for the Job resource, but it is
+particularly important because it determines what Pool will be examined for
+finding JobIds to migrate. The exception to this is when {\bf Selection
+Type = SQLQuery}, in which case no Pool is used, unless you
+specifically include it in the SQL query.
+
\item [Type = Migrate]
-This defines the job that is run as being a Migration Job.
-Migration Jobs do not have any Files associated with them, and in
-that sense they are more or less like an Admin job. They just
-check to see if there is anything to Migrate then possibly start
-and control new Backup jobs to move the data.
+{\bf Migrate} is a new type that defines the job that is run as being a
+Migration Job. A Migration Job is a sort of control job and does not have
+any Files associated with it, and in that sense they are more or less like
+an Admin job. Migration jobs simply check to see if there is anything to Migrate then
+possibly start and control new Backup jobs to migrate the data from the
+specified Pool to another Pool.
\item [Selection Type = \lt{}Selection-type-keyword\gt{}]
- Where \lt{}Selection-type-keyword\gt{} can be:
+ The \lt{}Selection-type-keyword\gt{} determines how the migration job
+ will go about selecting what JobIds to migrate. In most cases, it is
+ used in conjunction with a {\bf Selection Pattern} to give you fine
+ control over exactly what JobIds are selected. The possible values
+ for \lt{}Selection-type-keyword\gt{} are:
\begin{description}
- \item [SmallestVolume] This selection keyword selects the smallest volume in
- the Pool to be migrated.
- \item [OldestVolume] This selection keyword selects the oldest volume in
- the Pool to be migrated.
- \item [Client]
- \item [Volume]
- \item [Job]
- \item [SQLQuery]
+ \item [SmallestVolume] This selection keyword selects the volume with the
+ fewest bytes from the Pool to be migrated. The Pool to be migrated
+ is the Pool defined in the Migration Job resource. The migration
+ control job will then start and run one migration backup job for
+ each of the Jobs found on this Volume. The Selection Pattern, if
+ specified, is not used.
+ \item [OldestVolume] This selection keyword selects the volume with the
+ oldest last write time in the Pool to be migrated. The Pool to be
+ migrated is the Pool defined in the Migration Job resource. The
+ migration control job will then start and run one migration backup
+ job for each of the Jobs found on this Volume. The Selection
+ Pattern, if specified, is not used.
+ \item [Client] The Client selection type, first selects all the Clients
+ that have been backed up in the Pool specified by the Migration
+ Job resource, then it applies the {\bf Selection Pattern} (defined
+ below) as a regular expression to the list of Client names, giving
+ a filtered Client name list. All jobs that were backed up for those
+ filtered (regexed) Clients will be migrated.
+ The migration control job will then start and run one migration
+ backup job for each of the JobIds found for those filtered Clients.
+ \item [Volume] The Volume selection type, first selects all the Volumes
+ that have been backed up in the Pool specified by the Migration
+ Job resource, then it applies the {\bf Selection Pattern} (defined
+ below) as a regular expression to the list of Volume names, giving
+ a filtered Volume list. All JobIds that were backed up for those
+ filtered (regexed) Volumes will be migrated.
+ The migration control job will then start and run one migration
+ backup job for each of the JobIds found on those filtered Volumes.
+ \item [Job] The Job selection type, first selects all the Jobs (as
+ defined on the {\bf Name} directive in a Job resource)
+ that have been backed up in the Pool specified by the Migration
+ Job resource, then it applies the {\bf Selection Pattern} (defined
+ below) as a regular expression to the list of Job names, giving
+ a filtered Job name list. All JobIds that were run for those
+ filtered (regexed) Job names will be migrated. Note, for a given
+ Job named, they can be many jobs (JobIds) that ran.
+ The migration control job will then start and run one migration
+ backup job for each of the Jobs found.
+ \item [SQLQuery] The SQLQuery selection type, used the {\bf Selection
+ Pattern} as an SQL query to obtain the JobIds to be migrated.
+ The Selection Pattern must be a valid SELECT SQL statement for your
+ SQL engine, and it must return the JobId as the first field
+ of the SELECT.
\item [PoolOccupancy] Not yet implemented.
\item [PoolTime] Not yet implemented.
\end{description}
-\end{description}
-
\item [Selection Pattern = \lt{}Quoted-string\gt{}]
-The Selection Patterns permitted for each Selection-type-keyword
-are described above.
+ The Selection Patterns permitted for each Selection-type-keyword are
+ described above.
+
+ For the OldestVolume and SmallestVolume, this
+ Selection pattern is not used (ignored).
+
+ For the Client, Volume, and Job
+ keywords, this pattern must be a valid regular expression that will filter
+ the appropriate item names found in the Pool.
+
+ For the SQLQuery keyword, this pattern must be a valid SELECT SQL statement
+ that returns JobIds.
\end{description}
the specified PoolTime, then they will be migrated.
\item [Migration High Bytes = \lt{}byte-specification\gt{}]
+ This directive specifies the number of bytes in the Pool which will
+ trigger a migration if a {\bf PoolOccupancy} migration selection
+ type has been specified. Please note, that the fact that the Pool
+ usage goes above this level does not automatically trigger a migration
+ job.
\item [Migration Low Bytes = \lt{}byte-specification\gt{}]
+ This directive specifies the number of bytes in the Pool which will
+ stop a migration if a {\bf PoolOccupancy} migration selection
+ type has been specified and triggered by more than Migration High
+ Bytes being in the pool. In other words, once a migration job
+ is started with {\bf PoolOccupancy} migration selection and it
+ determines that there are more than Migration High Bytes, the
+ migration job will continue to run jobs until the number of
+ bytes in the Pool drop to or below Migration Low Bytes.
\item [Next Pool = \lt{}pool-specification\gt{}]
+ The Next Pool directive specifies the pool to which Jobs will be
+ migrated.
\item [Storage = \lt{}storage-specification\gt{}]
+ The Storage directive specifies what Storage resource will be used
+ for all Jobs that use this Pool. It takes precedence over any other
+ Storage specifications that may have been given such as in the
+ Schedule Run directive, or in the Job resource.
\end{description}
+
+\subsection*{Example Migration Jobs}
+\index[general]{Example Migration Jobs}
+\addcontentsline{toc}{subsection}{Example Migration Jobs}
+
+As an example, suppose you have the following Job that
+you run every night:
+
+\footnotesize
+\begin{verbatim}
+# Define the backup Job
+Job {
+ Name = "NightlySave"
+ Type = Backup
+ Level = Incremental # default
+ Client=rufus-fd
+ FileSet="Full Set"
+ Schedule = "WeeklyCycle"
+ Messages = Standard
+ Pool = Default
+}
+
+# Default pool definition
+Pool {
+ Name = Default
+ Pool Type = Backup
+ AutoPrune = yes
+ Recycle = yes
+ Next Pool = Tape
+ Storage = File
+ LabelFormat = "File"
+}
+
+# Tape pool definition
+Pool {
+ Name = Tape
+ Pool Type = Backup
+ AutoPrune = yes
+ Recycle = yes
+ Storage = DLTDrive
+}
+
+# Definition of File storage device
+Storage {
+ Name = File
+ Address = rufus
+ Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
+ Device = "File" # same as Device in Storage daemon
+ Media Type = File # same as MediaType in Storage daemon
+}
+
+# Definition of DLT tape storage device
+Storage {
+ Name = DLTDrive
+ Address = rufus
+ Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
+ Device = "HP DLT 80" # same as Device in Storage daemon
+ Media Type = DLT8000 # same as MediaType in Storage daemon
+}
+
+\end{verbatim}
+\normalsize
+
+Where we have included only the essential information -- i.e. the
+Director, FileSet, Catalog, Client, Schedule, and Messages resources are omitted.
+
+As you can see, by running the NightlySave Job, the data will be backed up
+to File storage using the Default pool to specify the Storage as File.
+
+Now, if we add the following Job resource to this conf file.
+
+\footnotesize
+\begin{verbatim}
+Job {
+ Name = "migrate-volume"
+ Type = Migrate
+ Level = Full
+ Client = rufus-fd
+ FileSet = "Full Set"
+ Messages = Standard
+ Storage = DLTDrive
+ Pool = Default
+ Maximum Concurrent Jobs = 4
+ Selection Type = Volume
+ Selection Pattern = "File"
+}
+\end{verbatim}
+\normalsize
+
+and then run the job named {\bf migrate-volume}, all volumes in the Pool
+named Default (as specified in the migrate-volume Job that match the
+regular expression pattern {\bf File} will be migrated to tape storage
+DLTDrive because the {\bf Next Pool} in the Default Pool specifies that
+Migrations should go to the pool named {\bf Tape}, which uses
+Storage {\bf DLTDrive}.
+
+If instead, we use a Job resource as follows:
+
+\footnotesize
+\begin{verbatim}
+Job {
+ Name = "migrate"
+ Type = Migrate
+ Level = Full
+ Client = rufus-fd
+ FileSet="Full Set"
+ Messages = Standard
+ Storage = DLTDrive
+ Pool = Default
+ Maximum Concurrent Jobs = 4
+ Selection Type = Job
+ Selection Pattern = ".*Save"
+}
+\end{verbatim}
+\normalsize
+
+All jobs ending with the name Save will be migrated from the File Default to
+the Tape Pool, or from File storage to Tape storage.
projects / bacula / docs / blobdiff