3 \label{_MigrationChapter}
4 \index[general]{Migration}
5 \addcontentsline{toc}{section}{Migration}
7 The term Migration, as used in the context of Bacula, means moving data from
8 one Volume to another. In particular it refers to a Job (similar to a backup
9 job) that reads data that was previously backed up to a Volume and writes
10 it to another Volume. As part of this process, catalog records associated with the first
11 backup job are purged. In other words, Migration moves Bacula Job data from one
12 Volume to another. Although we mention Volumes to simplify the subject,
13 in reality, Migration is based upon the concept of moving individual Jobs from one Pool to another.
15 Migrations can be based on quite a number of different criteria such as:
17 \item a single previous Job
20 \item a regular expression matching a Job, Volume, or Client name
21 \item the time a Job is on a Volume
22 \item high and low water marks (usage or occupation) of a Pool
26 The details of these selection criteria will be defined below.
28 To run a Migration job, you must first define a Job resource very similar
29 to a Backup Job but with {\bf Type = Migrate} instead of {\bf Type =
30 Backup}. One of the key points to remember is that the Pool that is
31 specified for the migration job is the only pool from which jobs will
32 be migrated, with one exception noted below.
34 The migration job normally is either manually started or starts
35 from a Schedule much like a backup job. It searches
36 for a previous backup Job or Jobs that match the parameters you have
37 specified in the migration Job resource, primiarily a {\bf Selection Type}
38 (detailed a bit later). Then for
39 each previous backup JobId found, the Migration Job will run a new Job which
40 copies the old Job data from the previous Volume to a new Volume in
41 the Migration Pool. It is possible that no prior Jobs are found for
42 migration, in which case, the Migration job will simply terminate having
43 done nothing, but normally at a minimum, three jobs are involved during a
47 \item The currently running Migration control Job
48 \item The previous Backup Job (already run)
49 \item A new Migration Backup Job that moves the data from the
50 previous Backup job to the new Volume.
53 If the Migration control job finds a number of JobIds to migrate (e.g.
54 it is asked to migrate one or more Volumes), it will start one new
55 migration backup job for each JobId found.
57 \subsection*{Migration Job Resource Directives}
58 \addcontentsline{toc}{section}{Migration Job Resource Directives}
60 The following directives can appear in a Director's Job resource, and they
61 are used to define a Migration job.
64 \item [Pool = \lt{}Pool-name\gt{}] The Pool specified in the Migration
65 control Job is not a new directive for the Job resource, but it is
66 particularly important because it determines what Pool will be examined for
67 finding JobIds to migrate. The exception to this is when {\bf Selection
68 Type = SQLQuery}, in which case no Pool is used, unless you
69 specifically include it in the SQL query.
71 \item [Type = Migrate]
72 {\bf Migrate} is a new type that defines the job that is run as being a
73 Migration Job. A Migration Job is a sort of control job and does not have
74 any Files associated with it, and in that sense they are more or less like
75 an Admin job. Migration jobs simply check to see if there is anything to Migrate then
76 possibly start and control new Backup jobs to migrate the data from the
77 specified Pool to another Pool.
79 \item [Selection Type = \lt{}Selection-type-keyword\gt{}]
80 The \lt{}Selection-type-keyword\gt{} determines how the migration job
81 will go about selecting what JobIds to migrate. In most cases, it is
82 used in conjunction with a {\bf Selection Pattern} to give you fine
83 control over exactly what JobIds are selected. The possible values
84 for \lt{}Selection-type-keyword\gt{} are:
86 \item [SmallestVolume] This selection keyword selects the volume with the
87 fewest bytes from the Pool to be migrated. The Pool to be migrated
88 is the Pool defined in the Migration Job resource. The migration
89 control job will then start and run one migration backup job for
90 each of the Jobs found on this Volume. The Selection Pattern, if
91 specified, is not used.
92 \item [OldestVolume] This selection keyword selects the volume with the
93 oldest last write time in the Pool to be migrated. The Pool to be
94 migrated is the Pool defined in the Migration Job resource. The
95 migration control job will then start and run one migration backup
96 job for each of the Jobs found on this Volume. The Selection
97 Pattern, if specified, is not used.
98 \item [Client] The Client selection type, first selects all the Clients
99 that have been backed up in the Pool specified by the Migration
100 Job resource, then it applies the {\bf Selection Pattern} (defined
101 below) as a regular expression to the list of Client names, giving
102 a filtered Client name list. All jobs that were backed up for those
103 filtered (regexed) Clients will be migrated.
104 The migration control job will then start and run one migration
105 backup job for each of the JobIds found for those filtered Clients.
106 \item [Volume] The Volume selection type, first selects all the Volumes
107 that have been backed up in the Pool specified by the Migration
108 Job resource, then it applies the {\bf Selection Pattern} (defined
109 below) as a regular expression to the list of Volume names, giving
110 a filtered Volume list. All JobIds that were backed up for those
111 filtered (regexed) Volumes will be migrated.
112 The migration control job will then start and run one migration
113 backup job for each of the JobIds found on those filtered Volumes.
114 \item [Job] The Job selection type, first selects all the Jobs (as
115 defined on the {\bf Name} directive in a Job resource)
116 that have been backed up in the Pool specified by the Migration
117 Job resource, then it applies the {\bf Selection Pattern} (defined
118 below) as a regular expression to the list of Job names, giving
119 a filtered Job name list. All JobIds that were run for those
120 filtered (regexed) Job names will be migrated. Note, for a given
121 Job named, they can be many jobs (JobIds) that ran.
122 The migration control job will then start and run one migration
123 backup job for each of the Jobs found.
124 \item [SQLQuery] The SQLQuery selection type, used the {\bf Selection
125 Pattern} as an SQL query to obtain the JobIds to be migrated.
126 The Selection Pattern must be a valid SELECT SQL statement for your
127 SQL engine, and it must return the JobId as the first field
129 \item [PoolOccupancy] Not yet implemented.
130 \item [PoolTime] Not yet implemented.
133 \item [Selection Pattern = \lt{}Quoted-string\gt{}]
134 The Selection Patterns permitted for each Selection-type-keyword are
137 For the OldestVolume and SmallestVolume, this
138 Selection pattern is not used (ignored).
140 For the Client, Volume, and Job
141 keywords, this pattern must be a valid regular expression that will filter
142 the appropriate item names found in the Pool.
144 For the SQLQuery keyword, this pattern must be a valid SELECT SQL statement
149 \subsection*{Migration Pool Resource Directives}
150 \addcontentsline{toc}{section}{Migration Pool Resource Directives}
152 The following directives can appear in a Director's Pool resource, and they
153 are used to define a Migration job.
156 \item [Migration Time = \lt{}time-specification\gt{}]
157 If a PoolTime migration is done, the time specified here in seconds (time
158 modifiers are permitted -- e.g. hours, ...) will be used. If the
159 previous Backup Job or Jobs selected have been in the Pool longer than
160 the specified PoolTime, then they will be migrated.
162 \item [Migration High Bytes = \lt{}byte-specification\gt{}]
163 This directive specifies the number of bytes in the Pool which will
164 trigger a migration if a {\bf PoolOccupancy} migration selection
165 type has been specified. Please note, that the fact that the Pool
166 usage goes above this level does not automatically trigger a migration
169 \item [Migration Low Bytes = \lt{}byte-specification\gt{}]
170 This directive specifies the number of bytes in the Pool which will
171 stop a migration if a {\bf PoolOccupancy} migration selection
172 type has been specified and triggered by more than Migration High
173 Bytes being in the pool. In other words, once a migration job
174 is started with {\bf PoolOccupancy} migration selection and it
175 determines that there are more than Migration High Bytes, the
176 migration job will continue to run jobs until the number of
177 bytes in the Pool drop to or below Migration Low Bytes.
179 \item [Next Pool = \lt{}pool-specification\gt{}]
180 The Next Pool directive specifies the pool to which Jobs will be
183 \item [Storage = \lt{}storage-specification\gt{}]
184 The Storage directive specifies what Storage resource will be used
185 for all Jobs that use this Pool. It takes precedence over any other
186 Storage specifications that may have been given such as in the
187 Schedule Run directive, or in the Job resource.
191 \subsection*{Example Migration Jobs}
192 \index[general]{Example Migration Jobs}
193 \addcontentsline{toc}{subsection}{Example Migration Jobs}
195 As an example, suppose you have the following Job that
200 # Define the backup Job
204 Level = Incremental # default
207 Schedule = "WeeklyCycle"
212 # Default pool definition
223 # Tape pool definition
232 # Definition of File storage device
236 Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
237 Device = "File" # same as Device in Storage daemon
238 Media Type = File # same as MediaType in Storage daemon
241 # Definition of DLT tape storage device
245 Password = "ccV3lVTsQRsdIUGyab0N4sMDavui2hOBkmpBU0aQKOr9"
246 Device = "HP DLT 80" # same as Device in Storage daemon
247 Media Type = DLT8000 # same as MediaType in Storage daemon
253 Where we have included only the essential information -- i.e. the
254 Director, FileSet, Catalog, Client, Schedule, and Messages resources are omitted.
256 As you can see, by running the NightlySave Job, the data will be backed up
257 to File storage using the Default pool to specify the Storage as File.
259 Now, if we add the following Job resource to this conf file.
264 Name = "migrate-volume"
272 Maximum Concurrent Jobs = 4
273 Selection Type = Volume
274 Selection Pattern = "File"
279 and then run the job named {\bf migrate-volume}, all volumes in the Pool
280 named Default (as specified in the migrate-volume Job that match the
281 regular expression pattern {\bf File} will be migrated to tape storage
282 DLTDrive because the {\bf Next Pool} in the Default Pool specifies that
283 Migrations should go to the pool named {\bf Tape}, which uses
284 Storage {\bf DLTDrive}.
286 If instead, we use a Job resource as follows:
299 Maximum Concurrent Jobs = 4
301 Selection Pattern = ".*Save"
306 All jobs ending with the name Save will be migrated from the File Default to
307 the Tape Pool, or from File storage to Tape storage.