4 \section*{Storage Media Output Format}
6 \index{Format!Storage Media Output }
7 \index{Storage Media Output Format }
8 \addcontentsline{toc}{section}{Storage Media Output Format}
12 \addcontentsline{toc}{subsection}{General}
14 This document describes the media format written by the Storage daemon. The
15 Storage daemon reads and writes in units of blocks. Blocks contain records.
16 Each block has a block header followed by records, and each record has a
17 record header followed by record data.
19 This chapter is intended to be a technical discussion of the Media Format and
20 as such is not targeted at end users but rather at developers and system
21 administrators that want or need to know more of the working details of {\bf
24 \subsection*{Definitions}
26 \addcontentsline{toc}{subsection}{Definitions}
32 A block represents the primitive unit of information that the Storage daemon
33 reads and writes to a physical device. Normally, for a tape device, it will
34 be the same as a tape block. The Storage daemon always reads and writes
35 blocks. A block consists of block header information followed by records.
36 Clients of the Storage daemon (the File daemon) normally never see blocks.
37 However, some of the Storage tools (bls, bscan, bextract, ...) may be use
38 block header information. In older Bacula tape versions, a block could
39 contain records (see record definition below) from multiple jobs. However,
40 all blocks currently written by Bacula are block level BB02, and a given
41 block contains records for only a single job. Different jobs simply have
42 their own private blocks that are intermingled with the other blocks from
43 other jobs on the Volume (previously the records were intermingled within
44 the blocks). Having only records from a single job in any give block
45 permitted moving the VolumeSessionId and VolumeSessionTime (see below) from
46 each record heading to the Block header. This has two advantages: 1. a block
47 can be quickly rejected based on the contents of the header without reading
48 all the records. 2. because there is on the average more than one record per
49 block, less data is written to the Volume for each job.
53 A record consists of a Record Header, which is managed by the Storage daemon
54 and Record Data, which is the data received from the Client. A record is the
55 primitive unit of information sent to and from the Storage daemon by the
56 Client (File daemon) programs. The details are described below.
60 A number assigned by the Director daemon for a particular job. This number
61 will be unique for that particular Director (Catalog). The daemons use this
62 number to keep track of individual jobs. Within the Storage daemon, the JobId
63 may not be unique if several Directors are accessing the Storage daemon
68 A Session is a concept used in the Storage daemon corresponds one to one to a
69 Job with the exception that each session is uniquely identified within the
70 Storage daemon by a unique SessionId/SessionTime pair (see below).
74 A unique number assigned by the Storage daemon to a particular session (Job)
75 it is having with a File daemon. This number by itself is not unique to the
76 given Volume, but with the VolSessionTime, it is unique.
78 \item [VolSessionTime]
79 \index{VolSessionTime }
80 A unique number assigned by the Storage daemon to a particular Storage daemon
81 execution. It is actually the Unix time\_t value of when the Storage daemon
82 began execution cast to a 32 bit unsigned integer. The combination of the
83 {\bf VolSessionId} and the {\bf VolSessionTime} for a given Storage daemon is
84 guaranteed to be unique for each Job (or session).
88 A sequential number beginning at one assigned by the File daemon to the files
89 within a job that are sent to the Storage daemon for backup. The Storage
90 daemon ensures that this number is greater than zero and sequential. Note,
91 the Storage daemon uses negative FileIndexes to flag Session Start and End
92 Labels as well as End of Volume Labels. Thus, the combination of
93 VolSessionId, VolSessionTime, and FileIndex uniquely identifies the records
94 for a single file written to a Volume.
98 While writing the information for any particular file to the Volume, there
99 can be any number of distinct pieces of information about that file, e.g. the
100 attributes, the file data, ... The Stream indicates what piece of data it
101 is, and it is an arbitrary number assigned by the File daemon to the parts
102 (Unix attributes, Win32 attributes, data, compressed data,\ ...) of a file
103 that are sent to the Storage daemon. The Storage daemon has no knowledge of
104 the details of a Stream; it simply represents a numbered stream of bytes. The
105 data for a given stream may be passed to the Storage daemon in single record,
106 or in multiple records.
109 \index{Block Header }
110 A block header consists of a block identification (``BB02''), a block length
111 in bytes (typically 64,512) a checksum, and sequential block number. Each
112 block starts with a Block Header and is followed by Records. Current block
113 headers also contain the VolSessionId and VolSessionTime for the records
114 written to that block.
116 \item [Record Header]
117 \index{Record Header }
118 A record header contains the Volume Session Id, the Volume Session Time, the
119 FileIndex, the Stream, and the size of the data record which follows. The
120 Record Header is always immediately followed by a Data Record if the size
121 given in the Header is greater than zero. Note, for Block headers of level
122 BB02 (version 1.27 and later), the Record header as written to tape does not
123 contain the Volume Session Id and the Volume Session Time as these two
124 fields are stored in the BB02 Block header. The in-memory record header does
125 have those fields for convenience.
129 A data record consists of a binary stream of bytes and is always preceded by
130 a Record Header. The details of the meaning of the binary stream of bytes are
131 unknown to the Storage daemon, but the Client programs (File daemon) defines
132 and thus knows the details of each record type.
135 \index{Volume Label }
136 A label placed by the Storage daemon at the beginning of each storage volume.
137 It contains general information about the volume. It is written in Record
138 format. The Storage daemon manages Volume Labels, and if the client wants, he
141 \item [Begin Session Label]
142 \index{Begin Session Label }
143 The Begin Session Label is a special record placed by the Storage daemon on
144 the storage medium as the first record of an append session job with a File
145 daemon. This record is useful for finding the beginning of a particular
146 session (Job), since no records with the same VolSessionId and VolSessionTime
147 will precede this record. This record is not normally visible outside of the
148 Storage daemon. The Begin Session Label is similar to the Volume Label except
149 that it contains additional information pertaining to the Session.
151 \item [End Session Label]
152 \index{End Session Label }
153 The End Session Label is a special record placed by the Storage daemon on the
154 storage medium as the last record of an append session job with a File
155 daemon. The End Session Record is distinguished by a FileIndex with a value
156 of minus two (-2). This record is useful for detecting the end of a
157 particular session since no records with the same VolSessionId and
158 VolSessionTime will follow this record. This record is not normally visible
159 outside of the Storage daemon. The End Session Label is similar to the Volume
160 Label except that it contains additional information pertaining to the
164 \subsection*{Storage Daemon File Output Format}
165 \index{Format!Storage Daemon File Output }
166 \index{Storage Daemon File Output Format }
167 \addcontentsline{toc}{subsection}{Storage Daemon File Output Format}
169 The file storage and tape storage formats are identical except that tape
170 records are by default blocked into blocks of 64,512 bytes, except for the
171 last block, which is the actual number of bytes written rounded up to a
172 multiple of 1024 whereas the last record of file storage is not rounded up.
173 The default block size of 64,512 bytes may be overridden by the user (some
174 older tape drives only support block sizes of 32K). Each Session written to
175 tape is terminated with an End of File mark (this will be removed later).
176 Sessions written to file are simply appended to the end of the file.
178 \subsection*{Overall Format}
179 \index{Format!Overall }
180 \index{Overall Format }
181 \addcontentsline{toc}{subsection}{Overall Format}
183 A Bacula output file consists of Blocks of data. Each block contains a block
184 header followed by records. Each record consists of a record header followed
185 by the record data. The first record on a tape will always be the Volume Label
188 No Record Header will be split across Bacula blocks. However, Record Data may
189 be split across any number of Bacula blocks. Obviously this will not be the
190 case for the Volume Label which will always be smaller than the Bacula Block
193 To simplify reading tapes, the Start of Session (SOS) and End of Session (EOS)
194 records are never split across blocks. If this is about to happen, Bacula will
195 write a short block before writing the session record (actually, the SOS
196 record should always be the first record in a block, excepting perhaps the
199 Due to hardware limitations, the last block written to the tape may not be
200 fully written. If your drive permits backspace record, Bacula will backup over
201 the last record written on the tape, re-read it and verify that it was
204 When a new tape is mounted Bacula will write the full contents of the
205 partially written block to the new tape ensuring that there is no loss of
206 data. When reading a tape, Bacula will discard any block that is not totally
207 written, thus ensuring that there is no duplication of data. In addition,
208 since Bacula blocks are sequentially numbered within a Job, it is easy to
209 ensure that no block is missing or duplicated.
211 \subsection*{Serialization}
212 \index{Serialization }
213 \addcontentsline{toc}{subsection}{Serialization}
215 All Block Headers, Record Headers, and Label Records are written using
216 Bacula's serialization routines. These routines guarantee that the data is
217 written to the output volume in a machine independent format.
219 \subsection*{Block Header}
220 \index{Header!Block }
221 \index{Block Header }
222 \addcontentsline{toc}{subsection}{Block Header}
224 The format of the Block Header (version 1.27 and later) is:
228 uint32_t CheckSum; /* Block check sum */
229 uint32_t BlockSize; /* Block byte size including the header */
230 uint32_t BlockNumber; /* Block number */
231 char ID[4] = "BB02"; /* Identification and block level */
232 uint32_t VolSessionId; /* Session Id for Job */
233 uint32_t VolSessionTime; /* Session Time for Job */
237 The Block header is a fixed length and fixed format and is followed by Record
238 Headers and Record Data. The CheckSum field is a 32 bit checksum of the block
239 data and the block header but not including the CheckSum field. The Block
240 Header is always immediately followed by a Record Header. If the tape is
241 damaged, a Bacula utility will be able to recover as much information as
242 possible from the tape by recovering blocks which are valid. The Block header
243 is written using the Bacula serialization routines and thus is guaranteed to
244 be in machine independent format. See below for version 2 of the block header.
247 \subsection*{Record Header}
248 \index{Header!Record }
249 \index{Record Header }
250 \addcontentsline{toc}{subsection}{Record Header}
252 Each binary data record is preceded by a Record Header. The Record Header is
253 fixed length and fixed format, whereas the binary data record is of variable
254 length. The Record Header is written using the Bacula serialization routines
255 and thus is guaranteed to be in machine independent format.
257 The format of the Record Header (version 1.27 or later) is:
261 int32_t FileIndex; /* File index supplied by File daemon */
262 int32_t Stream; /* Stream number supplied by File daemon */
263 uint32_t DataSize; /* size of following data record in bytes */
267 This record is followed by the binary Stream data of DataSize bytes, followed
268 by another Record Header record and the binary stream data. For the definitive
269 definition of this record, see record.h in the src/stored directory.
271 Additional notes on the above:
275 \item [The {\bf VolSessionId} ]
276 \index{VolSessionId }
277 is a unique sequential number that is assigned by the Storage Daemon to a
278 particular Job. This number is sequential since the start of execution of the
281 \item [The {\bf VolSessionTime} ]
282 \index{VolSessionTime }
283 is the time/date that the current execution of the Storage Daemon started. It
284 assures that the combination of VolSessionId and VolSessionTime is unique for
285 every jobs written to the tape, even if there was a machine crash between two
288 \item [The {\bf FileIndex} ]
290 is a sequential file number within a job. The Storage daemon requires this
291 index to be greater than zero and sequential. Note, however, that the File
292 daemon may send multiple Streams for the same FileIndex. In addition, the
293 Storage daemon uses negative FileIndices to hold the Begin Session Label, the
294 End Session Label, and the End of Volume Label.
296 \item [The {\bf Stream} ]
298 is defined by the File daemon and is used to identify separate parts of the
299 data saved for each file (Unix attributes, Win32 attributes, file data,
300 compressed file data, sparse file data, ...). The Storage Daemon has no idea
301 of what a Stream is or what it contains except that the Stream is required to
302 be a positive integer. Negative Stream numbers are used internally by the
303 Storage daemon to indicate that the record is a continuation of the previous
304 record (the previous record would not entirely fit in the block).
306 For Start Session and End Session Labels (where the FileIndex is negative),
307 the Storage daemon uses the Stream field to contain the JobId. The current
308 stream definitions are:
312 STREAM_UNIX_ATTRIBUTES 1 /* Generic Unix attributes */
313 STREAM_FILE_DATA 2 /* Standard uncompressed data */
314 STREAM_MD5_SIGNATURE 3 /* MD5 signature for the file */
315 STREAM_GZIP_DATA 4 /* GZip compressed file data */
316 STREAM_WIN32_ATTRIBUTES 5 /* Windows attributes (superset of Unix) */
317 STREAM_SPARSE_DATA 6 /* Sparse data stream */
318 STREAM_SPARSE_GZIP_DATA 7 /* Sparse data stream compressed by GZIP */
319 STREAM_PROGRAM_NAMES 8 /* program names for program data */
320 STREAM_PROGRAM_DATA 9 /* Data needing program */
321 STREAM_SHA1_SIGNATURE 10 /* SHA1 signature for the file */
322 STREAM_WIN32_DATA 11 /* Win32 BackupRead data */
323 STREAM_WIN32_GZIP_DATA 12 /* Gzipped Win32 BackupRead data */
328 \item [The {\bf DataSize} ]
330 is the size in bytes of the binary data record that follows the Session
331 Record header. The Storage Daemon has no idea of the actual contents of the
332 binary data record. For standard Unix files, the data record typically
333 contains the file attributes or the file data. For a sparse file the first
334 64 bits of the file data contains the storage address for the data block.
337 The Record Header is never split across two blocks. If there is not enough
338 room in a block for the full Record Header, the block is padded to the end
339 with zeros and the Record Header begins in the next block. The data record, on
340 the other hand, may be split across multiple blocks and even multiple physical
341 volumes. When a data record is split, the second (and possibly subsequent)
342 piece of the data is preceded by a new Record Header. Thus each piece of data
343 is always immediately preceded by a Record Header. When reading a record, if
344 Bacula finds only part of the data in the first record, it will automatically
345 read the next record and concatenate the data record to form a full data
348 \subsection*{Version BB02 Block Header}
349 \index{Version BB02 Block Header }
350 \index{Header!Version BB02 Block }
351 \addcontentsline{toc}{subsection}{Version BB02 Block Header}
353 Each session or Job has its own private block. As a consequence, the SessionId
354 and SessionTime are written once in each Block Header and not in the Record
355 Header. So, the second and current version of the Block Header BB02 is:
359 uint32_t CheckSum; /* Block check sum */
360 uint32_t BlockSize; /* Block byte size including the header */
361 uint32_t BlockNumber; /* Block number */
362 char ID[4] = "BB02"; /* Identification and block level */
363 uint32_t VolSessionId; /* Applies to all records */
364 uint32_t VolSessionTime; /* contained in this block */
368 As with the previous version, the BB02 Block header is a fixed length and
369 fixed format and is followed by Record Headers and Record Data. The CheckSum
370 field is a 32 bit CRC checksum of the block data and the block header but not
371 including the CheckSum field. The Block Header is always immediately followed
372 by a Record Header. If the tape is damaged, a Bacula utility will be able to
373 recover as much information as possible from the tape by recovering blocks
374 which are valid. The Block header is written using the Bacula serialization
375 routines and thus is guaranteed to be in machine independent format.
377 \subsection*{Version 2 Record Header}
378 \index{Version 2 Record Header }
379 \index{Header!Version 2 Record }
380 \addcontentsline{toc}{subsection}{Version 2 Record Header}
382 Version 2 Record Header is written to the medium when using Version BB02 Block
383 Headers. The memory representation of the record is identical to the old BB01
384 Record Header, but on the storage medium, the first two fields, namely
385 VolSessionId and VolSessionTime are not written. The Block Header is filled
386 with these values when the First user record is written (i.e. non label
387 record) so that when the block is written, it will have the current and unique
388 VolSessionId and VolSessionTime. On reading each record from the Block, the
389 VolSessionId and VolSessionTime is filled in the Record Header from the Block
392 \subsection*{Volume Label Format}
393 \index{Volume Label Format }
394 \index{Format!Volume Label }
395 \addcontentsline{toc}{subsection}{Volume Label Format}
397 Tape volume labels are created by the Storage daemon in response to a {\bf
398 label} command given to the Console program, or alternatively by the {\bf
399 btape} program. created. Each volume is labeled with the following information
400 using the Bacula serialization routines, which guarantee machine byte order
403 For Bacula versions 1.27 and later, the Volume Label Format is:
407 char Id[32]; /* Bacula 1.0 Immortal\n */
408 uint32_t VerNum; /* Label version number */
409 /* VerNum 11 and greater Bacula 1.27 and later */
410 btime_t label_btime; /* Time/date tape labeled */
411 btime_t write_btime; /* Time/date tape first written */
412 /* The following are 0 in VerNum 11 and greater */
413 float64_t write_date; /* Date this label written */
414 float64_t write_time; /* Time this label written */
415 char VolName[128]; /* Volume name */
416 char PrevVolName[128]; /* Previous Volume Name */
417 char PoolName[128]; /* Pool name */
418 char PoolType[128]; /* Pool type */
419 char MediaType[128]; /* Type of this media */
420 char HostName[128]; /* Host name of writing computer */
421 char LabelProg[32]; /* Label program name */
422 char ProgVersion[32]; /* Program version */
423 char ProgDate[32]; /* Program build date/time */
427 Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, ...)
428 is stored in the record FileIndex field of the Record Header and does not
429 appear in the data part of the record.
431 \subsection*{Session Label}
432 \index{Label!Session }
433 \index{Session Label }
434 \addcontentsline{toc}{subsection}{Session Label}
436 The Session Label is written at the beginning and end of each session as well
437 as the last record on the physical medium. It has the following binary format:
442 char Id[32]; /* Bacula Immortal ... */
443 uint32_t VerNum; /* Label version number */
444 uint32_t JobId; /* Job id */
445 uint32_t VolumeIndex; /* sequence no of vol */
446 /* Prior to VerNum 11 */
447 float64_t write_date; /* Date this label written */
448 /* VerNum 11 and greater */
449 btime_t write_btime; /* time/date record written */
450 /* The following is zero VerNum 11 and greater */
451 float64_t write_time; /* Time this label written */
452 char PoolName[128]; /* Pool name */
453 char PoolType[128]; /* Pool type */
454 char JobName[128]; /* base Job name */
455 char ClientName[128];
456 /* Added in VerNum 10 */
457 char Job[128]; /* Unique Job name */
458 char FileSetName[128]; /* FileSet name */
464 In addition, the EOS label contains:
468 /* The remainder are part of EOS label only */
471 uint32_t start_block;
479 In addition, for VerNum greater than 10, the EOS label contains (in addition
484 uint32_t JobStatus /* Job termination code */
488 : Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label,
489 ...) is stored in the record FileIndex field and does not appear in the data
490 part of the record. Also, the Stream field of the Record Header contains the
491 JobId. This permits quick filtering without actually reading all the session
494 \subsection*{Overall Storage Format}
495 \index{Format!Overall Storage }
496 \index{Overall Storage Format }
497 \addcontentsline{toc}{subsection}{Overall Storage Format}
501 Current Bacula Tape Format
503 Version BB02 added 28 September 2002
504 Version BB01 is the old deprecated format.
505 A Bacula tape is composed of tape Blocks. Each block
506 has a Block header followed by the block data. Block
507 Data consists of Records. Records consist of Record
508 Headers followed by Record Data.
509 :=======================================================:
511 | Block Header (24 bytes) |
513 |-------------------------------------------------------|
515 | Record Header (12 bytes) |
517 |-------------------------------------------------------|
521 |-------------------------------------------------------|
523 | Record Header (12 bytes) |
525 |-------------------------------------------------------|
528 Block Header: the first item in each block. The format is
530 Partial Data block: occurs if the data from a previous
531 block spills over to this block (the normal case except
532 for the first block on a tape). However, this partial
533 data block is always preceded by a record header.
534 Record Header: identifies the Volume Session, the Stream
535 and the following Record Data size. See below for format.
536 Record data: arbitrary binary data.
537 Block Header Format BB02
538 :=======================================================:
539 | CheckSum (uint32_t) |
540 |-------------------------------------------------------|
541 | BlockSize (uint32_t) |
542 |-------------------------------------------------------|
543 | BlockNumber (uint32_t) |
544 |-------------------------------------------------------|
545 | "BB02" (char [4]) |
546 |-------------------------------------------------------|
547 | VolSessionId (uint32_t) |
548 |-------------------------------------------------------|
549 | VolSessionTime (uint32_t) |
550 :=======================================================:
551 BBO2: Serves to identify the block as a
552 Bacula block and also servers as a block format identifier
553 should we ever need to change the format.
554 BlockSize: is the size in bytes of the block. When reading
555 back a block, if the BlockSize does not agree with the
556 actual size read, Bacula discards the block.
557 CheckSum: a checksum for the Block.
558 BlockNumber: is the sequential block number on the tape.
559 VolSessionId: a unique sequential number that is assigned
560 by the Storage Daemon to a particular Job.
561 This number is sequential since the start
562 of execution of the daemon.
563 VolSessionTime: the time/date that the current execution
564 of the Storage Daemon started. It assures
565 that the combination of VolSessionId and
566 VolSessionTime is unique for all jobs
567 written to the tape, even if there was a
568 machine crash between two writes.
569 Record Header Format BB02
570 :=======================================================:
571 | FileIndex (int32_t) |
572 |-------------------------------------------------------|
574 |-------------------------------------------------------|
575 | DataSize (uint32_t) |
576 :=======================================================:
577 FileIndex: a sequential file number within a job. The
578 Storage daemon enforces this index to be
579 greater than zero and sequential. Note,
580 however, that the File daemon may send
581 multiple Streams for the same FileIndex.
582 The Storage Daemon uses negative FileIndices
583 to identify Session Start and End labels
584 as well as the End of Volume labels.
585 Stream: defined by the File daemon and is intended to be
586 used to identify separate parts of the data
587 saved for each file (attributes, file data,
588 ...). The Storage Daemon has no idea of
589 what a Stream is or what it contains.
590 DataSize: the size in bytes of the binary data record
591 that follows the Session Record header.
592 The Storage Daemon has no idea of the
593 actual contents of the binary data record.
594 For standard Unix files, the data record
595 typically contains the file attributes or
596 the file data. For a sparse file
597 the first 64 bits of the data contains
598 the storage address for the data block.
600 :=======================================================:
602 |-------------------------------------------------------|
603 | VerNum (uint32_t) |
604 |-------------------------------------------------------|
605 | label_date (float64_t) |
606 | label_btime (btime_t VerNum 11 |
607 |-------------------------------------------------------|
608 | label_time (float64_t) |
609 | write_btime (btime_t VerNum 11 |
610 |-------------------------------------------------------|
611 | write_date (float64_t) |
612 | 0 (float64_t) VerNum 11 |
613 |-------------------------------------------------------|
614 | write_time (float64_t) |
615 | 0 (float64_t) VerNum 11 |
616 |-------------------------------------------------------|
617 | VolName (128 bytes) |
618 |-------------------------------------------------------|
619 | PrevVolName (128 bytes) |
620 |-------------------------------------------------------|
621 | PoolName (128 bytes) |
622 |-------------------------------------------------------|
623 | PoolType (128 bytes) |
624 |-------------------------------------------------------|
625 | MediaType (128 bytes) |
626 |-------------------------------------------------------|
627 | HostName (128 bytes) |
628 |-------------------------------------------------------|
629 | LabelProg (32 bytes) |
630 |-------------------------------------------------------|
631 | ProgVersion (32 bytes) |
632 |-------------------------------------------------------|
633 | ProgDate (32 bytes) |
634 |-------------------------------------------------------|
635 :=======================================================:
637 Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
638 (old version also recognized:)
639 Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
640 LabelType (Saved in the FileIndex of the Header record).
641 PRE_LABEL -1 Volume label on unwritten tape
642 VOL_LABEL -2 Volume label after tape written
643 EOM_LABEL -3 Label at EOM (not currently implemented)
644 SOS_LABEL -4 Start of Session label (format given below)
645 EOS_LABEL -5 End of Session label (format given below)
647 label_date: Julian day tape labeled
648 label_time: Julian time tape labeled
649 write_date: Julian date tape first used (data written)
650 write_time: Julian time tape first used (data written)
651 VolName: "Physical" Volume name
652 PrevVolName: The VolName of the previous tape (if this tape is
653 a continuation of the previous one).
656 MediaType: Media Type
657 HostName: Name of host that is first writing the tape
658 LabelProg: Name of the program that labeled the tape
659 ProgVersion: Version of the label program
660 ProgDate: Date Label program built
662 :=======================================================:
664 |-------------------------------------------------------|
665 | VerNum (uint32_t) |
666 |-------------------------------------------------------|
668 |-------------------------------------------------------|
669 | write_btime (btime_t) VerNum 11 |
670 |-------------------------------------------------------|
671 | 0 (float64_t) VerNum 11 |
672 |-------------------------------------------------------|
673 | PoolName (128 bytes) |
674 |-------------------------------------------------------|
675 | PoolType (128 bytes) |
676 |-------------------------------------------------------|
677 | JobName (128 bytes) |
678 |-------------------------------------------------------|
679 | ClientName (128 bytes) |
680 |-------------------------------------------------------|
682 |-------------------------------------------------------|
683 | FileSetName (128 bytes) |
684 |-------------------------------------------------------|
685 | JobType (uint32_t) |
686 |-------------------------------------------------------|
687 | JobLevel (uint32_t) |
688 |-------------------------------------------------------|
689 | FileSetMD5 (50 bytes) VerNum 11 |
690 |-------------------------------------------------------|
691 Additional fields in End Of Session Label
692 |-------------------------------------------------------|
693 | JobFiles (uint32_t) |
694 |-------------------------------------------------------|
695 | JobBytes (uint32_t) |
696 |-------------------------------------------------------|
697 | start_block (uint32_t) |
698 |-------------------------------------------------------|
699 | end_block (uint32_t) |
700 |-------------------------------------------------------|
701 | start_file (uint32_t) |
702 |-------------------------------------------------------|
703 | end_file (uint32_t) |
704 |-------------------------------------------------------|
705 | JobErrors (uint32_t) |
706 |-------------------------------------------------------|
707 | JobStatus (uint32_t) VerNum 11 |
708 :=======================================================:
709 * => fields deprecated
710 Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
711 LabelType (in FileIndex field of Header):
712 EOM_LABEL -3 Label at EOM
713 SOS_LABEL -4 Start of Session label
714 EOS_LABEL -5 End of Session label
717 write_btime: Bacula time/date this tape record written
718 write_date: Julian date tape this record written - deprecated
719 write_time: Julian time tape this record written - deprecated.
722 MediaType: Media Type
723 ClientName: Name of File daemon or Client writing this session
724 Not used for EOM_LABEL.
728 \subsection*{Unix File Attributes}
729 \index{Unix File Attributes }
730 \index{Attributes!Unix File }
731 \addcontentsline{toc}{subsection}{Unix File Attributes}
733 The Unix File Attributes packet consists of the following:
735 \lt{}File-Index\gt{} \lt{}Type\gt{}
736 \lt{}Filename\gt{}@\lt{}File-Attributes\gt{}@\lt{}Link\gt{}
737 @\lt{}Extended-Attributes@\gt{} where
742 represents a byte containing a binary zero.
746 is the sequential file index starting from one assigned by the File daemon.
750 is one of the following:
754 #define FT_LNKSAVED 1 /* hard link to file already saved */
755 #define FT_REGE 2 /* Regular file but empty */
756 #define FT_REG 3 /* Regular file */
757 #define FT_LNK 4 /* Soft Link */
758 #define FT_DIR 5 /* Directory */
759 #define FT_SPEC 6 /* Special file -- chr, blk, fifo, sock */
760 #define FT_NOACCESS 7 /* Not able to access */
761 #define FT_NOFOLLOW 8 /* Could not follow link */
762 #define FT_NOSTAT 9 /* Could not stat file */
763 #define FT_NOCHG 10 /* Incremental option, file not changed */
764 #define FT_DIRNOCHG 11 /* Incremental option, directory not changed */
765 #define FT_ISARCH 12 /* Trying to save archive file */
766 #define FT_NORECURSE 13 /* No recursion into directory */
767 #define FT_NOFSCHG 14 /* Different file system, prohibited */
768 #define FT_NOOPEN 15 /* Could not open directory */
769 #define FT_RAW 16 /* Raw block device */
770 #define FT_FIFO 17 /* Raw fifo device */
776 is the fully qualified filename.
778 \item [File-Attributes]
779 \index{File-Attributes }
780 consists of the 13 fields of the stat() buffer in ASCII base64 format
781 separated by spaces. These fields and their meanings are shown below. This
782 stat() packet is in Unix format, and MUST be provided (constructed) for ALL
787 when the FT code is FT\_LNK or FT\_LNKSAVED, the item in question is a Unix
788 link, and this field contains the fully qualified link name. When the FT code
789 is not FT\_LNK or FT\_LNKSAVED, this field is null.
791 \item [Extended-Attributes]
792 \index{Extended-Attributes }
793 The exact format of this field is operating system dependent. It contains
794 additional or extended attributes of a system dependent nature. Currently,
795 this field is used only on WIN32 systems where it contains a ASCII base64
796 representation of the WIN32\_FILE\_ATTRIBUTE\_DATA structure as defined by
797 Windows. The fields in the base64 representation of this structure are like
798 the File-Attributes separated by spaces.
801 The File-attributes consist of the following:
803 \addcontentsline{lot}{table}{File Attributes}
804 \begin{longtable}{|l|l|l|l|l|}
806 \multicolumn{1}{|c| }{\bf Field No. } & \multicolumn{1}{c| }{\bf Stat Name }
807 & \multicolumn{1}{c| }{\bf Unix } & \multicolumn{1}{c| }{\bf Win98/NT } &
808 \multicolumn{1}{c| }{\bf MacOS } \\
810 \multicolumn{1}{|c| }{1 } & {st\_dev } & {Device number of filesystem } &
811 {Drive number } & {vRefNum } \\
813 \multicolumn{1}{|c| }{2 } & {st\_ino } & {Inode number } & {Always 0 } &
816 \multicolumn{1}{|c| }{3 } & {st\_mode } & {File mode } & {File mode } &
817 {777 dirs/apps; 666 docs; 444 locked docs } \\
819 \multicolumn{1}{|c| }{4 } & {st\_nlink } & {Number of links to the file } &
820 {Number of link (only on NTFS) } & {Always 1 } \\
822 \multicolumn{1}{|c| }{5 } & {st\_uid } & {Owner ID } & {Always 0 } &
825 \multicolumn{1}{|c| }{6 } & {st\_gid } & {Group ID } & {Always 0 } &
828 \multicolumn{1}{|c| }{7 } & {st\_rdev } & {Device ID for special files } &
829 {Drive No. } & {Always 0 } \\
831 \multicolumn{1}{|c| }{8 } & {st\_size } & {File size in bytes } & {File
832 size in bytes } & {Data fork file size in bytes } \\
834 \multicolumn{1}{|c| }{9 } & {st\_blksize } & {Preferred block size } &
835 {Always 0 } & {Preferred block size } \\
837 \multicolumn{1}{|c| }{10 } & {st\_blocks } & {Number of blocks allocated }
838 & {Always 0 } & {Number of blocks allocated } \\
840 \multicolumn{1}{|c| }{11 } & {st\_atime } & {Last access time since epoch }
841 & {Last access time since epoch } & {Last access time -66 years } \\
843 \multicolumn{1}{|c| }{12 } & {st\_mtime } & {Last modify time since epoch }
844 & {Last modify time since epoch } & {Last access time -66 years } \\
846 \multicolumn{1}{|c| }{13 } & {st\_ctime } & {Inode change time since epoch
847 } & {File create time since epoch } & {File create time -66 years }
852 \subsection*{Old Depreciated Tape Format}
853 \index{Old Depreciated Tape Format }
854 \index{Format!Old Depreciated Tape }
855 \addcontentsline{toc}{subsection}{Old Depreciated Tape Format}
857 The format of the Block Header (version 1.26 and earlier) is:
861 uint32_t CheckSum; /* Block check sum */
862 uint32_t BlockSize; /* Block byte size including the header */
863 uint32_t BlockNumber; /* Block number */
864 char ID[4] = "BB01"; /* Identification and block level */
868 The format of the Record Header (version 1.26 or earlier) is:
872 uint32_t VolSessionId; /* Unique ID for this session */
873 uint32_t VolSessionTime; /* Start time/date of session */
874 int32_t FileIndex; /* File index supplied by File daemon */
875 int32_t Stream; /* Stream number supplied by File daemon */
876 uint32_t DataSize; /* size of following data record in bytes */
882 Current Bacula Tape Format
884 Version BB01 is the old deprecated format.
885 A Bacula tape is composed of tape Blocks. Each block
886 has a Block header followed by the block data. Block
887 Data consists of Records. Records consist of Record
888 Headers followed by Record Data.
889 :=======================================================:
892 | (16 bytes version BB01) |
893 |-------------------------------------------------------|
896 | (20 bytes version BB01) |
897 |-------------------------------------------------------|
901 |-------------------------------------------------------|
904 | (20 bytes version BB01) |
905 |-------------------------------------------------------|
908 Block Header: the first item in each block. The format is
910 Partial Data block: occurs if the data from a previous
911 block spills over to this block (the normal case except
912 for the first block on a tape). However, this partial
913 data block is always preceded by a record header.
914 Record Header: identifies the Volume Session, the Stream
915 and the following Record Data size. See below for format.
916 Record data: arbitrary binary data.
917 Block Header Format BB01 (deprecated)
918 :=======================================================:
919 | CheckSum (uint32_t) |
920 |-------------------------------------------------------|
921 | BlockSize (uint32_t) |
922 |-------------------------------------------------------|
923 | BlockNumber (uint32_t) |
924 |-------------------------------------------------------|
925 | "BB01" (char [4]) |
926 :=======================================================:
927 BBO1: Serves to identify the block as a
928 Bacula block and also servers as a block format identifier
929 should we ever need to change the format.
930 BlockSize: is the size in bytes of the block. When reading
931 back a block, if the BlockSize does not agree with the
932 actual size read, Bacula discards the block.
933 CheckSum: a checksum for the Block.
934 BlockNumber: is the sequential block number on the tape.
935 VolSessionId: a unique sequential number that is assigned
936 by the Storage Daemon to a particular Job.
937 This number is sequential since the start
938 of execution of the daemon.
939 VolSessionTime: the time/date that the current execution
940 of the Storage Daemon started. It assures
941 that the combination of VolSessionId and
942 VolSessionTime is unique for all jobs
943 written to the tape, even if there was a
944 machine crash between two writes.
945 Record Header Format BB01 (deprecated)
946 :=======================================================:
947 | VolSessionId (uint32_t) |
948 |-------------------------------------------------------|
949 | VolSessionTime (uint32_t) |
950 |-------------------------------------------------------|
951 | FileIndex (int32_t) |
952 |-------------------------------------------------------|
954 |-------------------------------------------------------|
955 | DataSize (uint32_t) |
956 :=======================================================:
957 VolSessionId: a unique sequential number that is assigned
958 by the Storage Daemon to a particular Job.
959 This number is sequential since the start
960 of execution of the daemon.
961 VolSessionTime: the time/date that the current execution
962 of the Storage Daemon started. It assures
963 that the combination of VolSessionId and
964 VolSessionTime is unique for all jobs
965 written to the tape, even if there was a
966 machine crash between two writes.
967 FileIndex: a sequential file number within a job. The
968 Storage daemon enforces this index to be
969 greater than zero and sequential. Note,
970 however, that the File daemon may send
971 multiple Streams for the same FileIndex.
972 The Storage Daemon uses negative FileIndices
973 to identify Session Start and End labels
974 as well as the End of Volume labels.
975 Stream: defined by the File daemon and is intended to be
976 used to identify separate parts of the data
977 saved for each file (attributes, file data,
978 ...). The Storage Daemon has no idea of
979 what a Stream is or what it contains.
980 DataSize: the size in bytes of the binary data record
981 that follows the Session Record header.
982 The Storage Daemon has no idea of the
983 actual contents of the binary data record.
984 For standard Unix files, the data record
985 typically contains the file attributes or
986 the file data. For a sparse file
987 the first 64 bits of the data contains
988 the storage address for the data block.
990 :=======================================================:
992 |-------------------------------------------------------|
993 | VerNum (uint32_t) |
994 |-------------------------------------------------------|
995 | label_date (float64_t) |
996 |-------------------------------------------------------|
997 | label_time (float64_t) |
998 |-------------------------------------------------------|
999 | write_date (float64_t) |
1000 |-------------------------------------------------------|
1001 | write_time (float64_t) |
1002 |-------------------------------------------------------|
1003 | VolName (128 bytes) |
1004 |-------------------------------------------------------|
1005 | PrevVolName (128 bytes) |
1006 |-------------------------------------------------------|
1007 | PoolName (128 bytes) |
1008 |-------------------------------------------------------|
1009 | PoolType (128 bytes) |
1010 |-------------------------------------------------------|
1011 | MediaType (128 bytes) |
1012 |-------------------------------------------------------|
1013 | HostName (128 bytes) |
1014 |-------------------------------------------------------|
1015 | LabelProg (32 bytes) |
1016 |-------------------------------------------------------|
1017 | ProgVersion (32 bytes) |
1018 |-------------------------------------------------------|
1019 | ProgDate (32 bytes) |
1020 |-------------------------------------------------------|
1021 :=======================================================:
1023 Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
1024 (old version also recognized:)
1025 Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
1026 LabelType (Saved in the FileIndex of the Header record).
1027 PRE_LABEL -1 Volume label on unwritten tape
1028 VOL_LABEL -2 Volume label after tape written
1029 EOM_LABEL -3 Label at EOM (not currently implemented)
1030 SOS_LABEL -4 Start of Session label (format given below)
1031 EOS_LABEL -5 End of Session label (format given below)
1032 label_date: Julian day tape labeled
1033 label_time: Julian time tape labeled
1034 write_date: Julian date tape first used (data written)
1035 write_time: Julian time tape first used (data written)
1036 VolName: "Physical" Volume name
1037 PrevVolName: The VolName of the previous tape (if this tape is
1038 a continuation of the previous one).
1041 MediaType: Media Type
1042 HostName: Name of host that is first writing the tape
1043 LabelProg: Name of the program that labeled the tape
1044 ProgVersion: Version of the label program
1045 ProgDate: Date Label program built
1047 :=======================================================:
1049 |-------------------------------------------------------|
1050 | VerNum (uint32_t) |
1051 |-------------------------------------------------------|
1052 | JobId (uint32_t) |
1053 |-------------------------------------------------------|
1054 | *write_date (float64_t) VerNum 10 |
1055 |-------------------------------------------------------|
1056 | *write_time (float64_t) VerNum 10 |
1057 |-------------------------------------------------------|
1058 | PoolName (128 bytes) |
1059 |-------------------------------------------------------|
1060 | PoolType (128 bytes) |
1061 |-------------------------------------------------------|
1062 | JobName (128 bytes) |
1063 |-------------------------------------------------------|
1064 | ClientName (128 bytes) |
1065 |-------------------------------------------------------|
1067 |-------------------------------------------------------|
1068 | FileSetName (128 bytes) |
1069 |-------------------------------------------------------|
1070 | JobType (uint32_t) |
1071 |-------------------------------------------------------|
1072 | JobLevel (uint32_t) |
1073 |-------------------------------------------------------|
1074 | FileSetMD5 (50 bytes) VerNum 11 |
1075 |-------------------------------------------------------|
1076 Additional fields in End Of Session Label
1077 |-------------------------------------------------------|
1078 | JobFiles (uint32_t) |
1079 |-------------------------------------------------------|
1080 | JobBytes (uint32_t) |
1081 |-------------------------------------------------------|
1082 | start_block (uint32_t) |
1083 |-------------------------------------------------------|
1084 | end_block (uint32_t) |
1085 |-------------------------------------------------------|
1086 | start_file (uint32_t) |
1087 |-------------------------------------------------------|
1088 | end_file (uint32_t) |
1089 |-------------------------------------------------------|
1090 | JobErrors (uint32_t) |
1091 |-------------------------------------------------------|
1092 | JobStatus (uint32_t) VerNum 11 |
1093 :=======================================================:
1094 * => fields deprecated
1095 Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
1096 LabelType (in FileIndex field of Header):
1097 EOM_LABEL -3 Label at EOM
1098 SOS_LABEL -4 Start of Session label
1099 EOS_LABEL -5 End of Session label
1102 write_btime: Bacula time/date this tape record written
1103 write_date: Julian date tape this record written - deprecated
1104 write_time: Julian time tape this record written - deprecated.
1107 MediaType: Media Type
1108 ClientName: Name of File daemon or Client writing this session
1109 Not used for EOM_LABEL.