--- /dev/null
+%%
+%%
+
+\chapter{Storage Media Output Format}
+\label{_ChapterStart9}
+\index{Format!Storage Media Output}
+\index{Storage Media Output Format}
+\addcontentsline{toc}{section}{Storage Media Output Format}
+
+\section{General}
+\index{General}
+\addcontentsline{toc}{subsection}{General}
+
+This document describes the media format written by the Storage daemon. The
+Storage daemon reads and writes in units of blocks. Blocks contain records.
+Each block has a block header followed by records, and each record has a
+record header followed by record data.
+
+This chapter is intended to be a technical discussion of the Media Format and
+as such is not targeted at end users but rather at developers and system
+administrators that want or need to know more of the working details of {\bf
+Bacula}.
+
+\section{Definitions}
+\index{Definitions}
+\addcontentsline{toc}{subsection}{Definitions}
+
+\begin{description}
+
+\item [Block]
+ \index{Block}
+ A block represents the primitive unit of information that the Storage daemon
+reads and writes to a physical device. Normally, for a tape device, it will
+be the same as a tape block. The Storage daemon always reads and writes
+blocks. A block consists of block header information followed by records.
+Clients of the Storage daemon (the File daemon) normally never see blocks.
+However, some of the Storage tools (bls, bscan, bextract, ...) may be use
+block header information. In older Bacula tape versions, a block could
+contain records (see record definition below) from multiple jobs. However,
+all blocks currently written by Bacula are block level BB02, and a given
+block contains records for only a single job. Different jobs simply have
+their own private blocks that are intermingled with the other blocks from
+other jobs on the Volume (previously the records were intermingled within
+the blocks). Having only records from a single job in any give block
+permitted moving the VolumeSessionId and VolumeSessionTime (see below) from
+each record heading to the Block header. This has two advantages: 1. a block
+can be quickly rejected based on the contents of the header without reading
+all the records. 2. because there is on the average more than one record per
+block, less data is written to the Volume for each job.
+
+\item [Record]
+ \index{Record}
+ A record consists of a Record Header, which is managed by the Storage daemon
+and Record Data, which is the data received from the Client. A record is the
+primitive unit of information sent to and from the Storage daemon by the
+Client (File daemon) programs. The details are described below.
+
+\item [JobId]
+ \index{JobId}
+ A number assigned by the Director daemon for a particular job. This number
+will be unique for that particular Director (Catalog). The daemons use this
+number to keep track of individual jobs. Within the Storage daemon, the JobId
+may not be unique if several Directors are accessing the Storage daemon
+simultaneously.
+
+\item [Session]
+ \index{Session}
+ A Session is a concept used in the Storage daemon corresponds one to one to a
+Job with the exception that each session is uniquely identified within the
+Storage daemon by a unique SessionId/SessionTime pair (see below).
+
+\item [VolSessionId]
+ \index{VolSessionId}
+ A unique number assigned by the Storage daemon to a particular session (Job)
+it is having with a File daemon. This number by itself is not unique to the
+given Volume, but with the VolSessionTime, it is unique.
+
+\item [VolSessionTime]
+ \index{VolSessionTime}
+ A unique number assigned by the Storage daemon to a particular Storage daemon
+execution. It is actually the Unix time\_t value of when the Storage daemon
+began execution cast to a 32 bit unsigned integer. The combination of the
+{\bf VolSessionId} and the {\bf VolSessionTime} for a given Storage daemon is
+guaranteed to be unique for each Job (or session).
+
+\item [FileIndex]
+ \index{FileIndex}
+ A sequential number beginning at one assigned by the File daemon to the files
+within a job that are sent to the Storage daemon for backup. The Storage
+daemon ensures that this number is greater than zero and sequential. Note,
+the Storage daemon uses negative FileIndexes to flag Session Start and End
+Labels as well as End of Volume Labels. Thus, the combination of
+VolSessionId, VolSessionTime, and FileIndex uniquely identifies the records
+for a single file written to a Volume.
+
+\item [Stream]
+ \index{Stream}
+ While writing the information for any particular file to the Volume, there
+can be any number of distinct pieces of information about that file, e.g. the
+attributes, the file data, ... The Stream indicates what piece of data it
+is, and it is an arbitrary number assigned by the File daemon to the parts
+(Unix attributes, Win32 attributes, data, compressed data,\ ...) of a file
+that are sent to the Storage daemon. The Storage daemon has no knowledge of
+the details of a Stream; it simply represents a numbered stream of bytes. The
+data for a given stream may be passed to the Storage daemon in single record,
+or in multiple records.
+
+\item [Block Header]
+ \index{Block Header}
+ A block header consists of a block identification (``BB02''), a block length
+in bytes (typically 64,512) a checksum, and sequential block number. Each
+block starts with a Block Header and is followed by Records. Current block
+headers also contain the VolSessionId and VolSessionTime for the records
+written to that block.
+
+\item [Record Header]
+ \index{Record Header}
+ A record header contains the Volume Session Id, the Volume Session Time, the
+FileIndex, the Stream, and the size of the data record which follows. The
+Record Header is always immediately followed by a Data Record if the size
+given in the Header is greater than zero. Note, for Block headers of level
+BB02 (version 1.27 and later), the Record header as written to tape does not
+contain the Volume Session Id and the Volume Session Time as these two
+fields are stored in the BB02 Block header. The in-memory record header does
+have those fields for convenience.
+
+\item [Data Record]
+ \index{Data Record}
+ A data record consists of a binary stream of bytes and is always preceded by
+a Record Header. The details of the meaning of the binary stream of bytes are
+unknown to the Storage daemon, but the Client programs (File daemon) defines
+and thus knows the details of each record type.
+
+\item [Volume Label]
+ \index{Volume Label}
+ A label placed by the Storage daemon at the beginning of each storage volume.
+It contains general information about the volume. It is written in Record
+format. The Storage daemon manages Volume Labels, and if the client wants, he
+may also read them.
+
+\item [Begin Session Label]
+ \index{Begin Session Label}
+ The Begin Session Label is a special record placed by the Storage daemon on
+the storage medium as the first record of an append session job with a File
+daemon. This record is useful for finding the beginning of a particular
+session (Job), since no records with the same VolSessionId and VolSessionTime
+will precede this record. This record is not normally visible outside of the
+Storage daemon. The Begin Session Label is similar to the Volume Label except
+that it contains additional information pertaining to the Session.
+
+\item [End Session Label]
+ \index{End Session Label}
+ The End Session Label is a special record placed by the Storage daemon on the
+storage medium as the last record of an append session job with a File
+daemon. The End Session Record is distinguished by a FileIndex with a value
+of minus two (-2). This record is useful for detecting the end of a
+particular session since no records with the same VolSessionId and
+VolSessionTime will follow this record. This record is not normally visible
+outside of the Storage daemon. The End Session Label is similar to the Volume
+Label except that it contains additional information pertaining to the
+Session.
+\end{description}
+
+\section{Storage Daemon File Output Format}
+\index{Format!Storage Daemon File Output}
+\index{Storage Daemon File Output Format}
+\addcontentsline{toc}{subsection}{Storage Daemon File Output Format}
+
+The file storage and tape storage formats are identical except that tape
+records are by default blocked into blocks of 64,512 bytes, except for the
+last block, which is the actual number of bytes written rounded up to a
+multiple of 1024 whereas the last record of file storage is not rounded up.
+The default block size of 64,512 bytes may be overridden by the user (some
+older tape drives only support block sizes of 32K). Each Session written to
+tape is terminated with an End of File mark (this will be removed later).
+Sessions written to file are simply appended to the end of the file.
+
+\section{Overall Format}
+\index{Format!Overall}
+\index{Overall Format}
+\addcontentsline{toc}{subsection}{Overall Format}
+
+A Bacula output file consists of Blocks of data. Each block contains a block
+header followed by records. Each record consists of a record header followed
+by the record data. The first record on a tape will always be the Volume Label
+Record.
+
+No Record Header will be split across Bacula blocks. However, Record Data may
+be split across any number of Bacula blocks. Obviously this will not be the
+case for the Volume Label which will always be smaller than the Bacula Block
+size.
+
+To simplify reading tapes, the Start of Session (SOS) and End of Session (EOS)
+records are never split across blocks. If this is about to happen, Bacula will
+write a short block before writing the session record (actually, the SOS
+record should always be the first record in a block, excepting perhaps the
+Volume label).
+
+Due to hardware limitations, the last block written to the tape may not be
+fully written. If your drive permits backspace record, Bacula will backup over
+the last record written on the tape, re-read it and verify that it was
+correctly written.
+
+When a new tape is mounted Bacula will write the full contents of the
+partially written block to the new tape ensuring that there is no loss of
+data. When reading a tape, Bacula will discard any block that is not totally
+written, thus ensuring that there is no duplication of data. In addition,
+since Bacula blocks are sequentially numbered within a Job, it is easy to
+ensure that no block is missing or duplicated.
+
+\section{Serialization}
+\index{Serialization}
+\addcontentsline{toc}{subsection}{Serialization}
+
+All Block Headers, Record Headers, and Label Records are written using
+Bacula's serialization routines. These routines guarantee that the data is
+written to the output volume in a machine independent format.
+
+\section{Block Header}
+\index{Header!Block}
+\index{Block Header}
+\addcontentsline{toc}{subsection}{Block Header}
+
+The format of the Block Header (version 1.27 and later) is:
+
+\footnotesize
+\begin{verbatim}
+ uint32_t CheckSum; /* Block check sum */
+ uint32_t BlockSize; /* Block byte size including the header */
+ uint32_t BlockNumber; /* Block number */
+ char ID[4] = "BB02"; /* Identification and block level */
+ uint32_t VolSessionId; /* Session Id for Job */
+ uint32_t VolSessionTime; /* Session Time for Job */
+\end{verbatim}
+\normalsize
+
+The Block header is a fixed length and fixed format and is followed by Record
+Headers and Record Data. The CheckSum field is a 32 bit checksum of the block
+data and the block header but not including the CheckSum field. The Block
+Header is always immediately followed by a Record Header. If the tape is
+damaged, a Bacula utility will be able to recover as much information as
+possible from the tape by recovering blocks which are valid. The Block header
+is written using the Bacula serialization routines and thus is guaranteed to
+be in machine independent format. See below for version 2 of the block header.
+
+
+\section{Record Header}
+\index{Header!Record}
+\index{Record Header}
+\addcontentsline{toc}{subsection}{Record Header}
+
+Each binary data record is preceded by a Record Header. The Record Header is
+fixed length and fixed format, whereas the binary data record is of variable
+length. The Record Header is written using the Bacula serialization routines
+and thus is guaranteed to be in machine independent format.
+
+The format of the Record Header (version 1.27 or later) is:
+
+\footnotesize
+\begin{verbatim}
+ int32_t FileIndex; /* File index supplied by File daemon */
+ int32_t Stream; /* Stream number supplied by File daemon */
+ uint32_t DataSize; /* size of following data record in bytes */
+\end{verbatim}
+\normalsize
+
+This record is followed by the binary Stream data of DataSize bytes, followed
+by another Record Header record and the binary stream data. For the definitive
+definition of this record, see record.h in the src/stored directory.
+
+Additional notes on the above:
+
+\begin{description}
+
+\item [The {\bf VolSessionId} ]
+ \index{VolSessionId}
+ is a unique sequential number that is assigned by the Storage Daemon to a
+particular Job. This number is sequential since the start of execution of the
+daemon.
+
+\item [The {\bf VolSessionTime} ]
+ \index{VolSessionTime}
+ is the time/date that the current execution of the Storage Daemon started. It
+assures that the combination of VolSessionId and VolSessionTime is unique for
+every jobs written to the tape, even if there was a machine crash between two
+writes.
+
+\item [The {\bf FileIndex} ]
+ \index{FileIndex}
+ is a sequential file number within a job. The Storage daemon requires this
+index to be greater than zero and sequential. Note, however, that the File
+daemon may send multiple Streams for the same FileIndex. In addition, the
+Storage daemon uses negative FileIndices to hold the Begin Session Label, the
+End Session Label, and the End of Volume Label.
+
+\item [The {\bf Stream} ]
+ \index{Stream}
+ is defined by the File daemon and is used to identify separate parts of the
+data saved for each file (Unix attributes, Win32 attributes, file data,
+compressed file data, sparse file data, ...). The Storage Daemon has no idea
+of what a Stream is or what it contains except that the Stream is required to
+be a positive integer. Negative Stream numbers are used internally by the
+Storage daemon to indicate that the record is a continuation of the previous
+record (the previous record would not entirely fit in the block).
+
+For Start Session and End Session Labels (where the FileIndex is negative),
+the Storage daemon uses the Stream field to contain the JobId. The current
+stream definitions are:
+
+\footnotesize
+\begin{verbatim}
+#define STREAM_UNIX_ATTRIBUTES 1 /* Generic Unix attributes */
+#define STREAM_FILE_DATA 2 /* Standard uncompressed data */
+#define STREAM_MD5_SIGNATURE 3 /* MD5 signature for the file */
+#define STREAM_GZIP_DATA 4 /* GZip compressed file data */
+/* Extended Unix attributes with Win32 Extended data. Deprecated. */
+#define STREAM_UNIX_ATTRIBUTES_EX 5 /* Extended Unix attr for Win32 EX */
+#define STREAM_SPARSE_DATA 6 /* Sparse data stream */
+#define STREAM_SPARSE_GZIP_DATA 7
+#define STREAM_PROGRAM_NAMES 8 /* program names for program data */
+#define STREAM_PROGRAM_DATA 9 /* Data needing program */
+#define STREAM_SHA1_SIGNATURE 10 /* SHA1 signature for the file */
+#define STREAM_WIN32_DATA 11 /* Win32 BackupRead data */
+#define STREAM_WIN32_GZIP_DATA 12 /* Gzipped Win32 BackupRead data */
+#define STREAM_MACOS_FORK_DATA 13 /* Mac resource fork */
+#define STREAM_HFSPLUS_ATTRIBUTES 14 /* Mac OS extra attributes */
+#define STREAM_UNIX_ATTRIBUTES_ACCESS_ACL 15 /* Standard ACL attributes on UNIX */
+#define STREAM_UNIX_ATTRIBUTES_DEFAULT_ACL 16 /* Default ACL attributes on UNIX */
+\end{verbatim}
+\normalsize
+
+\item [The {\bf DataSize} ]
+ \index{DataSize}
+ is the size in bytes of the binary data record that follows the Session
+Record header. The Storage Daemon has no idea of the actual contents of the
+binary data record. For standard Unix files, the data record typically
+contains the file attributes or the file data. For a sparse file the first
+64 bits of the file data contains the storage address for the data block.
+\end{description}
+
+The Record Header is never split across two blocks. If there is not enough
+room in a block for the full Record Header, the block is padded to the end
+with zeros and the Record Header begins in the next block. The data record, on
+the other hand, may be split across multiple blocks and even multiple physical
+volumes. When a data record is split, the second (and possibly subsequent)
+piece of the data is preceded by a new Record Header. Thus each piece of data
+is always immediately preceded by a Record Header. When reading a record, if
+Bacula finds only part of the data in the first record, it will automatically
+read the next record and concatenate the data record to form a full data
+record.
+
+\section{Version BB02 Block Header}
+\index{Version BB02 Block Header}
+\index{Header!Version BB02 Block}
+\addcontentsline{toc}{subsection}{Version BB02 Block Header}
+
+Each session or Job has its own private block. As a consequence, the SessionId
+and SessionTime are written once in each Block Header and not in the Record
+Header. So, the second and current version of the Block Header BB02 is:
+
+\footnotesize
+\begin{verbatim}
+ uint32_t CheckSum; /* Block check sum */
+ uint32_t BlockSize; /* Block byte size including the header */
+ uint32_t BlockNumber; /* Block number */
+ char ID[4] = "BB02"; /* Identification and block level */
+ uint32_t VolSessionId; /* Applies to all records */
+ uint32_t VolSessionTime; /* contained in this block */
+\end{verbatim}
+\normalsize
+
+As with the previous version, the BB02 Block header is a fixed length and
+fixed format and is followed by Record Headers and Record Data. The CheckSum
+field is a 32 bit CRC checksum of the block data and the block header but not
+including the CheckSum field. The Block Header is always immediately followed
+by a Record Header. If the tape is damaged, a Bacula utility will be able to
+recover as much information as possible from the tape by recovering blocks
+which are valid. The Block header is written using the Bacula serialization
+routines and thus is guaranteed to be in machine independent format.
+
+\section{Version 2 Record Header}
+\index{Version 2 Record Header}
+\index{Header!Version 2 Record}
+\addcontentsline{toc}{subsection}{Version 2 Record Header}
+
+Version 2 Record Header is written to the medium when using Version BB02 Block
+Headers. The memory representation of the record is identical to the old BB01
+Record Header, but on the storage medium, the first two fields, namely
+VolSessionId and VolSessionTime are not written. The Block Header is filled
+with these values when the First user record is written (i.e. non label
+record) so that when the block is written, it will have the current and unique
+VolSessionId and VolSessionTime. On reading each record from the Block, the
+VolSessionId and VolSessionTime is filled in the Record Header from the Block
+Header.
+
+\section{Volume Label Format}
+\index{Volume Label Format}
+\index{Format!Volume Label}
+\addcontentsline{toc}{subsection}{Volume Label Format}
+
+Tape volume labels are created by the Storage daemon in response to a {\bf
+label} command given to the Console program, or alternatively by the {\bf
+btape} program. created. Each volume is labeled with the following information
+using the Bacula serialization routines, which guarantee machine byte order
+independence.
+
+For Bacula versions 1.27 and later, the Volume Label Format is:
+
+\footnotesize
+\begin{verbatim}
+ char Id[32]; /* Bacula 1.0 Immortal\n */
+ uint32_t VerNum; /* Label version number */
+ /* VerNum 11 and greater Bacula 1.27 and later */
+ btime_t label_btime; /* Time/date tape labeled */
+ btime_t write_btime; /* Time/date tape first written */
+ /* The following are 0 in VerNum 11 and greater */
+ float64_t write_date; /* Date this label written */
+ float64_t write_time; /* Time this label written */
+ char VolName[128]; /* Volume name */
+ char PrevVolName[128]; /* Previous Volume Name */
+ char PoolName[128]; /* Pool name */
+ char PoolType[128]; /* Pool type */
+ char MediaType[128]; /* Type of this media */
+ char HostName[128]; /* Host name of writing computer */
+ char LabelProg[32]; /* Label program name */
+ char ProgVersion[32]; /* Program version */
+ char ProgDate[32]; /* Program build date/time */
+\end{verbatim}
+\normalsize
+
+Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, ...)
+is stored in the record FileIndex field of the Record Header and does not
+appear in the data part of the record.
+
+\section{Session Label}
+\index{Label!Session}
+\index{Session Label}
+\addcontentsline{toc}{subsection}{Session Label}
+
+The Session Label is written at the beginning and end of each session as well
+as the last record on the physical medium. It has the following binary format:
+
+
+\footnotesize
+\begin{verbatim}
+ char Id[32]; /* Bacula Immortal ... */
+ uint32_t VerNum; /* Label version number */
+ uint32_t JobId; /* Job id */
+ uint32_t VolumeIndex; /* sequence no of vol */
+ /* Prior to VerNum 11 */
+ float64_t write_date; /* Date this label written */
+ /* VerNum 11 and greater */
+ btime_t write_btime; /* time/date record written */
+ /* The following is zero VerNum 11 and greater */
+ float64_t write_time; /* Time this label written */
+ char PoolName[128]; /* Pool name */
+ char PoolType[128]; /* Pool type */
+ char JobName[128]; /* base Job name */
+ char ClientName[128];
+ /* Added in VerNum 10 */
+ char Job[128]; /* Unique Job name */
+ char FileSetName[128]; /* FileSet name */
+ uint32_t JobType;
+ uint32_t JobLevel;
+\end{verbatim}
+\normalsize
+
+In addition, the EOS label contains:
+
+\footnotesize
+\begin{verbatim}
+ /* The remainder are part of EOS label only */
+ uint32_t JobFiles;
+ uint64_t JobBytes;
+ uint32_t start_block;
+ uint32_t end_block;
+ uint32_t start_file;
+ uint32_t end_file;
+ uint32_t JobErrors;
+\end{verbatim}
+\normalsize
+
+In addition, for VerNum greater than 10, the EOS label contains (in addition
+to the above):
+
+\footnotesize
+\begin{verbatim}
+ uint32_t JobStatus /* Job termination code */
+\end{verbatim}
+\normalsize
+
+: Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label,
+...) is stored in the record FileIndex field and does not appear in the data
+part of the record. Also, the Stream field of the Record Header contains the
+JobId. This permits quick filtering without actually reading all the session
+data in many cases.
+
+\section{Overall Storage Format}
+\index{Format!Overall Storage}
+\index{Overall Storage Format}
+\addcontentsline{toc}{subsection}{Overall Storage Format}
+
+\footnotesize
+\begin{verbatim}
+ Current Bacula Tape Format
+ 6 June 2001
+ Version BB02 added 28 September 2002
+ Version BB01 is the old deprecated format.
+ A Bacula tape is composed of tape Blocks. Each block
+ has a Block header followed by the block data. Block
+ Data consists of Records. Records consist of Record
+ Headers followed by Record Data.
+ :=======================================================:
+ | |
+ | Block Header (24 bytes) |
+ | |
+ |-------------------------------------------------------|
+ | |
+ | Record Header (12 bytes) |
+ | |
+ |-------------------------------------------------------|
+ | |
+ | Record Data |
+ | |
+ |-------------------------------------------------------|
+ | |
+ | Record Header (12 bytes) |
+ | |
+ |-------------------------------------------------------|
+ | |
+ | ... |
+ Block Header: the first item in each block. The format is
+ shown below.
+ Partial Data block: occurs if the data from a previous
+ block spills over to this block (the normal case except
+ for the first block on a tape). However, this partial
+ data block is always preceded by a record header.
+ Record Header: identifies the Volume Session, the Stream
+ and the following Record Data size. See below for format.
+ Record data: arbitrary binary data.
+ Block Header Format BB02
+ :=======================================================:
+ | CheckSum (uint32_t) |
+ |-------------------------------------------------------|
+ | BlockSize (uint32_t) |
+ |-------------------------------------------------------|
+ | BlockNumber (uint32_t) |
+ |-------------------------------------------------------|
+ | "BB02" (char [4]) |
+ |-------------------------------------------------------|
+ | VolSessionId (uint32_t) |
+ |-------------------------------------------------------|
+ | VolSessionTime (uint32_t) |
+ :=======================================================:
+ BBO2: Serves to identify the block as a
+ Bacula block and also servers as a block format identifier
+ should we ever need to change the format.
+ BlockSize: is the size in bytes of the block. When reading
+ back a block, if the BlockSize does not agree with the
+ actual size read, Bacula discards the block.
+ CheckSum: a checksum for the Block.
+ BlockNumber: is the sequential block number on the tape.
+ VolSessionId: a unique sequential number that is assigned
+ by the Storage Daemon to a particular Job.
+ This number is sequential since the start
+ of execution of the daemon.
+ VolSessionTime: the time/date that the current execution
+ of the Storage Daemon started. It assures
+ that the combination of VolSessionId and
+ VolSessionTime is unique for all jobs
+ written to the tape, even if there was a
+ machine crash between two writes.
+ Record Header Format BB02
+ :=======================================================:
+ | FileIndex (int32_t) |
+ |-------------------------------------------------------|
+ | Stream (int32_t) |
+ |-------------------------------------------------------|
+ | DataSize (uint32_t) |
+ :=======================================================:
+ FileIndex: a sequential file number within a job. The
+ Storage daemon enforces this index to be
+ greater than zero and sequential. Note,
+ however, that the File daemon may send
+ multiple Streams for the same FileIndex.
+ The Storage Daemon uses negative FileIndices
+ to identify Session Start and End labels
+ as well as the End of Volume labels.
+ Stream: defined by the File daemon and is intended to be
+ used to identify separate parts of the data
+ saved for each file (attributes, file data,
+ ...). The Storage Daemon has no idea of
+ what a Stream is or what it contains.
+ DataSize: the size in bytes of the binary data record
+ that follows the Session Record header.
+ The Storage Daemon has no idea of the
+ actual contents of the binary data record.
+ For standard Unix files, the data record
+ typically contains the file attributes or
+ the file data. For a sparse file
+ the first 64 bits of the data contains
+ the storage address for the data block.
+ Volume Label
+ :=======================================================:
+ | Id (32 bytes) |
+ |-------------------------------------------------------|
+ | VerNum (uint32_t) |
+ |-------------------------------------------------------|
+ | label_date (float64_t) |
+ | label_btime (btime_t VerNum 11 |
+ |-------------------------------------------------------|
+ | label_time (float64_t) |
+ | write_btime (btime_t VerNum 11 |
+ |-------------------------------------------------------|
+ | write_date (float64_t) |
+ | 0 (float64_t) VerNum 11 |
+ |-------------------------------------------------------|
+ | write_time (float64_t) |
+ | 0 (float64_t) VerNum 11 |
+ |-------------------------------------------------------|
+ | VolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PrevVolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolType (128 bytes) |
+ |-------------------------------------------------------|
+ | MediaType (128 bytes) |
+ |-------------------------------------------------------|
+ | HostName (128 bytes) |
+ |-------------------------------------------------------|
+ | LabelProg (32 bytes) |
+ |-------------------------------------------------------|
+ | ProgVersion (32 bytes) |
+ |-------------------------------------------------------|
+ | ProgDate (32 bytes) |
+ |-------------------------------------------------------|
+ :=======================================================:
+
+ Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
+ (old version also recognized:)
+ Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
+ LabelType (Saved in the FileIndex of the Header record).
+ PRE_LABEL -1 Volume label on unwritten tape
+ VOL_LABEL -2 Volume label after tape written
+ EOM_LABEL -3 Label at EOM (not currently implemented)
+ SOS_LABEL -4 Start of Session label (format given below)
+ EOS_LABEL -5 End of Session label (format given below)
+ VerNum: 11
+ label_date: Julian day tape labeled
+ label_time: Julian time tape labeled
+ write_date: Julian date tape first used (data written)
+ write_time: Julian time tape first used (data written)
+ VolName: "Physical" Volume name
+ PrevVolName: The VolName of the previous tape (if this tape is
+ a continuation of the previous one).
+ PoolName: Pool Name
+ PoolType: Pool Type
+ MediaType: Media Type
+ HostName: Name of host that is first writing the tape
+ LabelProg: Name of the program that labeled the tape
+ ProgVersion: Version of the label program
+ ProgDate: Date Label program built
+ Session Label
+ :=======================================================:
+ | Id (32 bytes) |
+ |-------------------------------------------------------|
+ | VerNum (uint32_t) |
+ |-------------------------------------------------------|
+ | JobId (uint32_t) |
+ |-------------------------------------------------------|
+ | write_btime (btime_t) VerNum 11 |
+ |-------------------------------------------------------|
+ | 0 (float64_t) VerNum 11 |
+ |-------------------------------------------------------|
+ | PoolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolType (128 bytes) |
+ |-------------------------------------------------------|
+ | JobName (128 bytes) |
+ |-------------------------------------------------------|
+ | ClientName (128 bytes) |
+ |-------------------------------------------------------|
+ | Job (128 bytes) |
+ |-------------------------------------------------------|
+ | FileSetName (128 bytes) |
+ |-------------------------------------------------------|
+ | JobType (uint32_t) |
+ |-------------------------------------------------------|
+ | JobLevel (uint32_t) |
+ |-------------------------------------------------------|
+ | FileSetMD5 (50 bytes) VerNum 11 |
+ |-------------------------------------------------------|
+ Additional fields in End Of Session Label
+ |-------------------------------------------------------|
+ | JobFiles (uint32_t) |
+ |-------------------------------------------------------|
+ | JobBytes (uint32_t) |
+ |-------------------------------------------------------|
+ | start_block (uint32_t) |
+ |-------------------------------------------------------|
+ | end_block (uint32_t) |
+ |-------------------------------------------------------|
+ | start_file (uint32_t) |
+ |-------------------------------------------------------|
+ | end_file (uint32_t) |
+ |-------------------------------------------------------|
+ | JobErrors (uint32_t) |
+ |-------------------------------------------------------|
+ | JobStatus (uint32_t) VerNum 11 |
+ :=======================================================:
+ * => fields deprecated
+ Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
+ LabelType (in FileIndex field of Header):
+ EOM_LABEL -3 Label at EOM
+ SOS_LABEL -4 Start of Session label
+ EOS_LABEL -5 End of Session label
+ VerNum: 11
+ JobId: JobId
+ write_btime: Bacula time/date this tape record written
+ write_date: Julian date tape this record written - deprecated
+ write_time: Julian time tape this record written - deprecated.
+ PoolName: Pool Name
+ PoolType: Pool Type
+ MediaType: Media Type
+ ClientName: Name of File daemon or Client writing this session
+ Not used for EOM_LABEL.
+\end{verbatim}
+\normalsize
+
+\section{Unix File Attributes}
+\index{Unix File Attributes}
+\index{Attributes!Unix File}
+\addcontentsline{toc}{subsection}{Unix File Attributes}
+
+The Unix File Attributes packet consists of the following:
+
+\lt{}File-Index\gt{} \lt{}Type\gt{}
+\lt{}Filename\gt{}@\lt{}File-Attributes\gt{}@\lt{}Link\gt{}
+@\lt{}Extended-Attributes@\gt{} where
+
+\begin{description}
+
+\item [@]
+ represents a byte containing a binary zero.
+
+\item [FileIndex]
+ \index{FileIndex}
+ is the sequential file index starting from one assigned by the File daemon.
+
+\item [Type]
+ \index{Type}
+ is one of the following:
+
+\footnotesize
+\begin{verbatim}
+#define FT_LNKSAVED 1 /* hard link to file already saved */
+#define FT_REGE 2 /* Regular file but empty */
+#define FT_REG 3 /* Regular file */
+#define FT_LNK 4 /* Soft Link */
+#define FT_DIR 5 /* Directory */
+#define FT_SPEC 6 /* Special file -- chr, blk, fifo, sock */
+#define FT_NOACCESS 7 /* Not able to access */
+#define FT_NOFOLLOW 8 /* Could not follow link */
+#define FT_NOSTAT 9 /* Could not stat file */
+#define FT_NOCHG 10 /* Incremental option, file not changed */
+#define FT_DIRNOCHG 11 /* Incremental option, directory not changed */
+#define FT_ISARCH 12 /* Trying to save archive file */
+#define FT_NORECURSE 13 /* No recursion into directory */
+#define FT_NOFSCHG 14 /* Different file system, prohibited */
+#define FT_NOOPEN 15 /* Could not open directory */
+#define FT_RAW 16 /* Raw block device */
+#define FT_FIFO 17 /* Raw fifo device */
+\end{verbatim}
+\normalsize
+
+\item [Filename]
+ \index{Filename}
+ is the fully qualified filename.
+
+\item [File-Attributes]
+ \index{File-Attributes}
+ consists of the 13 fields of the stat() buffer in ASCII base64 format
+separated by spaces. These fields and their meanings are shown below. This
+stat() packet is in Unix format, and MUST be provided (constructed) for ALL
+systems.
+
+\item [Link]
+ \index{Link}
+ when the FT code is FT\_LNK or FT\_LNKSAVED, the item in question is a Unix
+link, and this field contains the fully qualified link name. When the FT code
+is not FT\_LNK or FT\_LNKSAVED, this field is null.
+
+\item [Extended-Attributes]
+ \index{Extended-Attributes}
+ The exact format of this field is operating system dependent. It contains
+additional or extended attributes of a system dependent nature. Currently,
+this field is used only on WIN32 systems where it contains a ASCII base64
+representation of the WIN32\_FILE\_ATTRIBUTE\_DATA structure as defined by
+Windows. The fields in the base64 representation of this structure are like
+the File-Attributes separated by spaces.
+\end{description}
+
+The File-attributes consist of the following:
+
+\addcontentsline{lot}{table}{File Attributes}
+\begin{longtable}{|p{0.6in}|p{0.7in}|p{1in}|p{1in}|p{1.4in}|}
+ \hline
+\multicolumn{1}{|c|}{\bf Field No. } & \multicolumn{1}{c|}{\bf Stat Name }
+& \multicolumn{1}{c|}{\bf Unix } & \multicolumn{1}{c|}{\bf Win98/NT } &
+\multicolumn{1}{c|}{\bf MacOS } \\
+ \hline
+\multicolumn{1}{|c|}{1 } & {st\_dev } & {Device number of filesystem } &
+{Drive number } & {vRefNum } \\
+ \hline
+\multicolumn{1}{|c|}{2 } & {st\_ino } & {Inode number } & {Always 0 } &
+{fileID/dirID } \\
+ \hline
+\multicolumn{1}{|c|}{3 } & {st\_mode } & {File mode } & {File mode } &
+{777 dirs/apps; 666 docs; 444 locked docs } \\
+ \hline
+\multicolumn{1}{|c|}{4 } & {st\_nlink } & {Number of links to the file } &
+{Number of link (only on NTFS) } & {Always 1 } \\
+ \hline
+\multicolumn{1}{|c|}{5 } & {st\_uid } & {Owner ID } & {Always 0 } &
+{Always 0 } \\
+ \hline
+\multicolumn{1}{|c|}{6 } & {st\_gid } & {Group ID } & {Always 0 } &
+{Always 0 } \\
+ \hline
+\multicolumn{1}{|c|}{7 } & {st\_rdev } & {Device ID for special files } &
+{Drive No. } & {Always 0 } \\
+ \hline
+\multicolumn{1}{|c|}{8 } & {st\_size } & {File size in bytes } & {File
+size in bytes } & {Data fork file size in bytes } \\
+ \hline
+\multicolumn{1}{|c|}{9 } & {st\_blksize } & {Preferred block size } &
+{Always 0 } & {Preferred block size } \\
+ \hline
+\multicolumn{1}{|c|}{10 } & {st\_blocks } & {Number of blocks allocated }
+& {Always 0 } & {Number of blocks allocated } \\
+ \hline
+\multicolumn{1}{|c|}{11 } & {st\_atime } & {Last access time since epoch }
+& {Last access time since epoch } & {Last access time -66 years } \\
+ \hline
+\multicolumn{1}{|c|}{12 } & {st\_mtime } & {Last modify time since epoch }
+& {Last modify time since epoch } & {Last access time -66 years } \\
+ \hline
+\multicolumn{1}{|c|}{13 } & {st\_ctime } & {Inode change time since epoch
+} & {File create time since epoch } & {File create time -66 years}
+\\ \hline
+
+\end{longtable}
+
+\section{Old Depreciated Tape Format}
+\index{Old Depreciated Tape Format}
+\index{Format!Old Depreciated Tape}
+\addcontentsline{toc}{subsection}{Old Depreciated Tape Format}
+
+The format of the Block Header (version 1.26 and earlier) is:
+
+\footnotesize
+\begin{verbatim}
+ uint32_t CheckSum; /* Block check sum */
+ uint32_t BlockSize; /* Block byte size including the header */
+ uint32_t BlockNumber; /* Block number */
+ char ID[4] = "BB01"; /* Identification and block level */
+\end{verbatim}
+\normalsize
+
+The format of the Record Header (version 1.26 or earlier) is:
+
+\footnotesize
+\begin{verbatim}
+ uint32_t VolSessionId; /* Unique ID for this session */
+ uint32_t VolSessionTime; /* Start time/date of session */
+ int32_t FileIndex; /* File index supplied by File daemon */
+ int32_t Stream; /* Stream number supplied by File daemon */
+ uint32_t DataSize; /* size of following data record in bytes */
+\end{verbatim}
+\normalsize
+
+\footnotesize
+\begin{verbatim}
+ Current Bacula Tape Format
+ 6 June 2001
+ Version BB01 is the old deprecated format.
+ A Bacula tape is composed of tape Blocks. Each block
+ has a Block header followed by the block data. Block
+ Data consists of Records. Records consist of Record
+ Headers followed by Record Data.
+ :=======================================================:
+ | |
+ | Block Header |
+ | (16 bytes version BB01) |
+ |-------------------------------------------------------|
+ | |
+ | Record Header |
+ | (20 bytes version BB01) |
+ |-------------------------------------------------------|
+ | |
+ | Record Data |
+ | |
+ |-------------------------------------------------------|
+ | |
+ | Record Header |
+ | (20 bytes version BB01) |
+ |-------------------------------------------------------|
+ | |
+ | ... |
+ Block Header: the first item in each block. The format is
+ shown below.
+ Partial Data block: occurs if the data from a previous
+ block spills over to this block (the normal case except
+ for the first block on a tape). However, this partial
+ data block is always preceded by a record header.
+ Record Header: identifies the Volume Session, the Stream
+ and the following Record Data size. See below for format.
+ Record data: arbitrary binary data.
+ Block Header Format BB01 (deprecated)
+ :=======================================================:
+ | CheckSum (uint32_t) |
+ |-------------------------------------------------------|
+ | BlockSize (uint32_t) |
+ |-------------------------------------------------------|
+ | BlockNumber (uint32_t) |
+ |-------------------------------------------------------|
+ | "BB01" (char [4]) |
+ :=======================================================:
+ BBO1: Serves to identify the block as a
+ Bacula block and also servers as a block format identifier
+ should we ever need to change the format.
+ BlockSize: is the size in bytes of the block. When reading
+ back a block, if the BlockSize does not agree with the
+ actual size read, Bacula discards the block.
+ CheckSum: a checksum for the Block.
+ BlockNumber: is the sequential block number on the tape.
+ VolSessionId: a unique sequential number that is assigned
+ by the Storage Daemon to a particular Job.
+ This number is sequential since the start
+ of execution of the daemon.
+ VolSessionTime: the time/date that the current execution
+ of the Storage Daemon started. It assures
+ that the combination of VolSessionId and
+ VolSessionTime is unique for all jobs
+ written to the tape, even if there was a
+ machine crash between two writes.
+ Record Header Format BB01 (deprecated)
+ :=======================================================:
+ | VolSessionId (uint32_t) |
+ |-------------------------------------------------------|
+ | VolSessionTime (uint32_t) |
+ |-------------------------------------------------------|
+ | FileIndex (int32_t) |
+ |-------------------------------------------------------|
+ | Stream (int32_t) |
+ |-------------------------------------------------------|
+ | DataSize (uint32_t) |
+ :=======================================================:
+ VolSessionId: a unique sequential number that is assigned
+ by the Storage Daemon to a particular Job.
+ This number is sequential since the start
+ of execution of the daemon.
+ VolSessionTime: the time/date that the current execution
+ of the Storage Daemon started. It assures
+ that the combination of VolSessionId and
+ VolSessionTime is unique for all jobs
+ written to the tape, even if there was a
+ machine crash between two writes.
+ FileIndex: a sequential file number within a job. The
+ Storage daemon enforces this index to be
+ greater than zero and sequential. Note,
+ however, that the File daemon may send
+ multiple Streams for the same FileIndex.
+ The Storage Daemon uses negative FileIndices
+ to identify Session Start and End labels
+ as well as the End of Volume labels.
+ Stream: defined by the File daemon and is intended to be
+ used to identify separate parts of the data
+ saved for each file (attributes, file data,
+ ...). The Storage Daemon has no idea of
+ what a Stream is or what it contains.
+ DataSize: the size in bytes of the binary data record
+ that follows the Session Record header.
+ The Storage Daemon has no idea of the
+ actual contents of the binary data record.
+ For standard Unix files, the data record
+ typically contains the file attributes or
+ the file data. For a sparse file
+ the first 64 bits of the data contains
+ the storage address for the data block.
+ Volume Label
+ :=======================================================:
+ | Id (32 bytes) |
+ |-------------------------------------------------------|
+ | VerNum (uint32_t) |
+ |-------------------------------------------------------|
+ | label_date (float64_t) |
+ |-------------------------------------------------------|
+ | label_time (float64_t) |
+ |-------------------------------------------------------|
+ | write_date (float64_t) |
+ |-------------------------------------------------------|
+ | write_time (float64_t) |
+ |-------------------------------------------------------|
+ | VolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PrevVolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolType (128 bytes) |
+ |-------------------------------------------------------|
+ | MediaType (128 bytes) |
+ |-------------------------------------------------------|
+ | HostName (128 bytes) |
+ |-------------------------------------------------------|
+ | LabelProg (32 bytes) |
+ |-------------------------------------------------------|
+ | ProgVersion (32 bytes) |
+ |-------------------------------------------------------|
+ | ProgDate (32 bytes) |
+ |-------------------------------------------------------|
+ :=======================================================:
+
+ Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
+ (old version also recognized:)
+ Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
+ LabelType (Saved in the FileIndex of the Header record).
+ PRE_LABEL -1 Volume label on unwritten tape
+ VOL_LABEL -2 Volume label after tape written
+ EOM_LABEL -3 Label at EOM (not currently implemented)
+ SOS_LABEL -4 Start of Session label (format given below)
+ EOS_LABEL -5 End of Session label (format given below)
+ label_date: Julian day tape labeled
+ label_time: Julian time tape labeled
+ write_date: Julian date tape first used (data written)
+ write_time: Julian time tape first used (data written)
+ VolName: "Physical" Volume name
+ PrevVolName: The VolName of the previous tape (if this tape is
+ a continuation of the previous one).
+ PoolName: Pool Name
+ PoolType: Pool Type
+ MediaType: Media Type
+ HostName: Name of host that is first writing the tape
+ LabelProg: Name of the program that labeled the tape
+ ProgVersion: Version of the label program
+ ProgDate: Date Label program built
+ Session Label
+ :=======================================================:
+ | Id (32 bytes) |
+ |-------------------------------------------------------|
+ | VerNum (uint32_t) |
+ |-------------------------------------------------------|
+ | JobId (uint32_t) |
+ |-------------------------------------------------------|
+ | *write_date (float64_t) VerNum 10 |
+ |-------------------------------------------------------|
+ | *write_time (float64_t) VerNum 10 |
+ |-------------------------------------------------------|
+ | PoolName (128 bytes) |
+ |-------------------------------------------------------|
+ | PoolType (128 bytes) |
+ |-------------------------------------------------------|
+ | JobName (128 bytes) |
+ |-------------------------------------------------------|
+ | ClientName (128 bytes) |
+ |-------------------------------------------------------|
+ | Job (128 bytes) |
+ |-------------------------------------------------------|
+ | FileSetName (128 bytes) |
+ |-------------------------------------------------------|
+ | JobType (uint32_t) |
+ |-------------------------------------------------------|
+ | JobLevel (uint32_t) |
+ |-------------------------------------------------------|
+ | FileSetMD5 (50 bytes) VerNum 11 |
+ |-------------------------------------------------------|
+ Additional fields in End Of Session Label
+ |-------------------------------------------------------|
+ | JobFiles (uint32_t) |
+ |-------------------------------------------------------|
+ | JobBytes (uint32_t) |
+ |-------------------------------------------------------|
+ | start_block (uint32_t) |
+ |-------------------------------------------------------|
+ | end_block (uint32_t) |
+ |-------------------------------------------------------|
+ | start_file (uint32_t) |
+ |-------------------------------------------------------|
+ | end_file (uint32_t) |
+ |-------------------------------------------------------|
+ | JobErrors (uint32_t) |
+ |-------------------------------------------------------|
+ | JobStatus (uint32_t) VerNum 11 |
+ :=======================================================:
+ * => fields deprecated
+ Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
+ LabelType (in FileIndex field of Header):
+ EOM_LABEL -3 Label at EOM
+ SOS_LABEL -4 Start of Session label
+ EOS_LABEL -5 End of Session label
+ VerNum: 11
+ JobId: JobId
+ write_btime: Bacula time/date this tape record written
+ write_date: Julian date tape this record written - deprecated
+ write_time: Julian time tape this record written - deprecated.
+ PoolName: Pool Name
+ PoolType: Pool Type
+ MediaType: Media Type
+ ClientName: Name of File daemon or Client writing this session
+ Not used for EOM_LABEL.
+\end{verbatim}
+\normalsize
projects / bacula / docs / blobdiff