in Job, Client, and Pool, with precedence such that Job overrides Client
which in turn overrides Pool.
- Print a message when a job starts if the conf file is not current.
-- To pass Include 1 or two letter commands
- I Name Include name - first record
- B Name Base name - repeat
- R "xxx" Regexp
- W "xxx" Wild Card
- E zzz Exclude expression (wild card)
- P "plugin" Plugin
- D "reader" Reader program
- T "writer" Writer program
- O Options In current commpressed format
- (compression, signature, onefs, recurse, sparse,
- replace, verify options, ...)
- N End option set
- B BaseName Start second option set any letter
- ...
- E
- F Number Number of filenames to follow
- <file-name>
- B Name
- ...
- N End option set
- F Number Number of filenames to follow
- <file-name>
- ...
- Spooling ideas taken from Volker Sauer's and other's emails:
> IMHO job spooling should be turned on
VolSessionId and VolSessionTime.
=========================================================
-=============================================================
-
- File Backup Options Project
- 28 April 2004
-
-
-Subject: File Backup Options
-
-Problem:
- Bacula users who are using compression would like to suppress
- compressing all .gz files since it was a waste of CPU time.
- Although Bacula currently permits using different options
- (compression, ...) on a directory by directory basis, it cannot
- do it on a file by file basis, which is clearly what was
- desired.
-
-Proposed Implementation:
- To solve this problem, I propose the following:
-
- - Add a new Director resource type called Options.
-
- - The Options resource will have records for all options that
- can currently be specified on the Include record (in a
- FileSet). Examples below.
-
- - The Options resource will permit an exclude option as well
- as a number of additional options.
-
- - The heart of the Options resource is the ability to supply
- any number of Regex records which specify POSIX regular
- expressions. These regular expressions are applied to the
- fully qualified filename (path and all). If one matches,
- then the Options will be used.
-
- - When a Regex specification matches an included file, the
- options specified in the Options resource will override the
- default options specified on the Include record.
-
- - Include records will be modified to permit referencing one or
- more Options resources. The Options will be used in the
- order listed on the Include record and the first one that
- matches will be applied.
-
- - Options (or specifications) currently supplied on the Include
- record will be deprecated (i.e. removed in a later version a
- year or so from now).
-
- - The Exclude record will be deprecated as the same functionality
- can be obtained by using an Exclude = yes in the Options or
- an Exclude = xxxx directive.
-
-Options records:
- The following records can appear in the Options resource. An
- asterisk preceding the name indicates a feature not currently
- implemented.
-
- - Regexp "xxx" - Match regular expression
- - Wild "xxx" - Do a wild card match
-
- For Backup Jobs:
- - Compression= (GZIP, ...)
- - Signature= (MD5, SHA1, ...)
- - *Encryption=
- - OneFs= (yes/no) - remain on one filesystem
- - Recurse= (yes/no) - recurse into subdirectories
- - Sparse= (yes/no) - do sparse file backup
- - *Exclude= (yes/no) - exclude file from being saved
- - *INclude= (yes/no) - include the file (needed??)
- - *Reader= (filename) - external read (backup) program
- - *Plugin= (filename) - read/write plugin module
-
- For Verify Jobs:
- - verify= (ipnougsamc5) - verify options
-
- For Restore Jobs:
- - replace= (always/ifnewer/ifolder/never) - replace options currently
- implemented in 1.31
- - *Writer= (filename) - external write (restore) program
-
-
-Implementation:
- Currently options specifying compression, MD5 signatures,
- recursion, ... of a FileSet are supplied on the Include
- record. These will now all be collected into a Options
- resource, which will be specified in the Include in place of
- the options. Multiple Options may be specified. Since the
- Options may contain regular expressions that are applied to the
- full filename, this will give the ability to specify backup
- options on a file by file basis to whatever level of detail you
- wish.
-
-Example:
-
- Today:
-
- FileSet {
- Name = "FullSet"
- Include = compression=GZIP signature=MD5 {
- /
- }
- }
-
- Proposal:
-
- FileSet {
- Name = "FullSet"
- Include {
- Options {
- Compression = GZIP;
- Signature = MD5
- Wild = /*.?*/ # matches all files.
- }
- Include = /
- }
- }
-
- That's a lot more to do the same thing, but it gives the ability to
- apply options on a file by file basis. For example, suppose you
- want to compress all files but not any file with extensions .gz or .Z.
- In that case, you will need to group two sets of options using
- the Options resource. Files may be anywhere except in an
- option set ??? All Options apply to all files in the order
- the Options were specified. To have files included with
- different option sets without using wild-cards, use two or more
- Includes -- each one is handled in turn using only the files and
- optionsets specified in the include.
-
- FileSet {
- Name = "FullSet"
- Include {
- Options {
- Signature = MD5
- # Note multiple Matches are ORed
- Wild = "*.gz" # matches .gz files
- Wild = "*.Z" # matches .Z files
- }
- Options {
- Compression = GZIP
- Signature = MD5
- Wild = "*.?*" # matches all files
- }
- Include = /
- }
- }
-
- Now, since the no Compression option is specified in the
- first group of Options, *.gz or *.Z file will have an MD5 signature computed,
- but will not be compressed. For all other files, the *.gz *.Z will not
- match, so the second group of options will be used which will include GZIP
- compression.
-
-Questions:
-
-Results:
- After implementing the above, the user will be able to specify
- on a file by file basis (using regular expressions) what options are
- applied for the backup.
-
-
-=============================================
-
==========================================================
Unsaved File design
====
-=== Done in 1.33
-- Change console to bconsole.
-- Change smtp to bsmtp.
-- Fix time difference problem between Bacula and Client
- so that everything is in GMT.
-- Fix TimeZone problem!
-- Mount a tape that is not right for the job (wrong # files on tape)
- Bacula asks for another tape, fix problems with first tape and
- say "mount". All works OK, but status shows:
- Device /dev/nst0 open but no Bacula volume is mounted.
- Total Bytes=1,153,820,213 Blocks=17,888 Bytes/block=64,502
- Positioned at File=9 Block=3,951
- Full Backup job Rufus.2003-10-26_16.45.31 using Volume "DLT-24Oct03" on device /dev/nst0
- Files=21,003 Bytes=253,954,408 Bytes/sec=2,919,016
- FDReadSeqNo=192,134 in_msg=129830 out_msg=5 fd=7
-- Upgrade to cygwin 1.5
-- Optimize fsf not to read.
-- Use ioctl() fsf if it exists. Figure out where we are from
- the mt_status command. Use slow fsf only if other does not work.
-- Enhance "update slots" to include a "scan" feature
- scan 1; scan 1-5; scan 1,2,4 ... to update the catalog
-- Allow a slot or range of slots on the label barcodes command.
-- Finish implementation of Verify=DiskToCatalog
-- Make sure that Volumes are recycled based on "Least recently used"
- rather than lowest MediaId.
-- Add flag to write only one EOF mark on the tape.
-- Implement autochanger testing in btape "test" command.
-- Implement lmark to list everyfile marked.
-- Make mark/unmark report how many files marked/unmarked.
-- Keep last 5 or 10 completed jobs and show them in a similar list.
-- Make a Running Jobs: output similar to current Scheduled Jobs:
-- Change "create_media_record in bscan to use Archive instead of Full.
-- Have some way to estimate the restore size or have it printed.
-- Volume problems occurs if you have valid volume, written, then it is
- truncated. You get 12-Nov-2003 11:48 rufus-sd: kernsave.2003-11-12_11.48.09 Warning: mount.c:228 Volume on /tmp is not a Bacula labeled Volume, because:
- block.c:640 Read zero bytes on device /tmp.
-- Make sure that 64 bit I/O packets are used on Cygwin.
-- Add to supported autochangers
- OS : FreeBSD-4.9
- Auto-Changer : QUALSTAR TLS-4210
- Manufufactur : Qualstar
- Tapes : 12 (AIT1: 36GB, AIT2: 50GB all uncompressed)
- Drives : 2xAIT2 (installed in the Qualstar: SONY SDX-500C AIT2)
-- Document estimate command in tree.
-- Document lsmark command in tree.
-- Setup a standard job that builds a bootstrap file and saves
- it with the catalog database.
-- See if a restore job can add a file to the tape (prohibit this).
-- Restrict characters permitted in a name.
-- In restore, provide option for limiting to a particular Pool.
-- In restore, list FileSets that only have different base names --
- i.e. any FileSet with the same name should be treated as the same.
-- Make Scheduler sort jobs by StartTime, Priority.
-- Make sure smtp and any other useful program is executable by the world
- in case Bacula is not running as root.
-- Look at Dan's field width problems in PostgreSQL.
-- Look at effect of removing GROUP BYs.
-- In restore take all filesets with same base name.
-- From Alan Brown <ajb2@mssl.ucl.ac.uk>
- BTW, there's a make install bug in 1.33 - with --enable-gnome,
- gnome-console is built, but the binary and .conf are not being installed.
-- Permit Bacula and apcupsd donations (not done for apcupsd).
-- Fix Ctl-C crashing the Console (readline?).
-- Look at code in recycle_oldes_purged_volume() recycle.c. Why not
- let SQL do ORDER BY LastWritten ASC?
-- Look at find_next_volume() algorithm. Currently, it selects:
- +---------+------------+---------------------+-----------+
- | MediaId | VolumeName | LastWritten | VolBytes |
- +---------+------------+---------------------+-----------+
- | 3 | Test13 | 0000-00-00 00:00:00 | 1 |
- | 4 | Test14 | 0000-00-00 00:00:00 | 1 |
- | 1 | test11 | 2003-12-03 18:39:55 | 4,004,926 |
- | 2 | test12 | 2004-01-04 15:25:56 | 2,078,691 |
- +---------+------------+---------------------+-----------+
- but perhaps it should fill already used Volumes first, and use
- Append volumes before Purged, or Recycled, ...
-- Possibly remove the "|| ap == NULL" on lines 123 and 207 of lib/var.c,
- which creates compile problems on alpha systems.
- var.c:123: no match for `va_list & == long int'
-- Check "restore" 3 (JobId), then it asks for Storage resource. Does
- it verify that the correct volume is chosen?
-- Make Bacula "poll a drive".
-- Notes for final checking of Nic's code:
- Could I get you to double check the switch () statements in the
- job_check_maxwaittime and job_check_maxruntime functions in
- src/dird/job.c?
-- Define week of year for scheduler. W01, W02, ...
- Week 01 of a year is per definition the first week that has the
- Thursday in this year, which is equivalent to the week that contains the
- fourth day of January. In other words, the first week of a new year is
- the week that has the majority of its days in the new year. Week 01
- might also contain days from the previous year and the week before week
- 01 of a year is the last week (52 or 53) of the previous year even if it
- contains days from the new year. A week starts with Monday (day 1) and
- ends with Sunday (day 7). For example, the first week of the year 1997
- lasts from 1996-12-30 to 1997-01-05 and can be written in standard
- notation as
- 1997-W01 or 1997W01
- The week notation can also be extended by a number indicating the day
- of the week. For example, the day 1996-12-31, which is the Tuesday (day
- 2) of the first week of 1997, can also be written as
- 1997-W01-2 or 1997W012
-- Either restrict the characters in a name, or fix the problem
- emailing with names containing / (smtp command line breaks).
-- Implement .consolerc for Console
-- Implement scan: for every slot it finds, zero the slot of
- Volume other volume having that slot.
-- Make restore job check if all the files are actually restored.
-- Look at 2Gb limit for SQLite.
-- Fix get_storage_from_media_type (ua_restore) to use command line
- storage=
-- Don't print "Warning: Wrong Volume mounted ..." if mounting second volume.
-- Write a mini-readline with history and editing.
-- Take a careful look a the Basic recycling algorithm. When Bacula
- chooses, the order should be:
- - Look for Append
- - Look for Recycle or Purged
- - Prune volumes
- - Look for purged
- Instead of using lowest media Id, find the least recently used
- volume.
-
- When the tape is mounted and Bacula requests the status
- - Do everything possible to use it.
-
- Define a "available" status, which is the currently mounted
- Volume and all volumes that are currently in the autochanger.
-- Is a pool specification really needed for a restore? Yes, and
- you may want to exclude archive Pools.
-- Implement a PostgreSQL driver.
-- Fix restore to list errors if Invalid block found, and if # files
- restored does not match # expected.
-- Something is not right in last block of fill command.
-- Add FileSet to command line arguments for restore.
-- Enhance time and size scanning routines.
-- Add Console usr permissions -- do by adding filters for
- jobs, clients, storage, ...
-- Put max network buffer size on a directive.
-- Why does "mark cygwin" take so long!!!!!!!!
-- Implement alist processing for ACLs from Console.
-- When a file is set for restore, walk back up the chain of
- directories, setting them to be restored.
-- Figure out a way to set restore on a directory without recursively
- decending. (recurse off?).
-- Fix restore to only pull in last Differential and later Incrementals.
-- Implement 3 Pools for a Job:
- Job {
- Name = ...
- Full Backup Pool = xxx
- Incremental Backup Pool = yyy
- Differential Backup Pool = zzz
- }
-- Look at ASSERT() at 384 src/lib/bnet.c
-- Dates are wrong in restore list from Win32 FD.
-- Dates are wrong in catalog from Win32 FD.
-- Remove h_errno from bnet.c by including proper header.
-- For "list jobs" order by EndTime.
-- Make two tape fill test work.
-- Add atime preservation.
-- Do not err job if could not write bootstrap file.
-- Save and restore last_job across executions.
-- Have each daemon save the last_jobs structure when exiting and
- read it back in when starting up.
-- "restore jobid=1 select" calls get_storage_xxx, which prints "JobId 1 is
- not running."
-- Make column listing for running jobs JobId Level Type Started Name Status
-- Why does Bacula need the drive open to do "autochanger list" ?
-- Add data compare on write/read in btape "test".
-- Rescue builds incorrect script files on Rufus.
-- Release SQLite 2.8.9
-- During install, copy any console.conf to bconsole.conf.
-- Check: Run = Level=Differential feb-dec 1 at 1:05 to see if wday is empty.
-- Look at installation file permissions with Scott so that make install
- and the rpms agree.
-- Finish code passing files=nnn to restore start.
-- Add ctl-c to console to stop current command and discard buffered
- output.
-- Estimate to Tibs never returns.
-- Symbolic link a directory to another one, then backup the symbolic link.
-- Check and possibly fix problems with hard links.
-- Fix query buffer clobber ua_query.c
-- Allow "delete job jobid=xx jobid=xxx".
-- Update volume FromPool (or FromPool=xxx) refreshes the Volume defaults from Pool.
-- Implement multiple Volume in "purge jobs volume=".
-- Test Qmsg() code to be used in bnet.c to prevent recursion. Queue the
- message. If dequeueing toss the messages. Lock while dequeuing so that
- it cannot be called recursively and set dequeuing flag.
-- On unknown client in restore "client=xxx"
- Could not find Client "Matou": ERR=Query failed: DROP TABLE temp1:
- ERR=no such table: temp1
-- Null dlist head and tail pointers if all items are removed.
-- Build console in client-only build.
-- Phil says that Windows file sizes mismatch in Verify when they should,
- and that either the file size or the catalog size was zero.
-- Check time/dates printed during restore when using Win32 API.
-- Once a job is canceled, make sure the status reflects that even if the
- job is "stuck" in the run queue.
-- Test work on conio.c -- particularly linking.
-- Complete Win32 installer
-- Add ms disk example
+=== Done in 1.35
+- To pass Include 1 or two letter commands
+ I Name Include name - first record
+ B Name Base name - repeat
+ R "xxx" Regexp
+ W "xxx" Wild Card
+ E zzz Exclude expression (wild card)
+ P "plugin" Plugin
+ D "reader" Reader program
+ T "writer" Writer program
+ O Options In current commpressed format
+ (compression, signature, onefs, recurse, sparse,
+ replace, verify options, ...)
+ N End option set
+ B BaseName Start second option set any letter
+ ...
+ E
+ F Number Number of filenames to follow
+ <file-name>
+ B Name
+ ...
+ N End option set
+ F Number Number of filenames to follow
+ <file-name>
+ ...
+
projects / bacula / bacula / blobdiff