Kern's ToDo List
- 25 January 2003
+ 28 April 2003
-Documentation to do: (a little bit at a time)
+Documentation to do: (any release a little bit at a time)
- Document running a test version.
- Document query file format.
- Document static linking
-- Document how to automatically backup all local partitions
- Document problems with Verify and pruning.
- Document how to use multiple databases.
+- Add a section to the doc on Manual cycling of Volumes.
-
+
Testing to do: (painful)
-- that console command line options work
+- that ALL console command line options work and are always implemented
- blocksize recognition code.
+- multiple simultaneous Volumes
+
+For 1.30a release:
+- Examine Bare Metal restore problem.
+- Test multiple simultaneous Volumes
+- Document FInclude ...
+
+- Figure out how to use ssh or stunnel to protect Bacula communications.
+
+After 1.30:
+- Fix command prompt in gnome-console by checking on Ready.
+- Implement HEART_BEAT while SD waiting for tapes.
+- Include RunBeforeJob and RunAfterJob output in the message
+ stream.
+- Check if Job/File retentions apply to multivolume jobs.
+- Change M_INFO to M_RESTORED for all restored files.
+- Remove subsysdir from conf files (used only in autostart scripts).
+- Implement console @echo command.
+- Implement global with DB name and add to btraceback.gdb
+- Bug: fix access problems on files restored on WinXP.
+- Implement a Slot priority (loaded/not loaded).
+- Implement "vacation" Incremental only saves.
+- Implement single pane restore (much like the Gftp panes).
+- Implement Automatic Mount even in operator wait.
+- Implement create "FileSet"?
+- Implement Release Device in the Job resource to unmount a drive.
+- Implement Acquire Device in the Job resource to mount a drive,
+ be sure this works with admin jobs so that the user can get
+ prompted to insert the correct tape. Possibly some way to say to
+ run the job but don't save the files.
+- Implement all command line args on run.
+- Implement command line "restore" args.
+- Implement "restore current select=no"
+- Fix watchdog pthread crash on Win32
+- Fix "access not allowed" for backup of files on WinXP.
+- Implement "scratch pool" where tapes are defined and can be
+ taken by any pool that needs them.
+- Implement restore "current system", but take all files without
+ doing selection tree -- so that jobs without File records can
+ be restored.
+- Make | and < work on FD side.
+- Pass prefix_links to FD.
+- Implement a M_SECURITY message class.
+- Implement disk spooling. Two parts: 1. Spool to disk then
+ immediately to tape to speed up tape operations. 2. Spool to
+ disk only when the tape is full, then when a tape is hung move
+ it to tape.
+- From Phil Stracchino:
+ It would probably be a per-client option, and would be called
+ something like, say, "Automatically purge obsoleted jobs". What it
+ would do is, when you successfully complete a Differential backup of a
+ client, it would automatically purge all Incremental backups for that
+ client that are rendered redundant by that Differential. Likewise,
+ when a Full backup on a client completed, it would automatically purge
+ all Differential and Incremental jobs obsoleted by that Full backup.
+ This would let people minimize the number of tapes they're keeping on
+ hand without having to master the art of retention times.
-For 1.30 release:
-- Fix restore of hard linked file.
+- Allow multiple Storage specifications (or multiple names on
+ a single Storage specification) in the Job record. Thus a job
+ can be backed up to a number of storage devices.
+- Implement dump label to UA
+- Add prefixlinks to where or not where absolute links to FD.
+- Look at Python for a Bacula scripting language -- www.python.org
+- Issue message to mount a new tape before the rewind.
+- Simplified client job initiation for portables.
+- If SD cannot open a drive, make it periodically retry.
+- Implement LabelTemplate (at least first cut).
+- Add more of the config info to the tape label.
+- Implement a Mount Command and an Unmount Command where
+ the users could specify a system command to be performed
+ to do the mount, after which Bacula could attempt to
+ read the device. This is for Removeable media such as a CDROM.
+ - Most likely, this mount command would be invoked explicitly
+ by the user using the current Console "mount" and "unmount"
+ commands -- the Storage Daemon would do the right thing
+ depending on the exact nature of the device.
+ - As with tape drives, when Bacula wanted a new removable
+ disk mounted, it would unmount the old one, and send a message
+ to the user, who would then use "mount" as described above
+ once he had actually inserted the disk.
+
+- Make some way so that if a machine is skipped because it is not up
+ that Bacula will continue retrying for a specified period of time --
+ periodically.
+- If tape is marked read-only, then try opening it read-only rather than
+ failing, and remember that it cannot be written.
+- Refine SD waiting output:
+ Device is being positioned
+ > Device is being positioned for append
+ > Device is being positioned to file x
+ >
+- Figure out some way to estimate output size and to avoid splitting
+ a backup across two Volumes -- this could be useful for writing CDROMs
+ where you really prefer not to have it split -- not serious.
+- Add RunBeforeJob and RunAfterJob to the Client program.
+- Have SD compute MD5 or SHA1 and compare to what FD computes.
+- Make VolumeToCatalog calculate an MD5 or SHA1 from the
+ actual data on the Volume and compare it.
- Implement FileOptions (see end of this document)
- Implement Bacula plugins -- design API
- Make bcopy read through bad tape records.
- Program files (i.e. execute a program to read/write files).
Pass read date of last backup, size of file last time.
- Add Signature type to File DB record.
+- Make Restore report an error if FD or SD term codes are not OK.
- CD into subdirectory when open()ing files for backup to
speed up things. Test with testfind().
-- Add prefixlinks to where or not where absolute links to FD.
-- Look at handling <> in smtp doesn't work with exim.
- Priority job to go to top of list.
-- Implement Bar code handling
-- Why is catreq.c:111 Find vol called twice for a job?
- Find out why Full saves run slower and slower (hashing?)
-- Figure out how to allow multiple simultaneous file Volumes on a single device.
- Why are save/restore of device different sizes (sparse?) Yup! Fix it.
- Implement some way for the Console to dynamically create a job.
- Restore to a particular time -- e.g. before date, after date.
-- Implement disk spooling
-- Implement finer multiprocessing options.
- Solaris -I on tar for include list
-- Enable avoid backing up archive device (findlib/find_one.c:128)
+- Prohibit backing up archive device (findlib/find_one.c:128)
- Need a verbose mode in restore, perhaps to bsr.
- bscan without -v is too quiet -- perhaps show jobs.
- Add code to reject whole blocks if not wanted on restore.
-- Implement multiple simultaneous file Volumes on a single device.
- Start working on Base jobs.
+- Check if we can increase Bacula FD priorty in Win2000
- Make sure the MaxVolFiles is fully implemented in SD
-- Flush all the daemon messages at the end of every job.
- Check if both CatalogFiles and UseCatalog are set to SD.
-- Check if we can increase Bacula FD priorty in Win2000
- Need return status on read_cb() from read_records(). Need multiple
records -- one per Job, maybe a JCR or some other structure with
a block and a record.
- Put system type returned by FD into catalog.
- Possibly add email to Watchdog if drive is unmounted too
long and a job is waiting on the drive.
-- Strip trailing slashes from Include directory names in the FD.
- Use read_record.c in SD code.
- Why don't we get an error message from Win32 FD when bootstrap
file cannot be created for restore command?
-- Need to specify MaximumConcurrentJobs in the Job resource.
- When Marking a file in Restore that is a hard link, also
mark the link so that the data will be reloaded.
- Restore program that errors in SD due to no tape reports
OK incorrectly in output.
- After unmount, if restore job started, ask to mount.
-- Make Restore report an error if FD or SD term codes are not OK.
- Convert all %x substitution variables, which are hard to remember
and read to %(variable-name). Idea from TMDA.
- Add JobLevel in FD status (but make sure it is defined).
- Make Pool resource handle Counter resources.
- Remove NextId for SQLite. Optimize.
-- Strip trailing / from Include
- Move all SQL statements into a single location.
-- Cleanup db_update_media and db_update_pool
- Add UA rc and history files.
- put termcap (used by console) in ./configure and
allow -with-termcap-dir.
- Fix Autoprune for Volumes to respect need for full save.
- Fix Win32 config file definition name on /install
- No READLINE_SRC if found in alternate directory.
-- Add Client FS/OS id (Linux, Win95/98, ...).
- Test a second language e.g. french.
- Compare tape to Client files (attributes, or attributes and data)
- Make all database Ids 64 bit.
- Complete code in Bacula Resources -- this will permit
reading a new config file at any time.
- Handle ctl-c in Console
-- Implement LabelTemplate (at least first cut).
- Implement script driven addition of File daemon to config files.
- Think about how to make Bacula work better with File (non-tape) archives.
- Write Unix emulator for Windows.
-
- Implement new serialize subroutines
send(socket, "string", &Vol, "uint32", &i, NULL)
- Audit all UA commands to ensure that we always prompt where possible.
- Put memory utilization in Status output of each daemon
if full status requested or if some level of debug on.
- Make database type selectable by .conf files i.e. at runtime
-- gethostbyname failure in bnet_connect() continues
- generating errors -- should stop.
- Set flag for uname -a. Add to Volume label.
- Implement throttled work queue.
- Check for EOT at ENOSPC or EIO or ENXIO (unix Pc)
-- Allow multiple Storage specifications (or multiple names on
- a single Storage specification) in the Job record. Thus a job
- can be backed up to a number of storage devices.
-- Implement dump label to UA
-- Concept of VolumeSet during restore which is a list
- of Volume names needed.
- Restore files modified after date
- Restore file modified before date
- Emergency restore info:
- Implement Restore FileSet=
- Create a protocol.h and protocol.c where all protocol messages
are concentrated.
-- If SD cannot open a drive, make it periodically retry.
- Remove duplicate fields from jcr (e.g. jcr.level and jcr.jr.Level, ...).
- Timout a job or terminate if link goes down, or reopen link and query.
- Find general solution for sscanf size problems (as well
- Need a structure for pending actions:
- buffered messages
- termination status (part of buffered msgs?)
-- Concept of grouping Storage devices and job can use
- any of a number of devices
- Drive management
Read, Write, Clean, Delete
- Login to Bacula; Bacula users with different permissions:
This could be the output of df; or perhaps some sort of /etc/mtab record.
Longer term to do:
-- Design at hierarchial storage for Bacula.
+- Design at hierarchial storage for Bacula. Migration and Clone.
- Implement FSM (File System Modules).
- Identify unchanged or "system" files and save them to a
special tape thus removing them from the standard
backup FileSet -- BASE backup.
-- Turn virutally all sprintfs into snprintfs.
- Heartbeat between daemons.
- Audit M_ error codes to ensure they are correct and consistent.
- Add variable break characters to lex analyzer.
Item 6: Write a regression script.
-Started
+Done -- Continue to expand its testing.
What: This is an automatic script that runs and tests as many features
of Bacula as possible. The output is compared to previous
- Get correct error status from run_program or open_bpipe().
- Restrict permissions on File Volumes (now 0640).
- Umasked 022 daemons
+- Fix restore of hard linked file.
+- Figure out how to allow multiple simultaneous file Volumes on a single device.
+- Implement multiple simultaneous file Volumes on a single device.
+- Cleanup db_update_media and db_update_pool
+- Flush all the daemon messages at the end of every job.
+- Change stat1= fgets()!=NULL to stat1=fgest()==NULL; in
+ run_program -- bpipe.c
+- Apparently cancel does not work for jobs waiting to be
+ scheduled.
+- Implement TCP/IP connection for MySQL
+- Pull a canceled job from the Scheduling queue.
+- Implement max_file_size in block.c (already done, just tweaked).
+- Look at purge jobs volume (at least document it, and see if it is
+ logical).
+- Add list volumes does all pools. list volumes pool=xxx now works.
+- Add pool= to "list media" in ua_output.c
+- Strip trailing slashes from Include directory names in the FD.
+- Fix Error: bnet.c:408 gethostbyname() for lpmatou failed:
+ ERR=Operation not permited loop.
+- Add code if there is no mtio.h (cannot do -- too many ioctl defines needed)
+- Produce better error messages in when error/eof writing block.
+- Cancelling of a queued job does NOT work!!!!!!
+- Get two
+rufus-dir: Volume used once. Marking Volume "File0003" as Used.
+rufus-sd: Recycled volume File0003 on device /home/kern/bacula/working, all previous data lost.
+rufus-dir: Volume used once. Marking Volume "File0003" as Used.
+- Ability to backup to a file then later transfer to a tape -- Migration.
+ Migration based on MaxJobs(MinJobs),MaxVols(MinVols),AgeJobs,MaxBytes(MinBytes)
+ (i.e. HighwaterMark, LowwaterMark).
+- Eugeny Fisher <efischer@vip-rus.com> wants to cycle through a
+ set of volumes recycling the oldest volume when it is needed.
+- gethostbyname failure in bnet_connect() continues
+ generating errors -- should stop.
+- Add chflags() code for FreeBSD file flags
+- Bevan Anderson suggests having a run queue for each device
+ so that multiple simultaneous jobs can run but each writing
+ to a different Volume.
+- Look at handling <> in smtp doesn't work with exim.
+- Need to specify MaximumConcurrentJobs in the Job resource.
+- ***test GetFileAttributexEx, and remove MessageBox at 335 of winservice.cpp ****
+- Implement finer multiprocessing options.
+- Implement | and < in Exclude statements.
+- Figure out some way to specify a retention period for files
+ that no longer exist on the machine -- so that we maintain
+ say backups for 30 days, but if the file is deleted, we maintain
+ the last copy for 1 year. -- answer Volume retention.
+- Make non-zero status from RunJobBefore/After error the job.
+- Need define of int_least16_t in sha1.h for SuSE.
+- Implement bar code reader for autochangers
+- Document new MaximumConcurrentJob records (Job, Client, Storage)
+- Write up how to use/manage disk Volume Storage. ******
+- Remove kern and kelvin from mysql_grant...
+- Install grant_mysql...
+- Strip trailing / from Include
+- add #define ENABLE_NLS for Gnome compile on SuSE.
+- Add Client FS/OS id (Linux, Win95/98, ...).
+- Concept of VolumeSet during restore which is a list
+ of Volume names needed.
+- Turn virutally all sprintfs into snprintfs.
+- Update volume=Test01 requests pool, then lists volumes.
+ **** Test select_pool_and_media ...
+- Document relabel
+- Add IP address to authentication failures.
+- Add a default File storage so that new users can do backup
+ and restores right away.
+- Forbid sbindir and with-subsys-dir from being the same (otherwise
+ the binary gets deleted when the daemon is stopped in the
+ rc.d/inid.d directory.
+- Do not ignore SIGCHLD
+- Add Cleaning to list of volume statuses
+- Implement run at "xxx"
+- Document new transparent Console commands and wait command.
+- Document . and @ commands
+- Document run when.
+- Document Lutz Kittler's trick of using "Run Before Job" to
+ abort a job on a particular day.
+- Document Ludovic Strappazon's Win32 raw device save/restore.
+- Document not to restore .journal .autofsck
+- Document labeling a whole magazine using "cat"
+- Document how to automatically backup all local partitions
+- Document logrotate
+- Document OPTIMIZE TABLE in MySQL
+- Document new immediate File save configuration (walk user
+ through first save to file Volume with automatic Volume labeling?).
+- Implement scheduling of one time "run" jobs (i.e. instead of
+ starting immediately start at some specified time).
+- Bug: up arrow prints garbage in command line on gnome-console!
projects / bacula / bacula / blobdiff