Kern's ToDo List
- 18 November 2004
+ 22 February 2005
Major development:
Project Developer
======= =========
IPv6_2 Meno Abels
-Data encryption Meno Abels (see projects)
-Communication encryption Meno Abels
Version 1.37 Kern (see below)
========================================================
-For 1.36.1:
-- Fix orphanned buffers:
- Orphaned buffer: 24 bytes allocated at line 808 of rufus-dir job.c
- Orphaned buffer: 40 bytes allocated at line 45 of rufus-dir alist.c
-- Add dump of VolSessionId/Time and FileIndex with bls.
-
-1.37 Projects:
+1.37 Major Projects:
#3 Migration (Move, Copy, Archive Jobs)
-#4 Embedded Python Scripting
-#5 Events that call a Python program
+#4 Embedded Python Scripting
+ (Implemented in Dir/SD)
+#5 Events that call a Python program
+ (Implemented in Dir/SD)
#6 Select one from among Multiple Storage Devices for Job
#7 Single Job Writing to Multiple Storage Devices
-Regression tests:
+## Integrate web-bacula into a new Bacula project with
+ bimagemgr.
+## Consider moving docs to their own project.
+
+Suggestions for Preben:
+- Look at adding Client run command that will use the
+ port opened by the client.
+- Implement WildFile and WildDir to solve problem of
+ saving only *.doc files.
+- Bootstrap from JobMedia records.
+- Optimized bootstrap.
+
+For 1.37:
+- Device resource needs the "name" of the SD.
+- Add and option to see if the file size changed
+ during backup.
+- Implement "update device" from SD so that DIR will
+ always have current version of device.
+- Add disk seeking on restore.
+- Add Python writable variable for changing the Priority,
+ Client, Storage, JobStatus (error), ...
+- SD Autochanger work
+ - Lock all devices when using changer script.
+ - Check if Volume is mounted on another device
+ - Find a free drive if Changer name used.
+- SD Python
+ - Solicit Events
+- FD Python
+ - Python script to save with Python, not save, save with Bacula.
+ - Python script to do backup.
+- Windows restore:
+ data-fd: RestoreFiles.2004-12-07_15.56.42 Error:
+ > ..\findlib\../../findlib/create_file.c:275 Could not open e:/: ERR=Der
+ > Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen
+ > Prozess verwendet wird.
+ Restore restores all files, but then fails at the end.
+- Add better documentation on how restores can be done
+ from failed jobs.
+- Resolve the problem between Device name and Archive name,
+ and fix SD messages.
+- Make sure SD deletes spool files on error exit.
+- Delete old spool files when SD starts.
+- When Python creates a new label, the tape is immediately
+ recycled and no label created. This happens when using
+ autolabeling -- even when Python doesn't generate the name.
+- Add a restore directory-x
+- When labeling tapes, if you enter 000026, Bacula uses
+ the tape index rather than the Volume name 000026.
+- Max Vols limit in Pool off by one?
+- Look at Preben's acl.c error handling code.
+- See multiple-store.txt for Multiple Storage implementation
+ design.
+- Create a new GUI chapter explaining all the GUI programs.
+- Tell the "restore" user when browsing is no longer possible.
+- Cancel command should include JobId in list of Jobs.
+- Require restore via the restore command or make a restore Job
+ get the bootstrap file.
+- Make bootstrap file handle multiple MediaTypes (SD)
+- Add offline tape command to Bacula console.
+- Add performance testing hooks
+- Document that Bootstrap files can be written with cataloging
+ turned off.
+- Look at adding full Volume and Pool information to a Volume
+ label so that bscan can get *all* the info.
+- Scratch Pool where the volumes can be re-assigned to any Pool.
+- Upgrade to MySQL 4.1.1 See:
+ http://dev.mysql.com/doc/mysql/en/Server_SQL_mode.html
+- Bug:
+ Enter MediaId or Volume name: 32
+ Enter new Volume name: DLT-20Dec04
+ Automatically selected Pool: Default
+ Connecting to Storage daemon DLTDrive at 192.168.68.104:9103 ...
+ Sending relabel command from "DLT-28Jun03" to "DLT-20Dec04" ...
+ block.c:552 Write error at 0:0 on device /dev/nst0. ERR=Bad file descriptor.
+ Error writing final EOF to tape. This tape may not be readable.
+ dev.c:1207 ioctl MTWEOF error on /dev/nst0. ERR=Permission denied.
+ askdir.c:219 NULL Volume name. This shouldn't happen!!!
+ 3912 Failed to label Volume: ERR=dev.c:1207 ioctl MTWEOF error on /dev/nst0. ERR=Permission denied.
+
+ Label command failed for Volume DLT-20Dec04.
+ Do not forget to mount the drive!!!
+- Bug: if a job is manually scheduled to run later, it does not appear
+ in any status report and cannot be cancelled.
+
+Regression tests (Jo):
- Add Pool/Storage override regression test.
- Add delete JobId to regression.
- Add a regression test for dbcheck.
- Add database test to regression. Test each function like delete,
purge, ...
+- AntiVir can slow down backups on Win32 systems.
+- Win32 systems with FAT32 can be much slower than NTFS for
+ more than 1000 files per directory.
+
1.37 Possibilities:
-- Require restore via the restore command or make a restore Job
- get the bootstrap file.
+=== From Carsten Menke <bootsy52@gmx.net>
+The bacula-devel list is CC'ed (I'm also subscribed to the list right now,
+so I will also get the responses).
+
+Following is a list of what I think in the situations where I'm faced with,
+could be a usefull enhancement to bacula, which I'm certain other users will
+benefit from as well.
+
+1. NextJob/NextJobs Directive within a Job Resource in the form of
+ NextJobs = job1,job2.
+
+ Why:
+ I currently solved the problem with running multiple jobs each after each
+ by setting the Max Wait Time for a job to 8 hours, and give
+ the jobs different Priorities. However, there scenarios where
+ 1 Job is directly depending on another job, so if the former job fails,
+ the job after it needn't to be run
+ while maybe other jobs should run despite of that
+
+Example:
+ A Backup Job and a Verify job, if the backup job fails there is no need to run
+ the verify job, as the backup job already failed. However, one may like
+ to backup the Catalog to disk despite of that the main backup job failed.
+
+Notes:
+ I see that this is related to the Event Handlers which are on the ToDo
+ list, also it is maybe a good idea to check for the return value and
+ execute different actions based on the return value
+
+2. Priorities also for the Schedule Resource
+
+ Why:
+
+ With this it is possible to run monthly jobs at the end of the month
+ instead of the beginning of the next month, so one could define
+ weekly and monthly pools to be run on Fridays and if there are
+ 2 Jobs resulting on the same time, the one with the higher priority
+ takes precedence. This is also helpfull to implement a daily
+ rotation scheme with an additional 2 weeks changing weekly pool
+
+Example:
+ Schedule {
+ Name = "NightlyBackup"
+ Run = Priority = 1,Pool=Weekly friday at 21:00
+ Run = Priority = 2,Pool=Weekly monthly on friday at 21:00
+ Run = Priority = 3,Pool=Weekly yearly on friday at 21:00
+ }
+
+3. offline capability to bconsole
+
+ Why:
+ Currently I use a script which I execute within the last Job via the
+ RunAfterJob Directive, to release and eject the tape.
+ So I have to call bconsole "release=Storage-Name" and afterwards
+ mt -f /dev/nst0 eject to get the tape out.
+
+ If I have multiple Storage Devices, than these may not be /dev/nst0 and
+ I have to modify the script or call it with parameters etc.
+ This would actually not be needed, as everything is already defined
+ in bacula-sd.conf and if I can invoke bconsole with the
+ storage name via $1 in the script than I'm done and information is
+ not duplicated.
+
+4. %s for Storage Name added to the chars being substituted in "RunAfterJob"
+
+ Why:
+
+ For the reason mentioned in 3. to have the ability to call a
+ script with /scripts/foobar %s and in the script use $1
+ to pass the Storage Name to bconsole
+
+5. Setting Volume State within a Job Resource
+
+ Why:
+ Instead of using "Maximum Volume Jobs" in the Pool Resource,
+ I would have the possibilty to define
+ in a Job Resource that after this certain job is run, the Volume State
+ should be set to "Volume State = Used", this give more flexibility (IMHO).
+
+6. Localization of Bacula Messages
+
+ Why:
+ Unfortunatley many,many people I work with don't speak english very well.
+ So if at least the Reporting messages would be localized then they
+ would understand that they have to change the tape,etc. etc.
+
+ I volunteer to do the german translations, and if I can convince my wife also
+ french and Morre (western african language).
+
+7. OK, this is evil, probably bound to security risks and maybe not possible
+ due to the design of bacula.
+
+ Implementation of Backtics ( `command` ) for shell comand execution to
+ the "Label Format" Directive.
+
+Why:
+
+ Currently I have defined BACULA_DAY_OF_WEEK="day1|day2..." resulting in
+ Label Format = "HolyBackup-${BACULA_DAY_OF_WEEK[${WeekDay}]}". If I could
+ use backticks than I could use "Label Format = HolyBackup-`date +%A` to have
+ the localized name for the day of the week appended to the
+ format string. Then I have the tape labeled automatically with weekday
+ name in the correct language.
+==========
+- Can one write tapes faster with 8192 byte block sizes?
+- Specify a single directory to restore.
+- Implement Maximum Job Spool Size
+- Document security problems with the same password for everyone in
+ rpm and Win32 releases.
+- Browse generations of files.
+- I've seen an error when my catalog's File table fills up. I
+ then have to recreate the File table with a larger maximum row
+ size. Relevant information is at
+ http://dev.mysql.com/doc/mysql/en/Full_table.html ; I think the
+ "Installing and Configuring MySQL" chapter should talk a bit
+ about this potential problem, and recommend a solution.
+- For Solaris must use POSIX awk.
+- Want speed of writing to tape while despooling.
+- Supported autochanger:
+OS: Linux
+Man.: HP
+Media: LTO-2
+Model: SSL1016
+Slots: 16
+Cap: 200GB
+- Supported drive:
+ Wangtek 6525ES (SCSI-1 QIC drive, 525MB), under Linux 2.4.something,
+ bacula 1.36.0/1 works with blocksize 16k INSIDE bacula-sd.conf.
+- Add regex from http://www.pcre.org to Bacula for Win32.
- Use only shell tools no make in CDROM package.
- Document PostgreSQL performance problems bug 131.
- Document testing
-- Add performance testing hooks
-- if 2 concurrent backups are attempted on the same tape
- drive (autoloader) into different tape pools, one of them will exit
- fatally instead of halting until the drive is idle
-- Add seconds to start and end times in the Job report output.
-- Add disk seeking on restore.
- Document that ChangerDevice is used for Alert command.
- Include within include does it work?
- Implement a Pool of type Cleaning?
- Implement VolReadTime and VolWriteTime in SD
- Modify Backing up Your Database to include a bootstrap file.
- Think about making certain database errors fatal.
-- Look at correcting the time jump in the sceduler for daylight
+- Look at correcting the time jump in the scheduler for daylight
savings time changes.
- Add a "real" timer to network connections.
-- Tell the "restore" user when browsing is no longer possible.
- Write non-optimized bsrs from the JobMedia and Media records,
even after Files are pruned.
-- Document that Bootstrap files can be written with cataloging
- turned off.
-- Look at adding full Volume and Pool information to a Volume
- label so that bscan can get *all* the info.
- Promote to Full = Time period
-- Scratch Pool where the volumes can be re-assigned to any Pool.
-- Update StartTime if job held in Job Queue.
- Despool attributes simultaneously with data in a separate
thread, rejoined at end of data spooling.
- Implement Files/Bytes,... stats for restore job.
- Minimal autochanger handling in Bacula and in btape.
- Look into how tar does not save sockets and the possiblity of
not saving them in Bacula (Martin Simmons reported this).
-- Add All Local Partitions = yes to new style saves.
-- localmounts=`awk '/ext/ { print $2 }' /proc/mounts` # or whatever
- find $localmounts -xdev -type s -ls
- Fix restore jobs so that multiple jobs can run if they
are not using the same tape(s).
- Allow the user to select JobType for manual pruning/purging.
-- Look at adding Client run command that will use the
- port opened by the client.
- bscan does not put first of two volumes back with all info in
bscan-test.
- Implement the FreeBSD nodump flag in chflags.
perhaps if password is undefined.
- Implement "from ISO-date/time every x hours/days/weeks/months" in
schedules.
+=== rate design
+ jcr->last_rate
+ jcr->last_runtime
+ MA = (last_MA * 3 + rate) / 4
+ rate = (bytes - last_bytes) / (runtime - last_runtime)
+
==== from Marc Schoechlin
- the help-command should be more verbose
(it should explain the paramters of the different
- any actions should be interuptable with STRG+C
- command-expansion would be pretty cool
====
+- When the replace Never option is set, new directory permissions
+ are not restored. See bug 213. To fix this requires creating a
+ list of newly restored directories so that those directory
+ permissions *can* be restored.
- Compaction of Disk space by "migrating" Volumes that have pruned
Jobs (what criteria? size, #jobs, time).
- Add prune all command
is contained in only one device.
In SD allow Device to have Multiple MediaTypes
-- Look at www.nu2.nu/pebuilder as a helper for full windows
- bare metal restore.
- Ideas from Jerry Scharf:
First let's point out some big pluses that bacula has for this
it's open source
=== Done
-- Add "Rerun failed levels = yes/no" to Job resource.
-- Fix CDROM make script to permit picking up already installed
- static FD.
-- Add date/time to each Jmsg.
+- Save mount point for directories not traversed with onefs=yes.
+- Add seconds to start and end times in the Job report output.
+- if 2 concurrent backups are attempted on the same tape
+ drive (autoloader) into different tape pools, one of them will exit
+ fatally instead of halting until the drive is idle
+- Update StartTime if job held in Job Queue.
+- Look at www.nu2.nu/pebuilder as a helper for full windows
+ bare metal restore. (done by Scott)
+- Fix orphanned buffers:
+ Orphaned buffer: 24 bytes allocated at line 808 of rufus-dir job.c
+ Orphaned buffer: 40 bytes allocated at line 45 of rufus-dir alist.c
+- Implement Preben's suggestion to add
+ File System Types = ext2, ext3
+ to FileSets, thus simplifying backup of *all* local partitions.
+- Try to open a device on each Job if it was not opened
+ when the SD started.
+- Add dump of VolSessionId/Time and FileIndex with bls.
projects / bacula / bacula / blobdiff