stoppostgre
startpostgre
kerns-postgresql-config
+run
+run_clean
Kern's ToDo List
- 28 February 2004
+ 9 March 2004
Documentation to do: (any release a little bit at a time)
- DB upgrade to version 5 in bacula-1.27b, DB upgrade to
Overland Neo2000 (LVD)
For 1.33
+From Chris Hull:
+ it seems to be complaining about 12:00pm which should be a valid 12
+ hour time. I changed the time to 11:59am and everything works fine.
+ Also 12:00am works fine. 0:00pm also works (which I don't think
+ should). None of the values 12:00pm - 12:59pm work for that matter.
- Use SHA1 on authentication if possible.
- Check: Run = Level=Differential feb-dec 1 at 1:05 to see if wday is empty.
- Look at installation file permissions with Scott so that make install
it cannot be called recursively and set dequeuing flag.
- Add a .list all files in the restore tree (probably also a list all files)
Do both a long and short form.
+- Finish work on conio.c
+- See comtest-xxx.zip for Windows code to talk to USB.
+- Phil says that Windows file sizes mismatch in Verify when they should,
+ and that either the file size or the catalog size was zero.
+- Check time/dates printed during restore when using Win32 API.
+- Add ctl-c to console to stop current command and discard buffered
+ output.
+- Estimate to Tibs never returns.
- Later:
+- Add Events and Perl scripting.
- Add John's appended files:
Appended = { /files/server/logs/http/*log }
and such files would be treated as follows.On a FULL backup, they would
- Add "limit=n" for "list jobs"
- Check new HAVE_WIN32 open bits.
- Check if the tape has moved before writing.
-- Implement restart of daemon.
- Handling removable disks -- see below:
- Multiple drive autochanger support -- see below.
- Keep track of tape use time, and report when cleaning is necessary.
-- Add Events and Perl scripting.
-- See comtest-xxx.zip for Windows code to talk to USB.
- Fix FreeBSD mt_count problem.
-- During install, copy any console.conf to bconsole.conf.
- Add FromClient and ToClient keywords on restore command (or
BackupClient RestoreClient).
- Automatic "update slots" on user configuration directive when a
those of the time of the restore, not those of the originalfile.
The dates you will find in your restore log seem to be the original
creation dates
-- Check why Phil's Verify exclude does not work.
-- Phil says that Windows file sizes mismatch in Verify when they should,
- and that either the file size or the catalog size was zero.
-- Check time/dates printed during restore when using Win32 API.
- Volume "add"ed to Pool gets recycled in first use. VolBytes=0
- Get rid of 0 dates in LastWritten, ...
- If a tape is recycled while it is mounted, Stanislav Tvrudy must do an
pointer (rec) to try_repositioning().
- Look at extracting Win data from BackupRead.
- Having dashes in filenames apparently creates problems for restore
- by filename.
+ by filename??? hard to believe.
- Implement RestoreJobRetention? Maybe better "JobRetention" in a Job,
which would take precidence over the Catalog "JobRetention".
- Implement Label Format in Add and Label console commands.
- Possibly up network buffers to 65K. Put on variable.
- Put email tape request delays on one or more variables. User wants
to cancel the job after a certain time interval. Maximum Mount Wait?
- Job, Client, Device, Pool, or Volume?
+- Job, Client, Device, Pool, or Volume?
Is it possible to make this a directive which is *optional* in multiple
resources, like Level? If so, I think I'd make it an optional directive
in Job, Client, and Pool, with precedence such that Job overrides Client
which in turn overrides Pool.
- Print a message when a job starts if the conf file is not current.
-- Finish work on conio.c
- To pass Include 1 or two letter commands
I Name Include name - first record
B Name Base name - repeat
After 1.33:
- Look at www.nu2.nu/pebuilder as a helper for full windows
bare metal restore.
-From Chris Hull:
- it seems to be complaining about 12:00pm which should be a valid 12
- hour time. I changed the time to 11:59am and everything works fine.
- Also 12:00am works fine. 0:00pm also works (which I don't think
- should). None of the values 12:00pm - 12:59pm work for that matter.
Ideas from Jerry Scharf:
First let's point out some big pluses that bacula has for this
it's open source
http://csrc.nist.gov/CryptoToolkit/aes/). It's
an officially adopted standard, has survived peer
review, and provides keys up to 256 bits.
-- Add ctl-c to console to stop current command and discard buffered
- output.
-- Estimate to Tibs never returns.
- Think about how space could be freed up on a tape -- perhaps this
is a Merge or Compact feature that is needed.
- Modify FileSet, did not upgrade the current Increment job, but
- Add data compare on write/read in btape "test".
- Rescue builds incorrect script files on Rufus.
- Release SQLite 2.8.9
-
+- During install, copy any console.conf to bconsole.conf.
mdb->pnl = 1;
}
- Dmsg2(100, "sllit path=%s file=%s\n", mdb->path, mdb->fname);
+ Dmsg2(400, "split path=%s file=%s\n", mdb->path, mdb->fname);
}
/*
Mmsg(&mdb->cmd, "SELECT JobId, MediaId FROM JobMedia WHERE \
JobId=%d AND MediaId=%d", jm->JobId, jm->MediaId);
- Dmsg0(30, mdb->cmd);
+ Dmsg0(300, mdb->cmd);
if (QUERY_DB(jcr, mdb, mdb->cmd)) {
mdb->num_rows = sql_num_rows(mdb);
if (mdb->num_rows > 0) {
jm->JobId, jm->MediaId, jm->FirstIndex, jm->LastIndex,
jm->StartFile, jm->EndFile, jm->StartBlock, jm->EndBlock,count);
- Dmsg0(30, mdb->cmd);
+ Dmsg0(300, mdb->cmd);
if (!INSERT_DB(jcr, mdb, mdb->cmd)) {
Mmsg2(&mdb->errmsg, _("Create db JobMedia record %s failed. ERR=%s\n"), mdb->cmd,
sql_strerror(mdb));
stat = 1;
}
db_unlock(mdb);
- Dmsg0(30, "Return from JobMedia\n");
+ Dmsg0(300, "Return from JobMedia\n");
return stat;
}
db_lock(mdb);
Mmsg(&mdb->cmd, "SELECT MediaId FROM Media WHERE VolumeName='%s'",
mr->VolumeName);
- Dmsg1(110, "selectpool: %s\n", mdb->cmd);
+ Dmsg1(300, "selectpool: %s\n", mdb->cmd);
if (QUERY_DB(jcr, mdb, mdb->cmd)) {
mdb->num_rows = sql_num_rows(mdb);
int db_create_file_attributes_record(JCR *jcr, B_DB *mdb, ATTR_DBR *ar)
{
- Dmsg1(100, "Fname=%s\n", ar->fname);
- Dmsg0(50, "put_file_into_catalog\n");
+ Dmsg1(300, "Fname=%s\n", ar->fname);
+ Dmsg0(500, "put_file_into_catalog\n");
/*
* Make sure we have an acceptable attributes record.
*/
}
Dmsg0(500, "db_create_file_record OK\n");
- Dmsg3(100, "CreateAttributes Path=%s File=%s FilenameId=%d\n", mdb->path, mdb->fname, ar->FilenameId);
+ Dmsg3(300, "CreateAttributes Path=%s File=%s FilenameId=%d\n", mdb->path, mdb->fname, ar->FilenameId);
db_unlock(mdb);
return 1;
}
/* Look to see if DB already open */
for (mdb=NULL; (mdb=(B_DB *)qnext(&db_list, &mdb->bq)); ) {
if (strcmp(mdb->db_name, db_name) == 0) {
- Dmsg2(100, "DB REopen %d %s\n", mdb->ref_count, db_name);
+ Dmsg2(300, "DB REopen %d %s\n", mdb->ref_count, db_name);
mdb->ref_count++;
V(mutex);
return mdb; /* already open */
}
}
- Dmsg0(100, "db_open first time\n");
+ Dmsg0(300, "db_open first time\n");
mdb = (B_DB *) malloc(sizeof(B_DB));
memset(mdb, 0, sizeof(B_DB));
mdb->db_name = bstrdup(db_name);
644, /* mode */
&mdb->sqlite_errmsg); /* error message */
- Dmsg0(50, "sqlite_open\n");
+ Dmsg0(300, "sqlite_open\n");
if (mdb->db == NULL) {
Mmsg2(&mdb->errmsg, _("Unable to open Database=%s. ERR=%s\n"),
bsendmsg(ua, " %s\n", ua->prompt[i]);
free(ua->prompt[i]);
}
+ if (ua->num_prompts == 0) {
+ bsendmsg(ua, _("No Volumes found to restore.\n"));
+ stat = 0;
+ }
ua->num_prompts = 0;
bsendmsg(ua, "\n");
free_pool_memory(fname);
&sdmr.VolWriteTime) == 17) {
db_lock(jcr->db);
- Dmsg3(100, "Update media %s oldStat=%s newStat=%s\n", sdmr.VolumeName,
+ Dmsg3(300, "Update media %s oldStat=%s newStat=%s\n", sdmr.VolumeName,
mr.VolStatus, sdmr.VolStatus);
bstrncpy(mr.VolumeName, sdmr.VolumeName, sizeof(mr.VolumeName)); /* copy Volume name */
unbash_spaces(mr.VolumeName);
mr.FirstWritten = jcr->start_time; /* use Job start time as first write */
}
/* If we just labeled the tape set time */
- Dmsg2(100, "label=%d labeldate=%d\n", label, mr.LabelDate);
+ Dmsg2(300, "label=%d labeldate=%d\n", label, mr.LabelDate);
if (label || mr.LabelDate == 0) {
mr.LabelDate = time(NULL);
}
- Dmsg2(100, "Update media: BefVolJobs=%u After=%u\n", mr.VolJobs, sdmr.VolJobs);
+ Dmsg2(300, "Update media: BefVolJobs=%u After=%u\n", mr.VolJobs, sdmr.VolJobs);
/* Copy updated values to original media record */
mr.VolJobs = sdmr.VolJobs;
mr.VolFiles = sdmr.VolFiles;
mr.VolWriteTime = sdmr.VolWriteTime;
bstrncpy(mr.VolStatus, sdmr.VolStatus, sizeof(mr.VolStatus));
- Dmsg2(100, "db_update_media_record. Stat=%s Vol=%s\n", mr.VolStatus, mr.VolumeName);
+ Dmsg2(300, "db_update_media_record. Stat=%s Vol=%s\n", mr.VolStatus, mr.VolumeName);
/*
* Check if it has expired, and if not update the DB. Note, if
* Volume has expired, has_volume_expired() will update the DB.
jm.JobId = jcr->JobId;
jm.MediaId = jcr->MediaId;
- Dmsg6(100, "create_jobmedia JobId=%d MediaId=%d SF=%d EF=%d FI=%d LI=%d\n",
+ Dmsg6(300, "create_jobmedia JobId=%d MediaId=%d SF=%d EF=%d FI=%d LI=%d\n",
jm.JobId, jm.MediaId, jm.StartFile, jm.EndFile, jm.FirstIndex, jm.LastIndex);
if (!db_create_jobmedia_record(jcr, jcr->db, &jm)) {
Jmsg(jcr, M_ERROR, 0, _("Catalog error creating JobMedia record. %s"),
db_strerror(jcr->db));
bnet_fsend(bs, "1991 Update JobMedia error\n");
} else {
- Dmsg0(100, "JobMedia record created\n");
+ Dmsg0(300, "JobMedia record created\n");
bnet_fsend(bs, OK_create);
}
Jmsg1(jcr, M_ERROR, 0, _("Invalid Catalog request: %s"), omsg);
free_memory(omsg);
}
- Dmsg1(120, ">CatReq response: %s", bs->msg);
+ Dmsg1(300, ">CatReq response: %s", bs->msg);
Dmsg1(200, "Leave catreq jcr 0x%x\n", jcr);
return;
}
unser_uint32(data_len);
p += unser_length(p);
- Dmsg1(99, "UpdCat msg=%s\n", bs->msg);
- Dmsg5(99, "UpdCat VolSessId=%d VolSessT=%d FI=%d Strm=%d data_len=%d\n",
+ Dmsg1(300, "UpdCat msg=%s\n", bs->msg);
+ Dmsg5(300, "UpdCat VolSessId=%d VolSessT=%d FI=%d Strm=%d data_len=%d\n",
VolSessionId, VolSessionTime, FileIndex, Stream, data_len);
if (Stream == STREAM_UNIX_ATTRIBUTES || Stream == STREAM_UNIX_ATTRIBUTES_EX) {
len = strlen(fname); /* length before attributes */
attr = &fname[len+1];
- Dmsg2(109, "dird<stored: stream=%d %s\n", Stream, fname);
- Dmsg1(109, "dird<stored: attr=%s\n", attr);
+ Dmsg2(300, "dird<stored: stream=%d %s\n", Stream, fname);
+ Dmsg1(300, "dird<stored: attr=%s\n", attr);
ar.attr = attr;
ar.fname = fname;
ar.FileIndex = FileIndex;
ar.link = NULL;
ar.JobId = jcr->JobId;
- Dmsg2(111, "dird<filed: stream=%d %s\n", Stream, fname);
+ Dmsg2(300, "dird<filed: stream=%d %s\n", Stream, fname);
Dmsg1(120, "dird<filed: attr=%s\n", attr);
if (!db_create_file_attributes_record(jcr, jcr->db, &ar)) {
{"prunefiles", store_yesno, ITEM(res_job.PruneFiles), 1, ITEM_DEFAULT, 0},
{"prunevolumes",store_yesno, ITEM(res_job.PruneVolumes), 1, ITEM_DEFAULT, 0},
{"spoolattributes",store_yesno, ITEM(res_job.SpoolAttributes), 1, ITEM_DEFAULT, 0},
+ {"spooldata", store_yesno, ITEM(res_job.spool_data), 1, ITEM_DEFAULT, 0},
{"runbeforejob", store_str, ITEM(res_job.RunBeforeJob), 0, 0, 0},
{"runafterjob", store_str, ITEM(res_job.RunAfterJob), 0, 0, 0},
{"runafterfailedjob", store_str, ITEM(res_job.RunAfterFailedJob), 0, 0, 0},
res->res_job.hdr.name, res->res_job.JobType,
level_to_str(res->res_job.level), res->res_job.Priority,
res->res_job.MaxConcurrentJobs);
- sendit(sock, " Resched=%d Times=%d Interval=%s\n",
+ sendit(sock, " Resched=%d Times=%d Interval=%s Spool=%d\n",
res->res_job.RescheduleOnError, res->res_job.RescheduleTimes,
- edit_uint64_with_commas(res->res_job.RescheduleInterval, ed1));
+ edit_uint64_with_commas(res->res_job.RescheduleInterval, ed1),
+ res->res_job.spool_data);
if (res->res_job.client) {
sendit(sock, " --> ");
dump_resource(-R_CLIENT, (RES *)res->res_job.client, sendit, sock);
};
#define R_FIRST R_DIRECTOR
-#define R_LAST R_JOBDEFS
+#define R_LAST R_JOBDEFS
/*
* Some resource attributes
*/
-#define R_NAME 1020
-#define R_ADDRESS 1021
-#define R_PASSWORD 1022
-#define R_TYPE 1023
-#define R_BACKUP 1024
+#define R_NAME 1020
+#define R_ADDRESS 1021
+#define R_PASSWORD 1022
+#define R_TYPE 1023
+#define R_BACKUP 1024
/* Used for certain KeyWord tables */
-struct s_kw {
+struct s_kw {
char *name;
- int token;
+ int token;
};
/* Job Level keyword structure */
struct s_jl {
- char *level_name; /* level keyword */
- int level; /* level */
- int job_type; /* JobType permitting this level */
+ char *level_name; /* level keyword */
+ int level; /* level */
+ int job_type; /* JobType permitting this level */
};
/* Job Type keyword structure */
struct RUN;
/*
- * Director Resource
+ * Director Resource
*
*/
struct DIRRES {
- RES hdr;
- int DIRport; /* where we listen -- UA port server port */
- char *DIRaddr; /* bind address */
- char *password; /* Password for UA access */
- int enable_ssl; /* Use SSL for UA */
- char *query_file; /* SQL query file */
- char *working_directory; /* WorkingDirectory */
- char *pid_directory; /* PidDirectory */
- char *subsys_directory; /* SubsysDirectory */
- int require_ssl; /* Require SSL for all connections */
- MSGS *messages; /* Daemon message handler */
- uint32_t MaxConcurrentJobs; /* Max concurrent jobs for whole director */
- utime_t FDConnectTimeout; /* timeout for connect in seconds */
- utime_t SDConnectTimeout; /* timeout in seconds */
+ RES hdr;
+ int DIRport; /* where we listen -- UA port server port */
+ char *DIRaddr; /* bind address */
+ char *password; /* Password for UA access */
+ int enable_ssl; /* Use SSL for UA */
+ char *query_file; /* SQL query file */
+ char *working_directory; /* WorkingDirectory */
+ char *pid_directory; /* PidDirectory */
+ char *subsys_directory; /* SubsysDirectory */
+ int require_ssl; /* Require SSL for all connections */
+ MSGS *messages; /* Daemon message handler */
+ uint32_t MaxConcurrentJobs; /* Max concurrent jobs for whole director */
+ utime_t FDConnectTimeout; /* timeout for connect in seconds */
+ utime_t SDConnectTimeout; /* timeout in seconds */
};
Command_ACL,
FileSet_ACL,
Catalog_ACL,
- Num_ACL /* keep last */
+ Num_ACL /* keep last */
};
/*
* Console Resource
*/
struct CONRES {
- RES hdr;
- char *password; /* UA server password */
- int enable_ssl; /* Use SSL */
- alist *ACL_lists[Num_ACL]; /* pointers to ACLs */
+ RES hdr;
+ char *password; /* UA server password */
+ int enable_ssl; /* Use SSL */
+ alist *ACL_lists[Num_ACL]; /* pointers to ACLs */
};
*
*/
struct CAT {
- RES hdr;
+ RES hdr;
- int db_port; /* Port -- not yet implemented */
- char *db_address; /* host name for remote access */
- char *db_socket; /* Socket for local access */
+ int db_port; /* Port -- not yet implemented */
+ char *db_address; /* host name for remote access */
+ char *db_socket; /* Socket for local access */
char *db_password;
char *db_user;
char *db_name;
*
*/
struct CLIENT {
- RES hdr;
+ RES hdr;
- int FDport; /* Where File daemon listens */
- int AutoPrune; /* Do automatic pruning? */
- utime_t FileRetention; /* file retention period in seconds */
- utime_t JobRetention; /* job retention period in seconds */
+ int FDport; /* Where File daemon listens */
+ int AutoPrune; /* Do automatic pruning? */
+ utime_t FileRetention; /* file retention period in seconds */
+ utime_t JobRetention; /* job retention period in seconds */
char *address;
char *password;
- CAT *catalog; /* Catalog resource */
- uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
- uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
- int enable_ssl; /* Use SSL */
+ CAT *catalog; /* Catalog resource */
+ uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
+ uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
+ int enable_ssl; /* Use SSL */
};
/*
*
*/
struct STORE {
- RES hdr;
+ RES hdr;
- int SDport; /* port where Directors connect */
- int SDDport; /* data port for File daemon */
+ int SDport; /* port where Directors connect */
+ int SDDport; /* data port for File daemon */
char *address;
char *password;
char *media_type;
char *dev_name;
- int autochanger; /* set if autochanger */
- uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
- uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
- int enable_ssl; /* Use SSL */
+ int autochanger; /* set if autochanger */
+ uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
+ uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
+ int enable_ssl; /* Use SSL */
};
* Job Resource
*/
struct JOB {
- RES hdr;
-
- int JobType; /* job type (backup, verify, restore */
- int level; /* default backup/verify level */
- int Priority; /* Job priority */
- int RestoreJobId; /* What -- JobId to restore */
- char *RestoreWhere; /* Where on disk to restore -- directory */
- char *RestoreBootstrap; /* Bootstrap file */
- char *RunBeforeJob; /* Run program before Job */
- char *RunAfterJob; /* Run program after Job */
- char *RunAfterFailedJob; /* Run program after Job that errs */
- char *ClientRunBeforeJob; /* Run client program before Job */
- char *ClientRunAfterJob; /* Run client program after Job */
- char *WriteBootstrap; /* Where to write bootstrap Job updates */
- int replace; /* How (overwrite, ..) */
- utime_t MaxRunTime; /* max run time in seconds */
- utime_t MaxWaitTime; /* max blocking time in seconds */
- utime_t MaxStartDelay; /* max start delay in seconds */
- int PrefixLinks; /* prefix soft links with Where path */
- int PruneJobs; /* Force pruning of Jobs */
- int PruneFiles; /* Force pruning of Files */
- int PruneVolumes; /* Force pruning of Volumes */
- int SpoolAttributes; /* Set to spool attributes in SD */
- uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
- int RescheduleOnError; /* Set to reschedule on error */
- int RescheduleTimes; /* Number of times to reschedule job */
- utime_t RescheduleInterval; /* Reschedule interval */
- utime_t JobRetention; /* job retention period in seconds */
+ RES hdr;
+
+ int JobType; /* job type (backup, verify, restore */
+ int level; /* default backup/verify level */
+ int Priority; /* Job priority */
+ int RestoreJobId; /* What -- JobId to restore */
+ char *RestoreWhere; /* Where on disk to restore -- directory */
+ char *RestoreBootstrap; /* Bootstrap file */
+ char *RunBeforeJob; /* Run program before Job */
+ char *RunAfterJob; /* Run program after Job */
+ char *RunAfterFailedJob; /* Run program after Job that errs */
+ char *ClientRunBeforeJob; /* Run client program before Job */
+ char *ClientRunAfterJob; /* Run client program after Job */
+ char *WriteBootstrap; /* Where to write bootstrap Job updates */
+ int replace; /* How (overwrite, ..) */
+ utime_t MaxRunTime; /* max run time in seconds */
+ utime_t MaxWaitTime; /* max blocking time in seconds */
+ utime_t MaxStartDelay; /* max start delay in seconds */
+ int PrefixLinks; /* prefix soft links with Where path */
+ int PruneJobs; /* Force pruning of Jobs */
+ int PruneFiles; /* Force pruning of Files */
+ int PruneVolumes; /* Force pruning of Volumes */
+ int SpoolAttributes; /* Set to spool attributes in SD */
+ int spool_data; /* Set to spool data in SD */
+ uint32_t MaxConcurrentJobs; /* Maximume concurrent jobs */
+ int RescheduleOnError; /* Set to reschedule on error */
+ int RescheduleTimes; /* Number of times to reschedule job */
+ utime_t RescheduleInterval; /* Reschedule interval */
+ utime_t JobRetention; /* job retention period in seconds */
- MSGS *messages; /* How and where to send messages */
- SCHED *schedule; /* When -- Automatic schedule */
- CLIENT *client; /* Who to backup */
- FILESET *fileset; /* What to backup -- Fileset */
- STORE *storage; /* Where is device -- Storage daemon */
- POOL *pool; /* Where is media -- Media Pool */
- POOL *full_pool; /* Pool for Full backups */
- POOL *inc_pool; /* Pool for Incremental backups */
- POOL *dif_pool; /* Pool for Differental backups */
- JOB *verify_job; /* Job name to verify */
- JOB *jobdefs; /* Job defaults */
- uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
+ MSGS *messages; /* How and where to send messages */
+ SCHED *schedule; /* When -- Automatic schedule */
+ CLIENT *client; /* Who to backup */
+ FILESET *fileset; /* What to backup -- Fileset */
+ STORE *storage; /* Where is device -- Storage daemon */
+ POOL *pool; /* Where is media -- Media Pool */
+ POOL *full_pool; /* Pool for Full backups */
+ POOL *inc_pool; /* Pool for Incremental backups */
+ POOL *dif_pool; /* Pool for Differental backups */
+ JOB *verify_job; /* Job name to verify */
+ JOB *jobdefs; /* Job defaults */
+ uint32_t NumConcurrentJobs; /* number of concurrent jobs running */
};
#define MAX_FOPTS 30
/* File options structure */
struct FOPTS {
- char opts[MAX_FOPTS]; /* options string */
- alist match; /* match string(s) */
- alist base_list; /* list of base names */
+ char opts[MAX_FOPTS]; /* options string */
+ alist match; /* match string(s) */
+ alist base_list; /* list of base names */
};
/* This is either an include item or an exclude item */
struct INCEXE {
- FOPTS *current_opts; /* points to current options structure */
- FOPTS **opts_list; /* options list */
- int num_opts; /* number of options items */
- alist name_list; /* filename list -- holds char * */
+ FOPTS *current_opts; /* points to current options structure */
+ FOPTS **opts_list; /* options list */
+ int num_opts; /* number of options items */
+ alist name_list; /* filename list -- holds char * */
};
/*
*
*/
struct FILESET {
- RES hdr;
+ RES hdr;
- int new_include; /* Set if new include used */
- INCEXE **include_items; /* array of incexe structures */
- int num_includes; /* number in array */
+ int new_include; /* Set if new include used */
+ INCEXE **include_items; /* array of incexe structures */
+ int num_includes; /* number in array */
INCEXE **exclude_items;
int num_excludes;
- int have_MD5; /* set if MD5 initialized */
- struct MD5Context md5c; /* MD5 of include/exclude */
- char MD5[30]; /* base 64 representation of MD5 */
+ int have_MD5; /* set if MD5 initialized */
+ struct MD5Context md5c; /* MD5 of include/exclude */
+ char MD5[30]; /* base 64 representation of MD5 */
};
*
*/
struct SCHED {
- RES hdr;
+ RES hdr;
RUN *run;
};
* Counter Resource
*/
struct COUNTER {
- RES hdr;
-
- int32_t MinValue; /* Minimum value */
- int32_t MaxValue; /* Maximum value */
- int32_t CurrentValue; /* Current value */
- COUNTER *WrapCounter; /* Wrap counter name */
- CAT *Catalog; /* Where to store */
- bool created; /* Created in DB */
+ RES hdr;
+
+ int32_t MinValue; /* Minimum value */
+ int32_t MaxValue; /* Maximum value */
+ int32_t CurrentValue; /* Current value */
+ COUNTER *WrapCounter; /* Wrap counter name */
+ CAT *Catalog; /* Where to store */
+ bool created; /* Created in DB */
};
/*
*
*/
struct POOL {
- RES hdr;
-
- char *pool_type; /* Pool type */
- char *label_format; /* Label format string */
- char *cleaning_prefix; /* Cleaning label prefix */
- int use_catalog; /* maintain catalog for media */
- int catalog_files; /* maintain file entries in catalog */
- int use_volume_once; /* write on volume only once */
- int accept_any_volume; /* accept any volume */
- int purge_oldest_volume; /* purge oldest volume */
- int recycle_oldest_volume; /* attempt to recycle oldest volume */
- int recycle_current_volume; /* attempt recycle of current volume */
- uint32_t max_volumes; /* max number of volumes */
- utime_t VolRetention; /* volume retention period in seconds */
- utime_t VolUseDuration; /* duration volume can be used */
- uint32_t MaxVolJobs; /* Maximum jobs on the Volume */
- uint32_t MaxVolFiles; /* Maximum files on the Volume */
- uint64_t MaxVolBytes; /* Maximum bytes on the Volume */
- int AutoPrune; /* default for pool auto prune */
- int Recycle; /* default for media recycle yes/no */
+ RES hdr;
+
+ char *pool_type; /* Pool type */
+ char *label_format; /* Label format string */
+ char *cleaning_prefix; /* Cleaning label prefix */
+ int use_catalog; /* maintain catalog for media */
+ int catalog_files; /* maintain file entries in catalog */
+ int use_volume_once; /* write on volume only once */
+ int accept_any_volume; /* accept any volume */
+ int purge_oldest_volume; /* purge oldest volume */
+ int recycle_oldest_volume; /* attempt to recycle oldest volume */
+ int recycle_current_volume; /* attempt recycle of current volume */
+ uint32_t max_volumes; /* max number of volumes */
+ utime_t VolRetention; /* volume retention period in seconds */
+ utime_t VolUseDuration; /* duration volume can be used */
+ uint32_t MaxVolJobs; /* Maximum jobs on the Volume */
+ uint32_t MaxVolFiles; /* Maximum files on the Volume */
+ uint64_t MaxVolBytes; /* Maximum bytes on the Volume */
+ int AutoPrune; /* default for pool auto prune */
+ int Recycle; /* default for media recycle yes/no */
};
CONRES res_con;
CLIENT res_client;
STORE res_store;
- CAT res_cat;
- JOB res_job;
+ CAT res_cat;
+ JOB res_job;
FILESET res_fs;
SCHED res_sch;
POOL res_pool;
MSGS res_msgs;
COUNTER res_counter;
- RES hdr;
+ RES hdr;
};
/* Run structure contained in Schedule Resource */
struct RUN {
- RUN *next; /* points to next run record */
- int level; /* level override */
- int Priority; /* priority override */
+ RUN *next; /* points to next run record */
+ int level; /* level override */
+ int Priority; /* priority override */
int job_type;
- POOL *pool; /* Pool override */
- POOL *full_pool; /* Pool override */
- POOL *inc_pool; /* Pool override */
- POOL *dif_pool; /* Pool override */
- STORE *storage; /* Storage override */
- MSGS *msgs; /* Messages override */
+ bool spool_data; /* Data spooling override */
+ bool spool_data_set; /* Data spooling override given */
+ POOL *pool; /* Pool override */
+ POOL *full_pool; /* Pool override */
+ POOL *inc_pool; /* Pool override */
+ POOL *dif_pool; /* Pool override */
+ STORE *storage; /* Storage override */
+ MSGS *msgs; /* Messages override */
char *since;
int level_no;
- int minute; /* minute to run job */
- time_t last_run; /* last time run */
- time_t next_run; /* next time to run */
+ int minute; /* minute to run job */
+ time_t last_run; /* last time run */
+ time_t next_run; /* next time to run */
char hour[nbytes_for_bits(24)]; /* bit set for each hour */
char mday[nbytes_for_bits(31)]; /* bit set for each day of month */
char month[nbytes_for_bits(12)]; /* bit set for each month */
jcr->catalog = job->client->catalog;
jcr->fileset = job->fileset;
jcr->messages = job->messages;
+ jcr->spool_data = job->spool_data;
if (jcr->RestoreBootstrap) {
free(jcr->RestoreBootstrap);
jcr->RestoreBootstrap = NULL;
JCR *jcr = ((wait_pkt *)arg)->jcr;
jobq_t *jq = ((wait_pkt *)arg)->jq;
- Dmsg0(010, "Enter sched_wait.\n");
+ Dmsg0(300, "Enter sched_wait.\n");
free(arg);
time_t wtime = jcr->sched_time - time(NULL);
/* Wait until scheduled time arrives */
}
/* Check every 30 seconds if canceled */
while (wtime > 0) {
- Dmsg2(010, "Waiting on sched time, jobid=%d secs=%d\n", jcr->JobId, wtime);
+ Dmsg2(300, "Waiting on sched time, jobid=%d secs=%d\n", jcr->JobId, wtime);
if (wtime > 30) {
wtime = 30;
}
jobq_add(jq, jcr);
V(jcr->mutex);
free_jcr(jcr); /* we are done with jcr */
- Dmsg0(010, "Exit sched_wait\n");
+ Dmsg0(300, "Exit sched_wait\n");
return NULL;
}
pthread_t id;
wait_pkt *sched_pkt;
- Dmsg3(010, "jobq_add jobid=%d jcr=0x%x use_count=%d\n", jcr->JobId, jcr, jcr->use_count);
+ Dmsg3(300, "jobq_add jobid=%d jcr=0x%x use_count=%d\n", jcr->JobId, jcr, jcr->use_count);
if (jq->valid != JOBQ_VALID) {
Jmsg0(jcr, M_ERROR, 0, "Jobq_add queue not initialized.\n");
return EINVAL;
jcr->use_count++; /* mark jcr in use by us */
- Dmsg3(010, "jobq_add jobid=%d jcr=0x%x use_count=%d\n", jcr->JobId, jcr, jcr->use_count);
+ Dmsg3(300, "jobq_add jobid=%d jcr=0x%x use_count=%d\n", jcr->JobId, jcr, jcr->use_count);
if (!job_canceled(jcr) && wtime > 0) {
set_thread_concurrency(jq->max_workers + 2);
sched_pkt = (wait_pkt *)malloc(sizeof(wait_pkt));
if (job_canceled(jcr)) {
/* Add job to ready queue so that it is canceled quickly */
jq->ready_jobs->prepend(item);
- Dmsg1(010, "Prepended job=%d to ready queue\n", jcr->JobId);
+ Dmsg1(300, "Prepended job=%d to ready queue\n", jcr->JobId);
} else {
/* Add this job to the wait queue in priority sorted order */
foreach_dlist(li, jq->waiting_jobs) {
- Dmsg2(010, "waiting item jobid=%d priority=%d\n",
+ Dmsg2(300, "waiting item jobid=%d priority=%d\n",
li->jcr->JobId, li->jcr->JobPriority);
if (li->jcr->JobPriority > jcr->JobPriority) {
jq->waiting_jobs->insert_before(item, li);
- Dmsg2(010, "insert_before jobid=%d before waiting job=%d\n",
+ Dmsg2(300, "insert_before jobid=%d before waiting job=%d\n",
li->jcr->JobId, jcr->JobId);
inserted = true;
break;
/* If not jobs in wait queue, append it */
if (!inserted) {
jq->waiting_jobs->append(item);
- Dmsg1(010, "Appended item jobid=%d to waiting queue\n", jcr->JobId);
+ Dmsg1(300, "Appended item jobid=%d to waiting queue\n", jcr->JobId);
}
}
stat = start_server(jq);
pthread_mutex_unlock(&jq->mutex);
- Dmsg0(010, "Return jobq_add\n");
+ Dmsg0(300, "Return jobq_add\n");
return stat;
}
bool found = false;
jobq_item_t *item;
- Dmsg2(010, "jobq_remove jobid=%d jcr=0x%x\n", jcr->JobId, jcr);
+ Dmsg2(300, "jobq_remove jobid=%d jcr=0x%x\n", jcr->JobId, jcr);
if (jq->valid != JOBQ_VALID) {
return EINVAL;
}
}
if (!found) {
pthread_mutex_unlock(&jq->mutex);
- Dmsg2(010, "jobq_remove jobid=%d jcr=0x%x not in wait queue\n", jcr->JobId, jcr);
+ Dmsg2(300, "jobq_remove jobid=%d jcr=0x%x not in wait queue\n", jcr->JobId, jcr);
return EINVAL;
}
/* Move item to be the first on the list */
jq->waiting_jobs->remove(item);
jq->ready_jobs->prepend(item);
- Dmsg2(010, "jobq_remove jobid=%d jcr=0x%x moved to ready queue\n", jcr->JobId, jcr);
+ Dmsg2(300, "jobq_remove jobid=%d jcr=0x%x moved to ready queue\n", jcr->JobId, jcr);
stat = start_server(jq);
pthread_mutex_unlock(&jq->mutex);
- Dmsg0(010, "Return jobq_remove\n");
+ Dmsg0(300, "Return jobq_remove\n");
return stat;
}
/* if any threads are idle, wake one */
if (jq->idle_workers > 0) {
- Dmsg0(010, "Signal worker to wake up\n");
+ Dmsg0(300, "Signal worker to wake up\n");
if ((stat = pthread_cond_signal(&jq->work)) != 0) {
Jmsg1(NULL, M_ERROR, 0, "pthread_cond_signal: ERR=%s\n", strerror(stat));
return stat;
}
} else if (jq->num_workers < jq->max_workers) {
- Dmsg0(010, "Create worker thread\n");
+ Dmsg0(300, "Create worker thread\n");
/* No idle threads so create a new one */
set_thread_concurrency(jq->max_workers + 1);
if ((stat = pthread_create(&id, &jq->attr, jobq_server, (void *)jq)) != 0) {
bool timedout;
bool work = true;
- Dmsg0(010, "Start jobq_server\n");
+ Dmsg0(300, "Start jobq_server\n");
if ((stat = pthread_mutex_lock(&jq->mutex)) != 0) {
Jmsg1(NULL, M_ERROR, 0, "pthread_mutex_lock: ERR=%s\n", strerror(stat));
return NULL;
struct timeval tv;
struct timezone tz;
- Dmsg0(010, "Top of for loop\n");
+ Dmsg0(300, "Top of for loop\n");
timedout = false;
gettimeofday(&tv, &tz);
timeout.tv_nsec = 0;
/*
* Wait 4 seconds, then if no more work, exit
*/
- Dmsg0(010, "pthread_cond_timedwait()\n");
+ Dmsg0(300, "pthread_cond_timedwait()\n");
stat = pthread_cond_timedwait(&jq->work, &jq->mutex, &timeout);
if (stat == ETIMEDOUT) {
- Dmsg0(010, "timedwait timedout.\n");
+ Dmsg0(300, "timedwait timedout.\n");
timedout = true;
break;
} else if (stat != 0) {
/* This shouldn't happen */
- Dmsg0(010, "This shouldn't happen\n");
+ Dmsg0(300, "This shouldn't happen\n");
jq->num_workers--;
pthread_mutex_unlock(&jq->mutex);
return NULL;
/*
* If anything is in the ready queue, run it
*/
- Dmsg0(010, "Checking ready queue.\n");
+ Dmsg0(300, "Checking ready queue.\n");
while (!jq->ready_jobs->empty() && !jq->quit) {
JCR *jcr;
je = (jobq_item_t *)jq->ready_jobs->first();
jcr = je->jcr;
jq->ready_jobs->remove(je);
if (!jq->ready_jobs->empty()) {
- Dmsg0(010, "ready queue not empty start server\n");
+ Dmsg0(300, "ready queue not empty start server\n");
if (start_server(jq) != 0) {
jq->num_workers--;
pthread_mutex_unlock(&jq->mutex);
}
}
jq->running_jobs->append(je);
- Dmsg1(010, "Took jobid=%d from ready and appended to run\n", jcr->JobId);
+ Dmsg1(300, "Took jobid=%d from ready and appended to run\n", jcr->JobId);
if ((stat = pthread_mutex_unlock(&jq->mutex)) != 0) {
Jmsg1(NULL, M_ERROR, 0, "pthread_mutex_unlock: ERR=%s\n", strerror(stat));
jq->num_workers--;
return NULL;
}
/* Call user's routine here */
- Dmsg1(010, "Calling user engine for jobid=%d\n", jcr->JobId);
+ Dmsg1(300, "Calling user engine for jobid=%d\n", jcr->JobId);
jq->engine(je->jcr);
- Dmsg1(010, "Back from user engine jobid=%d.\n", jcr->JobId);
+ Dmsg1(300, "Back from user engine jobid=%d.\n", jcr->JobId);
if ((stat = pthread_mutex_lock(&jq->mutex)) != 0) {
Jmsg1(NULL, M_ERROR, 0, "pthread_mutex_unlock: ERR=%s\n", strerror(stat));
jq->num_workers--;
*/
jcr->reschedule_count++;
jcr->sched_time = time(NULL) + jcr->job->RescheduleInterval;
- Dmsg2(010, "Rescheduled Job %s to re-run in %d seconds.\n", jcr->Job,
+ Dmsg2(300, "Rescheduled Job %s to re-run in %d seconds.\n", jcr->Job,
(int)jcr->job->RescheduleInterval);
jcr->JobStatus = JS_Created; /* force new status */
dird_free_jcr(jcr); /* partial cleanup old stuff */
if (jcr->JobBytes == 0) {
- Dmsg1(010, "Requeue job=%d\n", jcr->JobId);
+ Dmsg1(300, "Requeue job=%d\n", jcr->JobId);
V(jq->mutex);
jobq_add(jq, jcr); /* queue the job to run again */
P(jq->mutex);
njcr->pool = jcr->pool;
njcr->store = jcr->store;
njcr->messages = jcr->messages;
- Dmsg0(010, "Call to run new job\n");
+ Dmsg0(300, "Call to run new job\n");
V(jq->mutex);
run_job(njcr); /* This creates a "new" job */
free_jcr(njcr); /* release "new" jcr */
P(jq->mutex);
- Dmsg0(010, "Back from running new job.\n");
+ Dmsg0(300, "Back from running new job.\n");
}
/* Clean up and release old jcr */
if (jcr->db) {
db_close_database(jcr, jcr->db);
jcr->db = NULL;
}
- Dmsg1(010, "====== Termination job=%d\n", jcr->JobId);
+ Dmsg1(300, "====== Termination job=%d\n", jcr->JobId);
free_jcr(jcr);
free(je); /* release job entry */
}
* If any job in the wait queue can be run,
* move it to the ready queue
*/
- Dmsg0(010, "Done check ready, now check wait queue.\n");
+ Dmsg0(300, "Done check ready, now check wait queue.\n");
while (!jq->waiting_jobs->empty() && !jq->quit) {
int Priority;
je = (jobq_item_t *)jq->waiting_jobs->first();
jobq_item_t *re = (jobq_item_t *)jq->running_jobs->first();
if (re) {
Priority = re->jcr->JobPriority;
- Dmsg2(010, "JobId %d is running. Set pri=%d\n", re->jcr->JobId, Priority);
+ Dmsg2(300, "JobId %d is running. Set pri=%d\n", re->jcr->JobId, Priority);
} else {
Priority = je->jcr->JobPriority;
- Dmsg1(010, "No job running. Set Job pri=%d\n", Priority);
+ Dmsg1(300, "No job running. Set Job pri=%d\n", Priority);
}
/*
* Walk down the list of waiting jobs and attempt
/* je is current job item on the queue, jn is the next one */
JCR *jcr = je->jcr;
jobq_item_t *jn = (jobq_item_t *)jq->waiting_jobs->next(je);
- Dmsg3(010, "Examining Job=%d JobPri=%d want Pri=%d\n",
+ Dmsg3(300, "Examining Job=%d JobPri=%d want Pri=%d\n",
jcr->JobId, jcr->JobPriority, Priority);
/* Take only jobs of correct Priority */
if (jcr->JobPriority != Priority) {
jcr->acquired_resource_locks = true;
jq->waiting_jobs->remove(je);
jq->ready_jobs->append(je);
- Dmsg1(010, "moved JobId=%d from wait to ready queue\n", je->jcr->JobId);
+ Dmsg1(300, "moved JobId=%d from wait to ready queue\n", je->jcr->JobId);
je = jn;
} /* end for loop */
break;
} /* end while loop */
- Dmsg0(010, "Done checking wait queue.\n");
+ Dmsg0(300, "Done checking wait queue.\n");
/*
* If no more ready work and we are asked to quit, then do it
*/
if (jq->ready_jobs->empty() && jq->quit) {
jq->num_workers--;
if (jq->num_workers == 0) {
- Dmsg0(010, "Wake up destroy routine\n");
+ Dmsg0(300, "Wake up destroy routine\n");
/* Wake up destroy routine if he is waiting */
pthread_cond_broadcast(&jq->work);
}
break;
}
- Dmsg0(010, "Check for work request\n");
+ Dmsg0(300, "Check for work request\n");
/*
* If no more work requests, and we waited long enough, quit
*/
- Dmsg2(010, "timedout=%d read empty=%d\n", timedout,
+ Dmsg2(300, "timedout=%d read empty=%d\n", timedout,
jq->ready_jobs->empty());
if (jq->ready_jobs->empty() && timedout) {
- Dmsg0(010, "break big loop\n");
+ Dmsg0(300, "break big loop\n");
jq->num_workers--;
break;
}
work = !jq->ready_jobs->empty() || !jq->waiting_jobs->empty();
- Dmsg1(010, "Loop again. work=%d\n", work);
+ Dmsg1(300, "Loop again. work=%d\n", work);
} /* end of big for loop */
Dmsg0(200, "unlock mutex\n");
pthread_mutex_unlock(&jq->mutex);
- Dmsg0(010, "End jobq_server\n");
+ Dmsg0(300, "End jobq_server\n");
return NULL;
}
/* Commands sent to Storage daemon */
static char jobcmd[] = "JobId=%d job=%s job_name=%s client_name=%s \
-type=%d level=%d FileSet=%s NoAttr=%d SpoolAttr=%d FileSetMD5=%s\n";
+type=%d level=%d FileSet=%s NoAttr=%d SpoolAttr=%d FileSetMD5=%s SpoolData=%d";
static char use_device[] = "use device=%s media_type=%s pool_name=%s pool_type=%s\n";
/* Response from Storage daemon */
bnet_fsend(sd, jobcmd, jcr->JobId, jcr->Job, jcr->job->hdr.name,
jcr->client->hdr.name, jcr->JobType, jcr->JobLevel,
jcr->fileset->hdr.name, !jcr->pool->catalog_files,
- jcr->job->SpoolAttributes, jcr->fileset->MD5);
+ jcr->job->SpoolAttributes, jcr->fileset->MD5, jcr->spool_data);
Dmsg1(200, "Jobcmd=%s\n", sd->msg);
unbash_spaces(jcr->job->hdr.name);
unbash_spaces(jcr->client->hdr.name);
static void restore_cleanup(JCR *jcr, int TermCode)
{
char sdt[MAX_TIME_LENGTH], edt[MAX_TIME_LENGTH];
- char ec1[30], ec2[30];
+ char ec1[30], ec2[30], ec3[30];
char term_code[100], fd_term_msg[100], sd_term_msg[100];
char *term_msg;
int msg_type;
Client: %s\n\
Start time: %s\n\
End time: %s\n\
+Files Expected: %s\n\
Files Restored: %s\n\
Bytes Restored: %s\n\
Rate: %.1f KB/s\n\
jcr->client->hdr.name,
sdt,
edt,
- edit_uint64_with_commas((uint64_t)jcr->jr.JobFiles, ec1),
- edit_uint64_with_commas(jcr->jr.JobBytes, ec2),
+ edit_uint64_with_commas((uint64_t)jcr->ExpectedFiles, ec1),
+ edit_uint64_with_commas((uint64_t)jcr->jr.JobFiles, ec2),
+ edit_uint64_with_commas(jcr->jr.JobBytes, ec3),
(float)kbps,
jcr->Errors,
fd_term_msg,
{"storage", 'S'},
{"messages", 'M'},
{"priority", 'p'},
+ {"spooldata", 's'},
{NULL, 0}
};
/* NOT REACHED */
}
switch (RunFields[i].token) {
+ case 's': /* Data spooling */
+ token = lex_get_token(lc, T_NAME);
+ if (strcasecmp(lc->str, "yes") == 0) {
+ lrun.spool_data = true;
+ lrun.spool_data_set = true;
+ } else if (strcasecmp(lc->str, "no") == 0) {
+ lrun.spool_data = false;
+ lrun.spool_data_set = true;
+ } else {
+ scan_err1(lc, _("Expect a YES or NO, got: %s"), lc->str);
+ }
+ break;
case 'L': /* level */
token = lex_get_token(lc, T_NAME);
for (j=0; joblevels[j].level_name; j++) {
if (run->Priority) {
jcr->JobPriority = run->Priority;
}
+ if (run->spool_data_set) {
+ jcr->spool_data = run->spool_data;
+ }
Dmsg0(200, "Leave wait_for_next_job()\n");
return jcr;
}
bsendmsg(ua, _("Unable to construct a valid BSR. Cannot continue.\n"));
goto bail_out;
}
- write_bsr_file(ua, rx.bsr);
+ if (!write_bsr_file(ua, rx.bsr)) {
+ goto bail_out;
+ }
bsendmsg(ua, _("\n%u file%s selected to be restored.\n\n"), rx.selected_files,
rx.selected_files==1?"":"s");
} else {
char *where, *fileset_name, *client_name, *bootstrap, *replace;
char *when, *verify_job_name;
int Priority = 0;
- int i, j, opt;
+ int i, j, opt, files = 0;
bool kw_ok;
JOB *job = NULL;
JOB *verify_job = NULL;
verify_job_name = ua->argv[i];
kw_ok = true;
break;
- case 16: /* files -- ignore for now */
+ case 16: /* files */
+ files = atoi(ua->argv[i]);
kw_ok = true;
break;
jcr->client = client;
jcr->fileset = fileset;
jcr->pool = pool;
+ jcr->ExpectedFiles = files;
if (where) {
if (jcr->where) {
free(jcr->where);
set_jcr_job_status(jcr, JS_Running);
- Dmsg1(110, "bfiled: opened data connection %d to stored\n", sd->fd);
+ Dmsg1(300, "bfiled: opened data connection %d to stored\n", sd->fd);
LockRes();
CLIENT *client = (CLIENT *)GetNextRes(R_CLIENT, NULL);
jcr->compress_buf_size = jcr->buf_size + ((jcr->buf_size+999) / 1000) + 30;
jcr->compress_buf = get_memory(jcr->compress_buf_size);
- Dmsg1(100, "set_find_options ff=%p\n", jcr->ff);
+ Dmsg1(300, "set_find_options ff=%p\n", jcr->ff);
set_find_options((FF_PKT *)jcr->ff, jcr->incremental, jcr->mtime);
- Dmsg0(110, "start find files\n");
+ Dmsg0(300, "start find files\n");
start_heartbeat_monitor(jcr);
free_pool_memory(jcr->compress_buf);
jcr->compress_buf = NULL;
}
- Dmsg1(110, "end blast_data stat=%d\n", stat);
+ Dmsg1(300, "end blast_data stat=%d\n", stat);
return stat;
}
/* Now possibly extend the attributes */
attr_stream = encode_attribsEx(jcr, attribsEx, ff_pkt);
- Dmsg3(200, "File %s\nattribs=%s\nattribsEx=%s\n", ff_pkt->fname, attribs, attribsEx);
+ Dmsg3(300, "File %s\nattribs=%s\nattribsEx=%s\n", ff_pkt->fname, attribs, attribsEx);
P(jcr->mutex);
jcr->JobFiles++; /* increment number of files sent */
set_jcr_job_status(jcr, JS_ErrorTerminated);
return 0;
}
- Dmsg1(100, ">stored: attrhdr %s\n", sd->msg);
+ Dmsg1(300, ">stored: attrhdr %s\n", sd->msg);
/*
* Send file attributes to Storage daemon
* slash. For a linked file, link is the link.
*/
if (ff_pkt->type == FT_LNK || ff_pkt->type == FT_LNKSAVED) {
- Dmsg2(100, "Link %s to %s\n", ff_pkt->fname, ff_pkt->link);
+ Dmsg2(300, "Link %s to %s\n", ff_pkt->fname, ff_pkt->link);
stat = bnet_fsend(sd, "%ld %d %s%c%s%c%s%c%s%c", jcr->JobFiles,
ff_pkt->type, ff_pkt->fname, 0, attribs, 0, ff_pkt->link, 0,
attribsEx, 0);
ff_pkt->type, ff_pkt->fname, 0, attribs, 0, 0, attribsEx, 0);
}
- Dmsg2(100, ">stored: attr len=%d: %s\n", sd->msglen, sd->msg);
+ Dmsg2(300, ">stored: attr len=%d: %s\n", sd->msglen, sd->msg);
if (!stat) {
if (is_bopen(&ff_pkt->bfd)) {
bclose(&ff_pkt->bfd);
wbuf = sd->msg; /* write buffer */
- Dmsg1(100, "Saving data, type=%d\n", ff_pkt->type);
+ Dmsg1(300, "Saving data, type=%d\n", ff_pkt->type);
#ifdef HAVE_LIBZ
set_jcr_job_status(jcr, JS_ErrorTerminated);
return 0;
}
- Dmsg1(100, ">stored: datahdr %s\n", sd->msg);
+ Dmsg1(300, ">stored: datahdr %s\n", sd->msg);
if (ff_pkt->flags & FO_MD5) {
MD5Init(&md5c);
if (gotMD5 && ff_pkt->flags & FO_MD5) {
MD5Final(signature, &md5c);
bnet_fsend(sd, "%ld %d 0", jcr->JobFiles, STREAM_MD5_SIGNATURE);
- Dmsg1(100, "bfiled>stored:header %s\n", sd->msg);
+ Dmsg1(300, "bfiled>stored:header %s\n", sd->msg);
memcpy(sd->msg, signature, 16);
sd->msglen = 16;
bnet_send(sd);
/* Terminate any SHA1 signature and send it to Storage daemon and the Director */
SHA1Final(&sha1c, signature);
bnet_fsend(sd, "%ld %d 0", jcr->JobFiles, STREAM_SHA1_SIGNATURE);
- Dmsg1(100, "bfiled>stored:header %s\n", sd->msg);
+ Dmsg1(300, "bfiled>stored:header %s\n", sd->msg);
memcpy(sd->msg, signature, 20);
sd->msglen = 20;
bnet_send(sd);
{
bfd->fid = open(fname, flags, mode);
bfd->berrno = errno;
- Dmsg1(50, "Open file %d\n", bfd->fid);
+ Dmsg1(400, "Open file %d\n", bfd->fid);
return bfd->fid;
}
int bclose(BFILE *bfd)
{
int stat;
- Dmsg1(50, "Close file %d\n", bfd->fid);
+ Dmsg1(400, "Close file %d\n", bfd->fid);
if (bfd->fid == -1) {
return 0;
}
uint32_t SDErrors; /* Number of non-fatal errors */
volatile int SDJobStatus; /* Storage Job Status */
volatile int FDJobStatus; /* File daemon Job Status */
+ uint32_t ExpectedFiles; /* Expected restore files */
B_DB *db; /* database pointer */
uint32_t MediaId; /* DB record IDs associated with this job */
uint32_t PoolId; /* Pool record id */
bool acquired_resource_locks; /* set if resource locks acquired */
int NumVols; /* Number of Volume used in pool */
int reschedule_count; /* Number of times rescheduled */
+ bool spool_data; /* Spool data in SD */
#endif /* DIRECTOR_DAEMON */
bool ignore_label_errors; /* ignore Volume label errors */
bool spool_attributes; /* set if spooling attributes */
bool no_attributes; /* set if no attributes wanted */
+ bool spool_data; /* set to spool data */
int CurVol; /* Current Volume count */
uint32_t FileId; /* Last file id inserted */
{
DCR *dcr = (DCR *)malloc(sizeof(DCR));
memset(dcr, 0, sizeof(DCR));
- jcr->dcr = dcr;
+ if (jcr) {
+ jcr->dcr = dcr;
+ }
dcr->jcr = jcr;
dcr->dev = dev;
dcr->block = new_block(dev);
if (dcr->record) {
free_record(dcr->record);
}
- dcr->jcr->dcr = NULL;
+ if (dcr->jcr) {
+ dcr->jcr->dcr = NULL;
+ }
free(dcr);
}
DEVICE *dev = dcr->dev;
JCR *jcr = dcr->jcr;
- lock_device(dev);
if (dcr->spooling) {
stat = write_block_to_spool_file(dcr, block);
- unlock_device(dev);
return stat;
}
+ lock_device(dev);
+
/*
* If a new volume has been mounted since our last write
* Create a JobMedia record for the previous volume written,
dev->dev_errno = ENODEV;
}
Emsg2(M_FATAL, 0, _("%s is an unknown device type. Must be tape or directory. st_mode=%x\n"),
- dev_name, statp.st_mode);
+ device->device_name, statp.st_mode);
return NULL;
}
if (!dev) {
/* #define NEW_LOCK 1 */
-#define new_lock_device(dev) _new_lock_device(__FILE__, __LINE__, (dev))
+#define new_lock_device(dev) _new_lock_device(__FILE__, __LINE__, (dev))
#define new_lock_device_state(dev,state) _new_lock_device(__FILE__, __LINE__, (dev), (state))
-#define new_unlock_device(dev) _new_unlock_device(__FILE__, __LINE__, (dev))
+#define new_unlock_device(dev) _new_unlock_device(__FILE__, __LINE__, (dev))
#define lock_device(d) _lock_device(__FILE__, __LINE__, (d))
#define unlock_device(d) _unlock_device(__FILE__, __LINE__, (d))
#define give_back_device_lock(d, p) _give_back_device_lock(__FILE__, __LINE__, (d), (p))
/* Arguments to open_dev() */
-#define READ_WRITE 0
-#define READ_ONLY 1
+#define READ_WRITE 0
+#define READ_ONLY 1
#define OPEN_READ_WRITE 0
-#define OPEN_READ_ONLY 1
+#define OPEN_READ_ONLY 1
#define OPEN_WRITE_ONLY 2
/* Generic status bits returned from status_dev() */
-#define BMT_TAPE (1<<0) /* is tape device */
-#define BMT_EOF (1<<1) /* just read EOF */
-#define BMT_BOT (1<<2) /* at beginning of tape */
-#define BMT_EOT (1<<3) /* end of tape reached */
-#define BMT_SM (1<<4) /* DDS setmark */
-#define BMT_EOD (1<<5) /* DDS at end of data */
-#define BMT_WR_PROT (1<<6) /* tape write protected */
-#define BMT_ONLINE (1<<7) /* tape online */
-#define BMT_DR_OPEN (1<<8) /* tape door open */
-#define BMT_IM_REP_EN (1<<9) /* immediate report enabled */
+#define BMT_TAPE (1<<0) /* is tape device */
+#define BMT_EOF (1<<1) /* just read EOF */
+#define BMT_BOT (1<<2) /* at beginning of tape */
+#define BMT_EOT (1<<3) /* end of tape reached */
+#define BMT_SM (1<<4) /* DDS setmark */
+#define BMT_EOD (1<<5) /* DDS at end of data */
+#define BMT_WR_PROT (1<<6) /* tape write protected */
+#define BMT_ONLINE (1<<7) /* tape online */
+#define BMT_DR_OPEN (1<<8) /* tape door open */
+#define BMT_IM_REP_EN (1<<9) /* immediate report enabled */
/* Test capabilities */
#define dev_cap(dev, cap) ((dev)->capabilities & (cap))
/* Bits for device capabilities */
-#define CAP_EOF (1<<0) /* has MTWEOF */
-#define CAP_BSR (1<<1) /* has MTBSR */
-#define CAP_BSF (1<<2) /* has MTBSF */
-#define CAP_FSR (1<<3) /* has MTFSR */
-#define CAP_FSF (1<<4) /* has MTFSF */
-#define CAP_EOM (1<<5) /* has MTEOM */
-#define CAP_REM (1<<6) /* is removable media */
-#define CAP_RACCESS (1<<7) /* is random access device */
-#define CAP_AUTOMOUNT (1<<8) /* Read device at start to see what is there */
-#define CAP_LABEL (1<<9) /* Label blank tapes */
-#define CAP_ANONVOLS (1<<10) /* Mount without knowing volume name */
-#define CAP_ALWAYSOPEN (1<<11) /* always keep device open */
+#define CAP_EOF (1<<0) /* has MTWEOF */
+#define CAP_BSR (1<<1) /* has MTBSR */
+#define CAP_BSF (1<<2) /* has MTBSF */
+#define CAP_FSR (1<<3) /* has MTFSR */
+#define CAP_FSF (1<<4) /* has MTFSF */
+#define CAP_EOM (1<<5) /* has MTEOM */
+#define CAP_REM (1<<6) /* is removable media */
+#define CAP_RACCESS (1<<7) /* is random access device */
+#define CAP_AUTOMOUNT (1<<8) /* Read device at start to see what is there */
+#define CAP_LABEL (1<<9) /* Label blank tapes */
+#define CAP_ANONVOLS (1<<10) /* Mount without knowing volume name */
+#define CAP_ALWAYSOPEN (1<<11) /* always keep device open */
#define CAP_AUTOCHANGER (1<<12) /* AutoChanger */
#define CAP_OFFLINEUNMOUNT (1<<13) /* Offline before unmount */
-#define CAP_STREAM (1<<14) /* Stream device */
-#define CAP_BSFATEOM (1<<15) /* Backspace file at EOM */
-#define CAP_FASTFSF (1<<16) /* Fast forward space file */
-#define CAP_TWOEOF (1<<17) /* Write two eofs for EOM */
+#define CAP_STREAM (1<<14) /* Stream device */
+#define CAP_BSFATEOM (1<<15) /* Backspace file at EOM */
+#define CAP_FASTFSF (1<<16) /* Fast forward space file */
+#define CAP_TWOEOF (1<<17) /* Write two eofs for EOM */
#define CAP_CLOSEONPOLL (1<<18) /* Close device on polling */
/* Test state */
#define dev_state(dev, st_state) ((dev)->state & (st_state))
/* Device state bits */
-#define ST_OPENED (1<<0) /* set when device opened */
-#define ST_TAPE (1<<1) /* is a tape device */
-#define ST_FILE (1<<2) /* is a file device */
-#define ST_FIFO (1<<3) /* is a fifo device */
-#define ST_PROG (1<<4) /* is a program device */
-#define ST_LABEL (1<<5) /* label found */
+#define ST_OPENED (1<<0) /* set when device opened */
+#define ST_TAPE (1<<1) /* is a tape device */
+#define ST_FILE (1<<2) /* is a file device */
+#define ST_FIFO (1<<3) /* is a fifo device */
+#define ST_PROG (1<<4) /* is a program device */
+#define ST_LABEL (1<<5) /* label found */
#define ST_MALLOC (1<<6) /* dev packet malloc'ed in init_dev() */
-#define ST_APPEND (1<<7) /* ready for Bacula append */
-#define ST_READ (1<<8) /* ready for Bacula read */
-#define ST_EOT (1<<9) /* at end of tape */
-#define ST_WEOT (1<<10) /* Got EOT on write */
-#define ST_EOF (1<<11) /* Read EOF i.e. zero bytes */
-#define ST_NEXTVOL (1<<12) /* Start writing on next volume */
-#define ST_SHORT (1<<13) /* Short block read */
+#define ST_APPEND (1<<7) /* ready for Bacula append */
+#define ST_READ (1<<8) /* ready for Bacula read */
+#define ST_EOT (1<<9) /* at end of tape */
+#define ST_WEOT (1<<10) /* Got EOT on write */
+#define ST_EOF (1<<11) /* Read EOF i.e. zero bytes */
+#define ST_NEXTVOL (1<<12) /* Start writing on next volume */
+#define ST_SHORT (1<<13) /* Short block read */
/* dev_blocked states (mutually exclusive) */
#define BST_NOT_BLOCKED 0 /* not blocked */
-#define BST_UNMOUNTED 1 /* User unmounted device */
+#define BST_UNMOUNTED 1 /* User unmounted device */
#define BST_WAITING_FOR_SYSOP 2 /* Waiting for operator to mount tape */
#define BST_DOING_ACQUIRE 3 /* Opening/validating/moving tape */
#define BST_WRITING_LABEL 4 /* Labeling a tape */
#define BST_UNMOUNTED_WAITING_FOR_SYSOP 5 /* Closed by user during mount request */
-#define BST_MOUNT 6 /* Mount request */
+#define BST_MOUNT 6 /* Mount request */
/* Volume Catalog Information structure definition */
struct VOLUME_CAT_INFO {
/* Media info for the current Volume */
- uint32_t VolCatJobs; /* number of jobs on this Volume */
- uint32_t VolCatFiles; /* Number of files */
- uint32_t VolCatBlocks; /* Number of blocks */
- uint64_t VolCatBytes; /* Number of bytes written */
- uint32_t VolCatMounts; /* Number of mounts this volume */
- uint32_t VolCatErrors; /* Number of errors this volume */
- uint32_t VolCatWrites; /* Number of writes this volume */
- uint32_t VolCatReads; /* Number of reads this volume */
- uint64_t VolCatRBytes; /* Number of bytes read */
- uint32_t VolCatRecycles; /* Number of recycles this volume */
- int32_t Slot; /* Slot in changer */
- bool InChanger; /* Set if vol in current magazine */
- uint32_t VolCatMaxJobs; /* Maximum Jobs to write to volume */
- uint32_t VolCatMaxFiles; /* Maximum files to write to volume */
- uint64_t VolCatMaxBytes; /* Max bytes to write to volume */
+ uint32_t VolCatJobs; /* number of jobs on this Volume */
+ uint32_t VolCatFiles; /* Number of files */
+ uint32_t VolCatBlocks; /* Number of blocks */
+ uint64_t VolCatBytes; /* Number of bytes written */
+ uint32_t VolCatMounts; /* Number of mounts this volume */
+ uint32_t VolCatErrors; /* Number of errors this volume */
+ uint32_t VolCatWrites; /* Number of writes this volume */
+ uint32_t VolCatReads; /* Number of reads this volume */
+ uint64_t VolCatRBytes; /* Number of bytes read */
+ uint32_t VolCatRecycles; /* Number of recycles this volume */
+ int32_t Slot; /* Slot in changer */
+ bool InChanger; /* Set if vol in current magazine */
+ uint32_t VolCatMaxJobs; /* Maximum Jobs to write to volume */
+ uint32_t VolCatMaxFiles; /* Maximum files to write to volume */
+ uint64_t VolCatMaxBytes; /* Max bytes to write to volume */
uint64_t VolCatCapacityBytes; /* capacity estimate */
- uint64_t VolReadTime; /* time spent reading */
- uint64_t VolWriteTime; /* time spent writing this Volume */
- char VolCatStatus[20]; /* Volume status */
+ uint64_t VolReadTime; /* time spent reading */
+ uint64_t VolWriteTime; /* time spent writing this Volume */
+ char VolCatStatus[20]; /* Volume status */
char VolCatName[MAX_NAME_LENGTH]; /* Desired volume to mount */
-};
+};
typedef struct s_steal_lock {
- pthread_t no_wait_id; /* id of no wait thread */
- int dev_blocked; /* state */
- int dev_prev_blocked; /* previous blocked state */
+ pthread_t no_wait_id; /* id of no wait thread */
+ int dev_blocked; /* state */
+ int dev_prev_blocked; /* previous blocked state */
} bsteal_lock_t;
-struct DEVRES; /* Device resource defined in stored_conf.h */
+struct DEVRES; /* Device resource defined in stored_conf.h */
/*
* Device structure definition. There is one of these for
*/
struct DEVICE {
public:
- DEVICE *next; /* pointer to next open device */
- DEVICE *prev; /* pointer to prev open device */
- JCR *attached_jcrs; /* attached JCR list */
- dlist *attached_dcrs; /* attached DCR list */
- pthread_mutex_t mutex; /* access control */
- pthread_cond_t wait; /* thread wait variable */
+ DEVICE *next; /* pointer to next open device */
+ DEVICE *prev; /* pointer to prev open device */
+ JCR *attached_jcrs; /* attached JCR list */
+ dlist *attached_dcrs; /* attached DCR list */
+ pthread_mutex_t mutex; /* access control */
+ pthread_cond_t wait; /* thread wait variable */
pthread_cond_t wait_next_vol; /* wait for tape to be mounted */
- pthread_t no_wait_id; /* this thread must not wait */
- int dev_blocked; /* set if we must wait (i.e. change tape) */
- int dev_prev_blocked; /* previous blocked state */
- int num_waiting; /* number of threads waiting */
- int num_writers; /* number of writing threads */
- int use_count; /* usage count on this device */
- int fd; /* file descriptor */
- int capabilities; /* capabilities mask */
- int state; /* state mask */
- int dev_errno; /* Our own errno */
- int mode; /* read/write modes */
- char *dev_name; /* device name */
- char *errmsg; /* nicely edited error message */
- uint32_t block_num; /* current block number base 0 */
- uint32_t file; /* current file number base 0 */
- uint64_t file_addr; /* Current file read/write address */
- uint64_t file_size; /* Current file size */
- uint32_t EndBlock; /* last block written */
- uint32_t EndFile; /* last file written */
- uint32_t min_block_size; /* min block size */
- uint32_t max_block_size; /* max block size */
- uint64_t max_volume_size; /* max bytes to put on one volume */
- uint64_t max_file_size; /* max file size to put in one file on volume */
- uint64_t volume_capacity; /* advisory capacity */
- uint32_t max_rewind_wait; /* max secs to allow for rewind */
- uint32_t max_open_wait; /* max secs to allow for open */
- uint32_t max_open_vols; /* max simultaneous open volumes */
- utime_t vol_poll_interval; /* interval between polling Vol mount */
- DEVRES *device; /* pointer to Device Resource */
- btimer_t *tid; /* timer id */
-
- VOLUME_CAT_INFO VolCatInfo; /* Volume Catalog Information */
- VOLUME_LABEL VolHdr; /* Actual volume label */
+ pthread_t no_wait_id; /* this thread must not wait */
+ int dev_blocked; /* set if we must wait (i.e. change tape) */
+ int dev_prev_blocked; /* previous blocked state */
+ int num_waiting; /* number of threads waiting */
+ int num_writers; /* number of writing threads */
+ int use_count; /* usage count on this device */
+ int fd; /* file descriptor */
+ int capabilities; /* capabilities mask */
+ int state; /* state mask */
+ int dev_errno; /* Our own errno */
+ int mode; /* read/write modes */
+ POOLMEM *dev_name; /* device name */
+ char *errmsg; /* nicely edited error message */
+ uint32_t block_num; /* current block number base 0 */
+ uint32_t file; /* current file number base 0 */
+ uint64_t file_addr; /* Current file read/write address */
+ uint64_t file_size; /* Current file size */
+ uint32_t EndBlock; /* last block written */
+ uint32_t EndFile; /* last file written */
+ uint32_t min_block_size; /* min block size */
+ uint32_t max_block_size; /* max block size */
+ uint64_t max_volume_size; /* max bytes to put on one volume */
+ uint64_t max_file_size; /* max file size to put in one file on volume */
+ uint64_t volume_capacity; /* advisory capacity */
+ uint32_t max_rewind_wait; /* max secs to allow for rewind */
+ uint32_t max_open_wait; /* max secs to allow for open */
+ uint32_t max_open_vols; /* max simultaneous open volumes */
+ utime_t vol_poll_interval; /* interval between polling Vol mount */
+ DEVRES *device; /* pointer to Device Resource */
+ btimer_t *tid; /* timer id */
+
+ VOLUME_CAT_INFO VolCatInfo; /* Volume Catalog Information */
+ VOLUME_LABEL VolHdr; /* Actual volume label */
/* Device wait times ***FIXME*** look at durations */
char BadVolName[MAX_NAME_LENGTH]; /* Last wrong Volume mounted */
- bool poll; /* set to poll Volume */
+ bool poll; /* set to poll Volume */
int min_wait;
int max_wait;
int max_num_wait;
};
/*
- * Device Context (or Control) Record.
- * There is one of these records for each Job that is using
+ * Device Context (or Control) Record.
+ * There is one of these records for each Job that is using
* the device. Items in this record are "local" to the Job and
* do not affect other Jobs.
*/
struct DCR {
- dlink dev_link; /* link to attach to dev */
- JCR *jcr; /* pointer to JCR */
- DEVICE *dev; /* pointer to device */
- DEV_BLOCK *block; /* pointer to block */
- DEV_RECORD *record; /* pointer to record */
- bool spool_data; /* set to spool data */
- bool spooling; /* set when actually spooling */
- int spool_fd; /* fd if spooling */
- bool NewVol; /* set if new Volume mounted */
- bool WroteVol; /* set if Volume written */
- bool NewFile; /* set when EOF written */
- uint32_t VolFirstIndex; /* First file index this Volume */
- uint32_t VolLastIndex; /* Last file index this Volume */
- uint32_t FileIndex; /* Current File Index */
- uint32_t EndFile; /* End file written */
- uint32_t StartFile; /* Start write file */
- uint32_t StartBlock; /* Start write block */
- uint32_t EndBlock; /* Ending block written */
+ dlink dev_link; /* link to attach to dev */
+ JCR *jcr; /* pointer to JCR */
+ DEVICE *dev; /* pointer to device */
+ DEV_BLOCK *block; /* pointer to block */
+ DEV_RECORD *record; /* pointer to record */
+ bool spool_data; /* set to spool data */
+ bool spooling; /* set when actually spooling */
+ int spool_fd; /* fd if spooling */
+ bool NewVol; /* set if new Volume mounted */
+ bool WroteVol; /* set if Volume written */
+ bool NewFile; /* set when EOF written */
+ uint32_t VolFirstIndex; /* First file index this Volume */
+ uint32_t VolLastIndex; /* Last file index this Volume */
+ uint32_t FileIndex; /* Current File Index */
+ uint32_t EndFile; /* End file written */
+ uint32_t StartFile; /* Start write file */
+ uint32_t StartBlock; /* Start write block */
+ uint32_t EndBlock; /* Ending block written */
char VolumeName[MAX_NAME_LENGTH]; /* Volume name */
- VOLUME_CAT_INFO VolCatInfo; /* Catalog info for desired volume */
+ VOLUME_CAT_INFO VolCatInfo; /* Catalog info for desired volume */
};
* dependent. Arrgggg!
*/
#ifndef MTEOM
-#ifdef MTSEOD
+#ifdef MTSEOD
#define MTEOM MTSEOD
#endif
#ifdef MTEOD
void _lock_device(char *file, int line, DEVICE *dev)
{
int stat;
- Dmsg3(100, "lock %d from %s:%d\n", dev->dev_blocked, file, line);
+ Dmsg3(500, "lock %d from %s:%d\n", dev->dev_blocked, file, line);
P(dev->mutex);
if (dev->dev_blocked && !pthread_equal(dev->no_wait_id, pthread_self())) {
dev->num_waiting++; /* indicate that I am waiting */
void _unlock_device(char *file, int line, DEVICE *dev)
{
- Dmsg2(100, "unlock from %s:%d\n", file, line);
+ Dmsg2(500, "unlock from %s:%d\n", file, line);
V(dev->mutex);
}
*/
void _block_device(char *file, int line, DEVICE *dev, int state)
{
- Dmsg3(100, "block set %d from %s:%d\n", state, file, line);
+ Dmsg3(500, "block set %d from %s:%d\n", state, file, line);
ASSERT(dev->dev_blocked == BST_NOT_BLOCKED);
dev->dev_blocked = state; /* make other threads wait */
dev->no_wait_id = pthread_self(); /* allow us to continue */
*/
void _unblock_device(char *file, int line, DEVICE *dev)
{
- Dmsg3(100, "unblock %d from %s:%d\n", dev->dev_blocked, file, line);
+ Dmsg3(500, "unblock %d from %s:%d\n", dev->dev_blocked, file, line);
ASSERT(dev->dev_blocked);
dev->dev_blocked = BST_NOT_BLOCKED;
dev->no_wait_id = 0;
*/
void _steal_device_lock(char *file, int line, DEVICE *dev, bsteal_lock_t *hold, int state)
{
- Dmsg4(100, "steal lock. old=%d new=%d from %s:%d\n", dev->dev_blocked, state,
+ Dmsg4(500, "steal lock. old=%d new=%d from %s:%d\n", dev->dev_blocked, state,
file, line);
hold->dev_blocked = dev->dev_blocked;
hold->dev_prev_blocked = dev->dev_prev_blocked;
*/
void _give_back_device_lock(char *file, int line, DEVICE *dev, bsteal_lock_t *hold)
{
- Dmsg4(100, "return lock. old=%d new=%d from %s:%d\n",
+ Dmsg4(500, "return lock. old=%d new=%d from %s:%d\n",
dev->dev_blocked, hold->dev_blocked, file, line);
P(dev->mutex);
dev->dev_blocked = hold->dev_blocked;
static int use_device_cmd(JCR *jcr);
/* Requests from the Director daemon */
-static char jobcmd[] = "JobId=%d job=%127s job_name=%127s client_name=%127s \
-type=%d level=%d FileSet=%127s NoAttr=%d SpoolAttr=%d FileSetMD5=%127s\n";
+static char jobcmd[] = "JobId=%d job=%127s job_name=%127s client_name=%127s "
+ "type=%d level=%d FileSet=%127s NoAttr=%d SpoolAttr=%d FileSetMD5=%127s "
+ "SpoolData=%d";
static char use_device[] = "use device=%s media_type=%s pool_name=%s pool_type=%s\n";
/* Responses sent to Director daemon */
char auth_key[100];
BSOCK *dir = jcr->dir_bsock;
POOLMEM *job_name, *client_name, *job, *fileset_name, *fileset_md5;
- int JobType, level, spool_attributes, no_attributes;
+ int JobType, level, spool_attributes, no_attributes, spool_data;
struct timeval tv;
struct timezone tz;
struct timespec timeout;
* Get JobId and permissions from Director
*/
- Dmsg1(130, "Job_cmd: %s\n", dir->msg);
+ Dmsg1(100, "Job_cmd: %s\n", dir->msg);
job = get_memory(dir->msglen);
job_name = get_memory(dir->msglen);
client_name = get_memory(dir->msglen);
fileset_md5 = get_memory(dir->msglen);
if (sscanf(dir->msg, jobcmd, &JobId, job, job_name, client_name,
&JobType, &level, fileset_name, &no_attributes,
- &spool_attributes, fileset_md5) != 10) {
+ &spool_attributes, fileset_md5, &spool_data) != 11) {
pm_strcpy(&jcr->errmsg, dir->msg);
bnet_fsend(dir, BAD_job, jcr->errmsg);
Emsg1(M_FATAL, 0, _("Bad Job Command from Director: %s\n"), jcr->errmsg);
jcr->JobLevel = level;
jcr->no_attributes = no_attributes;
jcr->spool_attributes = spool_attributes;
+ jcr->spool_data = spool_data;
jcr->fileset_md5 = get_pool_memory(PM_NAME);
pm_strcpy(&jcr->fileset_md5, fileset_md5);
free_memory(job);
*/
void create_volume_label(DEVICE *dev, char *VolName, char *PoolName)
{
- struct date_time dt;
DEVRES *device = (DEVRES *)dev->device;
Dmsg0(90, "Start create_volume_label()\n");
memset(&dev->VolHdr, 0, sizeof(dev->VolHdr));
- /* ***FIXME*** we really need to get the volume name,
- * pool name, and pool type from the database.
- */
bstrncpy(dev->VolHdr.Id, BaculaId, sizeof(dev->VolHdr.Id));
dev->VolHdr.VerNum = BaculaTapeVersion;
dev->VolHdr.LabelType = PRE_LABEL; /* Mark tape as unused */
bstrncpy(dev->VolHdr.PoolType, "Backup", sizeof(dev->VolHdr.PoolType));
- /* Put label time/date in header */
- if (BaculaTapeVersion >= 11) {
- dev->VolHdr.label_btime = get_current_btime();
- dev->VolHdr.label_date = 0;
- dev->VolHdr.label_time = 0;
- } else {
- /* OLD WAY DEPRECATED */
- get_current_time(&dt);
- dev->VolHdr.label_date = dt.julian_day_number;
- dev->VolHdr.label_time = dt.julian_day_fraction;
- }
+ dev->VolHdr.label_btime = get_current_btime();
+ dev->VolHdr.label_date = 0;
+ dev->VolHdr.label_time = 0;
if (gethostname(dev->VolHdr.HostName, sizeof(dev->VolHdr.HostName)) != 0) {
dev->VolHdr.HostName[0] = 0;
static int open_data_spool_file(JCR *jcr);
static int close_data_spool_file(JCR *jcr);
static bool despool_data(DCR *dcr);
+static int read_block_from_spool_file(DCR *dcr, DEV_BLOCK *block);
+struct spool_hdr {
+ int32_t FirstIndex;
+ int32_t LastIndex;
+ uint32_t len;
+};
+
+enum {
+ RB_EOT = 1,
+ RB_ERROR,
+ RB_OK
+};
int begin_data_spool(JCR *jcr)
{
- if (jcr->dcr->spool_data) {
- return open_data_spool_file(jcr);
+ int stat = 1;
+ if (jcr->spool_data) {
+ Dmsg0(100, "Turning on data spooling\n");
+ jcr->dcr->spool_data = true;
+ stat = open_data_spool_file(jcr);
+ if (stat) {
+ jcr->dcr->spooling = true;
+ }
}
- return 1;
+ return stat;
}
int discard_data_spool(JCR *jcr)
{
- if (jcr->dcr->spool_data && jcr->dcr->spool_fd >= 0) {
+ if (jcr->dcr->spooling) {
+ Dmsg0(100, "Data spooling discarded\n");
return close_data_spool_file(jcr);
}
return 1;
int commit_data_spool(JCR *jcr)
{
bool stat;
- if (jcr->dcr->spool_data && jcr->dcr->spool_fd >= 0) {
- lock_device(jcr->dcr->dev);
+ if (jcr->dcr->spooling) {
+ Dmsg0(100, "Committing spooled data\n");
stat = despool_data(jcr->dcr);
- unlock_device(jcr->dcr->dev);
if (!stat) {
+ Dmsg1(000, "Bad return from despool WroteVol=%d\n", jcr->dcr->WroteVol);
close_data_spool_file(jcr);
return 0;
}
free_pool_memory(name);
return 0;
}
+ Dmsg1(100, "Created spool file: %s\n", name);
free_pool_memory(name);
return 1;
}
make_unique_data_spool_filename(jcr, &name);
close(jcr->dcr->spool_fd);
jcr->dcr->spool_fd = -1;
+ jcr->dcr->spooling = false;
unlink(name);
+ Dmsg1(100, "Deleted spool file: %s\n", name);
free_pool_memory(name);
return 1;
}
static bool despool_data(DCR *dcr)
{
- DEVICE *sdev;
- DCR *sdcr;
+ DEVICE *rdev;
+ DCR *rdcr;
dcr->spooling = false;
bool ok = true;
- DEV_BLOCK *block = dcr->block;
+ DEV_BLOCK *block;
JCR *jcr = dcr->jcr;
+ int stat;
+// lock_device(dcr->dev);
+ Dmsg0(100, "Despooling data\n");
/* Set up a dev structure to read */
- sdev = (DEVICE *)malloc(sizeof(DEVICE));
- memset(sdev, 0, sizeof(DEVICE));
- sdev->fd = dcr->spool_fd;
- lseek(sdev->fd, 0, SEEK_SET); /* rewind */
- sdcr = new_dcr(jcr, sdev);
+ rdev = (DEVICE *)malloc(sizeof(DEVICE));
+ memset(rdev, 0, sizeof(DEVICE));
+ rdev->dev_name = get_memory(strlen("spool")+1);
+ strcpy(rdev->dev_name, "spool");
+ rdev->errmsg = get_pool_memory(PM_EMSG);
+ *rdev->errmsg = 0;
+ rdcr = new_dcr(NULL, rdev);
+ rdcr->spool_fd = dcr->spool_fd;
+ rdcr->jcr = jcr; /* set a valid jcr */
+ block = rdcr->block;
+ lseek(rdcr->spool_fd, 0, SEEK_SET); /* rewind */
for ( ; ok; ) {
if (job_canceled(jcr)) {
ok = false;
break;
}
- if (!read_block_from_dev(jcr, sdev, block, CHECK_BLOCK_NUMBERS)) {
- if (dev_state(sdev, ST_EOT)) {
- break;
- }
- ok = false;
+ stat = read_block_from_spool_file(rdcr, block);
+ if (stat == RB_EOT) {
break;
- }
- if (!write_block_to_dev(dcr, block)) {
+ } else if (stat == RB_ERROR) {
ok = false;
break;
}
+ ok = write_block_to_device(dcr, block);
+ Dmsg3(100, "Write block ok=%d FI=%d LI=%d\n", ok, block->FirstIndex, block->LastIndex);
}
- lseek(sdev->fd, 0, SEEK_SET); /* rewind */
- if (ftruncate(sdev->fd, 0) != 0) {
+ lseek(rdcr->spool_fd, 0, SEEK_SET); /* rewind */
+ if (ftruncate(rdcr->spool_fd, 0) != 0) {
+ Dmsg1(000, "Bad return from ftruncate. ERR=%s\n", strerror(errno));
ok = false;
}
+ free_memory(rdev->dev_name);
+ free_pool_memory(rdev->errmsg);
+ free(rdev);
+ rdcr->jcr = NULL;
+ free_dcr(rdcr);
+// unlock_device(dcr->dev);
return ok;
}
+/*
+ * Read a block from the spool file
+ *
+ * Returns RB_OK on success
+ * RB_EOT when file done
+ * RB_ERROR on error
+ */
+static int read_block_from_spool_file(DCR *dcr, DEV_BLOCK *block)
+{
+ uint32_t rlen;
+ ssize_t stat;
+ spool_hdr hdr;
+
+ rlen = sizeof(hdr);
+ stat = read(dcr->spool_fd, (char *)&hdr, (size_t)rlen);
+ if (stat == 0) {
+ Dmsg0(100, "EOT on spool read.\n");
+ return RB_EOT;
+ } else if (stat != (ssize_t)rlen) {
+ if (stat == -1) {
+ Jmsg(dcr->jcr, M_FATAL, 0, "Spool read error. ERR=%s\n", strerror(errno));
+ } else {
+ Dmsg2(000, "Spool read error. Wanted %u bytes, got %u\n", rlen, stat);
+ Jmsg2(dcr->jcr, M_FATAL, 0, "Spool read error. Wanted %u bytes, got %u\n", rlen, stat);
+ }
+ return RB_ERROR;
+ }
+ rlen = hdr.len;
+ if (rlen > block->buf_len) {
+ Dmsg2(000, "Spool block too big. Max %u bytes, got %u\n", block->buf_len, rlen);
+ Jmsg2(dcr->jcr, M_FATAL, 0, "Spool block too big. Max %u bytes, got %u\n", block->buf_len, rlen);
+ return RB_ERROR;
+ }
+ stat = read(dcr->spool_fd, (char *)block->buf, (size_t)rlen);
+ if (stat != (ssize_t)rlen) {
+ Dmsg2(000, "Spool read error. Wanted %u bytes, got %u\n", rlen, stat);
+ Jmsg2(dcr->jcr, M_FATAL, 0, "Spool read error. Wanted %u bytes, got %u\n", rlen, stat);
+ return RB_ERROR;
+ }
+ /* Setup write pointers */
+ block->binbuf = rlen;
+ block->bufp = block->buf + block->binbuf;
+ block->FirstIndex = hdr.FirstIndex;
+ block->LastIndex = hdr.LastIndex;
+ block->VolSessionId = dcr->jcr->VolSessionId;
+ block->VolSessionTime = dcr->jcr->VolSessionTime;
+ Dmsg2(400, "Read block FI=%d LI=%d\n", block->FirstIndex, block->LastIndex);
+ return RB_OK;
+}
+
/*
* Write a block to the spool file
*
{
ssize_t stat = 0;
uint32_t wlen; /* length to write */
- DEVICE *dev = dcr->dev;
int retry = 0;
+ spool_hdr hdr;
ASSERT(block->binbuf == ((uint32_t) (block->bufp - block->buf)));
- wlen = block->binbuf;
- if (wlen <= WRITE_BLKHDR_LENGTH) { /* Does block have data in it? */
+ if (block->binbuf <= WRITE_BLKHDR_LENGTH) { /* Does block have data in it? */
Dmsg0(100, "return write_block_to_dev no data to write\n");
return true;
}
- /*
- * Clear to the end of the buffer if it is not full,
- * and on tape devices, apply min and fixed blocking.
- */
- if (wlen != block->buf_len) {
- uint32_t blen; /* current buffer length */
-
- Dmsg2(200, "binbuf=%d buf_len=%d\n", block->binbuf, block->buf_len);
- blen = wlen;
-
- /* Adjust write size to min/max for tapes only */
- if (dev->state & ST_TAPE) {
- if (wlen < dev->min_block_size) {
- wlen = ((dev->min_block_size + TAPE_BSIZE - 1) / TAPE_BSIZE) * TAPE_BSIZE;
- }
- /* check for fixed block size */
- if (dev->min_block_size == dev->max_block_size) {
- wlen = block->buf_len; /* fixed block size already rounded */
- }
+
+ hdr.FirstIndex = block->FirstIndex;
+ hdr.LastIndex = block->LastIndex;
+ hdr.len = block->binbuf;
+ wlen = sizeof(hdr);
+write_hdr_again:
+ stat = write(dcr->spool_fd, (char*)&hdr, (size_t)wlen);
+ if (stat != (ssize_t)wlen) {
+ if (!despool_data(dcr)) {
+ return false;
}
- if (wlen-blen > 0) {
- memset(block->bufp, 0, wlen-blen); /* clear garbage */
+ if (retry++ > 1) {
+ return false;
}
- }
+ goto write_hdr_again;
+ }
- ser_block_header(block);
- Dmsg1(300, "Write block of %u bytes\n", wlen);
+ wlen = block->binbuf;
+ Dmsg2(300, "Wrote block FI=%d LI=%d\n", block->FirstIndex, block->LastIndex);
write_again:
stat = write(dcr->spool_fd, block->buf, (size_t)wlen);
if (stat != (ssize_t)wlen) {
projects / bacula / bacula / commitdiff