PROBLEM DESCRIPTION:
The FreeBSD pthreads library does not properly handle End Of Tape.
-This problem will be fixed in FreeBSD 4.9. (note, the problem
-was apparently not fixed in 4.9, and version prior to 4.10-RELEASE
-and 5.2.1-RELEASE need to be patched. The bug results in more
-data being written to the tape than could be read. Any backup which
-involved more than one tape would be incomplete.
+This problem will be fixed in FreeBSD 4.9.
+
+UPDATE 2004/02/24: Note, the problem was apparently not fixed in
+ 4.9-RELEASE. 4.9-RELEASE contained a partial patch that did not
+ prevent data loss. To date, the latest FreeBSD -RELEASE versions
+ (4.9-RELEASE and 5.2.1-RELEASE) are *broken* as shipped. If
+ you are running one of these systems, please either patch
+ your system as described below or upgrade to -STABLE or
+ -CURRENT immediately.
+
+ We expect 4.10-RELEASE to be available within a few weeks
+ (written 24 Apr 2004) and 5.3-RELEASE to be available in a
+ few months. 4.10 and 5.3 *should* contain the fix, but we
+ can't know for sure until we've had a chance to test them.
+
+The bug results in more data being written than the tape will
+hold because of a lost status code. Any backup which involving
+more than one tape would have data lost.
DEMONSTRATION:
To demonstrate the problem, tapetest.c can be obtained from
http://www.freebsd.org/cgi/query-pr.cgi?pr=56274
+tapetest.c can also be found in the Bacula source distribution
+in <bacula-source>/platforms/freebsd/tapetest.c
+
This tests without pthreads:
* If you build this program with:
* Rewound /dev/nsa0
* *Begin writing blocks of 64512 bytes.
* ++++++++++++++++++++ ...
- * Write failed. Last block written=17294. stat=0 ERR=Unknown
-error: 0
+ * Write failed. Last block written=17294. stat=0 ERR=Unknown error: 0
* weof_dev
* Wrote EOF to /dev/nsa0
* *Rewound /dev/nsa0
* Total files=1, blocks=17913, bytes = 1155603456
* ========
*
- * which is incorrect because it wrote 17,926 blocks but read
- * back only 17,913 blocks AND the status on the last block
- * written was -1 when it should have been 0.
+ * which is incorrect because it wrote 17,926 blocks and the
+ * status on the last block written is stat=-1, which is incorrect.
+ * In addition only 17,913 blocks were read back.
+ *
+ * Similarly, if you ran this test on 4.9-RELEASE or 5.2.1-RELEASE
+ * (these versions contain an incomplete patch) then you would
+ * probably see something like this:
+ *
+ * ========
+ * Rewound /dev/nsa0
+ * *Begin writing blocks of 64512 bytes.
+ * +++++++++++++++ [...]
+ * weof_dev
+ * Wrote EOF to /dev/nsa0
+ * Write failed. Last block written=271163. stat=-1 ERR=No space left on device
+ * *Rewound /dev/nsa0
+ * *Starting scan at file 0
+ * Bad status from read -1. ERR=Input/output error
+ * 271163 blocks of 64512 bytes in file 0
+ * ========
+ *
+ * The above output is also incorrect. The block counts match,
+ * but note the -1 error code on the read and write. This is
+ * just as dangerous as the first example. If you see this
+ * output then you should patch or upgrade to -STABLE or -CURRENT
+ * immediately.
If you get the same number of blocks written and read WHEN using
-pthreads, and the test with pthreads enabled returns a stat=0
+pthreads, AND the test with pthreads enabled returns a stat=0
on the last write, and the scan operation returns no error
-code, then you've been correctly patched.
+code, then you've been correctly patched. It is important that
+stat=0 rather than -1 even if the correct number of blocks
+are read back. If the status is -1 on the pthreads test, you
+will lose data.
SOLUTION:
-For FreeBSD prior to and including 4.9, in other words, for
-FreeBSD prior to 4.10-RELEASE and 5.2.1-RELEASE you have two
-choices to ensure proper backups. These instructions assume you
-are familiar with patching FreeBSD and already have the FreeBSD
-source code installed on your machine.
+For FreeBSD versions prior to 4.10-RELEASE and 5.3-RELEASE you
+have two choices to ensure proper backups. These instructions
+assume you are familiar with patching FreeBSD and already have
+the FreeBSD source code installed on your machine.
+
+For FreeBSD 4.x:
Do one of the following:
To apply the patch, follow these instructions as root.
+ cd /usr/src/lib/libc_r/uthread/
fetch -o pthread.diff 'http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.16.2.6&r2=1.16.2.8'
+ patch < pthread.diff
+ cd ..
+ make all install
+
+For FreeBSD 5.x:
+
+Do one of the following:
+
+- cvsup and build your system to FreeBSD -CURRENT after the
+ date Wed Dec 17 16:44:03 2003 UTC
+
+- Apply this patch.
+
+ http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.22&r2=1.23
+
+ Wed Dec 17 16:44:03 2003 UTC
+
+ To apply the patch, follow these instructions as root.
cd /usr/src/lib/libc_r/uthread/
- patch < /path/to/pthread.diff
+ fetch -o pthread.diff 'http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.22&r2=1.23'
+ patch < pthread.diff
cd ..
make all install
-You should then recompile Bacula to get the new library
+After patching your system as shown above,
+you should then recompile Bacula to get the new library
code included by doing:
cd <bacula-source>
I suggest running tapetest on your patched system and then
conducting a backup which spans two tapes. Restore the data
and compare to the original. If not identical, please let us know.
-
-tapetest.c can also be found in the Bacula source distribution
-in <bacula-source>/platforms/freebsd/tapetest.c