-From bacula-users-admin@lists.sourceforge.net Mon Oct 20 23:44:13 2003
-Return-Path: <bacula-users-admin@lists.sourceforge.net>
-Received: from sc8-sf-list2.sourceforge.net (lists.sourceforge.net
- [66.35.250.206]) by matou.sibbald.com (8.11.6/8.11.6) with ESMTP id
- h9KLiDY13657 for <kern@sibbald.com>; Mon, 20 Oct 2003 23:44:13 +0200
-Received: from sc8-sf-list1-b.sourceforge.net ([10.3.1.13]
- helo=sc8-sf-list1.sourceforge.net) by sc8-sf-list2.sourceforge.net with
- esmtp (Exim 3.31-VA-mm2 #1 (Debian)) id 1ABhpS-00013k-00; Mon, 20 Oct 2003
- 14:44:42 -0700
-Received: from sc8-sf-mx1-b.sourceforge.net ([10.3.1.11]
- helo=sc8-sf-mx1.sourceforge.net) by sc8-sf-list1.sourceforge.net with esmtp
- (Cipher TLSv1:DES-CBC3-SHA:168) (Exim 3.31-VA-mm2 #1 (Debian)) id
- 1ABhnp-0007qu-00 for <bacula-users@lists.sourceforge.net>; Mon, 20 Oct 2003
- 14:43:01 -0700
-Received: from bast.unixathome.org ([66.11.174.150] ident=postfix) by
- sc8-sf-mx1.sourceforge.net with esmtp (Exim 4.22) id 1ABfiR-0002Iv-Q8 for
- bacula-users@lists.sourceforge.net; Mon, 20 Oct 2003 12:29:19 -0700
-Received: from wocker (wocker.unixathome.org [192.168.0.99]) by
- bast.unixathome.org (Postfix) with ESMTP id 258913F53 for
- <bacula-users@lists.sourceforge.net>; Mon, 20 Oct 2003 15:25:33 -0400 (EDT)
From: "Dan Langille" <dan@langille.org>
To: bacula-users@lists.sourceforge.net
-MIME-Version: 1.0
-Message-ID: <3F93FF4E.14552.13ACB682@localhost>
-Priority: normal
-X-mailer: Pegasus Mail for Windows (v4.02a)
-Content-type: text/plain; charset=US-ASCII
-Content-description: Mail message body
-X-Spam-Score: 0.0 (/)
-X-Spam-Report: 0.0/5.0 Spam Filtering performed by sourceforge.net. See
- http://spamassassin.org/tag/ for more details. Report problems to
- https://sf.net/tracker/?func=add&group_id=1&atid=200001
Subject: [Bacula-users] FreeBSD - large backups to tape
-Sender: bacula-users-admin@lists.sourceforge.net
-Errors-To: bacula-users-admin@lists.sourceforge.net
-X-BeenThere: bacula-users@lists.sourceforge.net
-X-Mailman-Version: 2.0.9-sf.net
-Precedence: bulk
-List-Help: <mailto:bacula-users-request@lists.sourceforge.net?subject=help>
-List-Post: <mailto:bacula-users@lists.sourceforge.net>
-List-Subscribe:
- <https://lists.sourceforge.net/lists/listinfo/bacula-users>,
- <mailto:bacula-users-request@lists.sourceforge.net?subject=subscribe>
-List-Id: Bacula user's email list for support and discussions
- <bacula-users.lists.sourceforge.net>
-List-Unsubscribe:
- <https://lists.sourceforge.net/lists/listinfo/bacula-users>,
- <mailto:bacula-users-request@lists.sourceforge.net?subject=unsubscribe>
-List-Archive:
- <http://sourceforge.net/mailarchive/forum.php?forum=bacula-users>
Date: Mon, 20 Oct 2003 15:29:18 -0400
-Content-Transfer-Encoding: 8bit
Kern and I have been working on a FreeBSD/Bacula problem.
He's asked me to post this to the list. The problem was within the
PROBLEM DESCRIPTION:
The FreeBSD pthreads library does not properly handle End Of Tape.
-This problem will be fixed in FreeBSD 4.9. The bug results in more
-data being written to the tape than could be read. Any backup which
-involved more than one tape would be incomplete.
+This problem will be fixed in FreeBSD 4.9.
+
+UPDATE 2004/02/24: Note, the problem was apparently not fixed in
+ 4.9-RELEASE. 4.9-RELEASE contained a partial patch that did not
+ prevent data loss. To date, the latest FreeBSD -RELEASE versions
+ (4.9-RELEASE and 5.2.1-RELEASE) are *broken* as shipped. If
+ you are running one of these systems, please either patch
+ your system as described below or upgrade to -STABLE or
+ -CURRENT immediately.
+
+ We expect 4.10-RELEASE to be available within a few weeks
+ (written 24 Apr 2004) and 5.3-RELEASE to be available in a
+ few months. 4.10 and 5.3 *should* contain the fix, but we
+ can't know for sure until we've had a chance to test them.
+
+The bug results in more data being written than the tape will
+hold because of a lost status code. Any backup which involving
+more than one tape would have data lost.
DEMONSTRATION:
To demonstrate the problem, tapetest.c can be obtained from
http://www.freebsd.org/cgi/query-pr.cgi?pr=56274
+tapetest.c can also be found in the Bacula source distribution
+in <bacula-source>/platforms/freebsd/tapetest.c
+
This tests without pthreads:
* If you build this program with:
* Rewound /dev/nsa0
* *Begin writing blocks of 64512 bytes.
* ++++++++++++++++++++ ...
- * Write failed. Last block written=17294. stat=0 ERR=Unknown
-error: 0
+ * Write failed. Last block written=17294. stat=0 ERR=Unknown error: 0
* weof_dev
* Wrote EOF to /dev/nsa0
* *Rewound /dev/nsa0
* Total files=1, blocks=17913, bytes = 1155603456
* ========
*
- * which is incroorect because it wrote 17,926 blocks but read
- * back only 17,913 blocks
+ * which is incorrect because it wrote 17,926 blocks and the
+ * status on the last block written is stat=-1, which is incorrect.
+ * In addition only 17,913 blocks were read back.
+ *
+ * Similarly, if you ran this test on 4.9-RELEASE or 5.2.1-RELEASE
+ * (these versions contain an incomplete patch) then you would
+ * probably see something like this:
+ *
+ * ========
+ * Rewound /dev/nsa0
+ * *Begin writing blocks of 64512 bytes.
+ * +++++++++++++++ [...]
+ * weof_dev
+ * Wrote EOF to /dev/nsa0
+ * Write failed. Last block written=271163. stat=-1 ERR=No space left on device
+ * *Rewound /dev/nsa0
+ * *Starting scan at file 0
+ * Bad status from read -1. ERR=Input/output error
+ * 271163 blocks of 64512 bytes in file 0
+ * ========
+ *
+ * The above output is also incorrect. The block counts match,
+ * but note the -1 error code on the read and write. This is
+ * just as dangerous as the first example. If you see this
+ * output then you should patch or upgrade to -STABLE or -CURRENT
+ * immediately.
If you get the same number of blocks written and read WHEN using
-pthreads, then you've been correctly patched.
+pthreads, AND the test with pthreads enabled returns a stat=0
+on the last write, and the scan operation returns no error
+code, then you've been correctly patched. It is important that
+stat=0 rather than -1 even if the correct number of blocks
+are read back. If the status is -1 on the pthreads test, you
+will lose data.
SOLUTION:
-For FreeBSD prior to 4.9, you have two choices to ensure proper
-backups. These instructions assume you are familiar with patching
-FreeBSD and already have the FreeBSD source code installed on your
-machine.
+For FreeBSD versions prior to 4.10-RELEASE and 5.3-RELEASE you
+have two choices to ensure proper backups. These instructions
+assume you are familiar with patching FreeBSD and already have
+the FreeBSD source code installed on your machine.
-1 - cvsup and build your system to FreeBSD 4.8-STABLE
+For FreeBSD 4.x:
-2 - Apply this patch.
+Do one of the following:
-http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.16.2.6&r2=1.16.2.7
+- cvsup and build your system to FreeBSD 4.x-STABLE after the
+ date Mon Dec 29 15:18:01 2003 UTC
-To apply the patch, follow these instructions as root.
+- Apply this patch.
-fetch -o pthread.diff http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.16.2.6\&r2=1.16.2.7
+ http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.16.2.6&r2=1.16.2.8
-cd /usr/src/lib/libc_r/uthread/
-patch < /path/to/pthread.diff
-cd ..
-make all install
+ To apply the patch, follow these instructions as root.
-I recommend restarting Bacula.
+ cd /usr/src/lib/libc_r/uthread/
+ fetch -o pthread.diff 'http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.16.2.6&r2=1.16.2.8'
+ patch < pthread.diff
+ cd ..
+ make all install
+For FreeBSD 5.x:
-TESTING:
+Do one of the following:
-I suggest running tapetest on your patched system and then
-conducting a backup which spans two tapes. Restore the data
-and compare to the original. If not identical, please let us know.
+- cvsup and build your system to FreeBSD -CURRENT after the
+ date Wed Dec 17 16:44:03 2003 UTC
+
+- Apply this patch.
+
+ http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.22&r2=1.23
-Thanks
+ Wed Dec 17 16:44:03 2003 UTC
---
-Dan Langille : http://www.langille.org/
+ To apply the patch, follow these instructions as root.
+ cd /usr/src/lib/libc_r/uthread/
+ fetch -o pthread.diff 'http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_write.c.diff?r1=1.22&r2=1.23'
+ patch < pthread.diff
+ cd ..
+ make all install
+After patching your system as shown above,
+you should then recompile Bacula to get the new library
+code included by doing:
--------------------------------------------------------
-This SF.net email is sponsored by OSDN developer relations
-Here's your chance to show off your extensive product knowledge
-We want to know what you know. Tell us and you have a chance to win $100
-http://www.zoomerang.com/survey.zgi?HRPT1X3RYQNC5V4MLNSV3E54
-_______________________________________________
-Bacula-users mailing list
-Bacula-users@lists.sourceforge.net
-https://lists.sourceforge.net/lists/listinfo/bacula-users
+ cd <bacula-source>
+ make clean
+ make
+ ...
+
+
+TESTING:
+
+I suggest running tapetest on your patched system and then
+conducting a backup which spans two tapes. Restore the data
+and compare to the original. If not identical, please let us know.
projects / bacula / bacula / blobdiff