Tuesday, September 25, 2007

Fun with Debian Source Packages II

On my last post I promised to present a use case for a Debian source package which doesn't have to do with debugging or patching a bad binary package. So here goes.

As part of my preparations for a rainy day - namely, the day on which my PC will suddenly die on me - I wanted to make sure that I could access the files on my USB backup disk, without Bacula. The files are stored in a set of large compressed archive files, so it may seem hopeless, but luckily Bacula comes with a file extraction utility bextract. The following command can extract the files from a given storage device, using a bootstrap file and the configuration file of the Bacula storage daemon:

bextract -b <bsr-file> -c <bacula-sd.conf> FileStorage <destination-directory>

So all I need to do is run a script at the end of each backup job, that copies the bootstrap file that Bacula generates to the backup disk. Furthermore, I need to make sure that up to date versions of both the storage daemon configuration file, and of the bextract executable are also stored on the backup disk.

But the bextract executable depends on a number of external shared libraries, as can be determined with ldd:

# ldd /usr/sbin/bextract
linux-gate.so.1 => (0xffffe000)
libacl.so.1 => /lib/libacl.so.1 (0xb7f67000)
libz.so.1 => /usr/lib/libz.so.1 (0xb7f52000)
libpython2.4.so.1.0 => /usr/lib/libpython2.4.so.1.0 (0xb7e41000)
libutil.so.1 => /lib/i686/cmov/libutil.so.1 (0xb7e3d000)
librt.so.1 => /lib/i686/cmov/librt.so.1 (0xb7e34000)
libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb7e1d000)
libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7e18000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d2d000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7d08000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7cfd000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7bb5000)
libattr.so.1 => /lib/libattr.so.1 (0xb7bb1000)
/lib/ld-linux.so.2 (0xb7f85000)


In order to avoid these dependencies, bextract has to be statically linked. The need for static linking has been anticipated by Bacula's authors and is supported as an option in the upstream build procedure.

But it's not enabled in the regular Debian build procedure, and getting this done entails some modification of the source package...

Looking at the package's debian/rules file I realized that it actually runs the upstream package's configure script with a set of options. I executed ./configure --help and got a list of all the available options - the relevant option in this case is --enable-static-tools.

So, in order to enable static linking of the Bacula tools, I added the option --enable-static-tools to the configuration options provided to the configure script (by modifying CONF_ALL in debian/rules). Instead of building the whole package with dpkg-buildpackage I launched the build of the sqlite3 version of the storage daemon (which is the version I have installed), like this:

fakeroot -- debian/rules build-stamp-sqlite3

This generated a statically linked bextract at debian/tmp-build-sqlite3/src/stored/bextract, as can verified with ldd:

# ldd debian/tmp-build-sqlite3/src/stored/bextract
not a dynamic executable


While useful at times, mutilation of source packages has some obvious downsides:
  • it may require quite a bit of experimentation before you get it right
  • you're basically on your own, pretty much the same situation that you face when downloading any non-Debian source package
  • maintenance can be quite a nightmare because source packages are not tracked by the Debian package management tools
Have fun.

Fun with Debian Source Packages I

Most of the time there's no need to build packages from source. But sometimes it can be useful. I did this on several occasions, for several reasons:
  • manually apply a patch from the Debian bug tracking system, so as to fix a bug
  • attempt to fix/debug a problem (this requires programming skills, and motivation - I happen to have both)
  • build binaries with different options than those used by the package maintainer
Why would anyone do the latter? Well, I'll have more to say about that next time.

For now I'll just spell out the procedure for building a typical Debian package from source (adapted from the APT HOWTO):
  • one-time step: add a deb-src line to /etc/apt/sources.list, like this:
    deb-src http://http.us.debian.org/debian/ testing main contrib non-free
  • run apt-get update
  • install build dependencies for the package you want to build (I'll use bacula-sd as an example):
    apt-get build-dep bacula-sd
  • cd to a temporary directory
  • get the source package:
    apt-get source bacula-sd
  • cd to the newly created source directory:
    cd bacula-2.2.0/
  • optional: make modifications to the source code
  • build the package:
    dpkg-buildpackage -rfakeroot -uc -b
  • optional: install the package:
    dpkg -i ../bacula-sd_2.2.0-1_i386.deb
    note that all the bacula packages are built from the same source package, so that the parent directory actually contains a bunch of binary packages.
The source directory is typically quite similar to the upstream source package, except that it contains a sub-directory named debian, where Debian specific files reside. Among these is the rules file which can be used to manually perform the various steps in the build process.

More fun to come.

Sunday, September 23, 2007

Printing Plain Text

Last time I wanted to print a document I got bitten quite hard. This time I was ready. Or so I thought. I mean, all I wanted to do was print a plain text document...

Way back when, during my time at the university, we used to print stuff with lpr, so I tried it out, and realized that it doesn't wrap around lines that are longer than the paper width. The document at hand contained newline characters only after each paragraph - it looked OK on screen, but horrible when printed out.

Well, I tried printing from within gedit (the default GNOME editor). It performs text wrapping on long lines at word boundaries, which is nice. But:
  • it prints the file name and page numbers on every page - and I couldn't find an obvious way to turn this feature off
  • there doesn't seem to be a simple way to insert a page break
  • it does not interpret or honor pagefeeds (^L characters) that are often used in plain text documents as page breaks
Next I tried gtklp: it can be made to wrap long lines, but not at word boundaries, so words get cut up between lines.

How about emacs? open the file, mark the whole document by hitting Ctrl-x-a , and then hit Meta-x (Alt-x on normal PC keyboards) and type fill-region - this does line wrapping at word boundaries, which makes the document printable with gtklp, or directly from emacs.

Except that region filling in emacs does some unexpected things - like indenting a whole paragraph if the first word is indented (I wanted just the first line to be indented). I could fix it by hand, but that didn't seem like the Right Thing To DoTM. There's probably a way to change this behavior, but I didn't bother looking for it.

I vaguely recalled using a2ps to convert plain text documents to PostScript, so I installed the a2ps package, read the manual page, and realized it didn't do line wrapping at word boundaries. Sigh.

So I searched for "word wrap a2ps", in the hope that I was missing something, and hit a blog entry which pointed to enscript as the right tool for the job.

The following command line does exactly what I want:

enscript --header='||Page $% of $=' --margin=72:72:72:72 -1 --word-wrap --media=A4 file.txt

(one inch margins on all sides, 1 up, word wrap, A4 page size, right aligned header showing page info)

Sheesh...

Sunday, September 2, 2007

Bacula to the Rescue

It finally happened - disastrous data corruption.

My mail client of choice - Icedove (the debianized Thunderbird) - started crashing for no apparent reason. It took me a few crashes to figure out that one of my email accounts was probably corrupted somehow - Icedove would crash whenever I tried to get messages for it. It even crashed when I tried to simply select its Inbox folder (I wanted to move the messages to a different folder and then delete and recreate the account).

This was definitely the right time to try bacula for real:
  1. Select "Edit->Account Settings..." menu item in Icedove's menu
  2. Find the bad account, record the path of the "Local directory" under "Server Settings"
  3. Exit Icedove
  4. Run bconsole - the bacula command console - in a terminal
  5. Enter the command restore and follow instructions to select the appropriate backup job that should be restored (I picked the latest)
  6. Eventually the command console will enter file selection mode, where you can mark files and directories to restore (hit ? and <Enter> to get a list of available commands). Select the files under the directory recorded at step 2 above.
  7. Run the restore job. Files are restored to a restore directory, so that there's no risk of overwrite.
  8. Replace the content of the bad mail directory with the restored files using

    cp -a <restore directory>/* <bad mail directory>

It's really that easy.