Actually, I haven't had a problem for quite a while, until last Thursday. It's not that I got an error, I didn't get any message at all - my mailbox was empty.
My initial guess was that the Bacula director daemon was stopped for some reason, so that the backup jobs were never started. I promptly started the Bacula console program bconsole, and used the run command three times, to start each backup job.
And then it hit me: bconsole would've reported an error if it could not connect to the Bacula director daemon. I typed list jobs and sure 'nuff - last night's jobs were on the list, marked as successfully completed.
By this time the backup jobs that I manually launched earlier were also completed, and no - there were no new messages in my Inbox. Something was broken. It wasn't critical - after all, backup jobs ran to completion - but my habits were disrupted, and I didn't like it at all.
That evening I sat down to fix the issue, or at least figure out what went wrong. I suspected that Bacula's configuration was somehow broken, probably due to a recent upgrade. I fired up emacs, opened /etc/bacula/bacula-dir.conf and /etc/bacula/bacula-dir.conf.dist, and compared them with M-x ediff-buffers.
The file with the .dist extension is the one that would've been installed upon a fresh install of Bacula. This file is left behind after an upgrade in order to allow the system administrator to do exactly what I was doing: compare the current configuration with the pristine configuration, that's shipped with the package.
Apart for the obviously different bacup/restore job definitions, there was indeed a difference in the command line used to launch bsmtp - the utility that's shipped with Bacula and is used to send email notifications. Aha! gotcha!
I "fixed" the command line, saved the file, restarted the director daemon
invoke-rc.d bacula-director restartand launched a backup job using bconsole.
The job was completed with no errors, but I got no email notification. I read the log messages carefully and noticed that bsmtp reported a fatal error (it could not connect to localhost). So I copied the bsmtp command line from the configuration file and pasted it into a root console window. I got the same error message.
So now it looked like a problem either in bsmtp or exim4 (the mail server). I tried restarting the mail server:
invoke-rc.d exim4 restartand got an error message:
Stopping MTA for restart:.So the culprit was exim4, not Bacula. I googled for the error message and hit Debain bug #476987. I read the thread of messages there, which mention several workarounds, till I got to the last message which simply stated that the bug has been fixed in version 4.69-3. I used
Restarting MTA:exim: incompatible command-line options or arguments
invoke-rc.d: initscript exim4, action "restart" failed.
apt-show-versions exim4to find out that the version installed on my machine is 4.69-2.
I could've stayed with this version, getting along with no email notifications from Bacula, until the fix trickled in from Debian/sid. But sticking to my habits is a motivation too strong to ignore - I decided instead to install the fixed (unstable) version of exim4 (please consult my previous post about maintaining a mixed testing/unstable Debian system).
I used the following command to upgrade any already installed package with the string "exim4" in its name, to the version that's in the unstable repository:
aptitude install -t unstable "~nexim4 ~i"I then restarted exim4, retried the backup job and finally got a notification email. Yay.
Fixing problems caused by recent upgrades is becoming another habit for me - it took maybe half an hour to sort this out, less than it took to write this blog post.
I was once told that pain is not habituating. Maybe I didn't get the point, but if you can't get used to pain then I guess this kind of activity can't be considered painful. But if this isn't pain then it must be fun, right?