Friday, November 27, 2009

Déjà vu: HPLIP Upgrade or Yet Another Printer Problem

I use hp-timedate to synchronize the clock on my Officejet 5510 All In One Printer with my PC clock.

Here's an error message that I found embedded in the output of hp-timedate a few days ago:
error: Unable to communicate with device (code=12): hpfax:/usb/officejet_5500_series?serial=MY3C1D11KS96
error: Unable to open device. Exiting.
I checked the USB cable connection and it looked OK. I listed the USB devices with lsusb and it seemed OK:
Bus 001 Device 004: ID 03f0:3a11 Hewlett-Packard OfficeJet 5500 series
I launched hp-toolbox and was surprised to discover that both the printer and fax icons appeared with a little red X-mark decoration. And sure enough, hp-toolbox insisted that both were unavailable.

I cycled the printer power, but to no avail. I considered rebooting my box, but then I had an idea - or, rather, I finally realized that I faced a very similar situation in the past. It all seemed very familiar. I tried running hp-timedate as root, and what-do-you-know - it worked.

So, this was a permissions issue. Again. I opened /usr/share/doc/hplip/NEWS.Debian.gz and found the following:
Access to the full functionality of hplip; ink check, toolbox,
printing and scanning is now provided for members of the 'lp'
group. The use of the scanner group is depreciated.
So I added myself to the lp group from the GNOME Control Center, using the "User and Groups" task, logged out and logged in, and then verified that I could finally manage the printer from my own account.

Unlike the previous time, the permissions issue was documented alright. Not that it helped, but it's progress nonetheless. I guess.

Friday, November 20, 2009

Reinventing a Wheel: Autostarting Applications

Once reinvented, a wheel is bound to be reinvented again.

I've already described how I use the ~/.xprofile script to launch applications when X starts. One issue with this method is that the applications are launched before the Window Manager is launched, which can and does cause all sorts of weird problems. Some applications need to be started from within the Window Manager.

GNOME, and, persumably, other desktop managers, conform with the freedesktop.org Desktop Application Autostart Specification. This means, among other things, that you can autostart applications by placing a file with a .desktop extension under ~/.config/autostart for each of these applications. The contents of the .desktop file has to conform with the Desktop Entry Specification.

All that technical Mumbo Jumbo is usually well hidden under the hood. A normal GNOME user is expected to configure auto-startup applications from the "Startup Applications" control center task. It's easy, clean and sane.

But I use an alternative Window Manager - awesome. It lets me autostart applications in a myriad of ways from the ~/.config/awesome/rc.lua configuration script. None of these conforms with the freedesktop standards, which means that whenever I install/remove a package with an associated autostart application (e.g. tray icon), I need to manually update my startup script.

Well, not anymore. Here's how I handle autostart applications in my rc.lua:
os.execute('grep -ie \'^exec=\' '..
'/etc/xdg/autostart/*.desktop '..
'$HOME/.config/autostart/*.desktop '..
'$HOME/.config/awesome/autostart/*.desktop '..
'| sed -e \'s@.*autostart/@@g\' -e\'s@Exec=@@g\' '..
'| awk -F: \'{e[$1]=$2}END{for(d in e) system(e[d]"&")}\'')
This hack autostarts the applications specified by the .desktop files in ~/.config/autostart and /etc/xdg/autostart, with the former taking precedence over the latter.

Being a hack, it only considers the Exec key in the .desktop file. It does not take into account the working directory set in the Path key and/or whether to start the application in a terminal (as specified by the Terminal key). There are other deficiencies too, but these are the ones that I might be inclined to address if the need arises.

As you may have noted from the code, I added a new autostart directory under ~/.config/awesome, which has the highest priority. It houses .desktop files that correspond to autostart applications, that I only want to start under awesome. I also use it to disable some of the GNOME autostart applications by copying their corresponding .desktop file to this directory, and replacing the command specified by the Exec key within the file, with the command /bin/true.

Saturday, November 14, 2009

Cellular Phone Backup Script

As promised, here it is (commentary follows):
#! /bin/bash

section=0

photos_root="d:/predefgallery/predefphotos"
backup="$HOME/backup/files/phones"
delay=2 # time (in seconds) to wait between each phone access

grep port= $HOME/.gammurc | cut -d= -f2 |
while read phone
do
name="$(grep -m $(( $section + 1 )) name= $HOME/.gammurc | tail -1 | sed s/name=//)"
echo $name

rm -rf /tmp/$phone
mkdir -p /tmp/$phone
mkdir -p "$backup/$phone"
cd /tmp/$phone

# backup photos
gammu -s $section getfolderlisting $photos_root | grep "$photos_root" | sort |
while read filerecord
do
phone_path=$(echo $filerecord | cut -d\; -f1)
file=$(echo $filerecord | cut -d\; -f3 | sed s/\"//g)
timestamp=$(date -d "$(echo $filerecord | cut -d\; -f4 | sed s/\"//g)" +'%s')
size=$(echo $filerecord | cut -d\; -f5)
local_path="$backup/$phone/$file"
echo $file

# copy file from phone if not found on disk or has different size and/or timestamp
if [[ ! ( -e $local_path && \
"$timestamp $size" == "$(stat --format='%Y %s' $local_path)" ) ]]; then
sleep $delay
gammu -s $section getfiles "$phone_path" 2>/dev/null
fi
done

# backup phone settings
sleep $(( $delay * 2 ))
# the first no is to save backup in ascii not unicode
# the second no is to disable broken backup of contacts in SIM (gammu bug?)
gammu -s $section --backup settings.backup 2>/dev/null <<EOF
no
yes
no
ALL
EOF
# # backup sms
# # disabled - seems to hang waiting for phone
# sleep $delay
# gammu -s $section --backupsms sms.backup
cd - > /dev/null
rsync -avz /tmp/$phone $backup
let section++
done
My initial plan was to backup photos, contacts, settings and SMS contents, but I've hit several problems which made the script so ugly:
  1. Gammu fails to copy contacts from the SIM card on our phones (Nokia 2600) - I decided to only backup the contacts from the phone memory, which meant that I had to synchronize the contacts list on the phone memory with that on the SIM card
  2. Gammu hangs while trying to retrieve SMS contents from my wife's phone - I've decided not backup SMS contents
  3. photo retrieval is slow - so I complicated my script to make sure that I only retrieve new photos
  4. photo retrieval is prone to failure - waiting a second or two before each transfer seems to make communications much more robust


Phone contents is copied over to a backup directory whose name is determined by the port entry in each phone's settings section inside ~/.gammurc - which, in my case, is the Bluetooth address of the phone. You may want to use the name entry instead.

And, in case it isn't obvious, you must have a settings section for each phone being backed up, inside ~/.gammurc. The easiest way to do this is to let the Wammu phone setup wizard guide you.

Gammu comes with a Python library, which I might use someday for rewriting this script in Python. The main benefit would be that communications with each phone would only need to be initiated once, potentially making the whole process both faster and more reliable.

Friday, November 6, 2009

On Being S.M.A.R.T

After reading "Watching a hard drive die", "Checking Hard Disk Sanity With Smartmontools" and the Wikipedia article about S.M.A.R.T, I decided to install smartmontools and test my hard drives for problems:
aptitude install smartmontools

The first hard drive that I diagnosed with smartctl was my laptop's primary hard disk:
smartctl -a /dev/hda
This showed a few unsettling results, but luckily no reallocation errors or other critical errors.

I got similar results for my external Western Digital Elements hard disk. That was good, because that's my backup disk. Phew.

My other external hard disk is an old Western Digital hard disk that's in a USB connected disk enclosure. I tried diagnosing it and got the following error:
root@machine-cycle:~# smartctl -a /dev/sda
smartctl 5.39 2009-10-10 r2955 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-9 by Bruce Allen, http://smartmontools.sourceforge.net

/dev/sda: Unsupported USB bridge [0x04b4:0x6830 (0x001)]
Smartctl: please specify device type with the -d option.

Use smartctl -h to get a usage summary
I did as I was told (i.e. read the usage summary) and then tried the following:
smartctl -a -d usbcypress /dev/sda
Cypress happens to be the manufacturer of this enclosure's USB to IDE bridge, but smartctl doesn't seem to recognize it without my help.

Well, now I got a report from smartctl but it showed that one DMA CRC error was logged.

I ran tests on all hard disks with smartctl -t short ... for each device and they were all completed successfully. Phew. /Me wiping cold sweat off brow/

Next thing to do was to enable smartd to monitor all my hard disks:
  1. edit /etc/default/smartmontools and make sure you have the following line in it:
    start_smartd=yes
  2. start the daemon:
    invoke-rc.d smartmontools start
On my system this doesn't work as is, and I had to edit the daemon configuration file /etc/smartd.conf, based on the examples in the comments and the manual page:
  1. comment out the line that starts with DEVICESCAN (i.e. prepend it with a sharp sign #)
  2. add lines per each hard disk to be tested:
    # primary disk                                                                                                                               
    /dev/hda -a -o on -S on -s (S/../.././05|L/../../7/04) -m root -M exec /usr/share/smartmontools/smartd-runner
    # /dev/gigapod (multimedia)
    /dev/disk/by-path/pci-0000:02:00.2-usb-0:1.4:1.0-scsi-0:0:0:0 -a -d usbcypress -d removable -o on -S on -s (S/../.././05|L/../../7/04) -m root -M exec /usr/share/smartmontools/smartd-runner
    # /dev/elements (backup)
    /dev/disk/by-path/pci-0000:02:00.2-usb-0:1.1:1.0-scsi-0:0:0:0 -a -d sat -d removable -o on -S on -s (S/../.././05|L/../../7/04) -m root -M exec /usr/share/smartmontools/smartd-runner
    Note that this schedules short self tests to run each morning at 5AM and long self tests to run on Sunday mornings at 4AM.

    Also note that I use the /dev/disk/by-path links to the external disk block device, in order not to be hit by udev's tendency to reorder device names.

Testing, 1, 2, 3 !