Friday, July 29, 2011

Configuring hpodder To Handle Compressed Podcast Feeds

My Nokia 2730 classic dumbphone is surprisingly smart. Unbeknownst to me at the time of purchase, was the fact that it has a 2GB MicroSD card, and can play MP3 files. But eventually I did stumble upon this feature and it wasn't too long before I started tuning in on podcasts.

I use hpodder (launched from a cron job) to follow podcast feeds and download episodes to my computer, and a semi-scripted procedure to move these files to my phone's memory card over a USB connection.

Lately, hpodder started complaining:
*** 12: Failure parsing feed: Lexical error in  file http://escapepod.org/feed/  at line 1 col 1:
  unrecognised token: ^_\213^H^@
So I tried to download the feed manually:
$ curl http://escapepod.org/feed/
and the terminal filled up with gibberish to the point that I had to blindly type reset in order to fix it.

That was weird: after all, the feed is nothing more than an XML file - a text file, which should be perfectly readable with the naked eye.

I saved the feed with
$ curl -o feed.bin http://escapepod.org/feed/
and then determined its type:
$ file -b -i feed.bin
application/x-gzip; charset=binary
$ zcat feed.bin | file -b -i - 
application/xml; charset=utf-8
I.e. a GZIP compressed XML file.

So hpodder choked on a compressed feed.

I consulted the manual and found out that hpodder delegates downloads to cURL. I skimmed through the cURL manpage, found about the --compressed command line option, tried the download again - and got the uncompressed XML contents. Hoozah!

Apparently, the server, that's serving that particular feed, is mis-configured to always compress its replies, even if not specifically asked to do it. The --compressed command line options tells cURL to request the server to compress its replies, and cURL decompresses them.

I tried downloading other feeds with the --compressed added, and it worked fine. So either this option is supported by all the other servers on my list of feeds, or that cURL does nothing when the reply is not compressed. I dunno.

All that I needed now was a way to convince hpodder to launch cURL the same way.

Turns out that hpodder is a rather nice piece of software (and rather well documented too). The hpodder manual pointed me to ~/.hpodder/curlrc:
$ echo compressed >> ~/.hpodder/curlrc
and now hpodder works like a charm again (and probably faster than before, because it always asks for compressed replies).

No comments:

Post a Comment