Archive for October 20, 2006

Mobile wget

Mobile wget

Mobile wget,
originally uploaded by Peru Tha Damaja.

I’m super excited about this. I got wget rolling again on my phone. wget is a utility that can slurp a web site with most of the flexibility you’d want to make it really powerful: link traversal with a custom link depth, control over whether it leaves the host site or only traverses links within that site, control to force it to only traverse links within some certain path or to exclude links within some other path, have it do link conversion (so that they work after you download), AND MORE!!!

On a Windows Mobile device it’s a lot trickier to get this rolling. First, it doesn’t come with a console so you need to get a console application. Microsoft’s CMD doesn’t seem to work for me any more but SymbolicTools’ PocketConsole does. But to get that to work, you need to tweak a reg key so that console applications work at all (Total Commander is a great tool with shell exploring, registry editing, and file viewing neatly integrated). Now that you’ve got a working console, you need Rainer Keuchel’s wget port. To use that port (as with all of his ports) you need a couple compatibility dll’s. Once you’ve got the console working and his port harness dll’s, you need a couple more tweaks to make wget and his other ports function properly since all these utilities rely on environment variables, which doesn’t exist in Windows Mobile. The shim is pretty straightforward - set up some more reg keys in HKLM/Environment which the port respects just like a regular environment. Now you can modify your _wgetrc in %home%, tell it http_proxy=http://216.155.165.50:8080 (the tzones proxy) and use_proxy=on and you’re ready to go to town!

Which would make anyone ask “why would I want to go to impossibly-difficult-ville?” Well, you probably wouldn’t if you’re asking, but if you’re like me, you subscribe to the New York Review of Books electronic edition and you think it’s the bees-knees to be able to read this on the go. But there is no convenient RSS format of this (yet) so with a little jiggery pokery, you should be able to run this command to retrieve the most recent issue to your device: “wget -nH -k -r -l1 -I /contents/,/articles/ http://www.nybooks.com/current-issue”.

In our next installment: since I haven’t done this yet and I suspect wget won’t know how to consume my cookies, I’ll try covering that.


Comments