automatically delete old things from the Downloads folder

Lots of OSs have a handy ~/Downloads folder where all downloaded files automatically go to. This is much cleaner than say, piling them up on the Desktop or elsewhere. I generally move something out of ~/Downloads if I need it, leaving the rest to pile up. Thanks to the very helpful tmpreaper, I have it automatically deleting any file older than 5 days from ~/Downloads. Now it manages itself. You can do this easily with Debian-based OSs (sudo apt-get install tmpreaper) or Mac OS X (fink install tmpreaper). You can cron it using a little script in the right folder:

/etc/cron.daily/tmpreaper_downloads (Debian)

#!/bin/sh
/usr/sbin/tmpreaper --all --protect .localized 5d /home/hans/Downloads

/etc/periodic/daily/900.tmpreaper_downloads (Mac OS X + Fink)

#!/bin/sh
/sw/sbin/tmpreaper --all --protect .localized 5d /Users/hans/Downloads

seeing the contents of the new .pkg format for Mac OS X

The original .pkg format for Mac OS X installer packages followed the old pattern established in NeXTSTEP of a special folder format with an extension to label it a file type. Inside of the .pkg folder, you could find the Contents/Archive.bom file, .bom meaning “Bill Of Materials”, i.e. a listing of the files that the package would install. Using the handy lsbom, you can get all of the info in a nice, scriptable way. The downside to this folder format is that in order to post it for download or other similar situations, it had to be turned into a single file, like a .zip or .tar.gz.

So Mac OS X 10.5 introduced a new format where it was already bundled up into a file, so there is no Contents/Archive.bom to use with lsbom. But there is a way! The new format is still a folder, but with a different layout, and then made into a file using xar, which is something like tar and it comes with Mac OS X. So we have to treat the new format .pkg like a tarball or zip:

tmp $ mkdir dnscrypt
tmp $ cd dnscrypt
dnscrypt $ xar -xzf ~/Downloads/dnscrypt-osx-client.pkg
dnscrypt $ ls
Distribution                            dnscrypt.pkg                  dnscryptproxy-1.pkg
Resources                               dnscryptClientPostflight.pkg  dnscryptproxy.pkg
comopendnsosxdnscryptconfigupdater.pkg  dnscryptClientPreflight.pkg   dnsupdater.pkg
comopendnsosxdnscryptmenubar.pkg        dnscryptmenubar.pkg

This package actually has multiple sub-packages, but they are all bundled together, so the one xar operation has expanded them all into folders again. There is no longer the extra Contents/ folder level, the files are just directly in the .pkg folder. And the Archive.bom file has been renamed to just Bom. And with a little bash script, we can see the whole contents:

for bom in */Bom; do echo --------------------; echo $bom; lsbom $bom; done

turning a series of tarballs into a series of git commits

I find that gitk, the git history browser, is immensely valuable for navigating code. I found myself wanting to navigate the history of code for which I only had the release tarballs. So I wanted to turn them into a series of git commits. The tricky thing here is that I wanted to include adds and deletes, so basically I rm -rf the contents of the git repo, then untar the tarball into the git repo, then you can see adds and deletes. Here’s the script that I actually used:

#!/bin/sh

for tarball in `\ls -1 Pd-0.42.5-extended-l2ork-dev-201*.tar.bz2 | sort`; do
	 date=`ls -l --time-style=long-iso $tarball | cut -b35-52`
	 echo --------------------------------
	 echo $tarball $date
	 rm -rf pd-l2ork.git/*
	 tar --exclude-vcs \
		  --exclude .svn \
		  --exclude .git \
		  --exclude pure-data/pd/.git \
		  --exclude \*~ \
		  --exclude pure-data/pd/bin \
		  --exclude pure-data/pd/obj \
		  --exclude \*.o \
		  --transform 's|pure-data/pd|pd-l2ork.git|' \
		  -xjf $tarball pure-data/pd
	 cd /export/storage/pd-l2ork/pd-l2ork.git/
	 echo "git add"
	 git add `find . -type f | grep -v .git`
	 echo "git commit"
	 git commit --date="$date" --author="Ivica Ico Bukvic " -am "$tarball"
	 cd /export/storage/pd-l2ork/
done

svn vs. HFS+ on GNU/Linux

I run Ubuntu and Mac OS X on a MacBook Pro as my main machine. I do most development work on Ubuntu and most emailing, calendaring, etc. on Mac OS X. But sometimes I want access to my source code in Mac OS X to do quick little fixes and the like. So I made a case-sensitive HFS+ partition that is mounted read/write in both Ubuntu and Mac OS X. Ubuntu can’t handle the HFS+ journalling yet, and HFS+ without journaling seems to explode every once in a while, so I didn’t want to mount my Mac OS X system read/write on Ubuntu, hence the separate partition.

It works quite well, especially when working with git, since the folder is the repo. With svn, its a bit of a different story. It mostly works, but there are some cases where SVN freaks out. If I do svn update in a folder, it all works fine. If I do svn update Makefile, i.e. only directly updating a file, I get:


hans@palatschinken externals $ svn up Makefile
svn: Can't open file 'Makefile/.svn/entries': Permission denied


Apparently this is a known issue to the SVN devs, but they don’t want to fix it: Re: svn diff issue: svn confuses files with dirs on hfsplus?

libtool error dumps

So GNU libtool and the whole autotools suite does seem to help a lot with complicated builds that need auto-detection of all sorts of things, but they can also be an exercise in totally obtuse errors, bizarre scripting, and frustration. I suppose all build systems really share those qualities in some respect. One thing that I’ve run into with puredata/pd-extended is this strange error dump from libtool that seems to be unfindable via search engines. Here’s a snippet:

../libtool: line 844: X--tag=CC: command not found
../libtool: line 844: X--tag=CC: command not found
../libtool: line 844: X--tag=CC: command not found
../libtool: line 844: X--tag=CC: command not found
../libtool: line 877: libtool: ignoring unknown tag : command not found
../libtool: line 877: libtool: ignoring unknown tag : command not found
../libtool: line 844: X--mode=compile: command not found
../libtool: line 844: X--mode=compile: command not found
../libtool: line 877: libtool: ignoring unknown tag : command not found
../libtool: line 877: libtool: ignoring unknown tag : command not found
../libtool: line 844: X--mode=compile: command not found
../libtool: line 844: X--mode=compile: command not found
../libtool: line 1011: *** Warning: inferring the mode of operation is deprecated.: command not found

Why on earth did libtool prepend that ‘X’ to the command line options and try to run them as commands? WTF? This has baffled me to no end… until I realized its because a one version of libtool does not like files generated by a different version of libtool! Arg. I solved this by deleting the m4 folder in my project, then restoring what files are needed in that folder by doing git reset --hard

backing up and restoring a Nokia N810

So I use a Nokia N810 running Maemo GNU/Linux as my main device, I send/receive SMS, IM, email, and phone calls with it. I not so long ago dropped my trusty N810 into a bucket of dirty mop water… if you check the archives, you’ll see that I revived it, and have been using it as my main device ever since. It made a miraculous recovery. But then… I was stupid enough to dropkick it into a wall. It was still running fine, but the screen had about a good 10% of it covered in technicolor gibberish. Then it started to heal! Well, the screen did at least, it healed completely, and the whole screen worked fine… but! The touchscreen started getting more and more out of whack until it finally died entirely. Hard to use a touchscreen device without a working touchscreen, even tho the keyboard is quite nice.

Enter William Ward, I put the call out for me to purchase someone’s old N810. He so kindly gave me his old N810 which he no longer used. Now I just wanted to clone my old device, its a heavily customized setup. Then I remembered, its just a linux box, sooo I can just rsync it! To make it a bit more reliable, I first rsynced the old device to my home server. I ran this command as root (to make sure the perms worked) on my home server:

rsync -axHAXv --progress -e ssh root@10.0.0.135:/ root/
rsync -axHAXv --progress -e ssh root@10.0.0.135:/media/mmc2 media_mmc2/

Then once that finished, I just rsynced over everything to my new N810 with working touchscreen, again running the command as root on my home server:

rsync -axHAXv --delete --progress -e ssh root/ root@10.0.0.140:/
rsync -axHAXv --delete --progress -e ssh media_mmc2/ root@10.0.0.140:/media/mmc2

Et viola! I had cloned my old install to the new hardware!

DO NOT try to run GNU/Linux on a MacBook without a working CD drive!

I’ve been happily dual booting between Ubuntu and Mac OS X on my 3rd gen MacBook Pro 3,1 for a while now. Its CD/DVD drive is barely functional, but I never use it, so I never thought about replacing it. Then I swapped out the hard drive for a new one, and messed up the grub 0.97 boot loader in the process. I tried installing grub on my new disk when it was in an external firewire/USB enclosure. I tried installing grub while the new disk was installed and booted into target disk mode. I tried making a bootable USB or Firewire drive to start Debian or Ubuntu. I tried, I tried, I tried… to no avail besides messing up my booting royally so I couldn’t boot to Mac OS X even without first booting to Target Disk Mode and running a Disk Utility “Repair Disk” on another MacBook on all my partitions. Sometimes my disk would not even show up in the other Mac’s Disk Utility, at all, like it didn’t even exist. Arg. Scary to say the least.

I tried booting CDs but failed. Arg. Then I finally found that my MacBook Pro’s hobbled “SuperDrive” could actually read and boot from a real, printed Ubuntu 9.10 install disk! Its nice and silvery since its printed in aluminum, not burned. So my hobbled read it find, and I did the standard grub install and voila! All is good:

grub-install --no-floppy --recheck --root-directory=/media/Ubuntu /dev/sda3

(yes indeed, that’s grub installing on the partition /dev/sda3, not the straight disk /dev/sda). And remember to read the instructions of these things and actually follow them. I kept wondering why rEFIt wasn’t installing again, with the cd /efi/refit/; ./enable.sh command. Duh, because I didn’t do as they say and reboot twice!.

Maybe now that I have finally made it out of the hell of trying to get my Ubuntu booting again, I might be willing to dive in again to get grub2 working. Booting .iso files directly would be pretty sweet!

sharing /home across Ubuntu and Mac OS X

I dual-boot between Ubuntu and Mac OS X on my MacBook Pro, and I like to have the setups parallel each other as much as possible. This means I can do things like share git repos and Android AVDs across OSes. One annoying detail is that the root of the home directories is different in each OS. In GNU/Linux, its /home. In Mac OS X, its /Users. That can be easily solved by using a symlink, i.e. ln -s /Users /home. But arg, Mac OS X’s NFS automounter gets in the way! It mounts /home and then if you use a networked home folder, it’ll mount it in /home. But who uses that on a laptop?

$ mount
/dev/disk0s2 on / (hfs, local, journaled)
devfs on /dev (devfs, local)
fdesc on /dev (fdesc, union)
map -hosts on /net (autofs, automounted)
map auto_home on /home (autofs, automounted)
/dev/disk0s3 on /Volumes/Ubuntu (fusefs_ext2, local, read-only, synchronous)
/dev/disk0s4 on /Volumes/Untitled (fusefs_ext2, local, read-only, synchronous)
/dev/disk0s5 on /Volumes/64-bit (fusefs_ext2, local, read-only, synchronous)
/dev/disk0s6 on /Volumes/share (hfs, local)

I say get rid of it! Luckily Apple hasn’t killed all the UNIXiness of Mac OS X, and you can still do useful things by editing simple text files. The NFS automounter is a good example, its settings are in /etc/auto_master and /etc/auto_home. So I edited /etc/auto_master as root and just commented out the line about automounting /home, then told the automounter to re-check its config, and voila! No more annoying /home mount! So in summary:

$ cd /
$ sudo emacs /etc/auto_master
$ sudo automount -vc
$ sudo umount /home
$ sudo rmdir /home
$ sudo ln -s /Users /home

supporting adb for Android phones on Ubuntu

Working with Eclipse/Android/adb on Ubuntu works quite well. There is one minor annoyance with it: the Android SDK does not set up the udev rules that allows the adb tool to find the phones when you plugin them in via USB. You need to add a one-liner udev rules file to /etc/udev/rules.d for each phone you want to support. Here is my collection:

10-archos-5-tablet.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0e79", ATTRS{idProduct} =="1361", MODE="0600", OWNER="hans", SYMLINK+="android_adb"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="0e79", ATTRS{idProduct} =="1361", MODE="0600", OWNER="hans", SYMLINK+="android_fastboot"

10-htc-eris.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct} =="0c98", MODE="0600", OWNER="hans"

10-htc-g1.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c01", MODE="0600", OWNER="hans"

10-htc-hero.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct}=="0c02", MODE="0600", OWNER="hans"

10-htc-wildfire.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0bb4", ATTRS{idProduct} =="0c8b", MODE="0600", OWNER="hans"

10-motorola-droid.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="22b8", ATTRS{idProduct} =="41db", MODE="0600", OWNER="hans"

10-nexus-one.rules

SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", MODE="0600", OWNER="hans", SYMLINK+="android_adb"

To use them, copy these all to /etc/udev/rules.d, then change hans to be your username. Then you need to restart udev and adb to make sure these changes take effect:

$ sudo restart udev
$ adb kill-server
$ adb start-server
$ adb devices

You should now see a report of your Android device(s) that are currently attached. Here’s an example with an HTC Wildfire attached:

$ adb devices
List of devices attached
HT0AFPY04124	device

download mp3s from Google Voice

So I use Google Voice for my voicemail, which gives often ridiculous computer transcriptions of the voicemail. I also use Vonage, which gives quite accurate human transcriptions, and includes an mp3 of the voicemail in the email. This is a much better situation, but Google Voice doesn’t let me turn of its voicemail so that only my Vonage voicemail service takes messages. Arg. So in my annoyance, I decided to try to write a script to download the mp3s from Google Voice. Turns out it was quite easy, especially because I was using python. Here’s my script, to use it, right click on the Play message link in the email, and copy it, then use that URL as the argument to this script (you’ll probaby want to change the dldir to something that works for your computer):

#!/usr/bin/python

import sys, re, time
from urllib2 import Request, urlopen, URLError, HTTPError

dldir = '/Users/hans/Desktop/'

url = sys.argv[1]
f = urlopen(url)
pagecontents = f.read()
f.close()

regex = re.compile("'messagePath': '(.*)'", re.MULTILINE)
for match in regex.finditer(pagecontents):
    mp3url = match.group(1)

regex = re.compile('gc-message-name">(.*)(.*)</span', re.MULTILINE)
for match in regex.finditer(pagecontents):
    msgtime = match.group(1)

msgdate = time.strftime('%Y-%m-%d %H.%M', time.strptime(msgtime, "%m/%d/%y %I:%M %p"))

filename = 'voicemail from ' + name + ' on ' + msgdate + '.mp3'
print "filename: " + filename
mp3 = urlopen(mp3url)
mp3contents = mp3.read()
mp3.close()
writefile = open(dldir + filename, 'w')
writefile.write(mp3contents)
writefile.close()