simple automated backup to a USB disk on Debian/Ubuntu/Mint

I used to use Mac OS X as my main machine, indeed for 18 years since I started when it was called NeXTSTEP.  Now Apple has turned it into a paradise for mindless consumption, driving creators away.  Now I use Linux Mint as my daily system, but one thing I was missing was a really simple, automatic backup to an external USB disk.  Apple did a good job with Time Machine.  Its simple to setup, happens automatically and in the background, provides a directory structure that you can browse like any standard file system and see the complete snapshots of your computer at the time of a given backup.

So I looked at a lot of apps and tools for doing this on Mint.  I looked at Back In Time, simplebackup, pybackpack, obnam, BoxBackup, rsnapshot, and more.  Back In Time seemed promising, and I probably should have considered it more. Its totally GUI focused, and oriented towards backing up the home folder as a regular user. I want a full disk backup as root. Also, the UI hit me with a ton of options, and I didn’t want to think that much.  simplebackup just seemed like a simple way to make a single backup, with no snapshots. pybackpack is no longer developed.  obnam sounds very promising as a command line tool, but it does not represent the backups as transparent files on a filesystem, so that nixed it for me.  BoxBackup has a client/server architecture, so far too complicated.  I think its oriented towards backing up a fleet of servers to a single backup host.

rsnapshot ended up being quite simple to set up for a command line person like me, and also easy to automate.  And most importantly, it backs up easily to an external disk and writes out each snapshot as a full image of the system at the time of the snapshot.  Sold!  So I just needed to tweak it a little bit.

I had to make a couple edits to /etc/rsnapshot.conf.  One annoying detail to note in this config file is that you have to use tabs between the config items in a line, spaces won't work. So first I pointed it to a folder on my external USB backup disk:

snapshot_root /media/ext4backup/palatschinken/

Then I told it not to create the root folder on the backup disk. If the backup disk is not present, it should not try to backup at all:

no_create_root 1

Then I enabled one fs mode (i.e. -x in rsync speak), which means that rsnapshot won't cross the boundaries of a disk. So if you tell it to backup /, and you have /home on a separate partition or disk, it won't backup /home. You could then add /home specifically, then it would be backed up. Here's the option:

one_fs 1

Then I specified what I want backed up. These are each separate partitions on my system:

backup / root/
backup /media/share share/

Now the rsnapshot should happily backup. You can test it by running rsnapshot hourly.So the next step is to cron it so it runs automatically in the back ground. I made a script like this for each of the cron folders (/etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly, /etc/cron.monthly):


# See ionice(1)
if [ -x /usr/bin/ionice ] &&
    /usr/bin/ionice -c3 true 2>/dev/null; then
    IONICE="/usr/bin/ionice -c3"

test -d /media/ext4backup/palatschinken && \
	nice $IONICE /usr/bin/rsnapshot hourly

Be sure to change the hourly in each script to match the folder its in. Adding in nice and ionice makes it run at very low CPU and disk priority, so you don't notice that its running really. Lastly, if you are using this on a machine that is shutodwn or goes to sleep often, then you probably want to install anacron to make sure the daily, weekly, monthy jobs run (apt-get install anacron, it usually configures itself to do the right thing once its installed).

Converting MediaWiki to Redmine

We had a MediaWiki install for our wiki needs, and we used it lots and filled it up with good stuff. Then we got Redmine going. Redmine has its own integrated wiki, but with a different wiki syntax. We had all this lovely scribblings in our mediawiki, but we wanted the tasty redmine integration. So conversion must be done. Thanks to python, it was easy. Here’s the script I wrote:


from BeautifulSoup import BeautifulSoup
import io
import os
import re
import sys
import urllib2

# def download_inline_images(text):
#     m = re.findall()
#     for inline in m:
#         print('Downloading ' + inline)
#         image = urllib2.urlopen('' + inline)
#         with open(inline, 'w') as f:
#             f.write(

def convert_and_save(title, text):
    contents = 'h1. ' + re.sub('_', ' ', title) + '\n\n'
    contents += text

    # headers
    contents = re.sub('===== (.+?) =====', 'h5. \\1\n', contents)
    contents = re.sub('==== (.+?) ====', 'h4. \\1\n', contents)
    contents = re.sub('=== (.+?) ===', 'h3. \\1\n', contents)
    contents = re.sub('== (.+?) ==', 'h2. \\1\n', contents)
    contents = re.sub('= (.+?) =', 'h1. \\1\n', contents)

    # fix bullet lists
    contents = re.sub('\n\*\*([^ ])', '\n** \\1', contents)
    contents = re.sub('\n\*([^ \*])', '\n* \\1', contents)

    # bold - needs to be after 'fix bullet lists'
    contents = re.sub("'''(.+?)'''", '*\\1*', contents)

    # italic
    contents = re.sub("''(.+?)''", '_\\1_', contents)

    # external links with mistaken double brackets
    contents = re.sub('\[\[(http.+?) (.+?)\]\]', '"\\2":\\1', contents)

    # external links
    contents = re.sub('\[(http.+?) (.+?)\]', '"\\2":\\1', contents)

    # inline image links
    contents = re.sub('\[\[File:([^|\]]+).*?\]\]', '!\\1!', contents)

    # make sure there's a trailing newline
    contents += '\n'

    filename = re.sub('/', '_', title) + '.redmine'
    with, 'w', encoding='utf8') as f:

def crawl_url(url):
    page = urllib2.urlopen(url)
    soup = BeautifulSoup(page)
    find = soup.find('textarea')
    if not find or len(find) == 0:
    textarea = find.contents[0]

    m = re.match('.*title=([^&]+)', url)
    title =
    convert_and_save(title, textarea)

    m = re.findall('\[\[([^|\]]+)', textarea)
    for page in m:
        print 'Trying: ' + page
        if not os.path.exists(page + '.redmine') \
                and not page.startswith('File:') \
                and not page.startswith('http'):
            print 'MATCH: ' + page
            crawl_url('' + re.sub(' ', '_', page) + '&action=edit')

def main(argv):
    if len(argv) == 0:
        print "Usage: " + sys.argv[0] + " [URL]"
    if len(argv) == 1:

if __name__ == "__main__":

Cannot run program “/opt/android-sdk/platform-tools/aapt”

It seems that Google did a big reorg of the Android SDK file tree, and some tools still expect things in the old locations, like our Jenkins build server:

FATAL: Cannot run program "/opt/android-sdk/platform-tools/aapt" (in directory ": error=2, No such file or directory Cannot run program "/opt/android-sdk/platform-tools/aapt" (in directory ": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(
	at hudson.Proc$LocalProc.(
	at hudson.Proc$LocalProc.(
	at hudson.Launcher$LocalLauncher.launch(
	at hudson.Launcher$ProcStarter.start(
	at hudson.Launcher$ProcStarter.join(
	at hudson.plugins.android_emulator.util.Utils.runAndroidTool(
	at hudson.plugins.android_emulator.util.Utils.runAndroidTool(
	at hudson.plugins.android_emulator.builder.AbstractBuilder.getPackageIdForApk(
	at hudson.plugins.android_emulator.builder.AbstractBuilder.uninstallApk(
	at hudson.plugins.android_emulator.InstallBuilder.perform(
	at hudson.tasks.BuildStepMonitor$1.perform(
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(
	at hudson.model.Build$
	at hudson.model.Build$BuildExecution.doRun(
	at hudson.model.AbstractBuild$
	at hudson.model.Run.execute(
	at hudson.model.ResourceController.execute(
Caused by: error=2, No such file or directory
	at java.lang.UNIXProcess.(
	at java.lang.ProcessImpl.start(
	at java.lang.ProcessBuilder.start(
	... 19 more

Luckily there are symlinks! This fixed it for me:

cd /opt/android-sdk
ln -s 18.0.1 build-tools/current
ln -s ../build-tools/current/aapt platform-tools/aapt
ln -s ../build-tools/current/lib platform-tools/lib

Then whenever the build-tools get updated, just the symlink in /opt/android-sdk/build-tools/current needs to be updated. This is a little improvement on the solution found here.

sign and encrypt while unencrypting using gnupg

I have a file that was encrypted to my OpenPGP key and I want to use gpg to decrypt then encrypt to someone else's OpenPGP key while signing it with my key. After reading some man pages and searching the internet, I figured this out:

gpg --decrypt secrets.gpg | gpg -o signed-secrets.gpg  --encrypt --sign -r

Then to make this file full paranoid, I'm also going to use symmetric encryption with a long generated passphrase. Here's how I did that:

pwgen=`pwgen 32 1` && echo $pwgen && echo $pwgen | gpg --batch --passphrase-fd 0 --symmetric signed-secrets.gpg; unset pwgen

Running a Jenkins Build slave in MinGW MSYS bash

I’m a UNIX guy but I have to build some of the software that I work on for Windows. Luckily there are lovely projects like Cygwin and MinGW for people like me. I’m using Jenkins to automate a lot of builds for Debian, Ubuntu, Mac OS X, and Windows, so I wanted the Jenkins slave on Windows to have a UNIX shell like all the rest of the slaves. Normally when you install the Jenkins slave as a service, it is running in a Windows environment by default, just like any other Windows service. The jenkins slave can be manually started in the MinGW MSYS bash shell, then the jenkins slave will be running jobs in the MinGW MSYS environment.

Luckily, there is an .XML config file for the Jenkins slave, and it can be configured to run the jenkins slave to run in MinGW MSYS bash. Here’s how I did it:

  1. follow the instructions for Install Slave as a Windows Service
  2. in cmd.exe run jenkins-slave.exe uninstall to uninstall the service
  3. edit these two lines jenkins-slave.xml to look like this:

    <arguments>-c "java -Xrs -jar /c/jenkins-build-path/slave.jar -jnlpUrl https://my.jenkins.server/computer/my.jenkins.slave/slave-agent.jnlp"</arguments>
  4. in cmd.exe run jenkins-slave.exe install to re-install the service

Then I can start it and ultimately have Jekins running in a nice, UNIXy, MinGW MSYS bash shell!

Setting up a chroot for Raspbian

I want to be able to build packages for Raspbian but on a regular machine.  Luckily, there is the very handy qemu-debootstrap which will let you setup a normal debootstrap chroot that runs via qemu.  Here’s how I did it (this is on Squeeze, for newer releases, you won’t need the backports.  You need qemu-user-static v1.1.2 or newer):

sudo apt-get -t squeeze-backports install qemu-user-static
sudo apt-get install dchroot schroot debootstrap wget
sudo dpkg -i raspbian-archive-keyring_20120528.2_all.deb
sudo mkdir /var/chroot/raspbian-armhf/
sudo qemu-debootstrap --keyring /usr/share/keyrings/raspbian-archive-keyring.gpg \
 --arch armhf wheezy /var/chroot/raspbian-armhf
echo raspbian-armhf | sudo tee /var/chroot/raspbian-armhf/etc/debian_chroot
sudo ln -s ../proc/mounts /var/chroot/raspbian-armhf/etc/mtab
sudo cp /etc/passwd /etc/group /etc/shadow /var/chroot/raspbian-armhf/etc

Then I edited my /etc/fstab to add the mount points:

# raspbian-armhf chroot
/home           /var/chroot/raspbian-armhf/home        none    bind            0       0
/tmp            /var/chroot/raspbian-armhf/tmp         none    bind            0       0
/dev            /var/chroot/raspbian-armhf/dev         none    bind            0       0
proc-chroot     /var/chroot/raspbian-armhf/proc        proc    defaults        0       0
devpts-chroot   /var/chroot/raspbian-armhf/dev/pts     devpts  defaults        0       0

I edited /etc/dchroot.conf to add the new chroot:

raspbian-armhf /var/chroot/raspbian-armhf

And then /etc/schroot/schroot.conf to add the new chroot there:

description=raspbian wheezy armhf

Now I'm ready for the last steps, then I can use my chroot:

sudo mount -a
dchroot -d -c raspbian-armhf

If it was successful, you should see your prompt change to:


fighting to simplify Android’s AssetManager

I like Android but I am not a fan of Java. I find that Java is overly pedantic and verbose, with no good reason unless you’re working on a team with hundreds of programmers. My guess is that it is exceedingly rare for Android apps to be written by teams larger than 10, so all this Java Enterprise thinking is constantly getting in the way.

The Android AssetManager is a good example of that. It does one thing nicely: whatever you drop into the assets/ folder of your Eclipse project is automatically bundled up into the .apk and can be retrieved using AssetManager. The way that they expect you to get each file out of the AssetManager is to explicitly request each file via

Most of the time, I'm using assets to deliver a bunch of native programs, scripts and libs, so not really things used directly in Java code directly like icons or soundfiles. In this situation, that means I have to manually maintain a list of files I want to extract in the Java code where I extract it. That is silly busy work when the apk will automatically include all of the files in the assets/ folder.

Luckily there is AssetManager.list() which lists the filenames inside of assets in the apk. Then you can use to open each asset, which returns an InputStream to the file. The pisser is that you can't tell whether the asset String you just got from AssetManager.list() is a path to a file or a directory. And will give you a FileNotFoundException if you try to open a directory. To make things even worse, the Android build system likes to throw in some empty folders just to make your life difficult. sounds/, webkit/, and images/ are the most common of these folders, but some vendors weren't satisfied to stop there, Samsung threw in an empty kioskmode/ and Motorola threw in an empty databases/.

So what I like to do is first check for those directory names and ignore them. Then catch any FileNotFoundExceptions when running

for (String asset : assetList) {
    if (asset.equals("images")
            || asset.equals("sounds")
            || asset.equals("webkit")
            || asset.equals("databases")
            || asset.equals("kioskmode"))

    int BUFFER = 2048;
    final File file = new File(NativeHelper.app_bin, asset);
    InputStream tmp;
    try {
        tmp =;
    } catch (FileNotFoundException e) {
    // do what you need to here

Here's an example of this in action: info/guardianproject/lildebi/


Get every new post delivered to your Inbox.