Joys of Gametrak

The Gametrak is a really nice, cheap device for tracking the location of two things in space. I got a few for $15 shipped, but they are Xbox and Playstation controllers, which means they are difficult to use with computers. Arg. In reality, Microsoft, Sony, Nintendo, etc. make it difficult to use their controllers in interesting ways because they use standards like HID they deliberately break part of them to ensure that they can get away with charging licensing fees for anyone who wants to make a controller that is compatible with their game consoles.

I am using an Xbox version with Mac OS X with the XBox HID driver, but that gives crappy results (one z doesn’t read at all, and the other gives values that are 23-89, i.e. very low resolution.). The two x and ys seem ok. There is supposed to be a similar driver for Playstation controllers, but I haven’t tried in on Mac OS X, or anything on Windows.

I was using a Playstation SIXAXIS very nicely with Pd (aka Pure Data) on Ubuntu. I wrote a Pd class for it called [sixaxis] There is a good chance that the Playstation controller support in the linux kernel supports the gametrak as well, but I haven’t tried it. Basically, the Playstation controllers are standard HID device, but you need to send them a specific command to get them to start sending data. You can see an example of that in the linux kernel drivers, but this could be done with libusb or other APIs I suppose.

My plan going forward is to replace the electronics of my gametraks with an AVR/HID. Then I can guarantee that I get at least 10-bit data, and it is a standard HID device.

 

downloading .gz files from archive.org

I am a huge fan of archive.org, it is a very useful website, and a extremely important mission. Recently, I was enjoying the Wayback Machine when trying to find some information on iPodLinux. It turns out that ipodlinux.org has been down for a long time, perhaps no one pays the hosting bills anymore. But thankfully, it was all archived by archive.org.

So I needed to get some source files like Pdpod.c.gz from ipodlinux.org/Pdpod, which was last archived on 2007-12-09. The files are there, but the downloading behaved strangely, like it ended too quickly or something. Then the resulting files were always corrupt. So I tried wget, it kept on retrying the download, saying Connection closed at byte 6372. Retrying.

I decided to look at the binary file with less to see if I could deduce anything. Eureka! I can see it, but it still doesn’t uncompress, gunzip says gzip: Pdpod.c.gz: unexpected end of file. Hmm, but less can see it, so gzip must work somehow. I ran it using -c to send the uncompressed data to stdout on the fly:

gunzip -c Pdpod.c.gz > Pdpod.c

And even though gzip still complains, the whole file is there. Yeeee haw…

 

rsync vs. FAT32

So I have an SD card formatted FAT32 for my Palm TX. I want to use rsync to sync mp3s, images, etc. to that card from Mac OS X, since Missing Sync has royally sucked and wasted lots of my time trying to do the same thing. I wrote my handy little script using rsync, but I noticed that it was copying all of the files each time. Strange…

It turns out that FAT32 has a very low time resolution, compared to ext2/3 or HFS+. Luckily, rsync has an option to solve that. So I added --modify-window=1 and it works now.

 

recovering from a MySQL meltdown

Joy of joys, my server is crashes for some ungodly reason, and it took out my MySQL server. Arg… The damn thing won’t start at all, it crashes badly everytime I start it. And it gives me a big, fat crash message, and even the backtrace is broken:

InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
080811 15:51:19  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 2057517825.
InnoDB: Doing recovery: scanned up to log sequence number 3 2057578233
080811 15:51:19  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=0
read_buffer_size=131072
max_used_connections=0
max_connections=100
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217599 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb511d4a8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81c06b9
0x83cba12
0x83cc69b
0x8390215
0x8392120
0x837f38a
0x83aa73c
0x82f0308
0xb7ed10bd
0xb7d0c01e
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

But thankfully, this MySQL was running on Debian. So through the magic of apt, I got everything running again. Basically, I renamed the key folders, then I purged MySQL server and reinstalled it, even deleting the databases (since they were moved elsewhere anyway):


mv /var/lib/mysql /var/lib/mysql-old
mv /var/log/mysql /var/log/mysql-old
aptitude purge mysql-server mysql-server-5.0
aptitude install mysql-server


Ok, now at least mysqld starts. Then I switched back the original files, keeping the Debian generated one, just in case:


/etc/init.d/mysql stop
mv /var/lib/mysql /var/lib/mysql-deb
mv /var/log/mysql /var/log/mysql-deb
mv /var/lib/mysql-old /var/lib/mysql
mv /var/log/mysql-old /var/log/mysql
/etc/init.d/mysql start


Wow, it worked… I wonder how borked the data is, it’s time for a backup, then we’ll see…. ah, scheiße, there is more recover pain to endure. Hmm… first one is a simple one. I tried deleting some cruft databases, but it wasn’t fully deleting them. And none of the databases were showing up in phpMyAdmin or SHOW DATABASES;. Turns out the perms were wrong. This helped that problem:


chown mysql.mysql /var/lib/mysql/


Ok, things were going swimmingly, until I got an error #1 - Can't create/write to file '/tmp/#sql_3c2e_0.MYI' (Errcode: 13). Crazy ass shit. Turns out that /tmp had strange permissions, only root can write:


root@blinky:mysql > ls -ld /tmp/
drwxr-xr-x 17 root root 262144 2008-08-11 18:19 /tmp//


So I rectified that with a little chmod 1777 /tmp/, but now, WTF!??!, mysql crashes again on startup, with the same error!

 

watch out with those mount point names!

So I am using Ubuntu Hardy more and more, and am quite impressed at how integrated things are. For a long time, Debian has been very well integrated on the command line for servers and things like that, but Ubuntu is really tackling giving all of the myriad apps a standard GUI. GNOME is helping a lot too, of course.

So I just set up a firewire drive to use for development work, and I saw in the Properties panel for the disk, under the Volume tab, I could set the mountpoint name. So I typed out a whole path in the Mount Point: field, /home/hans/code/openembedded, as I usually would when mounting on the command line. Then hit Close, and unmounted and remounted the disk.

When I plugged it back it, I got this obtuse message:

Cannot mount volume.
Unable to mount the volume.
Details
mount_point cannot contain the following characters: newline, G_DIR_SEPARATOR (usually /)

Hmm, WTF? Now what, the disk won’t mount, so I can’t bring up that Properties panel again… arg… hmm…. think, think. Let’s try running gconf-editor from the command line, for any (former?) Windows users, that’s like the Registry Editor. I try Edit -> Find…, type in openembedded, and check the boxes to search both key names and key values. That gives me the setting, where I change /home/hans/code/openembedded to openembedded, and it works! Phew…

 

wifi weirdness on my Palm TX

Ok, so you might start to notice a little trend here with these posts. In July, I started work on a big project to create easy-to-use firmwares for old embedded devices called Reware (thanks to Turtle Kalus for the name). So I’ve been hacking my own Palm TX, and of course, discovering annoying, technical details in the process.

So now for my latest issue: So now I’ve been booting my Palm TX using HackNDev’s GNU/Linux images off of an SD card. It works really well, thanks to things like Cocoboot, the little Palm app that starts booting the Linux kernel. But now I’ve noticed that whenever I reboot back into PalmOS from GNU/Linux, my wifi behaves badly. Basically, when I try to connect to a wifi network, it doesn’t use the network I select. Instead, it uses the name that is on top of the list of the previous networks that aren’t currently enabled. It’s maddening, no matter which active network I try, it always selects the topmost inactive network.

Luckily, I found a simple solution! Eureka! I merely turn on the wifi, it doesn’t need to be connected, just on. Then I hit the reset button on the back of my Palm TX. Voila! The wifi works properly again.

Just for fun, here’s my Palm TX booting HackNDev’s image:

 

How to format an SD card for a Palm TX…

So I just spent a joyous few hours trying to figure out how to format my 2GB SD card for my Palm TX and have it work both on my laptop and on the Palm. The Palm will happily format the card itself, and it’ll work just fine inside the Palm, but alas, nowhere else. So then I tried formatting the card on my laptop, both with Ubuntu and with Mac OS X, and then the Palm just wants to format it when I stick it in.

So I dove in deep into the various geeky tools to see what I could do. I tried to repartition the card using cfdisk /dev/sdb1 but this lovely Palm format creates something that cfdisk doesn’t recognize:

FATAL ERROR: Bad primary partition 0: Partition ends in the final partial cylinder

Using fdisk allowed me to see that the partition and the filesystem were marked as FAT16, so I could mount it if I told mount which filesystem it should see, like this: mount -t vfat /dev/sdb1 /mnt/tmp. So I can mount it ok, but not everything is happy:

[93139.773051] attempt to access beyond end of device
[93139.773067] sdb: rw=0, want=2797588, limit=2011136

Hmm, something is amiss. My guess is that this SD card is slightly bigger than 2GB in actual capacity, and FAT16 has a hard 2GB limit. The Palm is joyfully formatting the whole thing as FAT16, even though it should stop at 2GB. Anyway, back to making things work. I read lots of forums, tried some new tools like mkdosfs -F 16 /dev/sdb1 and dosfsck /dev/sdb1, but still it wasn’t working. I had read somewhere that Palms didn’t support FAT32, which is the FAT for disks bigger than 2GB, so I didn’t try that… until now.

So I created a new partition using the lovely cfdisk and set the partition type to 0B W95 FAT32, then I used mkdosfs /dev/sdb1 to create a FAT32 filesystem (mkdosfs tries to use the most appropriate version of FAT). And…. voila! It works on my Palm TX, it works on Ubuntu, it works on Mac OS X. And I am happy…

 

if: Expression Syntax.

Yay for the syntax error! Usually they are the simplest of problems, you forgot a semi-colon, or left off a parenthesis. But while I was building my OpenEmbedded build environment I ran into the error listed in the title here, but it seems far from simple. It is nested deeply within many layers of build systems:

make TARGETS=oneprocess SHELL=”/usr/bin/tcsh” CC=”ccache gcc” CFLAGS=” -isystem/home/hans/code/openembedded/stuff/tmp/staging/i686-linux/usr/include -Os -DIN_GCC -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -pedantic -Wno-long-long -Wno-error -DHAVE_CONFIG_H -DGENERATOR_FILE” LDFLAGS=”" LIBERTY=”/home/hans/code/openembedded/stuff/tmp/work/i686-linux/gcc-native-3.4.6-r9/gcc-3.4.6/build.i686-linux.i686-linux/gcc/../libiberty/libiberty.a” install-bin
make[2]: Entering directory `/home/hans/code/openembedded/stuff/tmp/work/i686-linux/gcc-native-3.4.6-r9/gcc-3.4.6/build.i686-linux.i686-linux/gcc/fixinc’
/usr/bin/tcsh /home/hans/code/openembedded/stuff/tmp/work/i686-linux/gcc-native-3.4.6-r9/gcc-3.4.6/gcc/fixinc/genfixes machname.h
SHELL=/bin/sh: Command not found.
export: Command not found.
if: Expression Syntax.
make[2]: *** [machname.h] Error 1
make[2]: Leaving directory `/home/hans/code/openembedded/stuff/tmp/work/i686-linux/gcc-native-3.4.6-r9/gcc-3.4.6/build.i686-linux.i686-linux/gcc/fixinc’
make[1]: *** [fixinc.sh] Error 2
make[1]: Leaving directory `/home/hans/code/openembedded/stuff/tmp/work/i686-linux/gcc-native-3.4.6-r9/gcc-3.4.6/build.i686-linux.i686-linux/gcc’
make: *** [all-gcc] Error 2
FATAL: oe_runmake failed

After much searching around, I found that this bug can be found in the original sources of gcc-3.4.6: NetBSD Problem Report #33238. Hmm, well, it looks like the root of the problem is this:

SHELL=/bin/sh: Command not found.
export: Command not found.

This is Bourne Shell (sh, bash, ksh, etc.) syntax and I am weird, old stickler for tcsh. So I ran bash, like basically everyone does these days, and voila, no more problem…

 

slow internet?

There is at least two parts of your net connection feeling ’slow’: one is responsiveness and the other is bitrate of downloading and uploading. Usually, when the internet feels ’slow’, it is not a problem of bitrate but rather responsiveness.

Cable and DSL modems are really the cause of most of the issues related to responsiveness. These modems have huge upload buffers, like 1 second or more. The idea is that they are tuned to making one user as fast as possible. But if that buffer is full, that mean any new packet you send will be delayed by at least one second.

So there is a simple way to regain the responsiveness, and that is by preventing the upload buffer from filling up ever. Basically, if you limit the total bitrate to be slightly less than the actual bitrate of the connection, then the buffer will always be empty. Then even when all your neighbors are using your connection for bittorrent, it’ll still be responsive. Your up/down bitrates will of course be slower.

I currently have about 10 regular users of my open wifi and I basically never even notice. I’ve messed with the ‘tc qdisc’ stuff quite a bit, and I have found that it is quite tricky to get things running smoothly. I currently use the most basic ‘tc qdisc’ filter, called a “Token Bucket Filter”, set up like this:


tc qdisc add dev eth1 root tbf rate 370kbit latency 50ms burst 2000


But if you have a WRT54G v1 through v4 (newer versions are much more limited), then you can run firmwares like Tomato and DD-WRT, and they’ll do this token bucket setup for you. I highly recommend Tomato, unless you want to be more 1337, then go with DD-WRT. In Tomato, go to QoS and you’ll see a nice web interface for playing with these things.

 

wrestling with Fink/unstable

Fink is a great tool for managing lots of software for Mac OS X. As with any untested software, it can be a bit painful when trying to use things from the unstable repository since the dependencies are not all cleanly worked out yet. I have found a number of tricks to smooth the process.

• always try to install as much as possible from binary packages first. If you find that fink wants to add more packages when it starts building a package from source, stop that build, and try installing the dependencies from binaries first. To force using binaries, use apt-get, or you can also set fink to try to use binaries from fink configure.
• sometimes when builds don’t go write, it can leave a lot of packages waiting to be installed. You can see which are in that state by running fink list | grep '^(i)'. Then try installing them one-by-one or in small batches.