credentiality

RobotShop sucks

2012-02-01T12:00:00.000-08:00

robotshop.com's customer service sucks. I paid extra for 2-day shipping, and 4 days later they emailed me back asking for my social security number. I had to work out on my own that it (probably) wasn't an identity theft scam, but something to do with import duties (they ship from Canada), because they didn't give any reason why they wanted it.

So I sent them my company's taxpayer ID, and the next day I get a voicemail asking me for it once again. So we're almost a week on, it still hasn't shipped, and they're apparently not even reading their email. Ugh.

Repeated strings in bash

2012-01-27T23:27:00.000-08:00

I just whipped up a little bar graph in bash using this trick: printf '=%.0s' $(seq 10)

A more complete example, using some file with lines containing a date string:

for d in {17..27} ; do
N=$(grep 2012_01_$d /tmp/somefile | wc -l)
echo -n "Jan $d: $N "
printf "=%.0s" $(seq $N)
echo
done

Jan 17: 28 ============================

Jan 18: 22 ======================

Jan 19: 43 ===========================================

Jan 20: 32 ================================

Jan 21: 0 =

Jan 22: 0 =

Jan 23: 46 ==============================================

Jan 24: 50 ==================================================

Jan 25: 50 ==================================================

Jan 26: 51 ===================================================

Jan 27: 41 =========================================

(Breaks for 0, as you can see)

If you're doing it for constant numbers, you can use {1..10} instead of $(seq 10), but {1..$N} doesn't seem to work.

Creating spectrograms in Linux using a ColorMunki

2012-01-06T16:31:00.000-08:00

The software that comes with the ColorMunki Photo is windows-only and is only about calibrating displays and printers. I wanted to get spectrograms out of it so that we can measure lighting and transmissive materials -- a chart of wavelength vs. intensity. BTW, check the argyll docs before you buy a ColorMunki -- apparently several of the models are the same hardware priced differently.

Install the argyll ubuntu package, then use the spotread command to take readings. The -S flag will display a graph after each reading. Other flags let you set which mode to use -- it has its own light for lighting a surface and measuring the reflected color, or you can have it read ambient light through a diffuser, or do spot readings without the light. I used -e to take spot readings.

It also prints out a list of intensities at regular wavelength intervals, so that should be easy to import into a spreadsheet for making your own charts.

Here's an example of a chart that spotread produced:

Walmart / vudu video rental sucks

2012-01-02T17:05:00.000-08:00

Just for the heck of it, we tried renting a video at walmart.com's vudu streaming service. It was unwatchable, freezing every few seconds for buffering. Checked our connection with speedtest.net, and the connection is fine.

And of course they've taken netflix's nonexistent customer service approach, with no way to ask for a refund. Thanks Walmart for wasting a half hour of my time.

Untangling gdata

2011-11-20T02:16:00.001-08:00

Update (2 Feb 2012): We got docs uploads working in GoogleCL with python gdata versions 2.0.15 and 2.0.16 after noticing a random page in Japanese (thanks Google Translate!) that mentioned changing all references of gdata.docs.data.DocsEntry to gdata.data.GDEntry. Also, the gdata.docs.data.MIMETYPES variable that was provided "for your convenience" went missing in 2.0.15 and 2.0.16. I've updated the table accordingly.

A brief history of python-gdata to upload files to Google Docs. (Much of this is also relevant to the other language interfaces to Google's gdata API.) Coila from the "Let's Get Technical" blog (where she has more details and links) and I worked all this out this evening trying to sort through some of the nastier open GoogleCL bugs.

Versions of python-gdata (as it's known in debian/ubuntu, or gdata-python-client on the code.google.com site):

2.0.0 .. 2.0.4: Uses old OAuth mechanism. Supports uploading limited file types to docs.

2.0.5: client.py interface added. Uses a new OAuth mechanism.

2.0.8: "Resumable upload" mechanism added.

2 May 2011:

Google starts allowing uploading arbitrary files to Google Docs. I believe this worked on versions 2.0.5 through 2.0.14 (2.0.15 wasn't released yet).

30 September 2011:

Google servers start returning "403.4 SSL Required" errors for versions 2.0.5 through 2.0.9, and also 2.0.11 for some reason. 2.0.10, 2.0.12, 2.0.13 and 2.0.14 seem to be fine.

Also, "403 Files must be uploaded using the resumable upload mechanism" errors start appearing. But only, apparently for versions 2.0.5 and later -- 2.0.0 through 2.0.5 still are able to upload limited file types.

One last wrinkle: GoogleCL is currently broken with version 2.0.15, failing due to gdata.docs.MIMETYPES having gone missing. So I don't yet know anything more about it.

Got all that? Here's a handy chart to help you keep it straight:

python-gdata version	OAuth	Can upload text files to docs as of 11/2011	Can upload arbitrary files to docs as of 11/2011	Arbitrary file upload without Resumable (5/2011 - 10/2011) ??	Supports Resumable Uploads	403: SSL error	403: Resumable upload required	Has gdata.docs.MIMETYPES	DocsEntry/GDEntry
2.0.0	old	X
2.0.1	old	X
2.0.2	old	X
2.0.3	old	X
2.0.4	old	X
2.0.5	new			X		X	X	X	DocsEntry
2.0.6	new			X		X	X	X	DocsEntry
2.0.7	new			X		X	X	X	DocsEntry
2.0.8	new			X	X	X	X	X	DocsEntry
2.0.9	new			X	X	X	X	X	DocsEntry
2.0.10	new	X	X	X	X		X	X	DocsEntry
2.0.11	new			X	X	X	X	X	DocsEntry
2.0.12	new	X	X	X	X		X	X	DocsEntry
2.0.13	new	X	X	X	X		X	X	DocsEntry
2.0.14	new	X	X	X	X		X	X	DocsEntry
2.0.15	new	X	X	?	X		X		GDEntry
2.0.16	new	X	X	?	X		X		GDEntry

Flat bottom table saw cuts

2011-10-28T21:19:00.000-07:00

If you've spent much time watching New Yankee Workshop, you've seen Norm cut a wide dado (slot) in a piece of wood by making multiple passes with the table saw, moving the wood a little to the side after each pass.

But when I tried it, I found that my saw left a V-shaped profile at the bottom of the cut, like the one on the left here:

That's because my blade, like most table saw blades, has an Alternating Top Bevel (ATB) grind. You can see a good picture at the bottom of Rockler's blade selection page. If you look carefully at a blade edge-on in the store, you can generally see the alternating profile, like you see on that page.

What I needed was a blade with a flat top grind. Apparently there are blades where all the teeth have a flat (or "raker") grind, but I've only ever seen them on Dado sets. (Ripping blades supposedly are ground this way, but I've never seen one that actually was.)

So instead of having all flat top grinds, I wanted an ATB/R (Alternating Top Bevel with Raker) blade. The rockler page calls this a "combination" grind. Remarkably, only two of the 10" blades they sell have this grind.

I ended up buying two blades: the Dewalt 7150PT ($36.04) and the Freud LU84R011 ($69.99). Both are 50 tooth "combination" blades, designed to be good at both ripping and crosscutting.

The Dewalt blade also happens to be a "thin kerf" blade -- it makes cuts that are 0.100" wide, compared to the 0.125" kerf of the Freud.

Once the blades arrived, I took some test cuts in a piece of 3/4" MDF to see how they compared to my original 80-tooth Diablo ATB blade. (Huh, turns out that's also a 0.100" thin-kerf blade. I never noticed before!)

I cut both single-pass and multi-pass slots:

Here you can see the profiles left by the blades in single-pass cuts:

The cut on the right is my ATB Diablo blade. It leaves the deepest V-groove. Measuring the groove in the wood with my calipers, it looks like the bottom of the groove is about 0.016" deeper than the top.

The cut on the left is the Dewalt ATB/R blade. Disappointingly, it also leaves a pretty significant V-profile, about 0.009".

The wider middle groove is from the Freud. It's the flattest, but still not perfect; I see about 0.005" difference between the top and bottom.

On to the multi-pass cuts. I should have been more careful to control the amount I moved between each pass, but the results agree pretty well with the single-pass tests.

The photo at the top of the page shows the Diablo ATB results. Here's the slot left by the Dewalt:

And here's the slot left by the Freud:

Conclusion: I'm not particularly satisfied with the dadoes from the Dewalt blade. The main reason I considered replacing my existing blade was to get flat bottomed channels, and the slots it leaves are bumpy enough that I'd still want to clean them out by hand with a chisel. On the other hand, it's half the price of the Freud, and cuts just as easily.

The Freud blade is the one I'll keep on my saw (there's another table saw at work I'll use with the Dewalt). I was hoping for completely flat slots, but these are good enough for my needs.

linux-alsa-driver-modules not up to date

2011-10-04T14:32:00.000-07:00

To get HDMI audio working on my lucid machine, I added some PPAs and then did:

$ sudo apt-get install linux-alsa-driver-modules-$(uname -r)

That fails currently, since the PPA hasn't kept up to date with the most recent kernel:

E: Couldn't find package linux-alsa-driver-modules-2.6.32-34-generic-pae

So for now, I've just downgraded to the old kernel. I also removed linux-image-generic so that it wouldn't automatically upgrade my kernels in the future:

$ sudo apt-get remove linux-image-generic-pae

$ sudo apt-get remove linux-image-2.6.32-34-generic-pae

Update (31 Jan 2012): Good grief, and now it's broken again. The latest kernel is 2.6.32-38. The fix this time was to add the lucid-proposed repository and install linux-backports-modules-alsa-lucid-generic-pae. That installs linux-backports-modules-alsa-2.6.32-38-generic-pae, which seems to take the place of linux-alsa-driver-modules-2.6.32-38-generic-pae.

To add lucid-proposed in synaptic, it's Settings... Software Sources... Updates... Pre-released updates.

That adds this line to the /etc/apt/sources.list:

deb http://us.archive.ubuntu.com/ubuntu/ lucid-proposed restricted main multiverse universe

Viewing Step (STP) files in Linux

2011-09-28T02:53:00.000-07:00

Mechanical engineers use step files to send their designs out for manufacturing. FreeCAD is Free Software that will view them. So tonight all I had to do on my Ubuntu Lucid machine was "sudo apt-get install freecad", then run it. It opened up the smaller step files in the liquid galaxy mechanical drawings just fine, although it did hang for quite a while when I tried the bigger ones (and I wasn't patient enough to see if it'd eventually figure it out).

The chasm between photo and video

2011-09-12T18:00:00.000-07:00

One of the reasons the web became so popular is that it made it easy to put photos and text together.

Now we're straining at the limits of static photos. We want them to be more lifelike, not just paintings on a wall. The sway of trees, our wandering gaze, the way we bob and turn our heads to get the shape of a thing. We get none of that with a static image.

Sure, it's getting a lot easier to embed video. But videos are like movies: they demand our whole attention from start to finish, and then they're done. Videos are entertainment, distractions. I'm talking about ambiance, digitized memories.

Here's what I see happening at the edges of what an image is.

Cinemagraphs are just animated GIFs, adding a tiny amount of motion to a scene to give it life. Cinematograph gallery, another one

Google Street View was one of the earlier players, showing panoramic photos on a map. Here's "indoor street view" from the Iraq Museum

Lytro has an after-the-fact refocusable lightfield camera. They call their photos "living pictures". Lytro gallery

Several news outlets have been playing with before/after photos of natural disasters. Click-to-toggle before/after / Slide before/after

"360 views" for products are gradually spreading through the online marketplace. Click "360 view" to see an example here: http://www.ford.com/cars/focus/?intcmp=fv-hpbb-dflt-40mpg-focus

A sackful of kludges

Except for the GIFs, every one of these innovative trends has something in common: somebody had to write code to make them work. Street View and Lytro, and many of the 360-degree product viewers require Flash. The before/after photos seem to have been one-off projects by the news outlets.

That's why the images I linked to are just static pictures -- only the .gif actually "works" without clicking on the links. (And the .GIFs themselves are also a huge kludge -- it's an ancient image format that only supports 256 colors, and has been a headache for browser vendors for years).

So none of them are really embeddable; you can't easily save a local copy of them, email them to your friends, or put them into a slide presentation. You can link to the pages where they live, but you're out of luck if the owners ever take the pages down.

Even ordinary product photos suffer from this kludge problem. How many times have you clicked on a product photo to "view bigger", only to get a popup window with the same image, at the same resolution? Some sites use flash, some use popups, some use javascript abominations that obliterate the product's description with a zoomed in image any time your cursor drifts across the product thumbnail.

What does it all mean?

The experimentation is good. People are exploring, seeing what catches on. There's a lot of space in the spectrum between static photos and video, and we're not done exploring it yet.

We don't even have good names yet. "360 view" is starting to catch on for drag-to-swivel product photos, but there doesn't seem to be any standard way to talk about the before-and-after photos, or any mental framework that lets us explain how they relate to 360 views or cinemagraphs.

And we need tools. Lots of creative people don't have the programming skills needed to build prototypes like these. And viewers get old and rickety quickly -- flash was the only way to do this stuff a few years ago, but everything you see here could be done just as well in html5.

My humble contribution to the space is the swivel viewer, which actually could be coerced to work for most of the things I described here. It proceeds from the notion that a set of images is better than a single one; drag your mouse to flip between images, scroll to zoom. But it's not embeddable, awkward for cinemagraphs and useless for panoramas.

Eventually we're going to need file formats that bundle these things -- whatever you call them -- into a single file, browsers that can embed them, and editors that let us create and tweak them.

I have no idea what that's going to look like, but whoever creates it is going to enable the expression of a lot of pent-up creativity.

Android ProgressDialog reappears and won't go away

2011-08-26T17:32:00.000-07:00

You'd think it'd be easy to create a progress dialog in your android app, but as far as I can tell, it's completely broken. Any number of things will cause it to reappear or never go away. Here's an apparently-ignored bug report:
http://code.google.com/p/android/issues/detail?id=11953

GWS S125 1T servo

2011-08-16T00:30:00.000-07:00

Acroname sells the "R298-1T-SERVO", which is actually a GWS S125 1T/2BB. It's different from most servos in that you can tell it to set any angle from 0-360 degrees, whereas most ordinary servos only move 90-180 degrees.

It has a 25-tooth spline, which should make it compatible with futaba arms.

Pololu sells it under the correct part number.

Arduino pro mini + bluesmirf + FTDI basic

2011-08-09T01:25:00.000-07:00

Tonight I got my Ubuntu Lucid machine to program a 3.3v Arduino Pro Mini using a 3.3v FTDI Basic USB->TTL serial interface, and then got them both to talk to a BlueSmirf bluetooth->TTL serial board.

To get the bluetooth board to work, I wired it to the USB board so that I could talk to it from both ends: its serial lines and over bluetooth. (See photo)

I ran minicom and connected to /dev/ttyUSB0, talking to the USB board and thus to the serial lines on the bluesmirf. This bluesmirf was set up to communicate at 57600 baud, which I had to find by trial and error. Once I got the right baud rate, typing "$$$" got me a "CMD" prompt as the manual suggests. I switched it over, so now it uses 115200 baud.

On the bluetooth side, I used these instructions to talk to the bluesmirf over bluetooth from my glaptop:

http://pratyeka.org/rfcomm/

Once I ran this: sudo rfcomm connect 0 00:06:66:04:B1:C2 1

I thought it was going to let me type to the bluetooth device, but really it's just creating /dev/rfcomm0 and then doing nothing. So with that command running in another window, I ran minicom and pointed it at /dev/rfcomm0, at which point I was able to type and see it in the other minicom window (pointed at /dev/ttyUSB0), and vice versa.

So that's how I knew the bluesmirf board was working.

To get the arduino running on my lucid machine, I had to apt-get install gcc-avr and avr-libc, make sure my JAVA_HOME was set right (because I have sun-java6-jre installed): export JAVA_HOME=/usr/lib/jvm/java-6-sun/jre

Then I downloaded the 64-bit linux install from here: http://arduino.cc/en/Main/Software and unpacked it into a folder in my home directory. Then I cd'ed into it and ran ./arduino. I tried the http://arduino.cc/en/Tutorial/Blink example, and it worked great.

Next, I wired the arduino to the bluesmirf as shown here:

I left the bluesmirf TX unconnected to the arduino RX so that it wouldn't interfere when I programmed the arduino. I used the "arduino" IDE to compile the following program, which just dumps some stuff over the serial port any time the reset button is hit. The only change from the stock arduino "ASCII table" example is the change to 115200 baud, and pulling pin 13 high to assert the CTS hardware flow control line on the bluesmirf. (I could also just as easily have run a wire from VCC to that line):

ASCII table

Prints out byte values in all possible formats:

* as raw binary values

* as ASCII-encoded decimal, hex, octal, and binary values

For more on ASCII, see http://www.asciitable.com and http://en.wikipedia.org/wiki/ASCII

The circuit: No external hardware needed.

created 2006

by Nicholas Zambetti

modified 18 Jan 2009

by Tom Igoe

This example code is in the public domain.

void setup()

{

Serial.begin(115200);

pinMode(13, OUTPUT);

digitalWrite(13, HIGH);

// prints title with ending line break

Serial.println("ASCII Table ~ Character Map");

}

// first visible ASCIIcharacter '!' is number 33:

int thisByte = 33;

// you can also write ASCII characters in single quotes.

// for example. '!' is the same as 33, so you could also use this:

//int thisByte = '!';

void loop()

{

// prints value unaltered, i.e. the raw binary version of the

// byte. The serial monitor interprets all bytes as

// ASCII, so 33, the first number, will show up as '!'

Serial.write(thisByte);

Serial.print(", dec: ");

// prints value as string as an ASCII-encoded decimal (base 10).

// Decimal is the default format for Serial.print() and Serial.println(),

// so no modifier is needed:

Serial.print(thisByte);

// But you can declare the modifier for decimal if you want to.

//this also works if you uncomment it:

// Serial.print(thisByte, DEC);

Serial.print(", hex: ");

// prints value as string in hexadecimal (base 16):

Serial.print(thisByte, HEX);

Serial.print(", oct: ");

// prints value as string in octal (base 8);

Serial.print(thisByte, OCT);

Serial.print(", bin: ");

// prints value as string in binary (base 2)

// also prints ending line break:

Serial.println(thisByte, BIN);

// if printed last visible character '~' or 126, stop:

if(thisByte == 126) { // you could also use if (thisByte == '~') {

// This loop loops forever and does nothing

while(true) {

continue;

}

// go on to the next character

thisByte++;

}

Once I flashed the arduino, I connected from my laptop to the bluesmirf over bluetooth, hit the reset button, and saw the table as expected in the minicom window connected to /dev/rfcomm0. First try, even. :)

Singleton bash script

2011-07-26T17:26:00.000-07:00

Sometimes it's important to make sure only one copy of a shell script runs at any given time. Here's an elegant way to find and kill any other processes running as the same name as your script:

kill $(pidof -x -o '%PPID' $0) >/dev/null 2>/dev/null

nVidia EVGA g210 card: HDMI audio and sync-to-vblank in lucid

2011-07-25T22:25:00.000-07:00

To get the HDMI audio to work on this g210 video card in Ubuntu Lucid (10.04), I first had to add this PPA and install updated alsa modules:

add-apt-repository ppa:team-iquik/alsa
add-apt-repository ppa:ubuntu-audio-dev/ppa
apt-get update
apt-get upgrade
apt-get install linux-alsa-driver-modules-$(uname -r) --force-yes

Then I had to set the module options correctly for my card:

$ more /etc/modprobe.d/sound.conf 
options snd-hda-intel enable_msi=0 probe_mask=0xffff,0xfff2

Once I did all that, I saw a single HDMI audio output:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: SB [HDA ATI SB], device 0: VT1828S Analog [VT1828S Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: SB [HDA ATI SB], device 1: VT1828S Digital [VT1828S Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: SB [HDA ATI SB], device 2: VT1828S HP [VT1828S HP]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: NVidia [HDA NVidia], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

And could test it:

speaker-test -c2 -twav -Dplughw:1,3

(Before I got the module options correct, I saw several HDMI audio outputs, and none of them would work.)

To get it to work with flash (for things like youtube), I had to create an /etc/asound.conf:

$ more /etc/asound.conf 
pcm.!default hdmi:NVidia
pcm:iec958 hdmi:NVidia

Then audio worked all the way. One annoyance is that the module hangs for ~5s when loading, so it increases my boot time.

Now that I could hear the youtube videos, I just had to get rid of the annoying tearing artifacts by telling the graphics to sync to the vertical blank interval. To do that, I enabled "Sync to VBlank" in "X Server XVideo Settings" and "OpenGL Settings", then installed compizconfig-settings-manager, ran "ccsm" and enabled it under General... General... Display Settings.

Building gnome from git on Ubuntu Lucid

2011-07-16T23:49:00.000-07:00

Lately I've been trying to build gedit from its source repository so that I can help improve it.

It's been hugely painful, I'm sad to say. Downloading from git and running autogen.sh, I got errors about my gnome libraries not being new enough. Fair enough, so I installed Ubuntu Natty on a spare hard disk. No luck, so I updated to the alpha of Ubuntu Oneiric. That actually worked OK, except that Oneiric is so alpha that it's very unstable.

Finally I asked on IRC, and the devs told me I needed to build with "jhbuild", the build manager for gnome. That failed on my Hardy machine, but that's no big deal since Hardy is pretty ancient now.

Lucid it is! I did a clean install on a separate hard disk, and started in with jhbuild. I used the sample.jhbuildrc file, pointing the install directories to my home directory instead of /opt. It chugged along for a while, checking out and building all of gedit's dependencies. Then it bombed while building gtk+/gdk with:

/usr/share/gir-1.0/GdkPixbuf-2.0.gir: Incompatible version 1.0 (supported: 1.2)

The folks at irc.gnome.org helped me trace the problem to jhbuild/config.py, which sets XDG_DATA_DIRS around line 368. Prepending /usr/share to XDG_DATA_DIRS means that the system data dirs take precedence, and thus the GdkPixbuf compiled by jhbuild doesn't get detected.

Commenting out the XDG_DATA_DIRS stuff and adding this to my .jhbuildrc solved the problem:
addpath('XDG_DATA_DIRS', os.path.join(prefix, 'share'))
addpath('XDG_DATA_DIRS', os.path.join(prefix, 'usr/share'))

Also, it looks to me like there's a bug in the XDG_CONFIG_DIRS around line 378, where it prepends /etc to XDG_DATA_DIRS instead of XDG_CONFIG_DIRS.

Apart from that, I've had to babysit jhbuild as it builds NetworkManager, libproxy and just about everything else in gnome, occasionally bombing out until I track down the right -dev packages to install.

Oneiric Ocelot installer failure

2011-07-15T00:50:00.000-07:00

Running the daily desktop build of the not-yet-released Ubuntu Oneiric (11.10), I kept getting install failures. /var/log/syslog showed that the problem was in /usr/share/ubiquity/install.py line 372. It was failing because the unlink / readlink / symlink wasn't working on /var/run and /var/lock.

Editing it as root, I changed it to:

try:
  os.unlink(targetpath)
except:
  print "Can't unlink " + targetpath

And:

try:
  linkto = os.readlink(sourcepath)
  os.symlink(linkto, targetpath)
except:
  print "symlink " + sourcepath + linkto + targetpath

And then it seemed to work.

Native android C program using ndk (update for android-ndk-r5c-linux-x86)

2011-07-06T20:38:00.001-07:00

Back in August 2010, I posted "Native android C program using ndk", wherein I described how to compile C programs as native Android binaries.

The paths changed a lot with android-ndk-r5c-linux-x86, so I updated agcc.pl and called it agcc2.pl.

Make sure you update the path to your NDK install in that script, but then you should be able to use it to compile the "hello, world" program as described in the original blog post.

Google App Engine for Business error with appcfg.py

2011-06-21T22:37:00.000-07:00

I have "App Engine for Business" access at work, which lets me create internal-only App Engine apps.

But upon creating a new internal app, I kept getting "Error 404" messages about "This application does not exist" when I tried to use appcfg update to upload it.

Turns out there's a special syntax for the application name in the app.yaml file. I changed the "application: myapp" line to "application: mycompany.com:myapp" and then it worked.

Regarding Bryce

2011-06-15T22:03:00.000-07:00

My apologies in advance.

Boyce: Bob, I'd like you to meet my brother, Bruce.
Bruce: Bob, I'm Bruce Bryce. It's a pleasure.
Bob: Bob Bluth, Bruce. Likewise. Where are you from?
Boyce: Bruce is from Blythe.
Bruce: Yes, I'm a lumber man. I grow Spruce in Blythe.
Bob: Blue Spruce?
Bruce: Both Blue and True Spruce. And in fact, I'm working on a new variety that's a mix of the two.
Bob: A True Blue Spruce?
Bruce: Exactly.
Bob: How's it going?
Bruce: It's back-breaking work. I lost half my trees this year to a blight.
Bob: That's terrible! What type of blight?
Bruce: Nobody knows. They're just calling it the Blythe Blight.
Bob: How did the True Blue do?
Bruce: It beat both the True and the Blue. In fact, it did so well I had to rename it. We're going to call it Bruce Bryce's Blythe Blight Beating True Blue Spruce.

Tax burden stats

2011-06-04T15:42:00.000-07:00

Here are tables for tax as a slice of GDP and tax rates by income level.

In terms of GDP, the last two years seem to be outliers, probably due to the economy tanking. The rest of the decade went from 8.0 to 10.2, pretty much exactly in line with the rest of the last 30 years.

The table of tax rates by income level is much more striking: the poorest 20% saw their taxes cut nearly in half since 1980, while the richest 20% saw only about 10% reduction in tax burden. So basically, the tax system got a whole lot more progressive in the last 30 years. (Progressive = the more you make, the higher your tax rate.) That's interesting in context of the stuff I've been seeing from the left lately about how we need to raise taxes on the rich, since it appears the burden has been shifting that way already for the last few decades.

Android "SetupTask cannot be found" error

2011-06-01T18:13:00.000-07:00

I recently checked in an android project, and when I checked it out elsewhere to check it over, I got the following error when trying to build it with "ant debug" per the command-line build instructions:

...
taskdef class com.android.ant.SetupTask cannot be found
 using the classloader AntClassLoader
...

The solution was to run "android update project --path ." to update the project to work in the new location.

Prepaid SIM in Spain

2011-04-04T09:58:00.000-07:00

As far as I can tell, the best way to get cellular/internet service for your unlocked GSM phone while in Spain is to head down to one of the ubiquitous Carrefour supermarkets and get a prepaid SIM card. I saw a variety of prices, like 10 Euro for a SIM card plus 10 Euro of credit on the account. We ended up paying 1EUR for the SIM card and putting 5EUR credit on it.

Calls are 0.08EUR/minute, but the best part is the internet access. After adding a "carrefourinternet" APN to my phone, 3g internet came right up. Their website has conflicting claims, but it appears that it costs either 0.50EUR/day for 20MB, or 1EUR/day for 100MB. Above that, it still works, but the rate is capped at 128Kb/sec, which still isn't bad. And for 19EUR, I can get 1GB of high speed connectivity on top of that, so that I don't hit the cap so much.

http://www.carrefour.es/telecom/internet-movil/index.html

Update: The Carrefour website is misleading. Apparently the 19EUR/1GB deal is only for USB cell modems. Instead I had to text "ALTA BONOPRE" to 22864, which buys 300MB for a month for 6EUR. And if it runs out, I think I can text "BAJA BONOPRE" to go back to 20MB/day.

Daily wtf

2011-03-21T23:39:00.000-07:00

My wife is taking a shell programming class, and one of the questions on the homework was:

How would you use fgrep to replace this command? grep '[a-z]\{9\}'

Now, fgrep is grep without the regular expressions. So this seems to be a very strange question. Nevertheless, let us assume that there is some hidden wisdom the problem is designed to help us uncover.

Approach 1: can we turn regexes back on with a flag? The manpage is unclear on this point; it tells us that fgrep is equivalent to grep -F, but doesn't specify what happens if we were to give, say, grep -F -E. At least in ubuntu lucid, it produces "grep: conflicting matchers specified".

Trying 'fgrep -E' produces 'fgrep: invalid option -- 'E'', which was my first hint that fgrep is /not/ in fact exactly equivalent to 'grep -F'. /bin/fgrep is in fact its own binary, about half the size of /bin/grep.

Now, the paragraph talking about fgrep in the manpage looks like this:

       In  addition,  three  variant  programs  egrep,  fgrep  and  rgrep  are
       available.   egrep  is  the  same  as  grep -E.   fgrep  is the same as
       grep -F.  rgrep is the same as grep -r.  Direct  invocation  as  either
       egrep  or  fgrep  is  deprecated,  but  is provided to allow historical
       applications that rely on them to run unmodified.

Note the last sentence, which tells us that if we call grep as fgrep, it should behave like fgrep. This turns out not to be the case. Symlinking or simply copying grep to fgrep doesn't appear to change grep's behavior at all. So that's a second bug in the manpage.

Approach 2: Perhaps the instructor really does want us to avoid regexes entirely. The manpage has this as the description of -F:

        -F, --fixed-strings
              Interpret PATTERN as a  list  of  fixed  strings,  separated  by
              newlines,  any  of  which is to be matched.  (-F is specified by
              POSIX.)

Separated by newlines? That's a little weird. I can't actually figure out how to put a newline into a command line argument. Inserting a literal one with ^V doesn't seem to do the trick, nor do things like fgrep 'a\nb'. So as far as I can tell, that's another bug, although fgrep -e pattern1 -e pattern2 does seem to work.

The other way to specify multiple patterns is to put them in a file, and then use fgrep -f pattern-file.

So we could, in theory, enumerate all the possible matching strings in a file and thus avoid the regex. A file with all 9-letter lowercase words, one per line, has 26^9 = 5.4T lines of 10 characters each, for a total of 54TB. Newegg currently has a 2TB disk for $69.99, so not counting tax, shipping, or filesystem overhead, a cool $1900.05 worth of disk would be enough to hold the pattern file. There do also appear to be several options for transparently compressed filesystems in linux, which makes the problem much more manageable.

You could also give fgrep -f - and let it read the patterns from stdin, piping in a list generated on the fly (although it's not clear if this would violate the terms of the question, since it requires more than just fgrep, unless you're a very determined typist).

The next problem is what fgrep does with the patterns. If it copies them into RAM or puts them into some sort of data structure (like, say, a search tree), you're going to need a whole lot of swap space. But this limitation might in turn be overcome by compressed swap.

And of course all of this assumes a 64-bit OS, and an implementation of fgrep that can handle trillions of patterns.

So my answer to the question How would you use fgrep to replace this command? grep '[a-z]\{9\}' would have to be:

Enumerate all 9-letter lowercase words to a 54 terabyte text file called 'patterns'. Then run fgrep -f patterns, ensuring there's enough memory available to store the set of matching patterns.

Xenarc 702tsv on Lucid with evtouch driver

2011-03-18T18:14:00.000-07:00

I got a Xenarc touchscreen working pretty well, except that intermittently (and usually after a reboot), the mouse pointer would behave erratically, while other times the touch behavior would work just fine.

The trick was to create an /etc/modprobe.d/xenarc.conf with:

# Stop messing with my Xenarc touch screen
blacklist usbtouchscreen

(I found that suggestion on Samiux's blog.

Now it seems to work consistently across reboots. The rest of the process was to set up a /usr/lib/X11/xorg.conf.d/11-touchkit.conf with:

Section "InputClass"
       Identifier "Touchkit Touch"
       MatchIsTablet   "on"
       MatchDevicePath "/dev/input/event*"
       Driver  "evtouch"
       Option  "ReportingMode" "Raw"
       Option  "Emulate3Buttons"
       Option  "Emultate3Timeout"      "50"
       Option  "SendCoreEvents"        "On"
       Option  "TapTimer"              "200"
       Option  "LongTouchTimer"        "400"
       Option  "MinX"                  "140"
       Option  "MinY"                  "193"
       Option  "MaxX"                  "1550"
       Option  "MaxY"                  "1750"
EndSection

Note that I've tried ~6 different 702TSV units, and they all seem to more or less work with these calibration parameters.

And then this is my /etc/X11/xorg.conf:

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
EndSection

Section "Files"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    Identifier     "Xenarc700"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Xenarc700"
    DefaultDepth    24
    SubSection     "Display"
        Depth       24
        # The native resolution of a Xenarc 702TSV is 800x480, but
        # I couldn't find any modes that'd work.
        Modes       "800x600"
    EndSubSection
EndSection

Section "ServerFlags"
    Option         "BlankTime" "0"
    Option         "StandbyTime" "0"
    Option         "SuspendTime" "0"
    Option         "OffTime" "0"
EndSection

Shopbot dresser plans

2011-02-27T15:00:00.000-08:00

A few weeks ago I built a pair of 5-drawer dressers using the shopbot.
I used apple-backed MDF for one set, and pecan-backed for the other.

You can find all the sketchup, partworks and shopbot files in the .zip file on this download page. Public domain, use them however you like. Note how I use wrapper.sbp files to control the feeds, speeds and depth of cut.

I cut the whole project with a 1/4" end mill, then chamfered the sides of the drawer fronts on the table saw. Tolerancing was the trickiest part of this project for me; I kept having to trim pieces or expand slots slightly. I improved things by the second dresser, but it still may not be perfect.

I believe these are the drawer slides I got from Home Depot. They're nice and smooth once I got the tolerances right.

Using rsync to copy only certain extensions

2011-02-25T15:22:00.000-08:00

I had a directory structure like this:

...
a/foo/bar/baz.txt
b/bar/asdf.jpg
c/qwerty/bletch.txt
...

and I wanted to rsync only the .txt files to my local machine:

a/foo/bar/baz.txt
c/qwerty/bletch.txt

There are lots of search results for the topic, but most didn't work right -- rsync -vv kept telling me that my --exclude="*" rule was excluding everything. This post gives the correct details. You need to explicitly include the directories you want, or just --include='*/', then --include='*.txt', then --exclude='*'. You may end up with a bunch of empty directories; I haven't figured out how to efficiently avoid that.

ant 1.8 on ubuntu lucid (10.04)

2011-02-15T23:23:00.000-08:00

UPDATE: Note the comment below which points out that you can simply "apt-get install ant1.8".

Trying to build a "Hello, World" app in Android, I built a skeletal project per Developing In Other IDEs page:

android create project \
--target 1 \
--name MyAndroidApp \
--path ./MyAndroidAppProject \
--activity MyAndroidAppActivity \
--package com.example.myandroid

After "apt-get install ant", I tried to build, but got this error:

$ ant debug
Unable to locate tools.jar. Expected to find it in /usr/lib/jvm/java-6-openjdk/lib/tools.jar

Android seems to want the sun version of the JDK, which I had installed earlier, but ant was pointed at the openjdk version. I fixed that error with:

$ export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.22

The next obstacle was:

$ ant debug
Buildfile: build.xml

BUILD FAILED
/home/jholt/android-sdk-linux_x86/asdf/MyAndroidAppProject/build.xml:82: The Android Ant-based build system requires Ant 1.8.0 or later. Current version is 1.7.1

Total time: 0 seconds

The ant 1.8 package from Maverick seems to install just fine. Don't forget the ant-optional package, or else you'll get errors like this:

$ ant build
Invalid implementation version between Ant core and Ant optional tasks.
 core    : 1.8.0 in file:/usr/share/ant/lib/ant.jar
 optional: 1.7.1 in file:/usr/share/ant/lib/ant-nodeps.jar

If you need newgrp, you dun goofed (and "setuidgid" may be the culprit)

2010-12-23T15:52:00.000-08:00

Recently, on a machine with custom upstart init scripts, I couldn't figure out why I couldn't access serial ports or USB devices, even though my username was in the dialout and plugdev groups in /etc/group.

Running the "groups" command, I could see that my groupset only contained my own personal group:

$ groups
credentiality

I found that I could run "newgrp dialout" to get it added to my groupset:

$ newgrp dialout
$ groups

dialout credentiality

And looking online, I found various tricks people would use to run the commands they needed using newgrp. (It's not trivial, since newgrp insists on creating a subshell).

But I kept getting the sense that newgrp should almost never be necessary anymore, now that modern unix systems have the notion of a groupset.

Turns out the culprit was "setuidgid", which we were using in the upstart scripts to run programs as me. They said things like:

exec setuidgid credentiality startx

Now that I know to look for it, setuidgid's manpage points out that it "remov[es] all supplementary groups". Which isn't very helpful if, for instance, a process needs to access files from both the "dialout" and "plugdev" groups.

Fortunately, there are several alternatives to setuidgid that preserve the group memberships you expect, and you can use the "groups" command to verify that:

# setuidgid credentiality groups

credentiality

# su -c groups credentiality

credentiality adm disk dialout cdrom floppy sudo audio dip video plugdev

# sudo -u credentiality groups

credentiality adm disk dialout cdrom floppy sudo audio dip video plugdev

# start-stop-daemon -c credentiality -S --exec /usr/bin/groups

credentiality adm disk dialout cdrom floppy sudo audio dip video plugdev

# # chpst is in package runit (yet another init system, ugh.), and lets you specify a list of groups explicitly:

# chpst -u credentiality:credentiality:plugdev:dialout groups

credentiality plugdev dialout

Carmichael numbers and Fermat primality

2010-12-02T14:13:00.000-08:00

I did some recreational mathematics today, and created this plot, showing the number of values for which a^(n-1) = 1 (mod n), for all composite n in the range [3..10000]:

The formula a^(n-1) mod n appears in Fermat's Little Theorem, which says that for prime values of n, the result will always be 1, no matter what a you choose. If you get a value other than 1, you know that n is composite.

That's neat because it lets us test whether a given n is composite much more quickly than by trying to factor it. We call that the Fermat Primality Test.

However, the converse doesn't always hold: sometimes a^(n-1)=1 (mod n) even if n is composite. When that happens for some a and n, we say that a is a false witness for n, since it incorrectly suggests that n might be prime.

And that's what the above graph shows: how many false witnesses there are for each value n up to 10,000. In particular, I scaled the y axis so that it shows the percentage of relatively prime a values that are false witnesses.

Carmichael Numbers

Most interesting are the vertical lines in the graph that reach all the way to the top. Those are the falsest of the pseudoprimes (with respect to the Fermat test). They're called the Carmichael Numbers, and I've always found them fascinating, because the smallest of them is 561! That's the leftmost line in the graph that reaches all the way to the top.

Just how prime are you?

The other thing I was surprised to find in this graph are what we might call the halfway-prime numbers -- numbers for which half of the witnesses will suggest primality with the Fermat test. And there are one-third-prime numbers, and so on.

Merry Christmas!

All that graphing got me wondering how often a^(n-1) produces numbers other than 1. So I created this image, which shows increasing values of n as you work downward, and shows how often we get a remainder r. r is shown on the X axis, while the number of times it came up is shown as a color:

0=black, 1=dark gray, 2=light gray, 3=blue, 4=orange, 5=red, 6 or more = white

The other trick is that in number theory, we sometimes talk about negative remainders. If you're working (mod 5), you might get remainders of 0,1,2,3,4, but since the next remainder after 4 is 0 again, sometimes it's easier to think of a remainder of 4 as -1. So instead of 0,1,2,3,4, we call them 0,1,2,-2,-1, and that's what I've done here.

Consider the topmost orange pixel in the above image. It's the 5th pixel from the top, so that whole row represents n=5. The orange pixel (and all the ones below it) are the "remainder=1" column. The column to the left of it is the "remainder=0" column, and the one to the right is "remainder=2". Since 5 is prime, we expect a^4 to be 1 (mod 5) for all 4 possible values of a: 1,2,3 and 4. And indeed, orange is the color for 4.

Two pixels down, you'll see a white pixel telling us that 7 is prime, since a^6=1 (mod 7) for a=1,2,3,4,5,6. In the above image, you can keep counting down from the orange pixel at n=5 and see all the primes as white pixels, with 31 at the very bottom.

Give me one of everything

One surprise to me was row 6 (just below the topmost orange pixel). It has a solid dark gray line extending all the way across. That means that the five values of a: 1,2,3,4,5 in the equation a^5 (mod 6) produced... the values 1,2,3,4,5! (Try it: 2^5=32, mod 6=2. 3^5=243, mod 6=3.)

And you can see that lots of other numbers have this behavior: 10, 14, (but not 18!), 22, 26, 30 and so on.

I need a bigger tree

Here's how the image looks if we keep going all the way to 100:

And if you really have a tall living room, you can see the numbers up to 1000, as well as the python source code I used to generate the images.

Bechdel Test: Aeon Flux

2010-10-03T01:52:00.000-07:00

I had forgotten what a great movie Aeon Flux was. I can't think of a more Bechdel-compliant action movie, it's visually stunning, and has a creative plot.

Escalation

2010-08-23T23:02:00.000-07:00

The answer was in the stars. For some reason, Earthlings had founded universities, and those universities had supported thousands of astronomers for hundreds of years. They refined their techniques over the centuries, collecting databases of statistics on distant stars which no human could ever visit.

And then a grad student in cryptography had gotten fed up with her adviser and switched to astronomy. To familiarize herself with the star database, Li had fed the coordinates of all the known stars in the galaxy into a PRNG test suite used to assess the predictability of cryptographic random number generators.

She hadn't been surprised when the database failed the test; stars aren't randomly distributed, and as they later found, at least 7 different astronomical techniques had also added artificial patterns to the database. However, Li thought it amusing when the test suite claimed the values were generated by a Linear Feedback Shift Register, the simplistic random number generator included in most programming libraries. It had even helpfully produced the LFSR parameters which, with a few lines of code, would generate a sequence of numbers that the test suite thought were somehow related to the database values.

Li had shown it to her advisor, and over a few months they had toyed with the results at odd hours, musing on a publication about systemic errors in sky survey techniques, or perhaps uncovering evidence of fabrication by some unscrupulous astronomer. When Li's model started predicting locations of stars before they were discovered, they worked full time on it for 18 months before telling another soul: surely it would be seen as plagiarism, not prediction.

But in the end, the systemic errors were identified and eliminated, and Li was confirmed to have made the greatest scientific discovery in all of history: the universe they lived in was incontrovertibly a simulation. And moreover, whatever Gods had placed the stars in their galaxies had been lazy in their choice of random number generators, just as human programmers are. (This did create a certain smugness among the kind of cryptographers who used cryptograpic-strength PRNGs to shuffle music playlists.)

As news-show pundits spouted absurdities and mankind came to grips with the hand of God, scientists in all fields now knew what to look for, and found bugs aplenty.

And as the fabric of the universe resolved under our microscopes, it was the cryptographers once again who made a profoundly disturbing hypothesis. The universe was made of bits, and manipulated by algorithms. Far more bits than our computers had, and running algorithms more complex than mankind had ever managed. But we knew there were bugs. What if some of those bugs were also security holes?

Teenagers had been exploiting complex systems for years. A carefully crafted string of a few hundred unexpected characters could cause a buggy subroutine to turn a user into an administrator. Or, if the string was slightly off, crash the server. This was normally no concern to the budding hackers. But then, they usually weren't hacking their own life support.

When the politicians finally understood, the strictest laws were passed. But studying really isn't much different than hacking, and eventually we learned all about our host machine. And several of the shortcomings of our universe started looking tantalizingly exploitable.

Ubuntu Lucid running alongside android on my phone

2010-08-22T16:49:00.000-07:00

At first I thought running ubuntu on my phone would require blowing away the android OS, but that's not the case. Instead, an ubuntu filesystem gets mounted on the SD card, with all the system files set up right so that things like apt-get work. Some things don't work, I think because the android kernel isn't quite what the ubuntu apps expect, but you know you're doing well when firefox runs!

The nexusonehacks Ubuntu disk image worked great for me, but took forever to download and had a ton of stuff that I didn't need, like firefox and openoffice.org.

So I stripped it down to bare bones and upgraded it to lucid. Check it out: run a bare-bones Ubuntu Lucid filesystem alongside android on a froyo phone.

Native vim for android

2010-08-20T23:35:00.000-07:00

Update: The vim binary has moved to the downloads page.

Update: Created a code.google.com project to keep stuff like this. Also, I got color syntax hilighting to work. See the README.txt in the vim/ directory of the repository.

Tonight I managed to compile vim to run natively on my android phone! My phone is rooted and running cyanogenmod CM6 rc3. Rooting is necessary to run vim, but cyanogen probably isn't necessary.

First I had to compile ncurses using the ndk. Follow that link and compile ncurses the way I did if you want to have any hope of compiling vim using these instructions.

Once I had ncurses built, I had to make just a few changes to vim to get it to work.

I downloaded vim from the mercurial archive, which as of today is pretty much the same as the 7.3 release, I think. I figured out how to call configure by trial and error:

vim_cv_memcpy_handles_overlap=no \
vim_cv_bcopy_handles_overlap=no \
vim_cv_memmove_handles_overlap=no \
vim_cv_stat_ignores_slash=no \
vim_cv_getcwd_broken=no \
vim_cv_tty_group=world \
vim_cv_terminfo=yes \
LDFLAGS="-L/path/to/ncurses-5.7/lib" \
CFLAGS="-I/path/to/ncurses-5.7/lib" \
vim_cv_toupper_broken=no \
CC=agcc.pl \
./configure --host=arm-eabi --with-tlib=ncurses

I got this error message: "could not compile program using uint32_t", and manually patched the configure script because I couldn't remember how to regenerate the script from the configure.in:

$as_echo_n "checking uint32_t is 32 bits... " >&6; }
if test "$cross_compiling" = yes; then :
  true
#  as_fn_error "could not compile program using uint32_t." "$LINENO" 5
else

I also got an "undefined reference to sysinfo" error due to an android bug that omitted the sysinfo() call. So I commented out the HAVE_SYSINFO and HAVE_SYSINFO_MEM_UNIT lines in src/auto/config.h.

I think that was it! Once it finished compiling, I copied over the runtime/ directory and the src/vim binary to /data/vim on my phone. I did an "export VIMRUNTIME=/data/vim/runtime", and vim worked great!

Once I got my VIMRUNTIME set, I was able to ":syntax on", although I'm not getting any color. But otherwise it seems to work great!

You can find a tarball of vim for android on the Downloads page: look for vim-7.3-android-binary.tar.gz. There's also a tarred-up copy of my build directory, although I'm not sure why anyone would want that.

Compile ncurses for android

2010-08-20T23:11:00.000-07:00

Update: The tarball of my ncurses build directory has moved to the downloads page. (Also, agcc -> agcc.pl thanks to Interarticle)

I got the crazy notion to try to compile a native vim binary for android, and one of its dependencies is libncurses.

Make sure you have the ndk installed and working.

You'll need the agcc wrapper, a perl script that creates appropriate command line arguments for arm-eabi-gcc. I learned today that the order of arguments to gcc can be quite significant while linking. I kept getting errors about the "__aeabi_unwind_cpp_pr0" symbol and common functions like printf not being found when it tried to link.

So I hacked Vlad's hacked agcc to make sure that libgcc.a and libc.a get put at the end of the command line, which seems to be necessary for static linking. Here's my hacked version of Vlad's agcc.

Make sure both agcc and arm-eabi-gcc are in your path.

Download ncurses-5.7.tar.gz.

Run configure:

CC=agcc.pl ./configure --host=arm-eabi --without-cxx-binding

When I tried to build, I got errors saying "'struct lconv' has no member named 'decimal_point'"", because android has a broken locale implementation.

So I commented out the HAVE_LOCALE_H line from include/ncurses_cfg.h. (Is there a better way to force configure to set a value like that during the ./configure process?)

It might be possible to build ncurses with the C++ binding, but I didn't try.

Once I had all that fixed, the make ran just fine, and I ended up with lib/libncurses.a.

Extract boot.img from an android phone

2010-08-20T01:01:00.000-07:00

(I did this on my rooted, boot-unlocked nexus one (HTC Passion) running cyanogen cm6. Cyanogen is probably not necessary, but boot-unlocking and rooting are).

This tutorial pointed me in the right direction.

I did:

$ adb pull /dev/mtd/mtd2 boot.img-from-phone

Then put it right back using:

$ adb reboot bootloader
[wait for bootloader to come up]
$ sudo ./fastboot flash boot boot.img-from-phone
$ sudo ./fastboot reboot

I could have also taken it apart and replaced the kernel with my own.

Nexus One (HTC Passion): compile and flash a kernel

2010-08-18T23:06:00.000-07:00

Update 2: my camera doesn't work with the custom kernel. no idea why. Also, I figured out how to extract the boot.img from the phone directly.

Update: mention how to update the bcm4329.ko wifi driver so that wifi will work again.

This page will tell you how to download and compile the android kernel, and try it out on your phone temporarily (without writing it to flash) using "fastboot boot zImage".

Assuming that worked, here's how to actually write the kernel to the phone's flash.

By this point you should already have fastboot. You'll also need a phone with an unlocked bootloader and probably rooted as well.

You'll also need the boot.img from your phone. It seems to be possible to pull it off a working phone, but I didn't try that. I'm running cyanogen 6, and the boot.img was in the update-cm-6.0.0-N1-RC3-signed.zip.

Next you need to unpack it using the split_bootimg perl script which I found in this tutorial.

That produced output like this:

$ ./split_bootimg.pl boot.img
Page size: 2048 (0x00000800)
Kernel size: 2206592 (0x0021ab80)
Ramdisk size: 167182 (0x00028d0e)
Second size: 0 (0x00000000)
Board name: 
Command line: no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5
Writing boot.img-kernel ... complete.
Writing boot.img-ramdisk.gz ... complete.

Next, you need mkbootimg. Supposedly you can use "fastboot flash:raw" to avoid this, but it didn't work for me. mkbootimg comes with the (multi-gigabyte) android sources, so I ended up downloading a pre-compiled mkbootimg binary, which made me feel dirty, but saved me many hours of downloading and compiling.

Constructing the right command line for mkbootimg was a pain, but eventually I was able to build a my-boot.img identical to the original:

./mkbootimg --kernel boot.img-kernel --ramdisk boot.img-ramdisk.gz -o my-boot.img --cmdline 'no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5' --base 0x20000000

The boot.img-kernel and boot.img-ramdisk.gz came from split_bootimg, as did the --cmdline string. The "--base 0x20000000" is specific to the N1 (aka nexus one aka HTC passion).

Once you get the boot.img constructed properly, reboot into the bootloader by holding down the trackball while you boot (or use "adb reboot-bootloader"). Then:

$ sudo ./fastboot flash boot my-boot.img
$ sudo ./fastboot reboot

If you get it wrong, your phone will hang at the screen with the unlocked padlock forever. Pull the battery to cold boot and reboot into the bootloader again.

(Nexus One / HTC Passion): If that worked, you may find that "wifi" under Settings... on the phone just says "error". That's because the bcm4329.ko kernel module doesn't match the kernel version.

Here's the bcm4329.ko that matches this 2.6.32 kernel (which has the serial port enabled).

The kernel module lives in /system/lib/modules/. The /system filesystem was mounted read-only on my phone, so from the root shell on my phone (adb shell) I remounted it read-write and then backed up the original kernel module, so that I can switch back to the old kernel if I need to:

# mount -oremount,rw /system
# cp /system/lib/modules/bcm4329.ko /sdcard/bcm4329.ko-orig

Then I copied over the file with "adb push bcm4329.ko /system/lib/modules/", then remounted /system back to read-only:

# mount -oremount,ro /system

It started working immediately.

Giving native Android C apps access to the Android API through SL4A

2010-08-15T22:22:00.000-07:00

UPDATE: this is slightly more mature now. Check out http://code.google.com/p/android-cruft/wiki/SL4AC

Today I played around with the Android NDK, a C compiler for Android. The Android folks make it very clear that it's only intended for making ordinary Java-based Android apps faster, by running a few selected functions down at the bare metal.

So it only comes with a few basic libraries which don't support things like reading the sensors and taking photos.

Enter SL4A, Scripting Languages for Android (formerly "ASE", the Android Scripting Environment). SL4A is a neat Android app that makes it very easy to write simple Android apps in many popular scripting languages like Python, Perl, Ruby and TCL. SL4A exposes a useful subset of the Android API in a clever way: by setting up a JSON RPC server. That way, each language only needs to implement a thin RPC client layer to access the whole SL4A API.

So today I managed to get the excellent Jansson C-language JSON library to compile with the NDK, making it possible for native C apps to access the Android API through SL4A, just as its normally supported languages can.

It works just fine as either a native Android binary or from a host machine, so you could also use it to control your phone from a C program running on a host computer.

Here's the ndk-to-sl4a.c source code, public domain. See the comments in the header for instructions on how to compile and run it.

If you're the trusting sort, you can also download a binary for froyo or for (ubuntu/lucid) linux. But you should still go through the comments in the source file to see how to set up SL4A first.

Native android C program using ndk

2010-08-15T18:11:00.000-07:00

Big update: I updated agcc.pl to work with the latest NDK release, and called it agcc2.pl. Here is the new blog post.

Update: link to agcc.pl works now. Update 2: link to agcc.pl *actually* works now?

The Android NDK isn't really intended to write standalone binaries, but this tutorial got me most of the way there. But some path changes in android-ndk-r4b broke the agcc wrapper, so I had to modify it to point to the right includes and libraries.

Here's what should be a complete HOWTO for linux. It probably requires a rooted phone at the least, in order to upload and run arbitrary binaries. I'm running a pre-release of cyanogenmod CM6.

1. Download and unpack android-sdk_r06-linux_x86.tgz and android-ndk-r4b-linux-x86.zip.

2. Add android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin and android-sdk-linux_86/tools/ to your PATH. (cd into each of those directories and then run "export PATH=$PATH:`pwd`")

3. Plug your phone in via USB and run "sudo adb start-server" to start the daemon in the background (as root so that it can access the USB device. If you accidentally start the daemon as yourself and can't talk to your phone, run "adb kill-server" to kill it, then "sudo adb start-server" to restart it as root.)

4. Make sure you can run "adb" (from the SDK) and "arm-eabi-gcc" (from the NDK) so that you know your PATH is set up right.

5. Grab my copy of the agcc wrapper hacked for android-ndk-r4b. "chmod 755" it so you can run it.

6. Create the "hello, world" .c file:

#include <stdio.h>

int main() {
  printf("Hello, world!\n");
}

7. Ready to compile! "$ agcc.pl -o hello hello.c"

8. Copy the binary to /data: "adb push hello /data/"

9. Run it! I did that by SSHing into the phone, but you can also just "adb shell".

# cd /data
# ./hello
Hello, world!

Simple kalman filter example

2010-08-06T00:59:00.000-07:00

There are a ton of Kalman filter overviews online, and lots of them give a general overview of what they do, but then they all hit a wall of variables and matrices, and fail to give good simple examples.

The best guide I found is a PDF scan of a much-faxed copy of Roger M. du Plessis' 1967 classic "Poor Man's Explanation of Kalman Filtering".

His paper is great because he starts with a single-variable filter (no matrices!) with no control component and no prediction algorithm. I'm going to try to give an even simpler example with softer terminology and more hand-waving.

Kalman filters are a way to take a bunch of noisy measurements of something, and perhaps also some predictions of how that something is changing, and maybe even some forces we're applying to that something, and to efficiently compute an accurate estimate of that something's true value.

Let's say we want to measure the temperature in a room. We think it's about 72 degrees, plus or minus 2 degrees. And we have a thermometer that gives uniformly random results within a range of +/-5 degrees of the true temperature.

We take a measurement with the thermometer and it reads 75. So what's our best estimate of the true temperature? Kalman filters use a weighted average to pick a point somewhere between our 72 degree guess and the 75 degree measurement. If the weight is large (approaching 1.0), we mostly trust our thermometer. If the weight is small, we mostly trust our guess and ignore the thermometer.

Here's how we choose the optimal weight, given the accuracy of our guess and the accuracy of the thermometer:

weight = temperature_variance / (temperature_variance + thermometer_variance)
0.29 = 2 / (2+5)

If temperature_variance were very large compared to thermometer_variance, weight would approach 1.0. That is, we'd ignore our guess and just use the measured value. Likewise, if thermometer_variance dominated, the weight would approach 0, and we'd put very little trust in our thermometer readings.

29% weight means we'll trust our guess more than the thermometer, which makes sense, because we think our guess is good to +/-2 degrees, whereas the thermometer was only good to +/-5.

Now we do the weighted average:

estimate = guess + weight*(measurement - guess)
72.87 = 72 + 0.29*(75-72)

or equivalently

estimate = (1-weight)*guess + weight*measurement
72.87 = 0.71*72 + 0.29*75

That is, we went 29% of the way from 72 to 75, or equivalently, we took 71% of the guess plus 29% of the measurement.

There's one last thing to compute: how confident we are in our estimate of 72.87 degrees. That equation is:

                      temperature_variance*thermometer_variance 
estimate_variance =   -----------------------------------------
                    (temperature_variance + thermometer_variance)

1.43 = 2*5 / (2+5)

So we think our estimate is correct to +/-1.43 degrees.

Now we have a guess that the temperature in the room is 72.87 degrees, +/-1.43 degrees. And we still have a thermometer that tells the temperature +/-5 degrees.

That's basically the situation where we started, so we can run the whole algorithm again:

First we compute the weight, using our new, more accurate guess confidence:

weight = temperature_variance / (temperature_variance + thermometer_variance)
0.22 = 1.43 / (1.43+5)

We take a measurement, and this time let's say it comes up as 71 degrees.

Now we can compute the weighted average of our old guess and our new measurement:

estimate = guess + weight*(measurement - guess)
72.46 = 72.87 + 0.22*(71-72.87)

And the new confidence level:

                      temperature_variance*thermometer_variance 
estimate_variance =   -----------------------------------------
                    (temperature_variance + thermometer_variance)

1.11 = 1.43*5 / (1.43+5)

So after the second measurement, we estimate that the actual temperature is 72.46 degrees, +/-1.11 degrees.

Kalman filters are nice because we don't have to remember the whole history of measurements and estimates. We just keep track of our most recent estimate and our confidence level in that estimate.

If we decide to turn on the air conditioner, so that the temperature in the room starts decreasing, we can expand our calculations to include that "control" variable, and use it to update our estimates by "predicting" how much colder it'll be at each measurement. Once I figure out how to do that, maybe I'll post a more complicated example :)

Plotting complex polynomials with octave and octaviz (vtk)

2010-07-27T02:25:00.000-07:00

I got to wondering what happens when you feed complex numbers into polynomials, so I poked around and found the awesome vtk library binding for the octave language, called octaviz. (octave is a free implementation of the MATLAB language).

When we plot z = f(x), where x and z are complex numbers, we get 4 dimensions to plot, so I shifted each point in the surface plot N units + or - in the Y direction, where N is the imaginary part of the output. The color also indicates how far each point was stretched in the + or - Y direction.

#!/usr/bin/octave

global a = 1
global b = 0
global c = 0

function z = polynomial (i,j)
  global a
  global b
  global c
  x = i + (j * 1j);
  z = a*x*x + b*x + c;
end

[I,J] = meshgrid(-1:0.2:1);
REAL = arrayfun(@real, arrayfun(@polynomial, I, J));
IMAG = arrayfun(@imag, arrayfun(@polynomial, I, J));
vtk_surf(I,J+IMAG,REAL, IMAG);

input ("Orient it the way you want it before I save.");
vtk_print('polynomial.jpg', '-djpeg');
input ("Done!");

Borges

2010-07-21T00:31:00.000-07:00

The stoics teach that we should not complain of life--the door of the prison is open.
-august 25, 1983

Things I hate: externalized costs in programming

2010-07-15T18:25:00.001-07:00

Bob adds a feature to handle a situation that someday might arise. Alice then discovers a production failure caused by that feature. Good luck removing the feature: doing that might cause a production failure someday!

python-tk: tkinter photo doesn't work inside a function

2010-07-15T14:46:00.000-07:00

Wow, python-tk sucks. When I create the photo inside a function, it gets garbage-collected because it's not smart enough to know that the label is using it. So I have to manually keep it from being deallocated (in this case, by making it global). Also, I'd really like to use after_idle instead of after, but it never gets around to actually updating the display.


#!/usr/bin/python

import Tkinter
import ImageTk,Image

root = Tkinter.Tk()
root.geometry('+640+480')

old_label = None
label = None
photo = None
def display_image(root, image):
  global old_label
  global label
  global photo

  root.geometry('%dx%d' % (image.size[0], image.size[1]))

  photo = ImageTk.PhotoImage(image)

  label = Tkinter.Label(root, image=photo)
  label.place(x=0,y=0,width=image.size[0],height=image.size[1])

  if old_label is not None:
    old_label.destroy()

  old_label = label

def image_loop():
  global root
  print '.'
  image = get_image()
  display_image(root, image)
  root.after(1000,image_loop)

image_loop()

root.mainloop()

python plumbing: PIL Image from data string

2010-07-15T12:58:00.001-07:00

Today I wanted to use netcat (nc) to send a JPEG image file over a network to a python script so that I can manipulate it with PIL, the python imaging library.

PIL has fromstring and frombuffer methods, but I couldn't get them to work. But I found the ImageFile module, which let me feed the data in as it came off the socket.

To serve up the images, I did this from the shell:


while true ; do nc -q 0 -l -p 12345 < foo.jpg  ; done

Then this python script received and displayed an image:


#!/usr/bin/python

import socket
from PIL import ImageFile

tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcp.connect(('127.0.0.1', 12345))
tcp.send("1000 4")

file = open("bar.jpg", "w")
parser = ImageFile.Parser()

while 1:
  jpgdata = tcp.recv(65536)
  if not jpgdata:
    tcp.close()
    break

  parser.feed(jpgdata)
  file.write(jpgdata)

file.close()
image = parser.close()
image.show()

Xorg window redraw: workaround by forcing update

2010-07-10T15:48:00.000-07:00

When using Sketchup in wine on my ubuntu linux machine, I find that it doesn't update the screen properly. For instance, if I hit control-A to select everything, I don't see the selection until I do something else. That is, the screen seems to be one update behind.

Here's a workaround that works with Sketchup: in a separate terminal, I run:


$ watch -n 0.3 xrefresh -geometry 1x1+500+500

That uses xrefresh to create a 1x1 pixel x11 window at 500,500 on the screen every 0.3 seconds. That's enough to get sketchup to redraw itself.

If that's not enough, you can run xrefresh without the -geometry option to have it draw a window the size of the entire screen. But that's a lot more distracting.

libgphoto2 / gphoto2 build from svn and install locally

2010-07-08T21:40:00.001-07:00

Today I wanted to make a patch against gphoto2 (the command-line interface to libgphoto2) without overwriting the version installed by ubuntu.

So I needed to download libgphoto2 and gphoto2 from subversion and then install it to a local directory.


$ sudo apt-get install automake autoconf libtool gettext pkg-config subversion build-essential
$ mkdir gphoto-svn
$ cd gphoto-svn
$ svn checkout https://gphoto.svn.sourceforge.net/svnroot/gphoto/trunk/libgphoto2
$ cd libgphoto2
$ autoreconf -is
$ mkdir -p /tmp/gphoto2/local
$ ./configure --prefix=/tmp/gphoto2/local
$ make
$ make install
$ # neat, libgphoto2 installed to /tmp/gphoto2/local
$ cd ..
$ svn checkout https://gphoto.svn.sourceforge.net/svnroot/gphoto/trunk/gphoto2
$ cd gphoto2
$ autoreconf -is
$ ./configure --prefix=/tmp/gphoto2/local --with-libgphoto2=/tmp/gphoto2/local
$ make
$ # make install is optional; I just ran ./gphoto2/gphoto2 directly
$ # edited files, then "make" to rebuild

Earth Source / pals4wood sucks

2010-07-08T16:43:00.000-07:00

American business is doomed. I needed to buy several thousand dollars worth of plywood. My friend recommended "PALS", a local distributor. Great, let's go to their website.

Hm, no price list online. That sucks. I guess I'll email them a list of what I need. Hm, the email bounced: mailbox full. Let's try one of the other sales reps, and mention the bounce so they can fix it. Nope, that one also bounces. Okaaayyy, well at least one of them has a @yahoo.com address listed. That's pretty sketchy, but maybe they just can't seem to get their mail service right.

Weeks go by, no answer. Thanks, guys. Something that I was ready to take care of weeks ago, credit card in hand, has been on my mind ever since, while I wait to see if you'll ever get around to answering.

I give up and order from their competitor. They don't have prices online either, so I get to go back and forth with their sales guy about what I want and what they have over the course of a week or so. And I finally give up on PALS, call in to the competitor, wait for him to pull up the email, give him my credit card info, and place the order.

Now, out of some perverse sense of business ethics, I decide to try one last time to reach PALS and let them know how broken their website is.

Searching again for "pals plywood", the first result is for "earth source forest products", whose website makes no mention of PALS. Maybe everything's fixed now?

I send them an email to info@pals4wood, the email address listed on the new website. That message bounces back (mailbox full) too, but also sends me an autoresponse... telling me to call them. Seriously, guys?

Okay, sure, why not? I'm already in it this far. I call the toll-free number listed in the email. Rings forever. Whee! Let's try the Oakland branch. Ah ha, somebody answers! Oops, wrong number.

That's right, the company with two websites, both of which list numerous email addresses, all broken, lists the wrong phone number to call when you email their info address. FML.

Borges

2010-07-07T22:25:00.000-07:00

The Secret Miracle

The date was set for March 29, at 9:00 A.M. That delay ... was caused by the administrative desire to work impersonally and deliberately, as vegetables do, or planets.

----

The Aleph

I do, however, recall these lines from a satire in which he lashed out vehemently against bad poets:


 This one fits the poem with a coat of mail
 Of erudition; that one, with gala pomps and circumstance.
 Both flail their absurd pennons to no avail,
 Neglecting, poor wretches, the factor sublime -- its LOVELINESS!

It was only out of concern that he might create an army of implacable and powerful enemies, he told me, that he did not fearlessly publish the poem.

----

The Aleph (1949)

"I picture him," he said with an animation that was rather unaccountable, "in his study, as though in the watchtower of a great city, surrounded by telephones, telegraphs, phonographs, the latest in radio-telephone and motion-picture and magic-lantern equipment, and glossaries and calendars and timetables and bulletins..."

He observed that for a man so equipped, the act of traveling was supererogatory; this twentieth century of ours had upended the fable Muhammad and the mountain -- mountains nowadays did in fact come to the modern Muhammad.

ubuntu hardy firefox 3.6.6 freezes

2010-07-03T16:12:00.000-07:00

After today's apt-get dist-upgrade on my hardy (ubuntu 8.04) machine, firefox would freeze almost immediately and had to be killed.

Turns out there are two copies of libflashplayer.so that can get installed, one by the flashplugin-nonfree package, and one by the adobe-flashplayer .deb package that adobe distributes.

sudo apt-get remove flashplugin-nonfree and then a reinstall of the install_flash_player_10_linux.deb from adobe fixes the freeze, and also upgraded me from flash9 to flash10.

conservation of responsibility

2010-07-02T17:02:00.000-07:00

Job descriptions usually specify that they want people with good teamwork skills. That can be good, but often it means sending out lots of emails asking other people for opinions.

In my last blog post I claimed that capacity for responsibility is a scarce resource.

So lately I've been trying to reduce the number of interrupts and decisions required for decisions that come along.

Example:

me: I'm in town. Want to have lunch?
Robert: sure!
me: [checks calendar] How about Thursday or Friday?
Robert: [checks calendar] Great, let's do Friday.
me: [checks calendar] 1pm sound okay?
Robert: sure
me: [adds it to our calendars]

Totals: 7 messages, 5 decisions, 4 uses of calendar. And that's just to *schedule* it. Without even talking about where to meet. Both of us still have to check our calendars on Friday and meet at the right time.

That could have been reduced to:
me: [adds lunch to our calendars with "hey, I'm in town" to the description]

Each time someone contacts us, we get a little anticipatory shot of stress as we get ready to read it and find out whether it's good, bad or boring. Reducing that can free up a lot of responsibility for more important tasks.

code.google.com bugs

2010-07-02T16:02:00.000-07:00

Trying to report a bug in code.google.com project hosting, I couldn't find anything by searching for "code.google.com bugs", "google code bugs", "code.google.com issues", etc. Turns out the bug tracker is under the "support" project. (Link goes to the issues page).

Maybe this will make it more findable.

Responsibility as a bottleneck

2010-07-02T15:12:00.000-07:00

I'm lucky enough to work at a company that tries give me all the resources I need to get my job done. Yet I've felt overwhelmed a lot lately, even, no especially when I find myself goofing off a lot. And it's because I've exceeded my capacity for responsibility.

I got the idea from hyperbole and a half, and it explains a lot of phenomena in our society.

Let's say I decide to take a for-fun class a the local college. And it's time to go to class now.

Oh wait, the car is parked at another building. And it's low on gas; should I fill up before I go? And my books are at home. Do without them?

You now have 4 things to do or decisions to make, all before class starts. Classes have about 16 - 48 class sessions. If going to class each time involves 4 little tasks, that's up to 200 little obstacles you'll have to handle, just to get to all the classes. Then there's registration, books, tests, and assignments. No wonder people don't take more classes on their own.

Calendars help, but how do you schedule keeping the gas tank full? Or remembering where you parked?

Here's the point: there's a limit to the number of things I can keep track of, and the number of decisions I can make in a day. And that holds me back more than my capacity to do the "actual" labor required to execute those decisions.

I don't surf the internet like a zombie for hours because the internet is so interesting or addicting. It's because it's stateless. I don't have to remember what I see, or take action on it in three days' time, or have it cascade into dozens of other pressing tasks.

"Unknown end tag for " error on code.google.com wiki page

2010-07-01T17:47:00.000-07:00

See http://code.google.com/p/support/issues/detail?id=4171. I was trying to link to an image, and it apparently wants a closing <img> before it can handle the </a>

Space-filling fabric

2010-06-30T17:35:00.000-07:00

I just got back my Hilbert Curve space-filling fabric from spoonflower, a site that'll print any image onto a variety of fabrics.

Space-filling fabric is suitable for tiling any plain spots on your walls, or for topological application. For storage, fold into a moebius strip and store in a Klein Bottle.

(The “Hilbert Curve” is a recursive space-filling function. If we color the corners of each square instead instead of drawing lines between them, we get the pattern you see below.)

Source code and images available at http://lunkwill.org/src/hilbert/.

Technical curriculum sucks

2010-06-29T14:32:00.001-07:00

My wife just posted to her blog about how much she's enjoying the math classes she's taking at the community college.

She always did well in math, and had a friend/rival in elementary school who she always competed with to be the best in her math studies. Then in 6th grade, her teacher wouldn't let her move on to the fast track because she hadn't done enough homework to get an A, even though she was acing the tests.

Apart from feeling cheated, she was now also behind in the expected progression for technical fields. She studied English in college, and never got past college algebra.

Now she's back in school, thinking of doing a graduate degree in engineering. So she's at the community college, where she's supposed to take several years worth of prerequisites outside her final department, to get the math and science core she feels like she needs just to apply to a good grad school.

Likewise, my cousin loved astronomy in high school, where they had a neat research-oriented pilot program. Her project was to study historical photos of nebulas to see if expansion could be tracked over decades.

I encouraged her to take more when she got to college, but when she saw how many math and other science classes she'd have to take as prerequisites, she also decided to major in English.

One last related anecdote: I just met Sridhar Vembu last weekend. He found such a brain drain from the indian universities that he had trouble hiring for his indian software company. So he started a trade school. He takes 17-year-old poor kids with interest and a cursory hint of aptitude, and teaches them programming (while giving them a stipend and feeding them). The curriculum covers programming, english and math in that order of importance, and in that order of preference. The kids hate abstraction and math notation, but later appreciate it more after they've spent a few months solving practical problems in web programming. After 6-9 months, almost all of them start working for the company, and they do just as well as the college grads. As he puts it, "math is the new sanskrit".

Oh, and I believe he's taking about half and half girls/boys. (Except that more of the girls get recruited away by the colleges.)

spreadshirt sucks

2010-06-29T14:08:00.000-07:00

You'd think that after 10 years of e-commerce, that shopping carts would be a solved problem. I just sent this feedback to spreadshirt.com.

Your shopping cart sucks. I bought a shirt from you through reddit.com, and I'm pretty sure I told it to order 3 shirts, but you only shipped and billed me for 1. So once I received it, I followed the link from the receipt to reorder. But clicking on the order number did nothing. My javascript console tells me that load_order() is undefined. I very nearly gave up at that point. So I went to spreadshirt.com directly, and clicked around until I found my order history there, and load_order() is defined from there. So somebody screwed up the javascript includes.
But it gets better. I ordered 5 shirts this time, looked for an "update" button and didn't find one, and clicked to continue the order. Only on the last page of the order process did I notice that the quantity was still at 1.

So I clicked on my shopping cart to back up the process and fix it. That showed me the item I was trying to order, and I clicked to proceed. But you were actually showing me the last page of the process again, so clicking committed the order... for one shirt.

Now on my fourth trip through the system, you at least gave me the option to cancel the order (thank you), and I once again repeated the process. And this time I found the "refresh" arrows next to the quantity. Protip: it's not actually that hard to have javascript code check the value of the quantity field when people click the button to go to the next page. You're throwing away orders and pissing people off when you expect your customers to find and click a button to tell your page something it should already know.

Finally, heading to the contact page, I get to enter a captcha (even though I'm already logged in) and type in a little tiny textbox to send you feedback, because you don't want to have to deal with the horrors of actually publishing an email address, even though you demand mine as part of the order process.

g++ error "does not exist in this scope" in karmic/lucid

2010-06-18T02:05:00.000-07:00

g++-4.4 breaks backward compatibility for some stupid reason, so things that compiled just fine on earlier systems now break with errors like "stdout does not exist in this scope".

To fix, sudo apt-get install g++-4.2, then make sure your Makefiles or whatever are set to use g++-4.2 instead of just g++. You could probably also just apt-get remove g++-4.4, but I'm not sure if that'd break anything.

OS statistics from wikimedia

2010-06-17T17:53:00.000-07:00

It's surprisingly hard to find statistics on linux distribution popularity by version number.

But wikipedia to the rescue. Their squid logs break down user-agent string statistics by OS. Here's the May 2010 report:

http://stats.wikimedia.org/archive/squid_reports/2010-05/SquidReportOperatingSystems.htm

Linux seems to account for about 2% of all browsing. It looks like some browsers just report "Linux" as their OS, unfortunately. But among os-versioned browser strings, it looks like Ubuntu dominates linux distros by about 10:1, and almost all Ubuntu users are running either Lucid or Karmic (about 50/50).

GoogleCL rocks

2010-06-16T14:38:00.001-07:00

indeed.

Intuitively simple way to calculate sin and cos

2010-06-12T02:15:00.001-07:00

A few years ago my brother in law asked how pi is calculated, and we found that it can be done as a reasonably intuitive algorithm, without any advanced math.

Ever since then, I've wondered if sines and cosines could also be calculated simply. Here's what I came up with. Please let me know if you know of a simpler way, where "simpler" means relying on fewer or more elementary mathematical principles.

Sines and cosines let us translate an angle in a right triangle into the length of one of is sides. But what is an angle? Well, on a unit circle, a 45 degree slice cuts off an arc that's 45/360ths the length of the circumference. We can cut that slice in half by finding the midpoint between the two ends of the arc, then extending that line out until it touches the edge of the circle. That slice represents 22.5 degrees. Since we have a point on the unit circle and we know its angle, the point's X and Y coordinates tell us the cosine and sine, respectively, of that angle. (And since the points are on a circle with a radius of 1, we know the hypotenuse of the right triangles is always of length 1).

Here's the same concept in pictures:

And here's python code that starts with the two points shown in the pictures, and keeps finding midpoints between them (recursively) until it gets a close enough approximation to any angle you want.


#!/usr/bin/python
"""
Released into the public domain, 12 June 2010

This program calculates the sine and cosine of an angle in a geometrically
intuitive way.

Here's how it works:

Start with a unit circle and observe that the points (1,0) and (0,1) on the
circumference of the circle are separated by 90 degrees.

The midpoint between those two points, (0.5, 0.5), bisects that 90
degree angle, forming two 45 degree angles.  We can extend the line from
the origin through (0.5, 0.5) until it reaches the circumference of the unit
circle (that is, until its distance from the origin is 1), and we get
(0.707, 0.707).  Thus we know that the cosine and sine of 45 degrees are both
0.707.

Likewise, we can find the cosine and sine of 22.5 degrees by taking
the midpoint between (0.707, 0.707) and (1,0), then scaling the line until
it reaches the circumference.  And we can find the cosine and sine of
67.5 degrees by taking the midpoint between (0.707, 0.707) and (0, 1).

We can continue to bisect angles until we get arbitrarily close to any
angle we choose.  The allowable angular error is set below as MAX_THETA_ERROR.

Note that this method is agnostic to the angular units we use.  If we start off
by telling the program that (0,1) and (1,0) are separated by an angle of 90,
then the program will return a result measured in degrees.

If instead we say that (0,1) and (1,0) are separated by pi/2, we'll get a
result in radians.
"""

import math

MAX_THETA_ERROR=0.01

def cos_sin(theta, x1,y1,theta1, x2,y2,theta2):
  """cos_sin returns the (cosine,sine) of a value within MAX_THETA_ERROR
  units of theta, given two points (x1,y1) and (x2,y2) that live
  at angular posisions theta1 and theta2 on the unit circle.  theta,
  theta1 and theta2 must all have the same kind of angular unit (degrees,
  radians, etc.), but this function doesn't care what that unit is or
  how many of them make up a circle.  

  theta2 must be greater than theta1, and (x1,y1) and (x2,y2) must be
  points on the circumference of the unit circle, less than 180 degrees
  apart.

  for example, cos_sin(30.0, 0.0,1.0,0.0, 1,0,0.0,90.0) returns
  (0.866089312575, 0.499889290387), which are the cosine and sine of 30.
  """

  # Uncomment this to see the successive approximations
  #print "target: %.2f.  (%.2f, %.2f)=%.2f (%.2f,%.2f)=%.2f" % \
  #  (theta, x1, y1, theta1, x2, y2, theta2)

  # if one of the points is close enough to theta, we're done!
  if (abs(theta1 - theta) < MAX_THETA_ERROR):
    return (x1,y1)
  if (abs(theta2 - theta) < MAX_THETA_ERROR):
    return (x2,y2)

  # the midpoint is just the average of the two points
  x_midpoint = (x1 + x2) / 2.0;
  y_midpoint = (y1 + y2) / 2.0;

  # scale (x_midpoint,y_midpoint) by 1/midpoint_length to get a point
  # exactly 1 unit away from the origin
  midpoint_length = math.sqrt(x_midpoint*x_midpoint + y_midpoint*y_midpoint)
  x_midpoint = x_midpoint / midpoint_length
  y_midpoint = y_midpoint / midpoint_length

  # the midpoint bisects the angle between the two points
  theta_midpoint = (theta1 + theta2) / 2.0;

  # figure out which side of the midpoint our target value theta lives on,
  # and bisect the extended midpoint with one of the original points to get
  # closer to the target
  if (theta >= theta_midpoint):
    return cos_sin(theta, x_midpoint, y_midpoint, theta_midpoint, x2, y2, theta2)
  else:
    return cos_sin(theta, x1, y1, theta1, x_midpoint, y_midpoint, theta_midpoint)

angle = 30.0
(cosine, sine) = cos_sin(angle, 1.0,0.0,0.0,  0.0,1.0,90.0)
print "The cosine of %.2f is %.2f" % (angle, cosine)
print "The sine of %.2f is %.2f" % (angle, sine)

Going from odometry to position in a two-wheeled robot

2010-06-03T17:08:00.000-07:00

My robot has two drive wheels on which I have odometry (plus some casters that we'll ignore). I want to use the odometry from the wheel encoders to keep track of where I've gone since I started counting.

Let's say the right wheel moves b inches while the left wheel moved a inches during some time interval. Assume b > a. Then if we let the robot keep moving forever with those wheel speeds, the robot will make a big circle counterclockwise on the floor about some point X. We want to find X so that we can use a and b to find the new wheel positions on the floor.

We want to find ra, the distance from the left wheel to X.

We start by imagining the circumference of the circle the left wheel would travel in: ca. ca = 2*pi*ra. The right wheel's circle will have a radius of ra+w, where w is the wheelbase of the robot.

Since the two wheels are circling the same point on the floor with the axle always pointing toward the center of the circles, we know the distances a and b will constitute the same proportion of their respective circles. (If a goes all the way around its circle, b must have gone all the way around its circle too. Likewise if a goes 1/10th of the way around, etc.)

So we know that a/ca = b/cb, which is also the proportion of the circle we traveled (if a/ca = 0.10 then we've gone 10% around the circle). Substitute and solve and we get ra = w*a / (b-a).

Great! Now we know our radius of curvature.

Next, we want to find X, the actual point on the floor that we're circling. We start by finding theta, the angle we moved around the circle, by simply multiplying a/ca by 360 (for example, 0.10 * 360 = 36 degrees). If we want it in radians instead of degrees, it's even simpler, since multiplying by 2*PI cancels out the 2*PI in ca, leaving just a/ra.

We know that the point X is ra inches to the left of Pa, along a line which also passes through Pb. (X, Pa and Pb are 2-dimensional points). We get a vector from Pb to Pa with (Pa-Pb), make it length 1 by dividing by w, then multiply by ra so it's the right length to reach from Pa to X. Then we add it to Pa so that the vector ends at X.

Now we can find the updated wheel locations on the floor by rotating Pa and Pb by theta degrees around the point X. How do we do that? Well, we translate everything so that X is at the origin, then use a rotation matrix to rotate by theta, then translate back so that X is where it started:

Here's code that implements all of that (with saner variable names), plus the case where both wheels move the same amount):


// Just some math to turn wheel odometry into position updates
// Released into the public domain 3 June 2010

#include <stdio.h>
#include <math.h>
#include <stdlib.h>

#define PI (3.14159)

#define WHEELBASE (12.0)

// left wheel
double Lx = -WHEELBASE/2.0;
double Ly = 0.0;

// right wheel
double Rx = WHEELBASE/2.0;
double Ry = 0.0;


// given distances traveled by each wheel, updates the
// wheel position globals
void update_wheel_position(double l, double r) {

  if (fabs(r - l) < 0.001) {
    // If both wheels moved about the same distance, then we get an infinite
    // radius of curvature.  This handles that case.

    // find forward by rotating the axle between the wheels 90 degrees
    double axlex = Rx - Lx;
    double axley = Ry - Ly;

    double forwardx, forwardy;
    forwardx = -axley;
    forwardy = axlex;

    // normalize
    double length = sqrt(forwardx*forwardx + forwardy*forwardy);
    forwardx = forwardx / length;
    forwardy = forwardy / length;

    // move each wheel forward by the amount it moved
    Lx = Lx + forwardx * l;
    Ly = Ly + forwardy * l;

    Rx = Rx + forwardx * r;
    Ry = Ry + forwardy * r;
    
    return;
  }

  double rl; // radius of curvature for left wheel
  rl = WHEELBASE * l / (r - l);

  printf("Radius of curvature (left wheel): %.2lf\n", rl);

  double theta; // angle we moved around the circle, in radians
  // theta = 2 * PI * (l / (2 * PI * rl)) simplifies to:
  theta = l / rl;

  printf("Theta: %.2lf radians\n", theta);

  // Find the point P that we're circling
  double Px, Py;

  Px = Lx + rl*((Lx-Rx)/WHEELBASE);
  Py = Ly + rl*((Ly-Ry)/WHEELBASE);

  printf("Center of rotation: (%.2lf, %.2lf)\n", Px, Py);

  // Translate everything to the origin
  double Lx_translated = Lx - Px;
  double Ly_translated = Ly - Py;

  double Rx_translated = Rx - Px;
  double Ry_translated = Ry - Py;

  printf("Translated: (%.2lf,%.2lf) (%.2lf,%.2lf)\n",
    Lx_translated, Ly_translated,
    Rx_translated, Ry_translated);

  // Rotate by theta
  double cos_theta = cos(theta);
  double sin_theta = sin(theta);

  printf("cos(theta)=%.2lf sin(theta)=%.2lf\n", cos_theta, sin_theta);

  double Lx_rotated = Lx_translated*cos_theta - Ly_translated*sin_theta;
  double Ly_rotated = Lx_translated*sin_theta + Ly_translated*sin_theta;

  double Rx_rotated = Rx_translated*cos_theta - Ry_translated*sin_theta;
  double Ry_rotated = Rx_translated*sin_theta + Ry_translated*sin_theta;

  printf("Rotated: (%.2lf,%.2lf) (%.2lf,%.2lf)\n",
    Lx_rotated, Ly_rotated,
    Rx_rotated, Ry_rotated);

  // Translate back
  Lx = Lx_rotated + Px;
  Ly = Ly_rotated + Py;

  Rx = Rx_rotated + Px;
  Ry = Ry_rotated + Py;
}

main(int argc, char **argv) {
  if (argc != 3) {
    printf("Usage: %s left right\nwhere left and right are distances.\n",
      argv[0]);
    return 1;
  }

  double left = atof(argv[1]);
  double right = atof(argv[2]);

  printf("Old wheel positions: (%lf,%lf) (%lf,%lf)\n",
    Lx, Ly, Rx, Ry);
  update_wheel_position(left, right);
  printf("New wheel positions: (%lf,%lf) (%lf,%lf)\n",
    Lx, Ly, Rx, Ry);
}

atmel atmega32 avr-libc interrupt example

2010-06-02T15:14:00.001-07:00

Edit: Fixed a bug in the example code where I tried to do _BV(foo|bar), which doesn't work at all. Changed them to (_BV(foo) | _BV(bar)). Also added a #define for _BV in case you don't already have it. I think this code is correct, but I haven't tried it.

I wanted to read a pair of quadrature encoders with my atmega32, so I needed to enable the INT0 and INT1 pins. Incidentally, if you don't mind throwing away half the resolution of your encoder, you can get away with using only one interrupt per encoder. (You put one of the pair of sensors on an interrupt, and the other one on a regular GPIO pin. You get interrupts half as often, and the other sensor tells you which way you moved).

Here's the code I used. The compiler complained about using obsolete avr-libc identifiers, but it seems to work fine.

#include <avr/interrupt.h>
#include <avr/signal.h>

// Just count the number of interrupts we see
int interrupt_counter = 0;
SIGNAL(SIG_INTERRUPT0) {
interrupt_counter++;
}

// I got lots of random resets until I added this handler.  So apparently you shouldn't enable
// an interrupt unless you have a handler installed.
SIGNAL(SIG_INTERRUPT1) {
interrupt_counter++;
}

And then in my init code:

#define _BV(x) (1 << (x))

// setup for servicing INT0, INT1 interrupt pins

// generate an interrupt on any logical change
MCUCR |= _BV(ISC10);
MCUCR &= ~_BV(ISC11);

MCUCR |= _BV(ISC00);
MCUCR &= ~_BV(ISC01);

// enable the interrupt pins
GICR |= _BV(INT1) | _BV(INT0);


// Set d2 and d3 to be inputs
DDRD &= ~(_BV(PD2) | _BV(PD3));

// Pullups off
PORTD &= ~(_BV(PD2) | _BV(PD3));

Another test post

2010-05-27T18:08:00.001-07:00

Wow, I love posting to blogger from the command line

Test post

2010-05-24T00:40:00.001-07:00

This post generated via the gdata API

Getting my ar5212 wifi card to search for an access point

2010-05-17T16:43:00.000-07:00

On ubuntu karmic, I uninstalled network-manager and tried to configure my Thinkpad T41p's ar5212 wireless connection manually.

iwconfig wlan0 essid MyEssid
iwconfig wlan0 txpower auto

When I run iwconfig, it says "not associated" and doesn't appear to be looking for an AP. It'll stay like this forever.

But if I run dhclient wlan0, within a few seconds it'll associate and get an address as expected.

sudo sucks

2010-05-14T17:48:00.001-07:00

sudo helpfully won't let you do anything as root anymore if you disturb its precious 440 permissions. Because, you know, having root be able to write the file would be INSECURE. FML.

Edit: Whee!! It even helpfully segfaults on karmic:


$ sudo bash
sudo: /etc/sudoers is mode 0640, should be 0440
Segmentation fault

Avoiding the annoying "no write since last change" vim message

2010-05-10T21:09:00.000-07:00

Whenever I'm on a new machine, it drives me nuts when the default vi install won't let me :bn without saving first. Adding ":set hidden" to my .vimrc fixes that, letting me switch buffers without saving first.

Working with named pipes in tcl

2010-04-25T14:32:00.000-07:00

Today I wanted to pipe lines of output from a C program into a TCL program using a named pipe, so that I didn't have to start them up at the same time. First I created the pipe with mkfifo:


$ mkfifo foo

Then here's the magic incantation in TCL to open the pipe and run a function every time something comes in:


set fifo [open "foo" {RDWR NONBLOCK}]
fconfigure $fifo -blocking 1
proc read_fifo {} {
  global fifo

  gets $fifo x
  puts "x is $x"
}
fileevent $fifo readable read_fifo

Then when I write to the fifo, my program prints them:


$ echo "asdf" > foo

Make sure you write newlines when you write to the fifo so that gets doesn't block (or use read instead).

Chicken tortilla casserole

2010-04-24T18:43:00.000-07:00

Quick and easy.

1 can cream of mushroom soup
1 can cream of celery soup
1 onion, chopped
3 boneless chicken breasts, diced
1 can chopped tomatoes (or use fresh tomatoes)
1 small can green chiles (or fresh)
1 small can olives
4 or more corn tortillas, cut into strips
8-16 ounces grated mozzarella or cheddar cheese
1 cup salsa

Mix up all the liquid and small stuff in a glass 9x13" casserole dish, then mix in the tortillas and chicken. Top with cheese. 350F for one hour.

v4l2 example

2010-04-18T15:23:00.000-07:00

Update: Updated the sample code to spit out a grayscale ASCII PGM on stdout so you can actually see what it's
capturing.

Looks like the video4linux project could use a convenience library for people who want to do simple things. But at least they have good example code.

The official v4l2 example worked pretty well for me. It worked fine with my capture card, but I got "VIDIOC_S_FMT error 22, Invalid argument" errors when I tried it with my webcam. Switching to "V4L2_PIX_FMT_YUV420" from "V4L2_PIX_FMT_YUYV" fixed it. I also simplified it a bit, taking out the command line options, MMAP code and the USERP (which didn't work for me anyway). It gets the example down from 675 lines to a little under 300, and makes it a little closer to my purposes.


/*
 *  V4L2 video capture example
 *
 *  This program can be used and distributed without restrictions.
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

#include <fcntl.h>        /* low-level i/o */
#include <unistd.h>
#include <errno.h>
#include <malloc.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <sys/ioctl.h>

#include <asm/types.h>    /* for videodev2.h */

#include <linux/videodev2.h>

#define CLEAR(x) memset (&(x), 0, sizeof (x))

struct buffer {
  void *start;
  size_t length;
};

static int fd = -1;
struct buffer *buffers = NULL;
static unsigned int n_buffers = 0;

static void errno_exit(const char *s) {
  fprintf (stderr, "%s error %d, %s\n", s, errno, strerror (errno));

  exit (EXIT_FAILURE);
}

static int xioctl(int fd, int request, void * arg) {
  int r;

  do r = ioctl (fd, request, arg);
  while (-1 == r && EINTR == errno);

  return r;
}

static void process_image(const void *p) {
  fputc ('.', stdout);
  fflush (stdout);
}

static int read_frame(void) {
  struct v4l2_buffer buf;
  unsigned int i;

  if (-1 == read (fd, buffers[0].start, buffers[0].length)) {
    switch (errno) {
      case EAGAIN:
          return 0;

    case EIO:
      /* Could ignore EIO, see spec. */
      /* fall through */

    default:
      errno_exit ("read");
    }
  }

  process_image (buffers[0].start);

  return 1;
}

static void mainloop(void) {
  unsigned int count;

  count = 100;

  while (count-- > 0) {
    for (;;) {
      fd_set fds;
      struct timeval tv;
      int r;

      FD_ZERO (&fds);
      FD_SET (fd, &fds);

      /* Timeout. */
      tv.tv_sec = 2;
      tv.tv_usec = 0;

      r = select (fd + 1, &fds, NULL, NULL, &tv);

      if (-1 == r) {
        if (EINTR == errno)
          continue;

        errno_exit ("select");
      }

      if (0 == r) {
        fprintf (stderr, "select timeout\n");
        exit (EXIT_FAILURE);
      }

      if (read_frame ())
            break;
  
      /* EAGAIN - continue select loop. */
    }
  }
}

static void uninit_device(void) {
  unsigned int i;

  free (buffers[0].start);
  free (buffers);
}

static void init_read(unsigned int buffer_size) {
  buffers = calloc (1, sizeof (*buffers));

  if (!buffers) {
    fprintf (stderr, "Out of memory\n");
    exit (EXIT_FAILURE);
  }

  buffers[0].length = buffer_size;
  buffers[0].start = malloc (buffer_size);

  if (!buffers[0].start) {
        fprintf (stderr, "Out of memory\n");
        exit (EXIT_FAILURE);
  }
}

static void init_device(char *dev_name) {
  struct v4l2_capability cap;
  struct v4l2_cropcap cropcap;
  struct v4l2_crop crop;
  struct v4l2_format fmt;
  unsigned int min;

  if (-1 == xioctl (fd, VIDIOC_QUERYCAP, &cap)) {
    if (EINVAL == errno) {
      fprintf (stderr, "%s is no V4L2 device\n",
         dev_name);
      exit (EXIT_FAILURE);
    } else {
      errno_exit ("VIDIOC_QUERYCAP");
    }
  }

  if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
    fprintf (stderr, "%s is no video capture device\n",
       dev_name);
    exit (EXIT_FAILURE);
  }

  if (!(cap.capabilities & V4L2_CAP_READWRITE)) {
    fprintf (stderr, "%s does not support read i/o\n",
       dev_name);
    exit (EXIT_FAILURE);
  }

  /* Select video input, video standard and tune here. */

  CLEAR (cropcap);

  cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

  if (0 == xioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
    crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    crop.c = cropcap.defrect; /* reset to default */

    if (-1 == xioctl (fd, VIDIOC_S_CROP, &crop)) {
      switch (errno) {
      case EINVAL:
        /* Cropping not supported. */
        break;
      default:
        /* Errors ignored. */
        break;
      }
    }
  } else {  
    /* Errors ignored. */
  }


  CLEAR (fmt);

  fmt.type    = V4L2_BUF_TYPE_VIDEO_CAPTURE;
  fmt.fmt.pix.width       = 640; 
  fmt.fmt.pix.height      = 480;

// This worked with my capture card, but bombed with
// "VIDIOC_S_FMT error 22, Invalid argument" on my Logitech QuickCam Pro 4000
// fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
// This worked on the logitech:
  fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUV420;

  fmt.fmt.pix.field       = V4L2_FIELD_INTERLACED;

  if (-1 == xioctl(fd, VIDIOC_S_FMT, &fmt))
    errno_exit("VIDIOC_S_FMT");

  /* Note VIDIOC_S_FMT may change width and height. */

  /* Buggy driver paranoia. */
  min = fmt.fmt.pix.width * 2;
  if (fmt.fmt.pix.bytesperline < min)
    fmt.fmt.pix.bytesperline = min;
  min = fmt.fmt.pix.bytesperline * fmt.fmt.pix.height;
  if (fmt.fmt.pix.sizeimage < min)
    fmt.fmt.pix.sizeimage = min;

  init_read (fmt.fmt.pix.sizeimage);
}

static void close_device(void) {
  if (-1 == close (fd))
    errno_exit ("close");

  fd = -1;
}

static void open_device(char *dev_name) {
  struct stat st; 

  if (-1 == stat (dev_name, &st)) {
    fprintf (stderr, "Cannot identify '%s': %d, %s\n",
       dev_name, errno, strerror (errno));
    exit (EXIT_FAILURE);
  }

  if (!S_ISCHR (st.st_mode)) {
    fprintf (stderr, "%s is no device\n", dev_name);
    exit (EXIT_FAILURE);
  }

  fd = open (dev_name, O_RDWR /* required */ | O_NONBLOCK, 0);

  if (-1 == fd) {
    fprintf (stderr, "Cannot open '%s': %d, %s\n",
       dev_name, errno, strerror (errno));
    exit (EXIT_FAILURE);
  }
}

int main(int argc, char **argv) {
  char *dev_name = "/dev/video1";

  open_device(dev_name);
  init_device(dev_name);
  mainloop();
  uninit_device();
  close_device();

  exit(EXIT_SUCCESS);
  return 0;
}

ubuntu karmic grub error: "no such device" uuid/search nonsense

2010-04-16T13:21:00.000-07:00

After cloning my recent karmic koala (9.10) install to another machine, I was annoyed that it wouldn't boot. It said:


error: no such device: ....

  Failed to boot default entries.

Karmic now locates disk by UUID instead of the more traditional /dev/sda notation, which sucks when you want to clone a disk. But it's easy enough to fix.

To get it to boot once, use "e" from the grub menu to edit the command line. Remove the entire line that says:


search --no-floppy --fs-uuid --set ...

Then change the root= option in the next line to use /dev/sda1 instead of mounting by UUID:


/libux/boot/vmlinuz-... root=/dev/sda1 ro quiet splash

Then hit control-x to boot with those settings.

To fix it for good once the system boots, sudo gedit /etc/default/grub and uncomment this line:


# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
GRUB_DISABLE_LINUX_UUID=true

Next, sudo gedit /etc/fstab and replace the UUID=... with /dev/sda1 for the / partition and /dev/sda5 for the swap partition (assuming you had ubuntu use the whole disk during install).

Finally, sudo gedit /usr/lib/grub/grub-mkconfig_lib and find this line (line 174 for me):


    echo "search --no-floppy --fs-uuid --set ${fs_uuid}"

Change it to:


    echo ""

Finally, sudo update-grub2 to write the changes to disk. Then cross your fingers and reboot.

Running gnome applications over ssh

2010-04-14T21:06:00.000-07:00

If you "ssh -X" into some machine and can run X apps fine, but can't run some gnome apps, and get errors about dbus sockets, run "export `dbus-launch`" from your ssh session to set up the environment variables you need.

Ubuntu Linux Sketchup using Wine

2010-04-11T18:16:00.000-07:00

Update2: I found a much more general solution to the redraw problem.

Update: I'm not sure if that worked or not. I think it improved the situation, but didn't fix the refresh problems entirely.

I've been using Sketchup successfully for quite a while now in Ubuntu linux. The trick was to install the wine PPA in order to get a more recent version of wine than Ubuntu distributes by default.

However, at home I had screen update problems: for instance, when I'd mouse over a face with the push/pull tool, it wouldn't draw in the crosshatching until I left the face.

But today I figured out how to solve the refresh problems, thanks to a comment in the wine appdb entry for sketchup 7. I ran nvidia-settings, then set:

NVIDIA X Server Settings manager checked
Antialiasing Settings - Override Application - Off
Anisotropic Filtering - Override Application - 1x

The next time I started up sketchup, it worked fine.

balancethebudget.com

2010-04-10T13:17:00.000-07:00

This is a pretty neat site that shows how the federal budget breaks down and lets you try your hand at balancing the budget:

http://balancethebudget.com/#

Seems to be unbiased. I was recently surprised by these numbers about state and local spending, which show, for instance, that we actually spend more on education tha...n defense. So it'd be interesting to see state and local budgets represented as well:

http://www.usgovernmentspending.com/year2006_0.html

I'd also like it to break down my own tax burden, since it's hard to contextualize billions and trillions. I'd rather see it as: "you spent $17 this year on farm subsidies, $7313 on defense, $7819 on public schools..."

I also think it'd be neat to let people see what would happen if individual tax checks went directly to specific programs, until those programs filled up. That is, I could enter my approximate income or how much my total tax burden was. Then a roulette wheel would spin, and it'd have a N percent chance of landing in the "defense" category, since defense uses N% of the budget. Then it'd drill down into the defense budget, making weighted-random choices until it tells me that my entire burden went to buying 8% of a single missile, for example.

Android Scripting Environment: launch a script from adb

2010-04-09T19:59:00.000-07:00

ASE is pretty slick; makes it very easy to do simple things in Android. Here's how to launch a script from the command line after using "adb shell" to get a shell prompt:


# Launch an activity in the background:
$ am start -a com.google.ase.action.LAUNCH_SCRIPT -n com.google.ase/.activity.AseServiceLauncher -e com.google.ase.extra.SCRIPT_NAME test.py

# Launch an activity in a new terminal:
$ am start -a com.google.ase.action.LAUNCH_TERMINAL -n com.google.ase/.activity.AseServiceLauncher -e com.google.ase.extra.SCRIPT_NAME test.py

And here's the java file where they name all the intents in ASE.

calcdeps.py "Missing provider" error

2010-03-16T18:23:00.000-07:00

Today I was trying to use the Google Closure compiler to build a single .js file out of several source files and their closure library dependencies:

closure-library-read-only/closure/bin/calcdeps.py -p closure-library-read-only -i bar.js -i foo.js -o compiled -c closure-compiler/compiler.jar > compiled.js

I kept getting errors like this:

Exception: Missing provider for (foo.Foo)

That sucked, since foo.js with its goog.Provide("foo.Foo") was right in the same directory with bar.js, and I even specified it explicitly on the command line.

Turns out I needed to specify "-p ." so that it'd look in the current directory for the dependency. So with a command line of:

closure-library-read-only/closure/bin/calcdeps.py -p . -p closure-library-read-only -i bar.js -i foo.js -o compiled -c closure-compiler/compiler.jar > compiled.js

I got:

Exception: Duplicate provide (goog.async.DeferredList) in ...

That seemed even weirder. Turns out calcdeps doesn't like having the same subdirectory show up twice in -p arguments. That is, "-p ." and "-p closure-library-read-only" was causing it to find the closure library twice. Okay, so we take out the second -p:

closure-library-read-only/closure/bin/calcdeps.py -p . -i bar.js -i foo.js -o compiled -c closure-compiler/compiler.jar > compiled.js

Now I get:

ERROR - namespace "foo.Foo" cannot be provided twice

Almost there: it doesn't like having foo.js explicitly provided. This actually worked:

closure-library-read-only/closure/bin/calcdeps.py -p . -i bar.js -o compiled -c closure-compiler/compiler.jar > compiled.js

Universal Muses

2010-03-14T15:02:00.000-07:00

The universe halted. This would have been traumatic for its inhabitants, had they been able to notice. But their brains had halted along with everything else, so they didn't mind.

"More elevator music. I'm telling you, happiness doesn't make for good art," said Cyn.

"I suppose you're right. I guess I just have a soft spot for them." Fob sighed, twiddling the values in the configuration file.

He pressed a button and erased the 10e80 particle simulation. 175 billion galaxies ceased to be simulated.

Cyn softened. "Look at it this way:the suffering is what gives their lives meaning. No suffering means no art, and no art means there's no point wasting computer time on them." Cyn and Fob shared time on a supercomputer with 10^73 gigabytes of memory, or about 100 times the number of particles in our universe. It had been respectable when it was built, but was starting to show its age.

"Yeah, I guess," said Fob, distracted. "How about this? I'm going to restore the water planet from the pre-life checkpoint, but turn up the resource saturation and sunlight, so they'll really eat each other alive."

"If you really think that'll help. At least turn up the quantum granularity so it doesn't take forever to run."

Fob laughed. "Okay, okay. The poor physicists. Maybe they'll start writing requiems."

Life bloomed and ended. Then bloomed again, and ended. On the third try it caught.

"Giant lizards. Cute."

Cyn and Fob searched the history books, as soon as there were history books. "Hey, some of this architecture is pretty good."

Fob checked another result. "And check out the music. There's a whole cluster of them right around 1800."

"Meh, it's okay. Make this one go deaf and see what happens."

XMPP / asmack / android / Google Talk

2010-03-01T20:34:00.000-08:00

OK, I finally figured out how to send and receive XMPP instant messages from my gmail account using asmack with an Android 2.1 AVD. For some reason, the event-driven message receive code doesn't work, but this polling example code does. Here's my latest source:


// DON'T FORGET TO INCLUDE THE INTERNET PERMISSION IN YOUR MANIFEST.XML

package com.example.HelloFormStuff;

import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;

import org.jivesoftware.smack.Chat;
import org.jivesoftware.smack.ChatManager;
import org.jivesoftware.smack.MessageListener;
import org.jivesoftware.smack.PacketCollector;
import org.jivesoftware.smack.XMPPConnection;
import org.jivesoftware.smack.XMPPException;
import org.jivesoftware.smack.filter.AndFilter;
import org.jivesoftware.smack.filter.FromContainsFilter;
import org.jivesoftware.smack.filter.PacketFilter;
import org.jivesoftware.smack.filter.PacketTypeFilter;
import org.jivesoftware.smack.packet.Message;
import org.jivesoftware.smack.packet.Packet;

public class HelloFormStuffActivity extends Activity {
    public int state = 0;
    private static final String TAG = "HelloFormStuffActivity"; 
    
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);       
        
        new Thread(new Runnable() {
          public void run() {            
            //XMPPConnection xmpp = new XMPPConnection("jabber.iitsp.com");
            XMPPConnection xmpp = new XMPPConnection("gmail.com");
            try {
              xmpp.connect();

              // for other jabber accounts, truncate after the @
              //xmpp.login("username", "password"); 

              // for gtalk / gmail, include the @
              xmpp.login("your-gmail-account@gmail.com", "your-gmail-password");

            } catch (XMPPException e) {
              Log.v(TAG, "Failed to connect to " + xmpp.getHost());
              e.printStackTrace();
            }
            ChatManager chatmanager = xmpp.getChatManager();
            Chat newChat = chatmanager.createChat("friend@gmail.com", new MessageListener() {
              // THIS CODE NEVER GETS CALLED FOR SOME REASON
              public void processMessage(Chat chat, Message message) {
                try {
                  Log.v(TAG, "Got:" + message.getBody());
                  chat.sendMessage(message.getBody());
                } catch (XMPPException e) {
                  Log.v(TAG, "Couldn't respond:" + e);
                }
                Log.v(TAG, message.toString());
              }
            });
                   
            // Send something to friend@gmail.com
            try {
              newChat.sendMessage("OMNOMNOM");
            } catch (XMPPException e) {
              Log.v(TAG, "couldn't send:" + e.toString());
            }
           
            // Accept only messages from friend@gmail.com
            PacketFilter filter 
                = new AndFilter(new PacketTypeFilter(Message.class), 
                                new FromContainsFilter("friend@gmail.com"));

            // Collect these messages
            PacketCollector collector = xmpp.createPacketCollector(filter);
            
            while(true) {
              Packet packet = collector.nextResult();
                
              if (packet instanceof Message) {
                Message msg = (Message) packet;
                // Process message
                Log.v(TAG, "Got message:" + msg.getBody());
              }
            }
              
          }
          
        }).start();
        
        //setContentView(this);
    }
}

Android XMPP

2010-02-26T16:53:00.000-08:00

Update: working code for send/receive over XMPP from gtalk / non-gtalk accounts

So, apparently XMPP support was briefly part of the official android SDK, but they took it out again.

There's a respected library called "smack", but it doesn't work with android out of the box.

There's a hack of smack for android called asmack, and that seems to work for me, at least for sending messages.

I haven't figured out how to make it work with talk.google.com, but I created an account at jabber.iitsp.com and was successful in sending an IM from my android.


package com.example.HelloFormStuff;

import android.app.Activity;
import android.media.MediaPlayer;
import android.os.Bundle;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.Toast;

import org.jivesoftware.smack.Chat;
import org.jivesoftware.smack.ChatManager;
import org.jivesoftware.smack.ConnectionConfiguration;
import org.jivesoftware.smack.MessageListener;
import org.jivesoftware.smack.XMPPConnection;
import org.jivesoftware.smack.XMPPException;
import org.jivesoftware.smack.packet.Message;

public class HelloFormStuffActivity extends Activity {
    public int state = 0;
  
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        //setContentView(R.layout.main);
        
        XMPPConnection xmpp = new XMPPConnection("jabber.iitsp.com");
        try {
          xmpp.connect();
          xmpp.login("username","password"); // just username, no @ sign
        } catch (XMPPException e) {
          // TODO Auto-generated catch block
          System.out.println("Failed to connect to " + xmpp.getHost());
          e.printStackTrace();
        }
        ChatManager chatmanager = xmpp.getChatManager();
        Chat newChat = chatmanager.createChat("destination@example.com", new MessageListener() {
          public void processMessage(Chat chat, Message message) {
            Toast.makeText(HelloFormStuffActivity.this, message.toString(), Toast.LENGTH_SHORT).show();
          }
        });
        
        try {
          newChat.sendMessage("IMing from my android");
        } catch (XMPPException e) {
          Toast.makeText(HelloFormStuffActivity.this, "ERROR", Toast.LENGTH_SHORT).show();
        }
        ...

Don't forget to add the INTERNET permission to the manifest.xml, or you'll get "not connected" exceptions.

picasa gdata API and face recognition

2010-02-17T23:11:00.000-08:00

Tonight I wanted to download a list of photos from picasaweb with a particular person in them, using the "subjectids" field. I don't see any official way to do this, but here's how I managed it:

In picasaweb (in my web browser), I loaded up the page showing all the photos my friend was in. At the bottom, there was a little "RSS" link. That link looked like this:

http://picasaweb.google.com/data/feed/base/user/credentiality?alt=rss&kind=photo&subjectids=SOME_BIG_LONG_STRING_OF_CHARACTERS&authkey=A_SHORTER_STRING&hl=en_US

I loaded that link in my browser, and it gave me the option to keep it as a live bookmark.

Instead, I trimmed the URL down and plunked it in this little python gdata script:


#!/usr/bin/python

import gdata.photos.service
import gdata.media
import gdata.geo

gd_client = gdata.photos.service.PhotosService()

gd_client.email = 'MY_EMAIL_ADDRESS'
gd_client.password = 'MY_PASSWORD'
gd_client.source = 'exampleCo-exampleApp-1'
gd_client.ProgrammaticLogin()

print "logged in"

photos = gd_client.GetFeed('/data/feed/base/user/credentiality?kind=photo&subjectids=THE_SAME_BIG_LONG_STRING')

for photo in photos.entry:
  print photo.content.src

That printed out a list of URLs to the images I wanted.

TED2010 Jane McGonigal on gamers

2010-02-11T15:04:00.001-08:00

Jane puts a really positive spin on the time people spend gaming. Games give us much more positive reinforcement than we get from real life, and people are consequently spending HUGE amounts of time gaming. 500 million gamers worldwide spend at least an hour a day gaming. Serious gamers spend 10,000 hours gaming by the time they turn 21 -- the same time they spent in K-12 education. People have spent 6 million man-years playing World of Warcraft.

I have long felt that we need to find a way to give people in real life the kind of positive stimulus they get in games, and she puts it in a much more optimistic way than I could. I'm really glad she's working on it.

#TED Wednesday morning session

2010-02-10T12:50:00.000-08:00

Very welcome scientific results on fighting poverty in Africa: handing out a kilogram of lentils with immunizations drasticaly increases rate of child innoculation. Deworming, iron supplements vastly more effective at keeping kids in school than hiring more teachers or other more conventional approaches.

Ukelele flamenco ftw.

And some worthwhile arguments about why our race is evolved to believe things and find patters where there isn't anything, since it's safer to be think there's a lion in the bush and be wrong than to think there isn't and be wrong.

Nady USB-1C doesn't get recognized

2010-02-09T01:09:00.000-08:00

I couldn't figure out why my Nady USB-1C USB mic didn't get detected in Linux. It showed up in lsusb as 17a0:0001 as it should, but I got errors in dmesg. Turns out it was the passive USB extension I was using with it. Works fine with an active extension.

App Engine: AttributeError: 'module' object has no attribute 'Message'

2010-01-26T18:07:00.001-08:00

Today I wasted over an hour trying to receive mail in App Engine. Every time I'd try to use it, I'd get errors like:

'module' object has no attribute 'Message'
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 509, in __call__
handler.post(*groups)
File "/base/python_lib/versions/1/google/appengine/ext/webapp/mail_handlers.py", line 58, in post
self.receive(mail.InboundEmailMessage(self.request.body))
File "/base/python_lib/versions/1/google/appengine/api/mail.py", line 546, in __init__
mime_message = _parse_mime_message(mime_message)
File "/base/python_lib/versions/1/google/appengine/api/mail.py", line 227, in _parse_mime_message
if isinstance(mime_message, email.Message.Message):
AttributeError: 'module' object has no attribute 'Message'

Turns out I'd named my file "email.py", which is a builtin module for python. Rename it and it works fine.

Don't forget to delete the email.pyc, or your dev_appserver will complain about "ImportError: No module named email.message".

Firefox crashes when using facebook chat

2010-01-25T18:59:00.000-08:00

Using firefox 3 for Linux, I was surprised to see it keep crashing when I chatted with a friend on facebook. It only happens when receiving messages with smileys :) and when the Chat... Options... "Play Sound for New Messages" option is enabled. So turn that off if you experience this bug.

Use vlc to find the title number for mplayer, or, Disney Sucks

2010-01-19T18:57:00.001-08:00

Some studios (Disney is one of the worst) put stupid copy protection measures on their DVDs that make them difficult to play on anything but a DVD player. That's a great way to tick off your completely legitimate paying customers.

One of the common ones is to make something other than the first title the start of the movie, or worse, to put one or more corrupt copies of the start of the movie, then have the actual movie start on title 74 or something. This makes it hard to play the DVDs on Linux with mplayer.

To work around this on a Linux machine, use vlc to find the true start of the movie. File... Open Disc... DVD (menus) and start playing the movie. (Feel free to simply watch the movie at this point). Wait until you get to the first scene. Then in the Playback... Navigation dropdown you can find the title number. Let's say it's 37. Then you should be able to play the movie with mplayer dvd://37

Netflix sucks

2010-01-11T19:33:00.000-08:00

Netflix is mostly doing the right things: today they sent me a very specific, very short survey about a movie I just watched. Good, don't waste my time! But they didn't even give me the option to type free text comments about the quality. It's actually something I was interested in and willing to give them some details on.

So I went to the account info page, and got the typical wild goose chase where companies don't want to admit that they won't let you email them. So now you've wasted my time when I was trying to help you out. That turned me from a happy, engaged customer into an annoyed one without an outlet. So it goes on the blog instead, and costs you other customers.

unable to enumerate usb device

2009-12-11T15:00:00.000-08:00

Today on an Ubuntu Karmic Koala machine, we plugged in a Canon camera and got a bunch of "unable to enumerate usb device" messages when we looked at dmesg. Turns out it was a bad cable. So check that first if you get that message.

Disable mouse or touchpad in xorg

2009-12-09T17:34:00.000-08:00

Update: Apparently figuring out how to disable x input devices like mice, keyboards and touch pads is hard to find on the search engines, so I'm updating the post to make it more findable. (This post was originally called "Ubuntu karmic: 3dconnexion Space Navigator and xorg", but I just helped a friend of mine disable his EEE PC's elantech touch pad, and realized that the solution is general purpose.)

My friend always uses an external mouse with his laptop, and accidentally brushes the touch pad when he's typing. So he wanted to find a way to shut off the touch pad.

In Ubuntu Karmic Koala (9.10), xorg now autodetects my 3dConnexion Space Navigator and uses its events for mouse X and Y, which is not what I want.

I just discovered xinput, though, which is extremely cool. Thanks Philip Langdale and Frederic Lepied for writing it!


root@lg1:~# xinput list
...
"3Dconnexion SpaceNavigator"    id=9    [XExtensionPointer]
        Type is MOUSE
        Num_buttons is 5
        Num_axes is 6
        Mode is Relative
        Motion_buffer is 256
        Axis 0 :
                Min_value is -1
                Max_value is -1
                Resolution is 1
        Axis 1 :
                Min_value is -1
                Max_value is -1
                Resolution is 1
        Axis 2 :
                Min_value is -1
                Max_value is -1
                Resolution is 1
        Axis 3 :
                Min_value is -1
                Max_value is -1
                Resolution is 1
        Axis 4 :
                Min_value is -1
                Max_value is -1
                Resolution is 1
        Axis 5 :
                Min_value is -1
                Max_value is -1
                Resolution is 1


root@lg1:~# xinput list-props "3Dconnexion SpaceNavigator"
Device '3Dconnexion SpaceNavigator':
        Device Enabled (93):    1
        Evdev Reopen Attempts (227):    10
        Evdev Axis Inversion (228):     0, 0
        Evdev Axis Calibration (229):
        Evdev Axes Swap (230):  0
        Evdev Middle Button Emulation (231):    2
        Evdev Middle Button Timeout (232):      50
        Evdev Wheel Emulation (233):    0
        Evdev Wheel Emulation Axes (234):       0, 0, 4, 5
        Evdev Wheel Emulation Inertia (235):    10
        Evdev Wheel Emulation Timeout (236):    200
        Evdev Wheel Emulation Button (237):     4
        Evdev Drag Lock Buttons (238):  0

root@lg1:~# xinput get-feedbacks "3Dconnexion SpaceNavigator"
1 feedback class
PtrFeedbackClass id=0
        accelNum is 2
        accelDenom is 1
        threshold is 4

root@lg1:~# xinput set-ptr-feedback "3Dconnexion SpaceNavigator" 4 8 1  # very sensitive X pointer
root@lg1:~# xinput set-ptr-feedback "3Dconnexion SpaceNavigator" 4 1 8  # much less sensitive.

The threshold parameter doesn't seem to do anything.

Ah, here's the magic to make it stop influencing the pointer:


root@lg1:~# xinput set-int-prop "3Dconnexion SpaceNavigator" 93 8 0  # property 93 from list-props

Also works as:


root@lg1:~# xinput set-int-prop "3Dconnexion SpaceNavigator" "Device Enabled" 8 0
root@lg1:~# xinput set-int-prop "3Dconnexion SpaceNavigator" "Device Enabled" 8 1  # reenable

Unfortunately, I don't see any way to re-zero if it drifts (often happens when they're cold or hot or brand new), or turn on the LED. Here's C code to do it, though:


#include <sys/ioctl.h>
#include <error.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <linux/input.h>

main(int argc, char **argv) {
  
  int fd;
  int retval;
  struct input_event ev; /* the event */

  if (argc != 3) {
    printf("Usage: %s device 1|0\n", argv[0]);
    exit(2);
  }

  if ((fd = open(argv[1], O_WRONLY)) < 0) {
      perror("opening the file you specified");
      exit(1);
  }
 
  ev.type = EV_LED;
  ev.code = LED_MISC;
  ev.value = (argv[2][0] == '1') ? 1:0;
  write(fd, &ev, sizeof(struct input_event));
}

Dell mini 10 with ubuntu jaunty xorg crash: stolenbase

2009-11-24T23:22:00.001-08:00

Good grief. Did a dist-upgrade on the Dell mini 10, and now xorg is broken again. Something about a stolenbase in Xorg.0.log.

Thanks to this blog for the fix:

sudo apt-get remove --purge psb-kernel-source
sudo apt-get install psb-kernel-source

Since when did I become a full time distro installer?

Ubuntu Karmic: install -pae kernel breaks nvidia driver

2009-11-23T13:37:00.000-08:00

Since I have > 3GB of RAM, I installed linux-image-generic-pae so that I could access up to 16GB on my Karmic Koala install. Note that installing a new kernel doesn't choose it as the default in grub (which is actually now grub2). So in /etc/default/grub I set GRUB_DEFAULT=saved so that it'd remember my choice, and then rebooted and hit SHIFT during the (very brief) grub blip to pull up the menu and choose the new kernel, which it then remembers.

Unfortunately, that broke my nvidia driver: when I rebooted into the kernel, the text login screen just flickered as xorg restarted constantly. This is due to 2 bugs:

https://bugs.launchpad.net/ubuntu/karmic/+source/jockey/+bug/451305

https://bugs.launchpad.net/ubuntu/+bug/454220

Summary: I had to

apt-get remove --purge linux-headers-2.6.31-15

And then, when I did an

apt-get install linux-headers-2.6.31-15-pae

it automatically rebuilt (successfully) the nvidia.ko.

Asus EEE pc 900 frequency scaling

2009-11-21T23:14:00.000-08:00

I just got Ubuntu Jaunty running on this cursed EEE 900, and noticed it was running hot. But I just turned on CPU frequency scaling, so now the machine can automatically slow itself down from 900mhz to as low as 112mhz.

Right click on the menu bar and click "Add to panel", then select "CPU Frequency Scaling Monitor". When I first did this, it complained "CPU Frequency scaling unsupported". Turns out the celeron needs a particular kernel module installed for that to work. Here's my source for this fix, translated from Italian.

sudo apt-get install cpufrequtils sysfsutils
sudo modprobe p4_clockmod

Then add p4_clockmod to your /etc/modules with your favorite text editor or by:

sudo bash
echo p4_clockmod >>/etc/modules
exit

Asus EEE pc 900 linux howto

2009-11-21T22:28:00.000-08:00

(Update: linked to my post on frequency scaling down at the bottom).

Wow, I can't believe how many hours I've spent trying to make Linux work on this EEE 900. Our 900A seems to be okay, but ubunty karmic is completely broken with the 900, and hardy is seriously deficient. I tried eeebuntu, moblin, and I don't even remember what other distros. All failed miserably. Here's how I got Jaunty to work reliably with the VESA video driver.

The main problem is the xserver-xorg-video-intel driver, which locks up quickly after (or during) boot. For some reason, the i915 just seems to cause everybody huge headaches. And unfortunately, switching to the (slower) vesa xorg driver in many distros (such as Karmic Koala) causes it to lose video sync and be even more unfixable, since then the VTs (alt-F1, etc.) won't work.

Ubuntu Jaunty is the only thing that even comes close to working for me. It installs okay, and uses the vesa video driver, but has the standard trouble with the mind-rendingly awful netbook-launcher, which has plagued every netbook install I've ever tried. netbook-launcher sucks in a big way. It tries to be fancy by using OpenGL for visual effects, but in doing so either breaks video drivers or just eats up 100% CPU all the time.

Worse, Jaunty has a bug when turning it off that leaves you without a working window manager. System... Preferences... Switch Desktop Mode turns it off, and everything looks great, but as soon as you restart, you'll get a blank desktop with no menu bar, task bar, or gnome menus. Here's how to fix it, summarizing from the above link. (The above link also has ideas about what to do if you've already switched and are in that broken state).

To shut off netbook-launcher and get back to the normal gnome menus in Jaunty UNR, first switch to "Classic" mode from System... Preferences... Switch Desktop Mode. Everything should look great, but it'll break if you logout or restart now. To make sure the gnome menus load the next time, open System... Preferences... Startup Applications. Make sure UNR Launcher and Maximus are unchecked. Now click "Add" and create a "GNOME Panel" entry, running the command "gnome-panel" (no path needed). Add another called "GNOME Window Manager", running command "gnome-wm".

That should do it! You'll be limited to 800x600, instead of the native 1024x600, and of course the VESA video won't be as fast as a working accelerated driver would be. If that bothers you, feel free to spend a few days downloading and installing distros that'll break the machine entirely. Then reinstall Jaunty and reapply this fix, and you'll appreciate the suboptimal yet stable performance. Youtube and hulu work with flashplugin-nonfree, although not so much in HD or fullscreen modes.

Next up, enabling CPU frequency scaling to extend battery life.

Carmichael orrery

2009-11-11T15:19:00.000-08:00

It occurs to me that it ought to be possible to demonstrate pseudoprimality with a purely mechanical device.

For instance, we could demonstrate 2^560%561 with 560 2:1 (or equivalently, 35 65536:1) gear amplifications followed by a 1:561 reduction, and rotate the input shaft once to see the output gear rest at 1. Of course, 2^560 ~ 3.8E168, so you'd have to turn the crank fairly hard.

You could also do a single 65536:1:561 train cranked n times for individual 65536n (mod 561) reductions, and repeat 35 iterations for 65536^35%561.

That would provide a (false) Fermat witness of 2. Simile for the other integers < 561. Not very interesting so far, but perhaps there's something more elegant.

Math and music

2009-11-05T23:54:00.000-08:00

If you represent the notes in the musical scale as the numbers 0 to 11 (0=A, 1=A#, 2=B, etc.), then the wheel of fifths is merely the multiples of 7 modulo 12. Likewise, the wheel of fourths is 5 mod 12. For some reason, the internets appear to be silent on this relationship.

Capturing sound in Linux using a USB headset (Plantronics DA40)

2009-11-02T16:57:00.000-08:00

I picked up a Plantronics DA40 USB headset today and figured out how to capture .wav files to disk and play them back through the headset.

There are lots of other options, but I like this one because it's USB and I could use it with any Linux system, whereas if I used the native souncard, I'd probably have to do more per-machine troubleshooting. dmesg showed that it was detected okay:

[14563.672694] input: Plantronics DA40 Adapter as /devices/pci0000:00/0000:00:1d.3/usb4/4-2/4-2:1.3/input/input6
[14563.728138] input,hidraw3: USB HID v1.00 Device [Plantronics DA40 Adapter] on usb-0000:00:1d.3-2
[14563.909787] usbcore: registered new interface driver snd-usb-audio

Usually the mic comes muted by default, so I pulled up alsamixer to check. It came up with my native sound card first. So I ran "alsamixer -c 1" to try it with the second card, and that was indeed the DA40. Hitting TAB let met toggle between record and playback settings.

In the playback tab, there's a Mic option. You want to mute that ("m"), since that just loops the mic back through the speakers. On the Capture tab, there's a "Mic" field, and that's the one that actually sets the mic gain.

To record, I ran:

arecord -r 44100 -f S16_LE -D hw:1,0 foo.wav

(Hit control-c to end capture.) hw:1,0 let me record from card 1, the second sound card. It refused to record at its default setting of 8000Hz, so the 44100 and S16_LE were necessary. Then I was able to play back foo.wav with:

mplayer -ao alsa:device=hw=1.0 foo.wav

So far so good. Next, I wanted to know how much deviation I was getting (was the gain turned up enough?) so I downloaded a nifty, very old program called Wavesurfer, available as a statically linked binary:

http://www.speech.kth.se/wavesurfer/

I selected a "waveform" panel and loaded up foo.wav, and could see that I was only getting around +-500 units max (out of +-32767). In File... Preferences... Sound I/O, I set the Input and Output device to /dev/dsp1 (which I believe is the OSS version of the second sound card). Now I can record and playback from wavesurfer. I used alsamixer to tweak the Mic setting (remember to use the Record tab!) until I got +-10000 or so on my recordings. Anything over +-1000 would probably have been fine.

(BTW, wavesurfer will also show you nifty spectrograms, from which I learned a lot about the human voice.)

Here's a trivial script to start arecord recording in the background, and another script to kill it when you've had enough.

$more start-recording.sh
#!/bin/bash

arecord -r 44100 -f S16_LE -D hw:1,0 /tmp/recording-in-progress.wav &
ARECORD_PID="$!"
echo Recording. Wrote arecord pid of $ARECORD_PID to /tmp/arecord.pid
echo $ARECORD_PID > /tmp/arecord.pid

$more stop-recording.sh
#!/bin/bash

kill "`cat /tmp/arecord.pid`"

Machine Monastery

2009-10-27T16:26:00.000-07:00

I've been reading 365 Tomorrows, and I like the idea of flash fiction. Here's a story that popped into my head while deep cleaning my carpets:

Cynthia was reluctant when it came time to leave the Machine Monastery. Nobody had predicted that machines would be Buddhists. Crazed killers, perhaps. Indifferent to humanity, perhaps. Cold calculators, almost certainly.

She had learned the tactile pleasures of sanding the walnut sides of an imperfect jewelry box she had made herself with hand tools. The visual pleasure of brushing a finish with a wet edge.

The empty contentedness of sweeping a floor. The ragged exhaustion of breaking out old concrete sidewalks with a sledgehammer and hauling them to a skip. The gleam of a toilet scrubbed clean.

The machines had done all these things, mostly better than humans could, and had found the same peace from their lessons. Cynthia would go back to her life in the city, where her finance skills would pay the bills, and where machine and human craftsmen continue to do their jobs with the labor-saving tools that made mass production cheap. But perhaps in the summer she would take another vacation to the mountains east of town, away from the noise, and rejuvenate with the joys of manual labor.

Ubuntu 8.04 (Hardy LTS) on HP EliteBook 8530w

2009-10-13T11:31:00.000-07:00

Trying to boot the laptop with my 8.04 live CD, it hangs with a message "BUG: soft lockup - CPU#0 stuck for 11s". Hit F6 at the ubuntu boot screen and edit the boot params: remove "splash" and "quiet", and add "acpi=off".

Shopbot joinery

2009-09-25T22:16:00.000-07:00

I tried my first joinery on the shopbot today. It's a concept for a sideboard to foot joint for a bed. I recessed the post 1/8" deeper than the plywood to give it a sense of depth when assembled, but I'm still deciding whether I like the effect. Maybe with a very gentle roundover on the post edges.

Here are my Sketchup, PartWorks and Shopbot partfiles. The partfiles are written for a 1/4" bit.

The raised sections of the post need a little rounding off at the tip to accommodate the 1/4" radius in the mating part; I should probably fix that in the sketchup model before I do any more of those. Also, in partworks, I relieved the plywood path by 0.030" so it had enough clearance to fit. I like how the fit came out with that amount of clearance.

squid reverse proxy (aka website accelerator) on ubuntu hardy

2009-09-04T19:34:00.000-07:00

We had a bunch of machines all hitting the same URLs on www.example.com, so we put up a squid reverse proxy on ourcache.example.com so that http://www.example.com/foo/bar would get served and cached by http://ourcache.example.com/foo/bar

sudo apt-get install squid squid-cgi # squid-cgi enables the cache manager web interface

Edit /etc/squid/squid.conf. It's very well documented, and we only had to modify a few lines from the default ubuntu hardy config:

Allow other local machines to use our cache:

acl our_networks src 10.42.42.0/24
http_access allow our_networks

instead of the default of:

http_access allow localhost

Have the cache listen on port 80 and forward all requests to www.example.com:

http_port 80 defaultsite=www.example.com
cache_peer www.example.com parent 80 0 no-query originserver

In our case, we wanted to cache pages that included "GET parameters" in the URL, such as http://www.example.com/search?query=foo (which is something you should only do in special cases):

# enable logging of the full URL, so you can see what's going on (though it's a potential privacy risk to your users)
strip_query_terms off

Comment out the lines that exclude cgi-bin and GET parameter URLs from being cached:

#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY

Then we went to: http://localhost/cgi-bin/cachemgr.cgi to see how well our cache was working (blank login and password by default).

After doing an "/etc/init.d/squid restart", we found that we could hit http://ourcache.example.com/foo/bar.html and get http://www.example.com/foo/bar.html, as expected.