Quadrotor Ball Juggling 2010-12-08
Quadrotor ball juggling : Autonomous drone helicopter playing keepsie-upsies with a ping-pong ball
Quadrotor ball juggling : Autonomous drone helicopter playing keepsie-upsies with a ping-pong ball
The New Pornographers are in the UK this week, playing Bowlie 2 , and more pertinently a show this Thursday , at what is apparently now called the O2 Shepherd's Bush Empire . What is more exciting is that this is their first time over here with Neko .
Here's a lovely interview with Carl about it. I have a ticket for Thursday night in my INBOX.
Heliotrope : This German rotating building design, was the first building in the world to generate more energy than it consumed.
Blame The Daily Mail : Adam Curtis in wonderful form on the roots of MI5.
IDW formally announces "The Strange Death Of Alex Raymond" : That is one busy cover.
This is what my final day at last.fm looked like.
In the morning, this.
Yes, I'm working on getting a MAME cab smuggled into Moonfruit.
Enron Email Dataset : cleaned up, organised, and presented as part of the Internet Archives dataset collection.
Self cleaning eggs : Guillemot eggs have a surface microstructure that makes them self-cleaning eggs, like a birds egg, EGG.
A potential exhibition of Yellowism :Tate Britain is organising an exhibition of art that has been attacked in publlc.
This afternoon I went on a short guided tour of the decomissioned Royal Navy submarine, HMS Ocelot . It's in a dry-dock at the Royal Dockyards working museum at Chatham , just 20 minutes down the road from home.
Apologies for the poor quality of the photos. I only had my iPhone, with 15% remaining charge, and submarines do not offer much in the way of natural lightning.
Despite having owned a year pass for the best part of a year, and frequently admired the Ocelot from the outside, this is the first time I've been aboard. The tour is short, cramped, and completely fascinating, although perhaps not for the squeamishly claustrophobic, and definitely not for the mobility impaired.
The Dockyards is a superb example of a modern lottery-assisted regeneration project. There's several large ships in dock you can wander around, huge warehouses full of boats and machinery to pore over, a ropery, an art gallery space, a working steam railway, several sub museums. Far more than you can do in a single visit, but your ticket, once purchased, is good for 12 months of repeat admission.
Fretboard Heatmaps : This is a really neat idea, although it's a pity it has so few artists.
Album Shuffle : I published my playlist generating python tool onto github
I've been having persistent niggles with my home router / 802.11x base station / DSL modem. It's a D-Link DSL-2740B , itself bought as a replacement for my ISP-provided machine, an O2 wireless III (a re-badged Thomson SpeedTouch) which proved itself a low performer at both wireless and routing, and particularly dismal at doing both simultaneously.
I picked up the D-link cheaply, in a clearance bin in John Lewis. In most respects it has been a splendid replacement for the O2. WiFi is fast, routing is consistent, ADSL sync is better. However, it does have one stupid bug. It can't do DHCP reliably. After a certain period, it starts sending out broken leases to clients; either issuing them with IP addresses that are already in use, or more commonly issuing a working address, but nullifying the nameserver settings. A reboot will restore sanity, but involves an irksome couple of minutes of network outage. Afterwards it is only a matter of time before the problem re-emerges, noticeably quicker if there's an increased rate of new leases issued, such as a group of visitors armed with smartphones popping in.
I'm consistently amazed at how flawed home router appliances are. How anyone 'normal' is supposed to cope with these things, I have no idea. I've updated the firmware to the last available revision, fiddled with the limited options in the admin interface, to little avail. Web searches turn up a few people commenting on the same problem, but no solutions offered. This leaves me with three straightforward, yet unappealing options.
/etc/bootpd.plist file this will create, e.g. / etc/bootpd.plist.template , and then disable internet sharing again, which will remove the /etc/bootpd.plist file if it still exists. Now rename your template back to /etc/bootpd.plist and edit it.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Subnets</key>
<array>
<dict>
<key>_creator</key>
<string>cms</string>
<key>allocate</key>
<true/>
<key>dhcpdomainname_server</key>
<string>208.67.222.222,208.67.220.220</string>
<key>dhcp_router</key>
<string>192.168.1.1</string>
<key>lease_max</key>
<integer>3600</integer>
<key>lease_min</key>
<integer>3600</integer>
<key>name</key>
<string>192.168.1</string>
<key>net_address</key>
<string>192.168.1.0</string>
<key>net_mask</key>
<string>255.255.255.0</string>
<key>net_range</key>
<array>
<string>192.168.1.12</string>
<string>192.168.1.254</string>
</array>
</dict>
</array>
<key>bootp_enabled</key>
<false/>
<key>detectotherdhcp_server</key>
<integer>0</integer>
<key>dhcp_enabled</key>
<array>
<string>en0</string>
</array>
<key>replythresholdseconds</key>
<integer>4</integer>
</dict>
</plist>
/var/db/dhcpd_leases , which will be a persistent database for issued leases. Now connect to the router, and disable it's DHCP server. /usr/libexec/bootpd . If you run it from a terminal with a -d flag, it will stay in the foreground and emit debugging info to stdout. You'll need root privileges for it to run, I just used sudo /usr/libexec/bootpd . Now request a dhcp address from a different network client. I used an iPad. It's a good idea to make a note of the network MAC address. If everything is working, you should see some output acknowledging the request, and then some more as a lease is issued. The client should then configure it's network interface with all the settings from your Subnet definition above. If it doesn't, and the output isn't helpful enough, there's also a further -v switch for more verbose logging. /System/Library/LaunchDaemons/bootps.plist . You can install this persistently into launchd like so sudo launchctl load -w /System/Library/LaunchDaemons/bootps.plist
sudo launchctl list should then show a com.apple.bootpd service enabled. If for some reason you need to disable it once again, you can uninstall the service using sudo launchctl unload -w /System/Library/LaunchDaemons/bootps.plist
How about some photos of squid flying through the air? I've heard anecdotal reports of this sort of thing happening, which on the face of it sound reasonable, if not a little far fetched. They do possess all the right sort of equipment, and controlled jet propulsion through the air isn't really that far from their usual method of locomotion at speed, which is controlled jet propulsion under the water, after all.
The full writeup in the parent post contains plenty of detail about a recent observation of groups of squid exhibiting fairly controlled, short flight. Not only does the article contain lots of interesting links to scientific write-ups of arial squid observation , but it also contains several high-resolution photo images of the buggers captured in the act.
It would make a lot of sense for them to use as an evasive action. Squid can manage impressive accelerations in their submarine environment, but through the air, they would perform even more rapidly, over short distances. "Short" is of course, relative. One of the write-ups based on observations estimates 20cm squid reaching 10m in a controlled flight. They seem to form their bodies into lifting, braking and stabilising shapes as they go. Squid are ace.
If you have a Mac, and you use Terminal.app to run UNIX commands, try executing this for a cool shell prompt
export PS1="\360\237\220\232 $ "
See what I did there?
If you are using a UTF-8 encoding for your terminal, which you probably are, and if you're using a recent OS X, and have the right fonts installed, which you probably do, you should have a little sea-shell graphic for your prompt. Literally a cool shell prompt.
In a recent revision to Unicode , code points were assigned for many emoji. Emoji-what-now? These are little emoticon glyphs that rose to popularity in Japan . Apple have included a nice typeface with full colour icons for a subset of these in the last couple of releases of both iOS and OS X, so you can use them in most applications that use the system type rendering library, like Messages. On OS X, this includes the bundled Terminal.app terminal emulator. So you can print little icons in your shell, if you know an encoding for a particular glyph.
Here's the ever popular 'pile of poo' ( U+1F4A9 )
Not sure what that is supposed to be used for, but it's terribly popular on the internet. "But how", I hear you ask, "do you find out the encoding sequences for these appealing novelties?"
Well, you can search for unicode code tables on the internet. On the Mac though, the easiest thing to do is probably to enable the Character Viewer tool via the Language and Text System preference pane.
This gets you a panel like this, where you can browse all the characters your computer knows how to render, including all the emoji sets, and find out their Unicode code points, and more importantly, a way to encode that code point in UTF-8.
So, as you can see in my fecal example, the UTF-8 byte sequence for 'pile of poo' ( U+1F4A9 ) is F0 9F 92 A9, and we can print that in a bash shell, using echo with the -e flag to enable interpreting of escape sequences, using the \x escape prefix to indicate bytes in hex.
Going back to the original shell trick, the shell emoji ( U+1F41A ) has the UTF-8 encoding F0 9F 90 9A. The bash shell doesn't seem to have an escape sequence for hex encoded bytes in it's prompt string, but it does interpret 3 digit codes prefixed with a plain \ as octal encoded literal bytes, so if we convert this hex string to four octal numbers, using bc or od, or emacs or just Calulator.app, we get the escape sequence from my initial shell example - "\360\237\220\232"
So far so cute. But is there anything vaguely useful you can do with this sort of thing? Sort of. A picture's worth a thousand words. So we could perhaps encode mnemonic information in icons, and somehow dynamically update the prompt to reflect this.
Bash will execute the contents of an environment variable PROMPT_COMMAND as a shell command immediately before the shell prompt is printed. Typically this is used to update terminal colours or title strings with escape sequences, or update PS1 to add some content that can't be printed using the built-in prompt escape functions. I decided to make my prompt respond to the result of my most recent command.
Here's the relevant shell glue I just stuck in my .bashrc
emoji ()
export PROMPT_COMMAND='PS1=$(emoji $?)'
This runs a shell function called emoji in a subshell, which returns a string based on the input argument. The input argument I'm using is the exit status of the last shell command. This gets me a smiley face in my shell prompt, unless the last command I ran returned a non-zero exit state, which in UNIX, indicates a problem happened. This makes my prompt draw as a 'confused smiley', if something has gone wrong.
Still cute, and almost useful!
I think I'll keep it for a while.
-AppleLanguages : Overload the locale for cocoa programs at the command line.
I'm amused that on porting his blog to wordpress jwz has seemingly reached the same level of disgruntlement with wordpress in about a day that it's taken me twelve months or so to reach.
Don't get me wrong, wordpress certainly gives you all sorts of awesome features OOTB, but at a certain cost of complexity, which makes things tricky to customise. Themes are hard to tweak, and the cost of entry to plugin-writing is large enough to put-off simple customisation in favour of out-sourcing to the lazyweb directory of plugins, which correspondingly increases the complexity of your install.
Most pertinently, there's the security record, a cynic might suggest it's a lack of security record. I'm gradually coming around to the line of thought that the frequency of updates actively contributes to the problem. The continual treadmill of manually updating drives people to investigate the auto-upgrade procedures, which are all built around interfaces that sound to me like designed-in exploit vectors, like having all the .php files in the software tree writeable by the httpd user, or running an FTPd service on the webhost that can chdir to the http script directories. Furthermore, the autoupgrade process is prone to terrifyingly unfriendly fail-states .
I'm not sure if there are any significantly appealing alternatives out there. I think there's probably a circular life cycle to the blog software used by any mildly technical person , that moves serially from 'simplest possible lazyweb solution', through 'this simple thing has been customised past the point of sanity, I'll write my own' all the way through to 'writing blogging software is hard, I'll just use wordpress' and subsequently right back to square one.
The elephant in the room is the simplest option. Just host your data in an fully managed service like wordpress.com , or tumblr or posterous . Or if you really don't care about handing every last bit of data you can generate about yourself into the possibly malevolent skynet-cum-panopticon Google-monster, you could get all oldskool with blogger As ever, I just can't get with the idea of giving all my content to an at-best disinterested third party. After all, that's where jwz started out , and look where that's got him. Manually migrating to wordpress, and grumbling.
Every HN Thread, ever : It's a particular flavour of Eternal September over there.
If you can read this message, it means I have properly configured blog posting by email. Is this useful? Is this a good idea? Time will tell.
In my piece yesterday, about my blog on last.fm I didn't link correctly to the article . Fixed now.
The other day at work , prompted by a shoutbox conversation with one of our users , I did a little bit of exploring some of the artist catalogue data. The idea was to find band names that were repeating words, such as ' Talk Talk ' and ' The The '. Coincidentally, I had a freshly installed database server with just this sort of information on it, and needed a good excuse to stress test it a little. PostgreSQL's regular expression support is brilliant , and it was a very trivial exercise to quickly knock up a query that returned promising data. In the process of refining it, I got a chance to play around with the Hadoop cluster. I wrote the whole thing up over on the company blog, if you'd like further details. Fame fame fatal fame, it can play hideous tricks on the brain, as the song goes .
Yesterday at work , I had to clean after up a particularly freaky Slony-I replication fault. I still haven't managed to understand quite what went wrong there. So this morning, I arrived at work in full diagnostic mode, jokingly grumbling about 'howfuckedismydatabase.com'. Laurie was particulary amused by this curmudgeonly joke, and we bantered about it. I pitched a few ideas about how such a joke site might operate, and we left it there and moved on.
Except Laurie didn't. Despite my attempts to dissuade him, he registered the domain, and started knocking together some pages based on the earlier jokes. I chipped in a couple more suggestions, and suggested some error messages, and within twenty minutes or so he had an operational site . Then we shared it with a couple of like-minded people, and left it be. A few of the other people at work passed it around, and a couple of people submitted it to reddit.
Within an hour or so things had started to really snowball. One of the reddit submissions gathered hundreds of upvotes, and for a period of time we were the number one story on hacker news . Laurie added a twitter button and a comment form to the site, and retweets and emails started accumulating fast. By mid-afternoon the site was approaching 200 hits a second, which it handled with aplomb, because he had coded it efficiently, and
configured the server sensibly.
It felt great to watch so many people comment positively about some of my dumb jokes, pretty much in real time. It gave me a really direct experience of something I'd always innately understood about the internet, but had not yet witnessed close to home; the ability to quickly reach an appropriate audience for almost any content, regardless of how specialised. Our little shared joke quickly reached out to thousands of people, who found something within it they also related to. This really amazes me.
It also showed me something about my own character. While I was perfectly happy to joke about the idea, it needed somebody like Laurie, with the skill and enthusiasm to pick up on it and make it into something tangible and exciting. I'd instinctively shied away from broadcasting it further than my desk, and my initial reaction was that developing it any further would be a waste of time and money. I was very wrong about that, it turned out to be an interesting experience, and enormous fun. I think this means I should endeavour to be a little less cynical.
I'm experimenting with desktop email clients again.
I like Apple Mail a lot, it's one of my favourite examples of GUI desktop application, but the last couple of iterations have made it a little more clumsy to use with keyboard navigation, and it doesn't scale terribly well to managing multiple, high-volume IMAP accounts. Particularly, I find refiling groups of similar emails to be more labour intensive than this task would seem to require. By means of contrast I love refiling mail on my iPhone using Apple Mail for iOS, in truth I love using Mail on my iPhone for any mail task way more than I'd expect, it's insanely usable for an email client on a tiny, squeezable hand-toy.
The real impetus for investigating a desktop alternative has come from our recent switch to using GMail for our corporate mail service at work. I hate google mail's not-quite-IMAP IMAP implementation, I hate it's sluggish IMAP performance through Mail.app, and I hate hate hate it's god-awful webmail interface. So I've been putting some thought into rethinking the way I process email. Naturally my first line of attack is to retreat to emacs .
I've used emacs for mail before, on and off. When I first switched to using linux for my desktop systems, way back in the 90s, I used gnus on emacs for mail for a while, then when I made the switch to XEmacs for a couple of years I discovered VM , which was my main INBOX on and off, following me back to GNU Emacs, with occasional experiments with Netscape Navigator , and Evolution up until I switched to a Mac full-time, around 2001. I do recall trying Thunderbird a couple of times, but I could never tolerate it for much longer than a half-hour. I also used Wanderlust for emacs for a few months when I first started working at last.fm, but I switched to using a Mac at work shortly after that, and added my work email to my Apple Mail setup.
This time around I'm trying to re-organise the way I approach mail fundamentally. A few years ago, I started deleting mail after I'd read it, unless I definitely felt it warranted keeping. I really liked the feeling of freedom that seemed to open up, releasing me from worrying about tidy filing of hierarchical mail archives that always needed archiving and backing up. Inspired by GMail's approach to tagging and searching, the mail I did keep I filed into a small set of IMAP buckets and indexed them in Apple Mail with labels and "smart folder" searches. So I'm trying to push that even further, and I'm trialling mu , a decidedly minimalist interface to email.
mu works over a local mail store, ideally Maildir . So I've started syncing my work GMail account to my laptop, using the mature, Free software syncing tool offlineimap ( I installed it from macports ). offlineimap has specific GMail support, and it's super-easy to set this up to sync to a GMail account, although I had to add a
folderfilter =
lambda foldername: foldername not in ['[Gmail]/All Mail']
to the account configuration in ~/.offlineimaprc to stop it syncing the Gmail "All Mail" filter as an IMAP folder, meaning I had 2 copies of every email going down. I set up a User launch agent via launchd to run offlineimap every 5 minutes, syncing to ~/Library/OfflineIMAP/lastfm/
Once the mail was syncing both ways, I ran
MAILDIR=~/Library/OfflineIMAP/lastfm/ mu index
to initialise the mu indexes. I can now explore the mail archive from the shell using commands like
mu find from:jira date:2w..today
which would return a summary list of emails matching the search criteria (i.e. all mail sent from JIRA in the last 2 weeks). mu is based on the xapian indexer library, and these queries run lightning-quick. The indexing process is thus entirely separate from the imap sync, and the indexes need to be updated by re-executing the 'mu index' command to keep them fresh. This takes fractions of a second after the original indexes are built.
I'm not really interested in running searches from the shell though. mu is really an archive browser ; ideal for integrating with other mail reading and sending utilities. mu ships with a nice keyboard friendly emacs interface called mu4e . mu4e offers quick navigation short cuts to browse IMAP folders, a simple syntax to mu searches, and a list of bookmarked searches, much like virtual folders. mu4e can be set to periodically update the mu index, and even run a Maildir sync, such as offlineimap. Here's the config elisp block from my startup files.
(setq-default
mu4e-maildir "~/Library/OfflineIMAP/lastfm"
mu4e-drafts-folder "/Drafts"
mu4e-trash-folder "/Deleted Messages"
mu4e-sent-folder "/Sent Messages"
mu4e-refile-folder "/Archive"
mu4e-mu-binary "/usr/local/bin/mu"
mu4e-sent-messages-behavior 'delete
mu4e-get-mail-command "true"
mu4e-update-interval 300)
all of which is quite straightforward. The root of the various folder paths is the top level Maildir. mu4e-sent-messages-behaviour is set to the symbol delete, which is recommended for GMail accounts, as GMail auto populates one of it's magical pretend folders with all sent messages. I have set mu4e-get-mail-command to true because I prefer to have the Maildir synced via my launch agent, independently from emacs.
There's a very nice mu4e manual which documents the package in detail, I haven't managed to work through it all yet. So far I'm managing quite well with manual searches, and the default set of keybindings and stored bookmarks. List view management follows the usual emacs semantics of building up 'marks' on list entries and then applying the actions in bulk, familiar to habituated emacs users from org-mode , wanderlust, dired etc.
The mail and editing and sending is borrowed from the usual emacs GNUS / smtpmail combination, which is fine, as these work perfectly well.
I've found only one tricksy wrinkle; mu4e, like any sensible thing expects email to be in plain text. If the viewer is summoned on a rich text ( usually HTML ) mail, it tries to convert it to plain text for viewing. By default is set up to use emacs built in html2text method, which frankly sucks, and failed to convert the majority of HTML mail in my INBOX. mu4e has a configuration variable mu4e-html2text-command option to use an external conversion command. This should be a utility that accepts HTML input on stdin, and returns converted text on stdout. The manual suggests using the python-html2text utilities, but I think on a Mac it makes more sense to use the mildly obscure, but occasionally useful, Apple provided shell tool - textutil
It needs to be invoked like this to work with mu4e.
(setq mu4e-html2text-command
"textutil -stdin -format html -convert txt -stdout")
And with that, everything works great. I'm going to try living with it for a few weeks before I customise it further, but I'm looking forwards to setting up Wanderlust-style dynamic refiles, and integrating crypto support, so I can return to GPG encrypting and signing my mail again, like I ought to, at my age. Never forgetting, of course, cms 1st law of software :- "All mail clients suck, intrinsically"
Wasps punish fake fighters : Science demonstrates that wasps made up to look tougher than they are receive punishment beatings.
Raspberry Pi Radio : Official last.fm scrobbler build for Raspberry Pi/Raspbian