I was running out of space in the hall to keep spare hard drives, so I put up a hard drive shelf. In order to achieve this, I used the wrong screws, applied those to the wrong hammers, and then carefully glued them into the wrong fasteners. It is level. It is mostly being held up by the pliant wall fasteners acting as springs. I used my splendid Bosch-PSB 1800 , which performed its role beautifully, injuring nobody.
Heliotrope : This German rotating building design, was the first building in the world to generate more energy than it consumed.
Ants micro-manage aphid populations : aphid farming extends to remarkable levels of population control.
Blame The Daily Mail : Adam Curtis in wonderful form on the roots of MI5.
Jeremy Corbyn fails to win backing of other UB40
Satire possibly dead.
A pro-Corbyn Labour source insisted they were unworried by the 50% endorsement. "We have the backing of the more popular and successful UB40" they said. "Proof that splitters don't prosper".
They actually tried to make #UB4Corbyn a thing.
blinks
IDW formally announces "The Strange Death Of Alex Raymond" : That is one busy cover.
The Chemist's war: how the U.S. government deliberately poisoned alcohol during Prohibition.
This is what my final day at last.fm looked like.
In the morning, this.
Yes, I'm working on getting a MAME cab smuggled into Moonfruit.
UK commuters crowdsource legal action : British commuters are raising money for legal action against rail franchise. Interesting times
Enron Email Dataset : cleaned up, organised, and presented as part of the Internet Archives dataset collection.
Self cleaning eggs : Guillemot eggs have a surface microstructure that makes them self-cleaning eggs, like a birds egg, EGG.
Quite a parameter list : @duncang is always telling me genomics is crufty
A potential exhibition of Yellowism :Tate Britain is organising an exhibition of art that has been attacked in publlc.
Wreck of HMS Tarpon found : Royal Navy submarine sunk in 1940 with a loss of at least 50 souls.
This afternoon I went on a short guided tour of the decomissioned Royal Navy submarine, HMS Ocelot . It's in a dry-dock at the Royal Dockyards working museum at Chatham , just 20 minutes down the road from home.
Apologies for the poor quality of the photos. I only had my iPhone, with 15% remaining charge, and submarines do not offer much in the way of natural lightning.
Despite having owned a year pass for the best part of a year, and frequently admired the Ocelot from the outside, this is the first time I've been aboard. The tour is short, cramped, and completely fascinating, although perhaps not for the squeamishly claustrophobic, and definitely not for the mobility impaired.
The Dockyards is a superb example of a modern lottery-assisted regeneration project. There's several large ships in dock you can wander around, huge warehouses full of boats and machinery to pore over, a ropery, an art gallery space, a working steam railway, several sub museums. Far more than you can do in a single visit, but your ticket, once purchased, is good for 12 months of repeat admission.
Fretboard Heatmaps : This is a really neat idea, although it's a pity it has so few artists.
Album Shuffle : I published my playlist generating python tool onto github
And just like that we're back. What happened cms?
It was never entirely my intention to go offline for such an extended hiatus. Even though the web is intrinsically brittle and ephemeral, I like to do my bit to keep my little backwater serving 200 OK s to the half-dozen people who stop by to check in regularly, and the couple of dozen who linked to something I put up at some point. It's basic web-citizenship as far as I'm concerned.
Before we went fully dark, I'd not posted for a long time already . And before that I'd slowed my posting down to something of a crawl. I think there's a few reasons for that. It's easy to get bored with blogging for the sake of blogging, especially in our current age where everyone shares profligately across many social platforms . It's fairly common to see blogs that have fallen into a recursion of no posts for months, then a post apologising about that, and then further disuse. I don't think this is one of those, but the proof is in the posting I suppose.
There's certainly been less time in real life for auxilliary pursuits like online rambling, and that's a big part of the reason. No time for any proper content posts, concomitant with a surge of alternative social platforms to play around with, meant it often seemed a bit redundant to post arrays of short-links , when I could just throw them up on twitter / adn / diaspora* / flickr / ello / imzy /whatever, with a bigger audience, and more interaction.
I was also feeling a bit self-conscious about standing up in public. After leaving last.fm (fairly amicably, as these things go, fwiw, albeit with a slightly battered heart), which felt like a fairly visible shift sideways, I was quite deliberately courting more obscure, maybe more unexpected job roles, and I remember feeling like I really didn't want to bare my thoughts to the internet judgement machine whilst I wasn't even entirely sure what I was doing myself a good deal of the time. Also busy! Young family plus startups really left little time for anything much else.
I also was really feeling the pain of Wordpress . I never quite managed to find an authoring approach to use with it that didn't make writing anything seem like far harder work than it ought to be, also because I always insist on self-hosting, the sheer weight of it for maintainence and security updates, and backups, and DBA -ing, and having to write PHP or perhaps even plugins to do the inevitable customisations someone like myself inevitably finds themselves suckered into doing. So Wordpress was a drag, which was feeding my reluctance to contribute much of substance. So I decided to pause on updating whilst, in true wannabe-hacker style, I whipped together some kind of alternative content publishing system.
I'll just take a paragraph out to stress that I actually admire WordPress a great deal. It's a very sophisticated and flexible web platform, and a great choice for site management, in either managed or self-hosted configuration. It kept this site ticking along for years. It just isn't a particularly good fit for my requirements, which are extremely simple
I thought about using another off-the-shelf blogging system, which would have been the sensible route, but I figured that would just lead to a similar frustrated stalemate. So I started to sketch out an application that would allow me to quickly fling out tagged and dated content without much overhead of hosting or writing. And I carried on intermittantly piecing this app together, often on trains, for a couple of years. As an exercise in procrastination, it worked out better than I expected, and I carried on posting short content to twitter and others, reasonably happy to continue to defer the responsibility.
But then the site went dark. I was hosting it all on a linode instance. I've been a very enthusiastic linode user for perhaps ten or more years, I think they have an excellent product, offering well-provisioned VPS instances , inexpensively, with an easy to use management site. Generally I've been very happy with them to date.
This changed somewhat last year, and my confidence deflated a little. There was an extended outage of service across linode in December 2015 , apparently as a result of a targetted DDOS . This lasted for many days, and the communications about it from linode were muted and suspiciously vague. This isn't really what I expect from a first-tier ISP. I came away with the impressions that there were some significant architectural problems with their infrastructure, probably from acrued technical debt , and potentially some exploitable vulnerabilities in their public facing application software . I decided it was time for a change.
I did some reasearch and rented a couple of new hosts. This time I've gone for low end, physical servers. This represented another procrastination opportunity, because when I originally set up the beatworm.co.uk linodes, almost ten years ago, I just hand configured everything by remote shell. Now I like to use the ansible configuration management system to set up hosts, and I took this opportunity to port my public infrastructure across to use repeatable playbooks. This turned into another major yak-shave , because there was slightly more to it than just a WordPress deployment, I was hosting mail, calendars, media streaming, IM, DNS, the works. After getting lost in this tarpit for a couple of months, I decided to move the application tier over to use the playbooks from the sovereign project , which covers much of the same ground, but is already written, and uses more modern components. Of course it wasn't entirely straightforward to integrate these plays over my existing base provisioning, and I ran into a couple of glitches and gotchas with some of the choices they'd made for configuration, but it only took a couple of weekends worth of fiddling to get it all running in a fairly acceptable shape. I moved the DNS across, at which point the wordpress site was left behind, and everything went dark.
I was surprised at how much this bothered me.
I like an outlet for sharing things. I enjoy the idea of having a stable internet identity . I don't like the way the modern web has folded these ideas into a handful of consumer products run by just a couple of corporate gatekeepers. That's not the web I grew up with, and it's not the web I want to see either. A very loosely federated ecosystem of ad-hoc resources, all mixed together as hypermedia, aggregated and accessed via an assorted bag of user-agents. That's how it works best . I like to write, because I like the practice and discipline of working toward articulating my thoughts for a general reader.
I like being able to curate an archive, and keep control over how that information persists and is presented. This is hard enough to do when you have primary jurisdiction over the medium and material (there is plenty of bitrot on view in my archive, particularly in the really old material, which has been migrated across multiple publishing platforms now), and basically impossible if you're relying on a third party service, which periodically re-invents itself to better serve it's own objectives, which are only ever to be tangentally aligned with your own, at best.
I don't like the sense of obligation I get from formal social media platforms. There's a subliminal sense of pressure to perform, to update, to observe the conventions, to consider and measure the implied audience. I'm not a joiner by nature. I just end up gently resenting the throng. I like to feel like I have a voice, but I don't want, or even expect to reach, an automatically provided audience.
So, I picked back up my now-neglected website platform experiment, and knocked it together enough to get an MVP out of the door. It serves HTML over HTTP. It has a relatively minimal set of style rules that should allow it to work gracefully across various screen dimensions. It has rudimentary support for RSS ( not that many people use newsreaders any more ). It's simple to run in a staging environment, and I can write posts in plain text in emacs , and edit and post them without much extra grief. It's only got about 22% of the functionality I had originally planned, but I feel the urge to ship it, use it, and hopefully I'll refine it in production.
There's a couple of interesting quirks to this new hosting setup. It's an ARM -based micro-blade, hosted on a scaleways C1 . The blogging software is semi-static , in as much as it serves generated content from the filesystem. It's written in common lisp , and deployed in a different lisp to the one it's developed on There's no frameworks (aside from using zurb foundation classes to base the CSS). There's no database. There's no comments, because I haven't yet decided on a productive way to support them.
Apple Vs GPL : Apple's attitude to GPLv3 is making OS X an increasingly shonky UNIX developer system
If you have a Mac, and you use Terminal.app to run UNIX commands, try executing this for a cool shell prompt
export PS1="\360\237\220\232 $ "
See what I did there?
If you are using a UTF-8 encoding for your terminal, which you probably are, and if you're using a recent OS X, and have the right fonts installed, which you probably do, you should have a little sea-shell graphic for your prompt. Literally a cool shell prompt.
In a recent revision to Unicode , code points were assigned for many emoji. Emoji-what-now? These are little emoticon glyphs that rose to popularity in Japan . Apple have included a nice typeface with full colour icons for a subset of these in the last couple of releases of both iOS and OS X, so you can use them in most applications that use the system type rendering library, like Messages. On OS X, this includes the bundled Terminal.app terminal emulator. So you can print little icons in your shell, if you know an encoding for a particular glyph.
Here's the ever popular 'pile of poo' ( U+1F4A9 )
Not sure what that is supposed to be used for, but it's terribly popular on the internet. "But how", I hear you ask, "do you find out the encoding sequences for these appealing novelties?"
Well, you can search for unicode code tables on the internet. On the Mac though, the easiest thing to do is probably to enable the Character Viewer tool via the Language and Text System preference pane.
This gets you a panel like this, where you can browse all the characters your computer knows how to render, including all the emoji sets, and find out their Unicode code points, and more importantly, a way to encode that code point in UTF-8.
So, as you can see in my fecal example, the UTF-8 byte sequence for 'pile of poo' ( U+1F4A9 ) is F0 9F 92 A9, and we can print that in a bash shell, using echo with the -e flag to enable interpreting of escape sequences, using the \x escape prefix to indicate bytes in hex.
Going back to the original shell trick, the shell emoji ( U+1F41A ) has the UTF-8 encoding F0 9F 90 9A. The bash shell doesn't seem to have an escape sequence for hex encoded bytes in it's prompt string, but it does interpret 3 digit codes prefixed with a plain \ as octal encoded literal bytes, so if we convert this hex string to four octal numbers, using bc or od, or emacs or just Calulator.app, we get the escape sequence from my initial shell example - "\360\237\220\232"
So far so cute. But is there anything vaguely useful you can do with this sort of thing? Sort of. A picture's worth a thousand words. So we could perhaps encode mnemonic information in icons, and somehow dynamically update the prompt to reflect this.
Bash will execute the contents of an environment variable PROMPT_COMMAND as a shell command immediately before the shell prompt is printed. Typically this is used to update terminal colours or title strings with escape sequences, or update PS1 to add some content that can't be printed using the built-in prompt escape functions. I decided to make my prompt respond to the result of my most recent command.
Here's the relevant shell glue I just stuck in my .bashrc
emoji ()
export PROMPT_COMMAND='PS1=$(emoji $?)'
This runs a shell function called emoji in a subshell, which returns a string based on the input argument. The input argument I'm using is the exit status of the last shell command. This gets me a smiley face in my shell prompt, unless the last command I ran returned a non-zero exit state, which in UNIX, indicates a problem happened. This makes my prompt draw as a 'confused smiley', if something has gone wrong.
Still cute, and almost useful!
I think I'll keep it for a while.
-AppleLanguages : Overload the locale for cocoa programs at the command line.
Every HN Thread, ever : It's a particular flavour of Eternal September over there.
I already mentioned in passing, St. Vincent , the band-shaped solo project brand thing of the super-engaging Annie Clark, was by far the best act I saw at Primavera Sound 2014. It was also the act I was most looking forward to seeing going in, it’s always nice when those line up.
I guess I’m a super-fan. I first spotted Annie playing with Sufjan Stevens ' touring band. I next encountered her playing solo support for the National , touring her first St. Vincent release , upon which occasion I bolted out of the auditorium by the third song, in order to make sure I got a copy of the CD she was plugging from the merch stall before she packed away. I saw another couple of shows in Bristol, with the full band, and bought all the records, including an interesting collaboration with David Byrne .
Last weekend, while idly browsing the Glastonbury live blog, I noticed that they’d just updated their description of the current iPlayer feeds to include St. Vincent streaming on the iPlayer from the park stage. I’d been avoiding the Glastonbury video feeds due to a combination of not being in the mood, and the dullness of the tv schedules, but I wasn’t going to miss out on this, so I whacked it on the TV. True to form, it was a great set, live, risky, and peppered with amusing crowd-surfing and hat theft . Even with a bit of sound problem, and some streaming glitches I enjoyed myself, and was amused to see my enthusiastic tweeting duly included in the Guardian live feed on the next page refresh.
“ That was a really good set ”, I thought to myself, afterwards, “ but it wasn’t nearly as exciting as the Barcelona one. True, that lacked crowd invasions, and nobody lost a hat, but the lighting, and the sound, and the staging, and the lack of daylight, and the crowd being really into it…A pity there’s no TV-broadcast quality stream of that night archived away somewhere ”.
Yes, I do really talk to myself like that sometimes. Especially when I’m pretending to transcribe my inner voice for a blog.
And then, I ran into this on Youtube.
Full set, multiple cameras, properly mixed sound, pretty good video quality. I have not yet watched it enough times to see if I can see myself ( front of house, stage left, VIP pen ) in the crowd, but I expect I will.
Fish Eating Spiders : Collated observational evidence identifies as many as five families of spiders that regularly hunt and consume fish.
Rusty green submarine : official news agency reported the leader “taught” the submarine’s captain a “good method of navigation”.
St. Vincent Interview : I respect her enormously. Best thing at Primavera Sound 14 by miles.
I'm experimenting with desktop email clients again.
I like Apple Mail a lot, it's one of my favourite examples of GUI desktop application, but the last couple of iterations have made it a little more clumsy to use with keyboard navigation, and it doesn't scale terribly well to managing multiple, high-volume IMAP accounts. Particularly, I find refiling groups of similar emails to be more labour intensive than this task would seem to require. By means of contrast I love refiling mail on my iPhone using Apple Mail for iOS, in truth I love using Mail on my iPhone for any mail task way more than I'd expect, it's insanely usable for an email client on a tiny, squeezable hand-toy.
The real impetus for investigating a desktop alternative has come from our recent switch to using GMail for our corporate mail service at work. I hate google mail's not-quite-IMAP IMAP implementation, I hate it's sluggish IMAP performance through Mail.app, and I hate hate hate it's god-awful webmail interface. So I've been putting some thought into rethinking the way I process email. Naturally my first line of attack is to retreat to emacs .
I've used emacs for mail before, on and off. When I first switched to using linux for my desktop systems, way back in the 90s, I used gnus on emacs for mail for a while, then when I made the switch to XEmacs for a couple of years I discovered VM , which was my main INBOX on and off, following me back to GNU Emacs, with occasional experiments with Netscape Navigator , and Evolution up until I switched to a Mac full-time, around 2001. I do recall trying Thunderbird a couple of times, but I could never tolerate it for much longer than a half-hour. I also used Wanderlust for emacs for a few months when I first started working at last.fm, but I switched to using a Mac at work shortly after that, and added my work email to my Apple Mail setup.
This time around I'm trying to re-organise the way I approach mail fundamentally. A few years ago, I started deleting mail after I'd read it, unless I definitely felt it warranted keeping. I really liked the feeling of freedom that seemed to open up, releasing me from worrying about tidy filing of hierarchical mail archives that always needed archiving and backing up. Inspired by GMail's approach to tagging and searching, the mail I did keep I filed into a small set of IMAP buckets and indexed them in Apple Mail with labels and "smart folder" searches. So I'm trying to push that even further, and I'm trialling mu , a decidedly minimalist interface to email.
mu works over a local mail store, ideally Maildir . So I've started syncing my work GMail account to my laptop, using the mature, Free software syncing tool offlineimap ( I installed it from macports ). offlineimap has specific GMail support, and it's super-easy to set this up to sync to a GMail account, although I had to add a
folderfilter =
lambda foldername: foldername not in ['[Gmail]/All Mail']
to the account configuration in ~/.offlineimaprc to stop it syncing the Gmail "All Mail" filter as an IMAP folder, meaning I had 2 copies of every email going down. I set up a User launch agent via launchd to run offlineimap every 5 minutes, syncing to ~/Library/OfflineIMAP/lastfm/
Once the mail was syncing both ways, I ran
MAILDIR=~/Library/OfflineIMAP/lastfm/ mu index
to initialise the mu indexes. I can now explore the mail archive from the shell using commands like
mu find from:jira date:2w..today
which would return a summary list of emails matching the search criteria (i.e. all mail sent from JIRA in the last 2 weeks). mu is based on the xapian indexer library, and these queries run lightning-quick. The indexing process is thus entirely separate from the imap sync, and the indexes need to be updated by re-executing the 'mu index' command to keep them fresh. This takes fractions of a second after the original indexes are built.
I'm not really interested in running searches from the shell though. mu is really an archive browser ; ideal for integrating with other mail reading and sending utilities. mu ships with a nice keyboard friendly emacs interface called mu4e . mu4e offers quick navigation short cuts to browse IMAP folders, a simple syntax to mu searches, and a list of bookmarked searches, much like virtual folders. mu4e can be set to periodically update the mu index, and even run a Maildir sync, such as offlineimap. Here's the config elisp block from my startup files.
(setq-default
mu4e-maildir "~/Library/OfflineIMAP/lastfm"
mu4e-drafts-folder "/Drafts"
mu4e-trash-folder "/Deleted Messages"
mu4e-sent-folder "/Sent Messages"
mu4e-refile-folder "/Archive"
mu4e-mu-binary "/usr/local/bin/mu"
mu4e-sent-messages-behavior 'delete
mu4e-get-mail-command "true"
mu4e-update-interval 300)
all of which is quite straightforward. The root of the various folder paths is the top level Maildir. mu4e-sent-messages-behaviour is set to the symbol delete, which is recommended for GMail accounts, as GMail auto populates one of it's magical pretend folders with all sent messages. I have set mu4e-get-mail-command to true because I prefer to have the Maildir synced via my launch agent, independently from emacs.
There's a very nice mu4e manual which documents the package in detail, I haven't managed to work through it all yet. So far I'm managing quite well with manual searches, and the default set of keybindings and stored bookmarks. List view management follows the usual emacs semantics of building up 'marks' on list entries and then applying the actions in bulk, familiar to habituated emacs users from org-mode , wanderlust, dired etc.
The mail and editing and sending is borrowed from the usual emacs GNUS / smtpmail combination, which is fine, as these work perfectly well.
I've found only one tricksy wrinkle; mu4e, like any sensible thing expects email to be in plain text. If the viewer is summoned on a rich text ( usually HTML ) mail, it tries to convert it to plain text for viewing. By default is set up to use emacs built in html2text method, which frankly sucks, and failed to convert the majority of HTML mail in my INBOX. mu4e has a configuration variable mu4e-html2text-command option to use an external conversion command. This should be a utility that accepts HTML input on stdin, and returns converted text on stdout. The manual suggests using the python-html2text utilities, but I think on a Mac it makes more sense to use the mildly obscure, but occasionally useful, Apple provided shell tool - textutil
It needs to be invoked like this to work with mu4e.
(setq mu4e-html2text-command
"textutil -stdin -format html -convert txt -stdout")
And with that, everything works great. I'm going to try living with it for a few weeks before I customise it further, but I'm looking forwards to setting up Wanderlust-style dynamic refiles, and integrating crypto support, so I can return to GPG encrypting and signing my mail again, like I ought to, at my age. Never forgetting, of course, cms 1st law of software :- "All mail clients suck, intrinsically"
Raspberry Pi Radio : Official last.fm scrobbler build for Raspberry Pi/Raspbian
