1. Last week I started a new job and with a new job comes a new computer. These days, I am once again a fairly comitted desktop linux user, for my sins, and so I asked for a Lenovo ThinkPad x270. Not without trepidation, because even though I'm a fairly expert user, it's been some time since I put a linux distribution onto a modern PC-laptop flat, and more recent hardware can present some driver challenges. It all went on pretty well, nonetheless, with just a moderate amount of tweaking, and a week or so in I can report that I'm pretty delighted with it. It's a wonderfully solid and useful piece of kit, everything works. Screen, keyboard and portability are spot on, battery life is a phenomenon, and there's at least one of every kind of useful port I care about.

    It's my first ThinkPad with a 'chiclet' keyboard. It's my first without a seven-row keyboard actually. I was a little bit worried about that. Keyboards are one of those things you like ThinkPads for, if you're the kind of person who likes ThinkPads, and of course I am. Actually, the keyboard is great. I think I can type faster on it than I can on my well-loved x220 model, which is basically my high-water mark for a laptop keyboard. Trackpoint is present and works as well as ever, trackpad is a huge improvement. I am not going to say that I wouldn't like the missing keys and ThinkLight back, but I'm not aggravated by their absence. After all I can use my 3l337 remapping skills to make sure I have everything I need somewhere that I can access it, and the less often used things can just go on mod key combinations and function shifts. It has the makings of a truly great keyboard if I'm honest, although I accept these things are subjective. There was just one amusing wrinkle though.


    For some reason they've put the PrtSc key in where the menu key was. This seemed pretty weird, but it could be worse. At least I still have a balanced group of three modifier keys either side of the space bar. It goes LCTRL WINDOWS ALT SPACE BAR ALTGR PRTSC RCTRL. I just modified my xkb settings very slightly to redefine PRTSC, and I was back to using my happy path of SUPER LMETA LCTRL SPACE BAR RCTRL RMETA SUPER, and emacsing about with gay abandon. Right up to the first time I hit backward-sexp whilst cheerfully editing code, and to my astonishment my laptop immediately rebooted without any warnings. I was so stunned I immediately tried that again. Same result. I was dumbfounded for maybe sixty seconds before I figured it out.

    PrtSc is an old key, although unlike many of the old dedicated PC buttons, (Scroll lock anyone?), it's managed to reinvent itself for modern generations. Typically it is used to trigger a screenshot. GNOME sets it up for that, and while I was remapping it I figured I would be able to manage just fine without a dedicated key for screenshotting. Print screen often used to share a key with another ancient button, SysReq, and System Request is a really interesting beast. Turns out, even though it's not labelled like that, the PrtSc key on my x270 was also a SysReq. And system requests are the key to this laptop narcolepsy.

    System Request was a button deliberately designed to bypass as much of your software as possible, and send a hardware interrupt direct to the operating system hardware event loop. Normal keyboard handling is entirely bypassed. It's a brain probe. No matter how elaborate your interface, or hotkey macros become, you have a dedicated batphone right there on your keyboard, a zap line into the mainframe. Even in it's most locked up system crash, this is a signal that could still get through.

    Originally, SysReq had it's own proud dedicated button. Then, as it's usage was a little bit esoteric, it became seconded to PrtSc. If you wanted to access the magic zap you still could. You just hit Alt in conjunction with PrtSc. And that happens to be the second piece of our puzzle. SysReq lingered on over there for some decades, largely entirely unused. A vestigial organ, like an appendix, or a supernumerary nipple. One of those dorky joke keys on a PC nobody understands or uses, that cool Apple systems condesncendingly wink at. Linux uses it though. Linux doesn't mind being dorky, and can always use a spare modifier key. Especially one with a hardware function.

    It's called the Magic SysRq key. Linux has a special interrupt handler sat there in the kernel listening for it. You can hit Alt + SysReq and then another key, and trigger special, super low level system recovery or debugging features, such as triggering a crash dump, forcing an OOM kill, or yes, rebooting the system. And that's where I was hitting it. With my remapping in place, ALT is CTRL, PRTSC is META, so when I am editing a lisp file, and hit C-M-b to move backwards one sexp, I'm actually banging on the chord that bypasses all my software stack, and pushes a reboot lever deep in my computer's lizard brain, which it dutifully obeys. A little bit frustrating, but honestly, as soon as I figured out what must be going on, it made me chuckle out loud.

    Linux being linux, it's entirely configurable of course. You can build a kernel with the feature missing, you can disable it in software, or you can configure a bitmask to define which key sequences are trapped and acted upon. I have opted to disable it for now. I would rather have my META key where I like it to be, than have an easy access debugger's powertool. Now everything is closer to perfect.

    posted by cms on
    tagged as
  2. Why I care about keyboard modifiers

    This is a merry dance.

    One of those things I generally expect to be part of the routine of running a linux desktop is a certain amount of manual effort necessary to keep things running smoothly. Sometimes this is the classic multi-decade horror story of "sound card" configuration. Sometimes its the inevitable friction between under-specified hardware and volunteer-maintained drivers. Sometimes it's the CADT principle, where everything changes around you, just because it can, as part of the generational cycle of collaborative software development. Sometimes it's just the self-induced consequence of having a system where you can tweak and configure everything to work however you'd like it to, and therefore you choose to, and sometimes that's quite a deep rabbit hole. This tale covers a small handful of these categories, although it's primarily a consequence of that lattermost case.

    Mod life

    Mod keys. Modifier keys, that is. Those would be the keys you hold down alongside other keys to change their behaviour. The most obvious and venerable of these is SHIFT. Hold that down and type an alpha-numeric key, and you generate a different character. With the alphabetical keys, you GET THE CAPITAL LETTER FORM. Other keys, like the numerals, get you punctuation. You may also be aware of the Alt/AltGr modifier keys, which hang around on most keyboards, and generally allow access to a different shift level, further symbols and accented characters. And then there's control, or CTRL. Maybe you know that guy as the menu-shortcut accelerator key, or maybe for a couple of shortcuts you might use in the shell, if you're a shell user. Actually the kids all call it 'terminal' these days, because that's what Apple does. And then they all use iTerm anyway, for I don't know why really but I'm sure it's great. Anyway, calm down Mac-loving readers, this story is about how terrible linux is, you're all wonderful. So CTRL in the shell - CTRL + C cancels things, CTRL + D ends a session. CTRL + A takes you to the start of the line. CTRL + E takes you back to the end. Assuming you're using a fairly standard bash shell. Those last two are slightly more interesting, and immensely relevant to this story.

    Enter Emacs

    They're readline bindings. Readline is a GNU library used to make command line shell editing a little more interactive. And because bash is the GNU shell, it uses readline by default. Those keybindings are the default readline bindings, and they work the same in any application that uses readline. Typically this means other interactive shells. These keybindings are taken from Emacs. Emacs is a text editor, that is to say it's an application for interactively working on so-called 'plain text' files. Not that's there's any such thing as a plain text file. Emacs is one of the most ancient, convoluted, complex, crufty, awkward pieces of software you're ever likely to encounter. It's one of the original fundamental components of the GNU system. You might say it was the standard editor. Emacs is also one of my all-time favourite things. So those readline keybindings we were discussing, are intended to bring some of the more capable text editing commands from the GNU text editor across to the GNU shell, making use of modifier keys. Emacs really really likes modifier keys.

    Because Emacs is a very old piece of software, it's design was heavily influenced by the keyboards typically used on the systems of its time. Computer use was a lot more text and command oriented, and the large, pre-PC era keyboards tended to reflect this by having a large amount of mod keys and function keys available. Commonly cited examples are the MIT or symbolics lisp machine 'space cadet' keyboards, and the Knight keyboard. As a consequence of this emacs can understand a lot of different modifier keys, and has a UI that is organised around layering functionality onto different keyboard 'chord' operations. You really need at least two distinct mod keys as a bare minimum to get emacs to do anything useful at all. We already met CTRL a couple of paragraphs back, but you also need another key called META. Emacs uses them in fundamental, and interestingly composable ways. For example, you can move the cursor forward one position by typing CTRL + F, but you can move the cursor one word forward by typing META + F. It's powerful, and sort of intuitive once you understand the fundamentals quite well. Unfortunately, they mostly stopped making keyboards with META keys on them some while back.

    Welcoming the X Window System to the fray

    Now I am getting quite old, but I'm not ancient enough to have run emacs on pre-Internet era hardware. I did use it a little bit on 7-bit serial terminals, and limped along using ESC as a prefix modifier, like a farmer, but by the time I started really learning how to use emacs to any serious degree, I'd made the jump to UNIX machines using X11 as a graphical user terminal. Some of the UNIX workstations had a META key. Some of them didn't, but had a few other modifier keys. Increasingly, UNIX graphical workstation started to mean 'Linux and XFree86 on PC hardware'. Now IBM-derived PC keyboards don't have a META key and never did. The original PC keyboards didn't really offer many modifier keys, but by this time period, everything had mostly standardised on the 101/102 key IBM extended model archetype. This doesn't have META keys, but it does have a pair of prominent ALT modifier keys. And so, we begin to remap.

    X11 is maybe one of the canonical reference points for design by committee. Fully intended to offer a portable graphical , networked user interface across a variety of dissimilar UNIX systems, it tries very hard to offer the broadest possible set of abstractions across similar base behaviours, trying to build a unifying API in all aspects. Screen dimensions and orientation, color model and layout, pointers, input devices, key-types, you name it. So you can usually configure your equipment in a bewildering, verging on frustratingly flexible manner. X11 is a very broad church and welcomes all kinds of keyboards. X11 allows you a whole byte for modifier keys (I think), so you can have Shift, Lock , Control, and then five others called Mod1 through Mod5. You can freely map key codes onto key symbols, and then assign key symbols to one or more modifiers. So obviously this all took fourteen hours to decipher in the first instance, but I gradually became reasonably adept at using the Xmodmap utility to set ALT to be both ALT and META, CAPSLK to be another CTRL and life was mostly good. You'd tweak your .Xmodmaprc file every time your keyboard changed significantly, load it in as part of your login, and everything would work. PC-104/105 keyboards came along, with windows keys, and this meant that you could perhaps add a SUPER or even a HYPER key, and bind those to other emacs macros. The system was working, and everyone got rich on the proceeds! Or not. Nonetheless, although linux desktop software was fairly terrible, it was a fine environment for running Emacs, and running Emacs was where most of the work got done after all.

    A detour into Macintosh

    Times have changed however, and systems have changed, and uses have changed, and so have I. Like everyone else, I started using laptops more. Modifier keys started getting scarce again, as the machines shrank down to be portable, and interfaces just grew ever more graphical. For about a decade, I used Macintosh systems, which represent their own series of keyboard configuration challenges. Macs are actually pretty good for modifiers, if I'm being fair - they have their own dedicated command key for all the system key shortcuts, and so you're free to map control and option as you see fit to control and meta. They even give you a little GUI configurator for managing and assigning modifier keys, which is way more convenient than spending hours searching for information about xmodmap. The main suckitude is that they don't have a symmetrical set of them on their laptop keyboards. You don't get a right hand CTRL. Symmetry is important for healthy typing habits with key chords, because it's vastly better for your hands if you use both of them for combinations. So you ideally want to be able to hold down CTRL META SUPER or some combination of them with one hand whilst you type the activation keys. So CTRL + C is best expressed as a right hand finger holding CTRL, whilst your left middle finger taps the C. So my life as a Macintosh Emacs user was constantly blighted by crazy-ass schemes to find keyboard layouts that allowed unstressful ways to type CTRL key combinations.

    Desktop Linux in the present day

    For the last few years though, I've been back on the linux horse (and why is a different story, for another day), and my main laptop, a battered lenovo ThinkPad, has a full set of three modifiers either side of the space bar, where they were intended to go. The Debian GNOME 3 desktop is configured to use the windows and menu keys for desktop commands, and the ALTGR key, which I have on the right, as some kind of compose prefix. Even thought it's X.org now, not XFree86, and Xmodmap is heavily deprecated in favour of the XKeyboard and Xinput extensions, using the GNOME configurator and then some of my old Xmodmap ways, I could make this go away, and map ALTGR to a right meta, ALT to a left meta and the windows and menu key to SUPER and HYPER. The lenovo x220 I use has a particularly excellent keyboard and all was right in the world.

    And then GNOME 3.22 switched to Wayland as a display server, rather than X. And this year's Debian defaulted to this. Even though there is an X11 compatibility layer, GTK+ and GNOME on Wayland do not talk to X11 directly for mediated key events any more, and this meant that Xmodmap can't be used to universally set modifier maps. GNOME 3 on wayland will still use xkb for key configurations, and this meant another fourteen hours of fiddling about in order to come up with a keyboard scheme that works for both GNOME and legacy X using the XKeyboard extension (XKB). This was not made any easier by the fact that all the attempts to search for information on this get bogged down in legacy explanations about Xmodmap or how to enable XKB for X11. But I got there in the end

    It seems like there's not actually any supported, or easily documented way to load user configurations into GNOME 3 + Wayland's XKB environment, so I ended up slightly disappointedly hacking them into the system options files. Of course this meant that several months after I did this, a system upgrade overwrote all of my changes, and I was left without a keyboard, and a scant recollection of how I ever did it, or what any of the bits were even called.

    Finally I fix it

    So this morning I figured out how to assemble it all again from first principles. To make it more worth my while, this time I decided to transpose all of the mod keys as I went, so I can have CTRL on the inside of META as it was originally intended to be, and push the other modifiers to the outside edge. To save myself the bother the next time this breaks underneath me, I thought I'd write down the exact sequences here. I am not going to try and attempt to explain XKB here. There are a several documents on the web that do that job, to varying degrees of success. I'm not going to pretend that I understand how it all works, I just experimented with xsetkbmap and xkbcomp under an X11 desktop until I understood how to express what I needed to work under Wayland. Here are the steps.

    System-wide keyboard configuration is fine for configuring the basic keyboard layout - using the Debian keyboard configurator, I can pick either a ThinkPad or a pc-105 model with a gb layout. The modifier layout can then be selected using xkboptions. I can tell GNOME what XKB options to apply from its database, using the dconf configuration key /org/gnome/desktop/input-sources/xkb-options.

    dconf editor in all it's glory

    If you're playing along at home, you may have spotted that cmswin is not the name of any valid xkb layout. The wrinkle is that none of the built-in options offer quite the right set of combinations. So this is how I added my own custom XKB option.

    1: Define an option

    I added a new file usr/share/X11/xkb/symbols/cmswin to define my partial keymap.

    Its contents:

    // alts are ctrls, winkeys are metas, ctrls are supers  
    partial modifier_keys  
    xkb_symbols "cms_modkeys" {  
                replace key <LALT> { [ Control_L, Control_L ] };  
                replace key <LWIN> { [ Alt_L, Meta_L ] };  
                replace key <LCTL> { [ Super_L ] };  
                replace key <RALT> { [ Control_R, Control_R ] };  
                replace key <MENU> { [ Alt_R, Meta_R ] };  
                replace key <RCTL> { [ Super_R ] }; };  
    }; // end  

    that defines the option.

    2: Add it to the rules database

    Further to this, I modified /usr/share/X11/xkb/rules/evdev

    adding the line

       cmswin:cms_modkeys            =       +cmswin(cms_modkeys)  

    to the section

      ! option        =       symbols  

    I believe this is adding an option named cmswin:cms_modkeys to the dataset assigning it to parsing the 'cms_modkeys' entry from the 'cmswin' file, in the symbols subdirectory. The convention in xkb is to name all the different symbols using the same substrings, and it's terribly confusing when you're trying to remember which part does what, although slightly helpful when you're trying to perform the reverse map and locate which file is responsible for which option, I suppose.

    3: Make it available for GNOME

    The final step is to add the line

    cmswin:cms_modkeys   fix keys for emacs  

    into the file /usr/share/X11/xkb/rules/evdev.lst

    I think this does something like import the option into the environment. There is also an evdev.xml file in the rules directory, which looks like it marks up the options to be used by the GNOME gui, but I didn't bother with that one, because life is too short to hand write XML for computers to parse, and I'd already spent half a day setting this all up. To give you an idea of how tedious this all was, for a while I'd added the evdev option into the section marked !option = types rather than symbols, and this caused wayland to stick to a crash loop as soon as I loaded the XKB option into the dconf key (with no visible error logs! yum!)

    4: Retire, rich from the proceeds

    With all of this in place however, everything works fine. For now. GNOME seems to be in a bit of a transitory phase with regards to keyboard and input configuration, it looks like they're reworking everything to use IBUS in the long term, so I expect I'll be doing some form of this dance again within a year. Until then though, this document can serve as a reference for the next time I, or anyone else interested enough needs to figure out how to do this.

    2017 then, and nothing seems to have really changed that much at all. Desktop Linux is still terrible, and desktop linux is still awesome. Emacs is still terrible, and Emacs is still the best tool I have.

    posted by cms on
    tagged as