Generating an index of Haskell haddock docs automatically

Haskell’s great, and has a lovely documentation generation system called haddock. Even better, if you install one of Haskell’s many third-party libraries using the excellent cabal install, it will (if you configure it to do so) generate these docs for you. Having local copies of documentation like this is always good for when you’re offline, so this is A Good Thing.

Sadly, there’s no master index of what’s installed. I’ve got it configured to install globally, so the docs go into /usr/local/share/doc, and each (version of each) package gets a folder of its own there; if you’ve got cabal calling haddock, that folder will contain an html folder with the docs in, but it’s tedious to click through an (otherwise empty) folder to get to it each time, and the whole setup’s not very pretty or informative (and the lexicographic sorting is case-senstivie, which I don’t much like). Eg:

Bare index of Haskell docs

People have attacked this problem before, but PHP makes my skin itch, and I can’t be bothered with apache, so a simpler, static solution seemed right for me.

Thus, I’ve knocked up a (quick, dirty, nasty) python script to generate the index. As a happy side-effect, I’ve pointed it at the hackage CSS file, so it’s pleasingly pretty and familiar:

Pretty index of Haskell docs

I did it in python because that’s still my go-to “hack stuff together in a hurry” language, I guess; but I was very nearly did it in Haskell. Mainly it was library knowledge that drove me. Also perhaps I fancied a break. Hmmm. Next time, for sure.

If anyone’s interested in the code (the doc path is hardcoded, so if you’re installing user docs, change it), you can view it pretty/download it via this link, or just check it out here. Oh, and what about the “automatically” part? Well, just stick it in a cron job… ;-)

Update: I realised I wasn’t linking to the actual index.html file, which kinda defeated the point of the script! However, it’s an easy fix. The line that says:


                s.write('<a href="file://%s">' % package.path)
</a>

should actually say:


                s.write('<a href="file://%s">' % os.path.join(package.path, 'html', 'index.html'))
</a>

Aaaanyway…

Read the rest of this entry »

Really nice git cheatsheet

Here’s a nice git cheatsheet, with examples: Everyday GIT With 20 Commands Or So [via brunns]. Really good — and to think that I didn’t know about git fsck and git gc!

A Haskell shell for computing musical scales, in 58 lines of code.

On my way home from work I had an idea for something I thought I’d find handy. Two hours later, I’ve written it — in Haskell of course. :-)

Context: I occasionally make what might kindly be called music, using computers and the like, and I don’t have any formal musical training. Consequently, I sometimes run into trouble regarding keys and scales. The idea I had was for something which would tell me, given a set of notes (ie the ones I’ve already used in a piece), what scales those notes are in — and consequently what other notes might also fit well with them. Long ago when I used Bars & Pipes on the Amiga, it had a tool for doing this. I could buy one for my Mac for about 8UKP (I may yet — it has a pretty GUI, which I might not be bothered to do), but it seemed I should be able to knock something up. The data types aren’t exactly complicated.

So I did it. I’ll put the code at the end of this post, or you can easily download it from here, in 133 lines of literate Haskell (58 lines of actual code). Here’s a screenshot showing typical use:

Haskell scale finder screenshot

Once again, I’ve been super impressed by how easy Haskell made this — and I’m particularly loving the Shellac library which gives me an interactive shell with history and graceful recovery from errors — all in 4 lines of code!

> react = mapM_ (uncurry printScale) . checkFit . parseNotes
>     where printScale :: String -> [Note] -> Sh () ()
>           printScale n ns = shellPutStrLn (n ++ ": " ++ ppNotes ns)
> main = runShell (mkShellDescription [] react) haskelineBackend ()

OK, I’m not using any of its juicier features like completion, state, automatic help, etc., but even so, it’s a great little library.

I could wrap it in a GUI, but to be honest, this’ll probably do the trick for me, assuming I haven’t made any big screw-ups, or got the scale definitions wrong. If anyone’s looking for a minimal Shellac example, maybe it’ll be useful.

(Update 2009-09-16 13:37: To get this up and running, if you’re not already a Haskeller, the following should suffice: 1. Download and install The Haskell Platform – nice ‘n’ easy. 2. Open a shell and issue cabal install Shellac-haskeline or possibly sudo cabal install --global Shellac-haskeline. 3. Download the code (from hpaste.org) and (still in a shell) issue ghci Scales, and when that’s loaded issue main. Or if you want to compile an executable, try ghc --make -main-is Scale.lhs Scale.lhs, just like in the screenshot. That should do it — I’d love to know if I’ve missed anything and/or if that works!)

Here’s the code (but, again, it’s prettier and easier to download from hpaste):

Read the rest of this entry »

Typogridphy

Typogridphy — A Typographical and Grid Layout CSS Framework [via hacker newspaper via raganwald].

ToneMatrix: funnnn!

Much fun to play with: ToneMatrix, a Tenori-on-like Flash jobbie [via rhodri]. Go! Now! Play!

The Algorithmic Beauty of Plants

The Algorithmic Beauty of Plants.

How to use git and svn together

How to use git and svn together — or more properly, how to use git on an svn repository. I’ve been doing this for a couple of things, having switched my brain and day-to-day work patterns from svn to git, and it works nicely. I haven’t tried anything nontrivial such as branching, mind. Anyway, putting it here because some friends were asking about it.

Stone Buddha on my bedroom wall

Stone Buddha, rasterbated

Original, rasterbated.

sshfs with RSA keys on OS X using MacFUSE

MacFUSE is nifty, and in particular supports sshfs for mounting filesystems from ssh-accessible remote boxes.

However, I found today that using the GUI always asks you for your “ssh password”. I don’t use a ssh password; indeed, all my boxes have password tunneling disabled: you can only log in via keys. The GUI didn’t/doesn’t seem to be key-aware.

Thankfully, the command-line version (sshfs-static) mentioned on the wiki page linked above is aware – so I succeeded in mounting my remote fs using:

Applications/sshfs.app/Contents/Resources/sshfs-static om: om \
-oreconnect,volname=om

I’ve set up an alias for that in my .zshrc and am now in sshfs happiness land.

(Oh yes, perhaps I should have mentioned: I’m using OS X now, for my desk/laptop needs at least. More on this as time progresses, no doubt.)

Other good reasons: the existence of Python, Ruby, Haskell, Smalltalk, …

There are many, many reasons why you should never ever accept a job which involves programming in C++ [via TR].

Where and how far away is the beer?

Two neato sites/apps spotted recently via fellow moderately-curmudgeonly-beer-drinkers’ blogs:

gmap pedometer [rhodri] — plot a course on a google map and find out how far you’ll walk. For example, on Thursday I was able to use this map to ensure that of the sixty-odd people I invited (via Facebook magic) to join me for a walk on Saturday, only one actually turned up, the rest presumably thinking “10 miles? Bugger that!”. As it was we skipped Mumbles Head and chopped it down to eight or so, but still, we look hardcore. Or foolish. My only complaint about gmap-pedometer is that there doesn’t seem to be a way to say “stop recording”, so you (yes, you, reader) could, presumably, go there now and add extra points to my walk, causing anyone coming after you to think I’d spent Saturday yomping enthusiastically from Mumbles to, say, Chicago. Untrue.

Secondly, and maybe more usefully: Beer In The Evening [smallcool] — search pubs by locality and various other (mainly binary) metrics. For example: pubs near my house that serve real ale. Hmmm, I’m sure there are really more than three, but at least it got The Park and would thus presumably help find something good, when in “a tight spot”. While in Llantrisant recently wondering if there were any nice pubs nearby serving food, a beautiful woman suggested to me that it’s a good idea to keep a copy of The Good Pub Guide or similar (eg this one on my wishlist) in your car. We were in her car so I didn’t feel inadequate or ashamed for my lack of pub-finding skills, but seeing this site makes me think that with an iPhone, such a book could be unnecessary. Plus, chicks dig iPhones. And so the ongoing quest to replace all printed media with collaboratively-generated content marches on, driven, as all human activity, by the desire to impress cute friends…

Hmmm, I think I might be overdoing it with the commas…

One way not to conduct Internet voting

USA Democratic Party “global primary” for Democrats Abroad badly run, insecure, untrustworthy — just like almost all (all?) electronic voting systems in use today.

There are well-known risks at every stage of the episode, so I repeat: that whole process was neither secure nor well-run; moreover, its collection of personal information using unsecured Web pages exposed participants to the risk of information theft, and delivering notionally secure information by email is painfully bad judgment. The episode proves nothing except that well-intentioned people continue to make elementary but serious errors in designing and setting up processes that must be safe at every step if they are to be meaningful.

Don’t like getting to sleep at night? Read the RISKS digest avidly.

Some handy tools: cstream, lsof, iftop, …

Unix tools, and how I use them at Andrew Birkett’s blog, the latest addition to my feedspace.

There’s some really nice stuff here I didn’t know about. cstream, iftop, and less -S are all new to me. watch is, of course, indispensible.

Slide40 – presentations with 8 bit style

Slide40 example slide

From the same good people who brought you the lambda calculcus in a can [ultimate].

Easily fixing stuck pixels

Fix stuck/dead pixels on your monitor with Killdeadpixel

Awesome — supposedly can often fix stuck pixels with minimal effort on the part of the human. Almost makes me wish I had one to try it out on… ;-)

Traffic flow simulation applet

Traffic flow simulation applet — traffic as in “cars and trucks” not “data” [reddit]. German, so the top speed limit is 140kph. :-)

Parsing Expression Grammars in Ruby with Treetop

A couple of years ago I read and greatly enjoyed Bryan Ford’s 2004 paper “Parsing Expression Grammars: A Recognition-Based Syntactic Foundation” [PDF, 134Kb] (wow, it’s still on my desk, in fact); while to my disappointment it proved inapplicable to my research (I’m constrained in the parsing and language description technologies I can use by what I have to integrate with), it was clearly Cool Stuff.

Yeah, so, apparently some Ruby hacker read that paper too [reddit].

See your filesystem space-eaters with filelight

KDE-heads may already know about this, but since I live in stripped-down land, it took LifeHacker to tell me about filelight. Lush tool.

Visualizing regular expressions with reAnimator

reAnimator — a very cool tool for visualizing regular expressions. Given an RE, renders the corresponding NFA and DFA and animates acceptance (or not) of an input string. Try out the “a|ab|abc|abcd” example with input “a” for a neato example. A flash app, written using python.

lshw – neato

lshw — very nice.

(Example HTML output: my laptop today.)

SAGE: open source mathematics software with python scripting

It’s possible (with fingers etc. crossed) that by the end of the year my job will have altered somewhat and I’ll be spending some of my time, amongst other things, doing the kinds of things one generally uses Mathematica and Matlab for. This would be a departure, but no doubt an interesting one.

And what should come steaming over the horizon into my view at this moment? Ah look, it’s SAGE, “Open Source Mathematics Software” in exactly that flavour, with scripting in Python. Maybe it’s half-baked rubbish, maybe it’s slick and up to the job; maybe I’ll never know. Maybe, just maybe, I’ll be checking it out by the end of the year. For now: noted.

A Subversion Pre-Commit Hook in Python

A Subversion Pre-Commit Hook (in Python) [smallcool]. That’s bound to come in handy some time… :-)

The urge to play

I recently discovered Kongregate, thanks to the twin magics of Psychopixi and Facebook. It’s a handy-dandy ubersite of online (mainly (exclusively?)) flash games, just the ticket for tickling that urge to game that comes over me now and then, especially when reading Penny Arcade from start to finish as I did for the second time in December. ;-)

In particular, I can’t heap enough praise on 3D Logic and 3D Logic 2 for infuriating puzzliness. In fact, I’m still stuck on level 30 of the latter.

The sound of the quietness

Sigh… Surely not another “gosh, haven’t I been quiet lately” post? Well, yes. These always give me the same feeling as phoning my parents when I should have phoned them a week ago, replying to messages that have got buried at the top of my inbox (ha! buried at the top!), and seeing Rich at Christmas for the first time in Jesus was it over a year? Actually no, that didn’t feel guilty and crap, just good to see him again. Anyway…

It’s partially because of Facebook, and a recent tendency to post stuff there instead of here. Of course, the sensible thing to do is post it here and then post a link to that on Facebook. So that’s what I’m going to do from now on.

Of late, I’d say it’s increasingly been because I’ve started fairly enthusiastically using lovely ickle super microbloggy Twitter, with its lovely 140 character limit that gets me burning on my SMS-honed character-saving neurons, oh such joy. Actually it’s really great, especially the text message integration… I realise I could probably post to this (WordPress) blog by SMS if I was willing to jump through enough hoops, sacrifice a goat, set my penis on fire, etc. but frankly life’s too short and fire hurts. With Twitter someone else has taken care of making it work, and all of a sardine I can “blog” from anywhere. (Yeah, I know you can with Facebook too but I dunno, I’m still somewhat and actually increasingly ambivalent about the closedness of that particular community.) Getting tweety updates of what everyone else is up to is super fun too. It’s quite random, in fact: at some point, and for no reason I have been able to discern other than that I am cool and handsome, a Danish girl called Nina started following my tweets. Naturally I reciprocated, leading to the somewhat surreal experience, around New Year’s, of every now and then over the course of her 4-day bender, getting little snippets of the Edingburgh hogmanay experience (I infer that her bloke, “the imported alien” hails from thereabouts, also by his reported sporting of a kilt in Copenhagen on Christmas Day). All very odd, but also strangely encouraging for the future of the humanity.

So, yeah; Twitter: it’s jolly good. What I need to do now is find a satisfactory way of integrating it here (work in progress). Probably I’ll just arrange for every tweet to appear as a blog post, though that might annoy anyone who follows me both on Twitter and via the Gimboland RSS. The vain fools.

My other big occupation of late has been my thesis. I have about another month or so to complete it (ie completely). The word count limit for an MPhil at Swansea is 60,000 – this doesn’t look like it’s going to be a problem as mine is currently around 23,000, but it’s hard to tell with LaTeX, particular given the large amount of mathematical notation in chapter 11. Probably it’s even less… Picking another dimension to measure, it’s 93 pages long; I expect it’ll be about 120 to 130 by the time I finish, so you can see there’s still a chunk to do. Still, I’ve been working on this project for two and a half years now, and researching in the area for four, so I must say it’s a real pleasure to finally see some chapters which feel complete and well written, and to have the feeling that I might actually know and have done something. It’s also of course warm and fuzzies that the end is in sight, certainly with a telescope and possibly with binoculars. Every now and then I get a hint of the feeling of a future in which I have a Masters degree (hopefully not being too presumptuous), all smug and validated. I could do with a bit of that…

Finally, I’d just like to mention that I started using Google Reader for my RSS needs today, having got annoyed with trying to run and synchronise two slightly different versions of Liferea on my laptop and work box. Screw it, life is (again) too short. Con: it doesn’t seem to do HTTP authentication, which makes Basecamp feeds out of scope. Pro: what happens when you hit “?” and how pretty that looks. Main pro: declaring “newsfreed bankruptcy” and starting again. Ah, the fresh winds of change!

Chore wars

Chore Wars.

Desired outcome.

It’s All Text!

Loving it: It’s All Text, a Firefox add-on to enable you to edit textarea contents in the editor of your choice. It works!

Ruby vs Java/PHP ads

Rails vs Java in style of PC vs Apple, PS3 vs wii, etc. [digg]

Full set on youtube: 1. Rails vs Java; 2. Rails vs PHP; 3. Rails vs PHP; 4. Rails vs PHP.

Double header

Quiet here lately because it’s exam processing time in the real world, so I’ve been busy busy busy marking, check-marking, and munging. Next week’s busy too but then I can settle into research gear for the summer and do lots of Haskell — woo.

I’m still loving the hell out of xmonad, which just keeps getting better every day. Yesterday I went dual head through the magic of Xinerama; at first it made my brain hurt, trying to keep track of where each workspace was, but now I’m getting to grips with it, I’d say. It’s nice to be able to work on one thing (a blog post, say) while keeping an eye on a background job, and while programming it’s cool to have multiple emacsen filling my field of vision. What could be better?

There is surely much more to say about many things, and I will surely say them soon.

Xmonad does status bars right by not doing them at all

The first thing I complained about the first time I saw Xmonad was the lack of a status bar. As Xmonad approaches its 0.2 release, it has become clear to me that this absence is in fact a feature, not a bug. Here’s why…

It’s now possible in Xmonad to define gaps on the edges of the screen, over which it will not arrange windows. This immediately and beautifully solves the status bar problem, because now you just run a separate status bar app, filling in whatever gap you’ve defined. If the app sets the “override-redirect” X flag (which anything claiming to be a status bar should), it’ll appear on every desktop and be excluded from Xmonad’s window management.

This is a stroke of genius, and a fine illustration of that principle at the beating heart of Unix, namely that you should write apps which do one thing well, and use them in synergetic combination. xmonad manages windows, but it is not in fact its job to provide a status bar: only the space for one. Factoring the status bar out in this manner gives us choice and orthogonality, both of which are A Good Thing.

For now, I’m going for the easy option: dzen, a minimal text-only status bar (looking a bit like this but at the bottom of the screen). Alternatives exist, for example the KDE kicker, but I haven’t really explored that space yet. In all honesty, I’ll probably just stick with dzen. What I want next: a way to display, on the status bar, info about which workspaces are in use, and which is visible. For this, Xmonad will need to report its current status somehow, perhaps like wmii does with its plan-9 inspired pseudo-filesystem.

Update 2007-06-15: very quickly, I got my wish; xmonad now has configurable logging hooks which can write arbitrary data to stdout every time the internal state changes — pipe that to dzen and hey pesto, problem solved. By default there’s a basic logger which simply writes a list of workspaces which currently have windows on, with the current workspace surrounded by square brackets. Lovely jubbly.

Haskell packages gotcha: global vs. per-user package databases

This morning, I was having a problem involving Haskell’s Cabal build system. Since it’s something which could conceivably affect others, and since googling on the error text didn’t help me, I’m going to report my experience for the benefit of future travellers/googlers.

I was trying to (re)build xmonad, and got the following error:

[gimbo@orb xmonad] runghc -v Setup configure 
   Could not find module `Distribution.Simple':
       Use -v to see a list of the files searched for.
Read the rest of this entry »

xmonad – a minimal window manager in Haskell

Wow, synchronicity. Yesterday I was browsing dons‘ space (because I spotted something like my string split in his h4sh stuff), and was very intruiged to spot xmonad. It’s another lightweight tiling window manager in the style of ion, wmii, dwm, and appears to be most like dwm in fact. It’s written in Haskell. Today, they announced the 0.1 release, so I just had to take a look.

It wasn’t too hard to install… A few dependencies, but all handled by Cabal, tvm. Since I already use wmii I didn’t find the keybindings too alien, and of course I can configure them at leisure should I wish to (and unlike dwm, it apparently doesn’t need a recompile when the config changes). Its (dwm-like) style of window management doesn’t sit completely easily with me, I have to say. I guess I’ve got used to the wmii way. I’m going to give it a go for a while, though, as I have an inkling I could grow to love it. It’s very snappy.

I miss three things.

First, a statusbar visible on every workspace. I just want to be able to see some things at a glance, always. Examples: time/date; battery life and charging/discharging status; xmms time remaining and shuffle status. Looking at the mailing list archives, it smells like there is or will be room for user-supplied extensions, and I guess this is an obvious candidate. If I don’t write it I’m sure someone else will. :-) Related to this is the ability to name workspaces, so they’re not just 1-9 – not critical but very handy. I don’t care about the whole tagging thing wmii does – I just want named workspaces as an aide memoir in the statusbar. “Ah yes, workspace 6 is hets.” That kind of thing.

Second, and routable-around: floating/unmanaged workspaces. Tiling/management is great 95% of the time, but sometimes it’s just a pain. The Gimp is the main culprit here, with its myriad windows of joy, but there are other cases. Xnest is probably the answer here. Here’s a little script to run an embedded icewm session:

#!/bin/sh

Xnest :1 -name "Xnest" +kb -ac -geometry 1280x800 &
DISPLAY=:1
xsetroot -display :1 -solid black &
exec icewm

Third, I miss my window bars (the bits along the top where most wms put close buttons, etc.). Primarily I miss them in my terminals, where they report the full path of the current working directory. Another handy at-a-glance feature. I guess I can live without this though, for the sake of precious screen space. :-)

None of these are killers/blockers, though, so I’m definitely going to give xmonad a serious go.

Installing Haskore (Haskell/CSound) under FreeBSD

I’ve spent most of today looking at Haskore, which appears to be the most well-developed kid on the block for doing Csound type things using Haskell (and indeed, doing music type things using Haskell). Here are my notes on finding/installing it, which experience wasn’t completely straightforward…

Read the rest of this entry »

netrenderer

Handy & cunning: netrenderer: how does such-and-such-a-URL look in IE7? Handy for those of us who avoid Windows like the plague as much as possible…

(And yes, I know Gimboland’s still a bit broken. Ah well.)

woof

woof – very handy for occasional one-shot (or multi-shot) file serving. Put it somewhere in your path and every now and then it will make you happy.

Woof (Web Offer One File) tries a different approach. It assumes that everybody has a web-browser or a commandline web-client installed. Woof is a small simple stupid webserver that can easily be invoked on a single file. Your partner can access the file with tools he trusts (e.g. wget). No need to enter passwords on keyboards where you don’t know about keyboard sniffers, no need to start a huge lot of infrastructure, just do a

$ woof filename

and tell the recipient the URL woof spits out. When he got that file, woof will quit and everything is done.

autopushd in zsh – make cd behave like pushd always

I’ve just been exposed to one of those delightful little hacks that are both forehead-slappingly simple/obvious and yet so powerful and clearly “right” that I just know I’m going to be using it for the next n years. Allow me to share.

In every Unix shell I’ve ever seen, one uses the “cd” command to change directory, moving around the directory structure.

cd has no memory of where you were before. Some shells add a stack-based history via the commands “pushd” and “popd” (which I only discovered in 2004). If you pushd to a directory (eg pushd /some/distant/directory), you can subsequently get back to where you were before very easily by just entering popd; furthermore, such calls can be nested arbitrarily deeply (that’s what the stack is for). This is, on occasion, tremendously helpful, and can save a lot of typing/thinking.

The insight: why bother with cd, then? Why not just always pushd around the place? That way, you’ve always got the possibility of using popd to get back to where you were before easily and quickly. There are two possible reasons why not. First the stack takes up some space in memory. Feh, no problem – it’s not that big, and I never have a shell open long enough for it to become a problem. Second, pushd is a few more keystrokes than cd, and my habits have cd hardwired into them.

Thus, the hack: alias cd to pushd. But wait! Don’t do that!

Since I use zsh, I don’t even need to alias it. I can use the zsh config option “autopushd” to make cd always behave like pushd. Hell, yeah! I just add the following to ~/.zshrc:

setopt autopushd

and hey pesto:

# Start by going to some fairly deeply nested directory
[gimbo@mane ~] cd work/teach/cs-238/current 

# Now I cd (but really pushd) to some other, also deeply nested, directory
[gimbo@mane current] cd ~/work/research/mphil/tools/HetCATS/CspCASL
[gimbo@mane current] pwd
/home/gimbo/work/research/mphil/tools/HetCATS/CspCASL

# In the words of Baloo: take me home, daddy!  (This next line doesn't work without the autopushd trickery)
[gimbo@mane current] popd
~/work/teach/cs-238/current ~
[gimbo@mane current] pwd
/home/gimbo/work/teach/cs-238/current

Update 2007-02-24: Will has just pointed out to me that “cd -” will take you back to where you were before, so repeatedly issuing cd - allows you to flip between two places easily – which is of course a common use case. Neato!

[gimbo@orb ~] cd /tmp 
[gimbo@orb /tmp] cd -
~
[gimbo@orb ~] cd -
/tmp
[gimbo@orb /tmp]

hpDJ: an automated DJ with floorshow feedback

Here’s a little gem for anyone else out there interested in dance music, DJ’ing, and computers: hpDJ: An automated DJ with floorshow feedback. Basically, it describes a system which allows one to define a “qualitative tempo trajectory” for a DJ set, then chooses tracks to fit the trajectory (the words “partial ordering” occur at this point, but that’s as close as we get to TCS), and automatically beatmatches/timestretches/mixes the tracks accordingly. Towards the end it considers the performance and “collaborative consumption as composition” possibilities, with genetic algorithms and the like, but the bit which impressed me the most was on the mixing, where the opportunities afforded by doing the job in software are recognised and made the most of.

However, because hpDJ operates in the pure software realm of digital signal processing (DSP), it is possible to create as many sweepable band-pass/cut filters as is desired for any particular cross-fade from one track to another. As with traditional hardware mixers, each DSP filter can have variables that control the degree of attenuation or boost, and its center-frequency. In addition to this, the shape of the DSP filter’s transfer function (e.g. the nature and rate of the fall-off or boost) and its bandwidth can also be under automatic control. Recording studios do have filters with these added controls, but such filters (known as a Parametric EQ) are too expensive to be built into each channel of professional mixing desks on a many-per-channel basis.

Thus, it becomes possible to specify hpDJ so that it analyses the audio frequency-time spectrogram for the incoming and outgoing tracks in the cross-fade, and uses a number of heuristics to determine how many DSP Parametric EQ filters are necessary and what their settings should be. This can be used to, for instance, selectively suppress the frequencies for a synthesizer melody-line in one track, attempting to make that melody “disappear” while keeping the bass-guitar and percussion elements in place during the cross-fade. By employing simple heuristics for detecting when one component of one track “clashes” with another component of the other track, such aesthetically unpleasant clashes (which may remain despite perfect beat-matching) could be automatically eliminated by hpDJ.

Note that this is all happening automatically (except maybe setting some initial loose parameters at the start) – very impressive.

Although this was published only in 2005 (I spotted it in this book on this man‘s desk), much of this was apparently done around 2000 or so. Thus, I wonder what goodies they’re cooking up in Bristol even as I type. Dave Cliff’s “inventor profile” page doesn’t really bring us up to speed, I feel… Aha! He’s moved to Southhampton.

Finally, I note that this is the first paper I’ve ever seen which opens with a Sisters of Mercy quote. Rock on.

Autoattach / automount encrypted home partition in FreeBSD using GBDE

I’ve just set up one of my laptops, running FreeBSD, so that /home is encrypted using GBDE, and is auto-attached/mounted/fsck’d at boot time. The instructions in the FreeBSD handbook aren’t completely clear, so here are some notes on how I did it.

Actually, most of those instructions are very clear, provided you don’t care about attaching at boot time. Indeed, I’ve had an encrypted partition which I attach/detach by hand, for about a year now, for which those instructions were perfectly adequate. Unfortunately, the section on “Automatically Mounting Encrypted Partitions” leaves out two important details which conflict with the rest of the chapter: your lock file’s name needs to end in the text “.lock”, and you need to be careful where you put it. This was confusing, and in some ways what you have to do to get automounting working contradicts the other examples in the chapter.

Anyway, long story short, I had to dig into /etc/rc.d/gbde to work out what had to be done, and here it is… A summary of how to set up an auto-attaching encrypting home partition on FreeBSD using GBDE:

(I did this as pretty much the first thing after a fresh install, before I’d even created any users. Needless to say, then, all of this happens as root.)

Here’s how /etc/fstab looked before I started:

# cat /etc/fstab
# Device        Mountpoint         FStype    Options   Dump Pass#
/dev/ad0s1b     none               swap      sw        0    0
/dev/ad0s1a     /                  ufs       rw        1    1
/dev/ad0s1e     /tmp               ufs       rw        2    2
/dev/ad0s1f     /usr               ufs       rw        2    2
/dev/ad0s1g     /usr/home          ufs       rw        2    2
/dev/ad0s1d     /var               ufs       rw        2    2
/dev/acd0       /cdrom             cd9660    ro,noauto 0    0

Unmount /home because I’m about to blat it completely:

# umount /usr/home

Create a directory to contain the lock files:

# mkdir /etc/gbde

Initialise the partition for GBDE. I used a use sector size of 2048 (which matches the UFS fragment size). Note that the lock file’s name ends in .lock; this is not how the main body of the GBDE instructions in the handbook does it, but it’s necessary to get /etc/rc.d/gbde to attach it properly on boot up:

# gbde init /dev/ad0s1g -i -L /etc/gbde/ad0s1g.lock
Enter new passphrase:
Reenter new passphrase: 

Attach the encrypted partition to the kernel for the first time (entering the passphrase previously specified), and write the filesystem:

# gbde attach /dev/ad0s1g -l /etc/gbde/ad0s1g.lock
Enter passphrase: 
# newfs -U /dev/ad0s1g.bde 
/dev/ad0s1g.bde: 6632.5MB (13583360 sectors) block size 16384,
                 fragment size 2048 using 37 cylinder groups of
                 183.77MB, 11761 blks, 23552 inodes.  with soft
                 updates
super-block backups (for fsck -b #) at:
 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624,

*SNIP*

Test the mount:

# mount /dev/ad0s1g.bde /usr/home

Assuming that looks good (eg in df), unmount it and detach it:

# umount /usr/home
# gbde detach /dev/ad0s1g

Now we’re ready to set it up for auto-attaching. We need to alter /etc/rc.conf and /etc/fs.tab

# tail -n 3 /etc/rc.conf
gbde_autoattach_all="YES"
gbde_lockdir="/etc/gbde"
gbde_devices="ad0s1g"

We need the gbde_lockdir line because otherwise it looks for the lock files just in /etc, I think.

# grep home /etc/fstab
/dev/ad0s1g.bde /usr/home          ufs       rw        2    2

Now when I boot, it asks for the passphrase, attaches the encrypted partition, and mounts it – automatically – then it gets fsck’d with everything else. It doesn’t automatically detach upon halting the system, but I guess that’s no problem. :-)

The only thing I don’t totally like is that if I get the passphrase wrong (3 times), it doesn’t attach or mount the encrypted partition (obviously), but then of course it fails filesystem checks completely and the boot process dumps down to single user mode, messily. Not completely unreasonable, I guess, but still a bit annoying.

Anyway, anyone now stealing my (crappy) laptop will have a much harder time of getting at the data on it (the interesting data, anyway). Whee!

OK, I’ll stop being such a fucking snob

Yesterday, I saw how this blog looks in IE, and I was dismayed. So OK, I’ll try to do something about it. Sigh…

But seriously, IE users, what are you doing? Haven’t you seen Firefox? It’s so much nicer it’s just not true – and the plugins, oh, the plugins! Just having adblock should be reason enough for migrating, not to mention DownThemAll, gmail manager, PDF download, and VideoDownloader. Oh yeah, and IE Tab for those pesky pages that just have to be viewed in IE. So – no excuses, nothing to lose and everything to gain… Come! Join us!

Hacking lightstats for ultimate tagness

I’ve hacked the lightstats plugin to forget about categories, and instead display “posts per tag” and “bytes per tag”, querying the tables created/maintained by ultimate tag warrior. Examples here. Code available by request.

Arguably the “posts per tag” is a bit redundant given the w00ty uber tag cloud – particularly once I have that displaying post counts in the tooltips. I’m quite proud of the “bytes per tag”, though… One big SQL statement gives me all I need:

SELECT tag, sum(length(post_content)) AS bytes
  FROM wp_post2tag
  INNER JOIN wp_tags ON wp_post2tag.tag_ID=wp_tags.tag_ID
  INNER JOIN wp_posts ON wp_post2tag.post_ID=wp_posts.ID
GROUP BY (wp_tags.tag_ID)

Well, OK, not so big. Quite big as a one-liner though. :-)

I’ve also installed wp-cache, which may or may not be pointless… (And, the following morning, disabled it, because it would prevent comments appearing for up to an hour after they’re posted!)

Operational Semantics of a Simple Process Algebra in Python and Haskell

As promised, though I’m still working on the shiny LaTeX article which actually explains all this stuff…

From the README:

This is an investigative/learning exercise in transforming process algebraic expressions into labelled transition systems according to their operational semantics, using Python and Haskell.

It started out just as an exercise in operational semantics, using Python because it’s the language I get things done fastest in. That proved interesting enough that I wanted to re-do it in Haskell, which I’m learning and which arguably handles this sort of thing better.

The Python version is more advanced. It includes mu-recursion, which the Haskell doesn’t, and is quite tidy IMHO. OTOH the Haskell is less developed, and in particular the functions for actually creating the graphs could, I’m sure, be improved. Always nice to have some future work to do…

I’m publishing this in case anyone is interested in the code. In particular, it is perhaps useful as an example of:

I’m working on a paper describing the problem, the semantics, and the approaches taken in the two languages, but it’s very much an ongoing WIP and isn’t ready to publish yet.

Homepage for this work: http://www.cs.swan.ac.uk/~csandy/research/op_sem_py_hs/

Operating Systems in Python

More project ideas for next year: further to this post about writing an operating system kernel in Haskell, it looks like some crazy dudes are doing similar things using Python. Ladies and gentlemen, we present: Cleese, and the delightfully-named Unununium [python-list].

Apparently Cleese started with the idea to “make the Python intepreter a micro-kernel and boot directly to the Python prompt.”, but it doesn’t look like there’s been any movement there over a year, and hasn’t released any files.

Unununium looks interesting, and very much under development. The introduction states that components and a unified filesystem namespace (a la Plan 9) are key goals.

OK, maybe all this is too flakey and out-there for a decent project, but nonetheless, nice to know it’s happening.