Or if you prefer, how about a 200MB zip containing it all — plus some recent even-more-half-baked stuff which hasn’t made it to last.fm yet.

Go! Listen! Scrobble!

Somewhat vexingly, there appears to be music by someone other than me doing the rounds but tagged as “Combinator” in the artist field: some of the top-listened tracks on the Combinator page aren’t by me! Come on, adoring public, together we can right this wrong.

Enjoy!

This is an implementation of the Observer design pattern in Haskell. I am not the originator or even main author of the code in this package; that is Dutch computer scientist Bastiaan Heeren, who wrote the original as part of an exercise for some students. But recently I was doing some GUI programming with wxHaskell (which I’ve found to be great, by the way) and found myself hankering for an observer or something MVC-ish, and my googling led me to his code. I used it, and then thought it might be worth releasing to the world at large — particular as it’s not wx-specific, or even GUI-specific, really. Bastiaan tells me he’s happy for me to do so, so here it is. I’ve changed names, separated it into two modules, and replaced IORef with MVars in order to better support multi-threaded use — but at heart it’s identical to Bastiaan’s code.

The basic idea is that an observable is a piece of mutable state, and when it’s changed, a list of observer functions are called in sequence, and passed the new value. For wx programming – which is anyway quite stateful and not very idiomatically functional – I’ve found it to be quite effective. Here’s a small example which illustrates basic use…

My GUI has a start/stop button, which toggles a running flag. When running is true, the button says “stop”, and certain things happen in the app’s idle handler (to run whatever is supposed to be running); when running is false, the button says “start”, and certain other things happen in the app’s idle handler. So here’s how we set it up:

In my app’s setup phase, I create running, which is initially false, using createSub :: a -> IO (Sub a):

    running <- createSub False

Later on (actually in a different function, which receives running as a parameter), I create the start/stop button and use running’s current value to initialise its appearance:

     r <- getValue running
     btnRunStop <- button p [text := runStopBtnTxt r, on command := runStopClicked running]

Here, runStopBtnTxt :: Bool -> String just returns “start” or “stop” appropriately. We use it elsewhere, which is why it’s factored out. When the button is clicked, runStopClicked toggles the state of running (and does other stuff not shown here):

    runStopClicked :: Sub Bool -> -> IO ()
    runStopClicked running = do r <- getValue running
                                running `setValue` not r

(This would actually be neater using changeValue :: Sub a -> (a -> a) -> IO (), but in the real version I use r later on, so this I need the getValue anyway.)

So this setValue call will cause all of running’s observers to be called with not r. The last thing we should do, then, is attach an observer:

    addObserver running (\x -> btnRunStop `set` [text := runStopBtnTxt x])

addObserver :: Sub a -> (a -> IO ()) -> IO () takes an observable subject and an observer function, and attaches that function to the subject’s list of observers. Here, the observer function just updates the text.

(It’s worth noting that the observer’s type is a -> IO () not Sub a -> IO () — i.e. observers receive values, not subjects. The subtext is that an observer shouldn’t modify the subject it’s observing, lest ye loop infinitely; you can get round this by passing the subject proper as an earlier argument, and provided it used setValue' not setValue you’d be fine — but I don’t know of any reason why you’d want to do this.)

The first win here is that the button’s on command function doesn’t need the button as a parameter, because it’s not directly updating the button text. The deeper win is relatively cheap and clean separation of concerns: the button only toggles the running value, and (separately) stuff happens when running changes. If some other mechanism (eg some network event) could also cause running to change, we’d already be dealing with it The Right Way: just add another observer. Without observers, if our reaction code was in the button’s on command handler, we’d need to call that, or factor the reaction code out, in order to deal with it then. This way’s clearly nicer. But hey, you’re reading a blog post containing type signatures, so I presumably don’t really need to explain separation of concerns, or the observer pattern, to you. :-)

I’m aware that there are more sophisticated approaches to this problem. I’m reliably informed that Functional Reactive Programming (FRP) subsumes observer quite naturally, but I don’t understand it enough to comment. Tom Lokhorst tells us that Erik Meijer & Co. at Microsoft have a good handle on this, as discussed in this entertaining video. Alternatively, if we like message-passing as our basic computational model, an implementation in CHP might be just the ticket — and that certainly looks like an impressively powerful approach. However, in my case, I wanted to keep things in a single thread as much as possible, because Haskell’s GUI frameworks (and indeed wx across all languages?) seem to interact with multi-threading in non-simple manners. This is also pretty much why I decided against implementing Neil Brown’s suggestion that the observers should be run in their own threads. Of course, if an observer wants to fork a thread, it absolutely can, and I may find myself doing this in my app soon — but I didn’t quite see the utility of baking it in. That’s probably my sequential imperative background showing through, and hopefully one day I’ll shake it off and welcome our new, multi-threaded masters. To conclude, everything mentioned in this paragraph is why I decided to release this as simple-observer.

Anyway, hopefully somebody will find this useful.

I see it as, when the limestone of imperative programming is worn away, the granite of functional programming will be observed.

Simon Peyton Jones

(I’m still not quite sure why I wrote this. You should just go and buy the book. But maybe someone will be interested.)

I got my copy of Peter Seibel’s new book Coders At Work yesterday, and the first chapter I turned to was, naturally, the interview with Simon Peyton Jones. I found it to be excellent and informative; of course, as anyone who’s seen him in action can testify, SPJ’s a careful and entertaining communicator of ideas, so that’s no surprise — if the other chapters are as good as this one, well, it’s a winner of a book.

Here’s a summary of the interview. Hopefully it contains enough to tantalise/encourage purchase, but not enough to give away the farm. :-) (And remember, this is just one chapter of the book.)

Seibel starts as he seems to start every chapter (modulo small variations), by asking “When did you learn to program?”. SPJ tells us about his school days, and his time as an undergraduate at Cambridge, through into industry and back to academia at UCL, and then on to a Professorship at Glasgow. Along the way we hear about some of the machines he programmed, and impressive things he did on them — such as “a program to extract 24-digit square roots” on his school’s computer with 100 8-digit memory locations, entered directly in machine code, because that’s all it had. Hearing about heroic stuff like this always makes me feel a bit bad, for having merely human levels of curiousity and drive as a schoolchild, and for a while I was worried the interview would be dominated by this historical stuff, since it went on for a few pages. Then I realised quite how long the chapter is (35 pages), and felt better. :-)

One thing I didn’t know was that SPJ doesn’t have a PhD. (Another thing I didn’t know was that neither does Robin Milner, apparently. Wow.) He discusses how that happened, and why one does a PhD, what its value is, etc., and also some personal philosophy of his approach to research. As someone who enrolled on a PhD just last week, I found this bit quite inspirational:

He [John Washbrook] said, “Just start something, no matter how humble.” This is not really about programming, this is about research. But no matter how humble and unoriginal and unimportant it may seem, start something and wroite a paper about it. So that’s what I did. It turned out to be a very significant piece of advice.

I’ve told that to every research student I’ve ever had since. Because it’s how you get started. Once you start the mill turning then computer science is very fractal — almost everything turns out to be interesting, because the subject grows ahead of you. It’s not like a fixed thing that’s there that you’ve got to discover. It just expands.

This leads into a few pages about functional programming in general, its popularity in the research community, its practicality, and the ways this research is feeding ideas back into the mainstream. There’s a bit of discussion of the vexed question of how to do experiments on programmer productivity/experience: how do you verify, other than anecdotally, that such-and-such a language (or feature) actually benefits the programmer or program? He mentions some interesting sounding work at Microsoft (”Steve Clarke and his colleagues at Redmond”) on this, at the API level. Then it’s back to how SPJ got into FP; he says he was influenced by being taught by Arthur Norman at Cambridge, where seeing a doubly-linked list implemented without side-effects was an ‘aha’ moment, and that David Turner’s work on S-K combinators was also really inspiring and exciting to him — and says a bit about what that means for FP implementation.

Then laziness comes up: it was another interesting thing which motivated him, and has turned out to be a big idea, not least because it kept the focus on purity, which is the Big Win, he now realises. He talks about the old embarassing I/O situation, and the stream-based approach to ’solving’ this, and just mentions monads here before being steered elsewhere by Seibel, never to return… :-)

He discusses his work on compiler writing (and GHC in particular of course), and the pros and cons of type-checking and static checks, including the problems of generic programming in a statically-typed world. This leads to a discussion of how he thinks about and designs software (unsurprisingly, its driven by types), and his “terribly primitive” programming environment (about the same as mine, except I’m one of the people he mentions that “just live their whole lives in ghci”), and his approaches to debugging. (Here he mentions some work Pepe Iborra did on debugging for Haskell — could he mean this? Pepe’s blog mentions that in 2006, but SPJ says he did the work “earlier this year”. So has Pepe done something more recent, or did the interview take place in 2006/7? I suspect the latter?)

On the subject of writing correct software, Seibel asks what he thinks about formal proofs, and his replies are as sensible as you’d expect — skeptical about ‘big up front’ specification/proof for (now) well-rehearsed reasons, but hopeful about “partial specifications” of program properties, e.g. in the style of QuickCheck.

Then we get six pages about STM. :-) Seibel starts by passing on a question from Guy Steele: “Is STM going to save the world?”. The typically sensible answer is of course not: No Silver Bullet, there are various ways to skin the concurrency cat, but that compared to locks & variables STM is a clear win every time. He talks about problems of starvation, and why that’s particularly problematic with optimistic approaches to concurrency, and comes back to the partial specification idea, discussing how STM lets you think more compositionally in terms of pre- and post-conditions than some other approaches. The STM discussion ends with him telling how he came to it via a talk by Tim Harris about STM in Java, and the realisation that STM could be made to fly so much more easily in Haskell than Java — leaving implementor brainspace free to invent mechanisms such as orElse which the Java guys hadn’t spotted (and example of FP research feeding back into the mainstream). “There was less crap, basically, so the cool idea stood out in high relief.”

Seibel then asks, “What’s your desert-island list of books for programmers?” I don’t want to give it away here (buy the book! It’s great!), but I’m sad to say I only own one of the books mentioned.

For the last six pages of the interview, discussion turns to SPJ’s everyday programming practice (he still loves it), programming as craft vs engineering (a false dichotomy, to SPJ), programming in the large, what it means for code to be beautiful, and code longevity. SPJ discusses how working in research allows you to put more time and energy into keeping things beautiful, and the tensions in industry that lead to ‘big systems full of goop’, and broad (rather than beautiful) APIs. It’s all insightful stuff, and his preferences, likes and dislikes echo my own so strongly that I’m glad I’m still working in research too, at least for now.

Overall, as I said, I really enjoyed it. I learnt a lot about the man, and a few things about programming I didn’t know (I haven’t really looked at STM yet, in particular). Cool.

Oh, and for a nice perspective on the relative longevities of limestone and granite alluded to in the quote I opened this with, check out this 1986 documentary taking a 30-minute tour round the entire British coastline: Round Britain Whizz. :-)

(Also, if you haven’t seen it yet, this is good, too.

Sadly, there’s no master index of what’s installed. I’ve got it configured to install globally, so the docs go into /usr/local/share/doc, and each (version of each) package gets a folder of its own there; if you’ve got cabal calling haddock, that folder will contain an html folder with the docs in, but it’s tedious to click through an (otherwise empty) folder to get to it each time, and the whole setup’s not very pretty or informative (and the lexicographic sorting is case-senstivie, which I don’t much like). Eg:

People have attacked this problem before, but PHP makes my skin itch, and I can’t be bothered with apache, so a simpler, static solution seemed right for me.

Thus, I’ve knocked up a (quick, dirty, nasty) python script to generate the index. As a happy side-effect, I’ve pointed it at the hackage CSS file, so it’s pleasingly pretty and familiar:

I did it in python because that’s still my go-to “hack stuff together in a hurry” language, I guess; but I was very nearly did it in Haskell. Mainly it was library knowledge that drove me. Also perhaps I fancied a break. Hmmm. Next time, for sure.

If anyone’s interested in the code (the doc path is hardcoded, so if you’re installing user docs, change it), you can view it pretty/download it via this link, or just check it out here. Oh, and what about the “automatically” part? Well, just stick it in a cron job… ;-)

Update: I realised I wasn’t linking to the actual index.html file, which kinda defeated the point of the script! However, it’s an easy fix. The line that says:

                s.write('<a href="file://%s">' % package.path)

should actually say:

                s.write('<a href="file://%s">' % os.path.join(package.path, 'html', 'index.html'))

Aaaanyway…

#!/usr/bin/env python

# Horribly hacked-together script to generate an HTML index page for
# Haddock-generated Haskell package docs.  Note that the docs
# directory is hard-coded, below: see the comments there.

import os
import re
import StringIO

# This is the path to the docs.  Any subdirectory which contains an
# 'html/index.html' will get an entry in the index (all the other are
# listed separately at the end).

# The index is written to 'index.html' at this path.

PATH = "/usr/local/share/doc"

class Package:

    def __init__(self, name, path):
        self.name = name
        self.path = path
        self.title = self._seekTitle()

    def __cmp__(self, other):
        return cmp(self.name.lower(), other.name.lower())

    def index(self):
        return os.path.join(self.path, 'html', 'index.html')

    titleRE = r'>.*: (.*)</TITLE'

    def _seekTitle(self):
        f = open(self.index())
        d = f.read()
        f.close()
        m = re.search(self.titleRE, d)
        if not m:
            raise ValueError, "Can't extract title from %s" % self.index()
        return m.group(1)

class Builder:

    def __init__(self, path):
        self.path = path
        libs = [l for l in os.listdir(self.path)
                if os.path.isdir(os.path.join(path, l))]
        self.has_html = []
        self.no_html = []
        for lib in libs:
            full = os.path.join(path, lib)
            if self.seek_html(full):
                self.has_html.append(Package(lib, full))
            else:
                self.no_html.append((lib, full))
        self.has_html.sort()
        p = os.path.join(path, 'index.html')
        o = open(p, 'w')
        o.write(self.html())
        o.close()

    def html(self):
        s = StringIO.StringIO()
        s.write('<html>\n')
        s.write('<head>\n')
        s.write('<title>Local Haskell package docs (from Haddock)</title>\n')
        s.write(('<link rel="stylesheet" type="text/css" '
                 'href="http://hackage.haskell.org/packages/hackage.css"/>\n'))
        s.write('</head>\n')
        s.write('<body>\n')
        s.write('<h2>Local packages with docs</h2>\n')

        split = self.splitAlpha()
        keys = split.keys()
        keys.sort()

        s.write('<p class="toc">\n')
        links = ['<a href="#%s">%s</a>' % (start, start) for start in keys]
        s.write(' &bull; '.join(links))
        s.write('</p>\n')
        
        for start in keys:
            packages = split[start]
            s.write('<h3 class="category"><a name="%s">%s</a></h3>\n' % (start, start))
            s.write('  <ul class="packages">\n')
            for package in packages:
                s.write('    <li>')
                s.write('<a href="file://%s">' % package.path)
                s.write(package.name)
                s.write('</a>: %s</li>\n' % package.title)
            s.write('  </ul>\n')
            
        s.write('<hr />')
        s.write('<h2>Directories without HTML docs</h2>\n')
        s.write('<ul>\n')
        for (name, path) in self.no_html:
            s.write('  <li><a href="file://%s">%s</li>\n' % (path, name))
        s.write('</ul>\n')
        s.write('</body>\n')
        s.write('</html>\n')
        h = s.getvalue()
        return h

    def splitAlpha(self):
        al = {}
        for package in self.has_html:
            start = package.name[0].upper()
            try:
                al[start].append(package)
            except KeyError:
                al[start] = [package]
        return al
            
    def seek_html(self, path):
        items = os.listdir(path)
        if 'html' not in items:
            return False
        html = os.path.join(path, 'html')
        if not os.path.isdir(html):
            return False
        return 'index.html' in os.listdir(html)
        
if __name__ == '__main__':
    builder = Builder(PATH)

Context: I occasionally make what might kindly be called music, using computers and the like, and I don’t have any formal musical training. Consequently, I sometimes run into trouble regarding keys and scales. The idea I had was for something which would tell me, given a set of notes (ie the ones I’ve already used in a piece), what scales those notes are in — and consequently what other notes might also fit well with them. Long ago when I used Bars & Pipes on the Amiga, it had a tool for doing this. I could buy one for my Mac for about 8UKP (I may yet — it has a pretty GUI, which I might not be bothered to do), but it seemed I should be able to knock something up. The data types aren’t exactly complicated.

So I did it. I’ll put the code at the end of this post, or you can easily download it from here, in 133 lines of literate Haskell (58 lines of actual code). Here’s a screenshot showing typical use:

Once again, I’ve been super impressed by how easy Haskell made this — and I’m particularly loving the Shellac library which gives me an interactive shell with history and graceful recovery from errors — all in 4 lines of code!

> react = mapM_ (uncurry printScale) . checkFit . parseNotes
>     where printScale :: String -> [Note] -> Sh () ()
>           printScale n ns = shellPutStrLn (n ++ ": " ++ ppNotes ns)
> main = runShell (mkShellDescription [] react) haskelineBackend ()

OK, I’m not using any of its juicier features like completion, state, automatic help, etc., but even so, it’s a great little library.

I could wrap it in a GUI, but to be honest, this’ll probably do the trick for me, assuming I haven’t made any big screw-ups, or got the scale definitions wrong. If anyone’s looking for a minimal Shellac example, maybe it’ll be useful.

(Update 2009-09-16 13:37: To get this up and running, if you’re not already a Haskeller, the following should suffice: 1. Download and install The Haskell Platform – nice ‘n’ easy. 2. Open a shell and issue cabal install Shellac-haskeline or possibly sudo cabal install --global Shellac-haskeline. 3. Download the code (from hpaste.org) and (still in a shell) issue ghci Scales, and when that’s loaded issue main. Or if you want to compile an executable, try ghc --make -main-is Scale.lhs Scale.lhs, just like in the screenshot. That should do it — I’d love to know if I’ve missed anything and/or if that works!)

Here’s the code (but, again, it’s prettier and easier to download from hpaste):

> module Scale where

> import Data.List

I'm going to use Shellac for interactivity...

> import System.Console.Shell
> import System.Console.Shell.Backend.Haskeline
> import System.Console.Shell.ShellMonad

... HughesPJ for pretty printing (in a small way, but it's just so pretty)...

> import Text.PrettyPrint.HughesPJ

... and Text.Regex for splitting a string on spaces.

> import Text.Regex

So the idea is to compute the possible scales a given set of notes
might be part of.

Scales can be represented simply as lists of intervals, naturally
denoting the tonic as the value 0.

> type Interval = Int
> data Scale = Scale {
>       scaleName :: String,
>       scaleIntervals :: [Interval]
>       } deriving Show

We know a few different scales, namely the ones whose Wikipedia
articles made a reasonable amount of sense to us... :-) This may be
stupid and/or wrong, and there are certainly nicer ways to represent
these scales, but it'll do for now.

> knownScales :: [Scale]
> knownScales = [Scale "Major" [0, 2, 4, 5, 7, 9, 11]
>               ,Scale "Minor"            [0, 2, 3, 5, 7, 8, 10]
>               ,Scale "Harmonic Minor"   [0, 2, 3, 5, 7, 8, 11]
>               ,Scale "Major Pentatonic"          [0, 2, 4, 7, 9]
>               ,Scale "Relative Minor Pentatonic" [0, 3, 5, 7, 10]
>               ,Scale "Yo" [0, 2, 5, 7, 9]
>               ]

Notes (A, B, C, etc.) can also be represented as integers.

> type Note = Int

Then we can use modular arithmetic (Z12) without any hassles.

> noteMod :: Note -> Note
> noteMod n = n `mod` 12

Given a base note and a scale, we can compute the notes in that scale.

> scaleNotes :: Note -> Scale -> [Note]
> scaleNotes base (Scale _ scale) =
>     nub . sort . map noteMod $ zipWith (+) (repeat base) scale

We will also want to refer to notes by their names.

> type NoteName = String

And so we need to know the order those names come in.  We'll
arbitrarily set C to 0.

> chromatic :: [NoteName]
> chromatic = ["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A",
>              "A#", "B"]

We need to be able to convert between note names and numbers easily.
We just die if anything goes wrong (i.e. unknown note name).

> namedNote :: NoteName -> Note
> namedNote name = case elemIndex name chromatic of
>                  Just a -> a
>                  Nothing -> error $ "unknown note " ++ name

> noteName :: Note -> NoteName
> noteName note = chromatic !! noteMod note

Given a set of base notes and a set of scales, we can compute all
possible combinations, for each one returning its name (e.g. "C
Major") and the notes found in it.

> mkScales :: [Note] -> [Scale] -> [(String, [Note])]
> mkScales notes scales =
>     do note <- notes
>        scale <- scales
>        return (noteName note ++ " " ++ scaleName scale,
>                scaleNotes note scale)

Then we can compute all the scales we know about.

> allScales :: [(String, [Note])]
> allScales = mkScales [0..11] knownScales

For I/O we want to be able to parse strings of space-separated note
names into lists of note numbers.

> parseNotes :: String -> [Note]
> parseNotes = map namedNote . splitRegex (mkRegex " +")

... and we want to be able to pretty print the same.

> ppNotes :: [Note] -> String
> ppNotes = show . hsep . map (text . noteName) . nub . sort . map noteMod

Then, to compute the scales a given set of notes belong to, we can
just stupidly brute force iterate over every note in the octave, and
every scale we know, and just decide if the set of notes we've got
fits or not.

> checkFit :: [Note] -> [(String, [Note])]
> checkFit notes = filter ((notes `isSubList`) . snd) allScales
>     where isSubList :: (Eq a) => [a] -> [a] -> Bool
>           isSubList x y = null $ x \\ y

OK, let's do some interactivity, courtesy of Shellac.

We interpret whatever the user enteres as a space-separated list of
note names, and in response we print the names (and notes) of every
scale those notes are in.

> react :: String -> Sh () ()
> react = mapM_ (uncurry printScale) . checkFit . parseNotes
>     where printScale :: String -> [Note] -> Sh () ()
>           printScale n ns = shellPutStrLn (n ++ ": " ++ ppNotes ns)

Our main function just runs that function in an interactive shell.

> main :: IO ()
> main = runShell (mkShellDescription [] react) haskelineBackend ()

Here’s my solution.

It was a lot of fun. I started out feeling like I should be able to do it, but not quite knowing what direction to take, so first of all I was writing some data types for things like Door = Car | Goat, etc. but pretty quickly realised it was just a matter of door numbers. My first working solution was actually a bit longer, because of some intermediate steps (for example the first thing I figured out, in its own function, was how to compute which goat door would get shown to the player given a (car, choice) pair). Then hlint gave me a couple of hints, like using replicateM, and I compacted it down to what we see here. I’m sure it has naive aspects, but I’m pretty happy with it — and I’m loving doing the Praxis problems.

(Shame the sound quality on the video’s so poor though.)

Nice to see that Wittgenstein (or at least, one of the Wittgensteins) is also a Haskell man…