Spamboland

Curses – the good name of the gimbo.org.uk domain is being sullied by spammers.

Today I found about 200 “Mail delivery failure” reports in my gimbo.org.uk “catchall” mailbox. It looks like the spammers are abusing me indirectly, in a way which can’t be stopped. (Though I can fairly easily ignore them, tbh.)

Since I own the gimbo.org.uk domain, I get to look after any email sent to that domain. There are a few boxes set up, but if you email, say, randomgibberish@gimbo.org.uk (for which there isn’t a mailbox set up), it’ll fall through to a “catchall” account. This is incredibly useful for, eg, signing up to websites. I don’t need to re-use a “real” email address, I can just use something_appropriate@gimbo.org.uk. If that site ever sells on that address and it starts getting spam, I can filter it easily. Nice, no problemo, etc.

Unfortunately, this means that if anyone sends you an email with a forged reply address of (say) ArchieGorney@gimbo.org.uk, and you bounce it (because it’s obviously spam, or the recipient address doesn’t exist), your bounce notification ends up in my inbox.

It appears that some spammer has started using gimbo.org.uk addresses for this purpose. I wonder if it’s someone I know having a go at me. Possible, but unlikely (I guess). Occam’s Razor says it’s probably just my random bad luck today. I wonder how long it’ll last… I don’t think there’s anything I can do about it, unfortunately… Ah well!

Later… TR suggested I was being Joe Jobbed, which I agree is possible. The profile of the spam is “normal contemporary spam”, however (ie nonsense text and attached gifs actually selling something), which perhaps doesn’t fit the Joe Job profile. *shrug*.

eBay architecture notes

Quite interesting rough notes on eBay’s architecture – (horizontal) scalability is the watchword, with J2EE on Oracle being the central technologies. Interesting (but not entirely surprising) to see that most of the work is done application-level [raganwalk].

Muhammad Yunus – Nobel Lecture: Poverty is a Threat to Peace

Awe inspiring.

Ninety four percent of the world income goes to 40 percent of the population while sixty percent of people live on only 6 per cent of world income. Half of the world population lives on two dollars a day. Over one billion people live on less than a dollar a day. This is no formula for peace.

Poverty is the absence of all human rights. The frustrations, hostility and anger generated by abject poverty cannot sustain peace in any society. For building stable peace we must find ways to provide opportunities for people to live decent lives.

Today, Grameen Bank gives loans to nearly 7.0 million poor people, 97 per cent of whom are women, in 73,000 villages in Bangladesh. Grameen Bank gives collateral-free income generating, housing, student and micro-enterprise loans to the poor families and offers a host of attractive savings, pension funds and insurance products for its members. Since it introduced them in 1984, housing loans have been used to construct 640,000 houses. The legal ownership of these houses belongs to the women themselves. We focused on women because we found giving loans to women always brought more benefits to the family.

In a cumulative way the bank has given out loans totaling about US $6.0 billion. The repayment rate is 99%. Grameen Bank routinely makes profit. Financially, it is self-reliant and has not taken donor money since 1995. Deposits and own resources of Grameen Bank today amount to 143 per cent of all outstanding loans. According to Grameen Bank’s internal survey, 58 per cent of our borrowers have crossed the poverty line.

The Quest of the Algorithm of the Chimes of the Bells of the Clock of the Long Now

I’m on a bit of a quest at the moment, which is turning up all sorts of interesting bits and bobs (and distracting me nicely from the work I should be doing today).

Yesterday I listened for the first time to Danny Hillis‘ 2004 Long Now seminar on “Progress on the 10,000 Year Clock” (listen here). Like most of these seminars (which I really should write about in more detail some time) it’s well worth a listen, bristling with neato facts, insights, and mind-expanding ideas. (A less time-consuming but still usefully detailed introduction to the clock project can be found here.) One of the things which particularly caught my imagination was the discussion of the bells. Apparently His Enoship noticed that the number of days in 10,000 years is almost exactly equal to the number of permutations of ten things (365 x10000 = 3650000; 10! = 3628800) – so the idea is that once per day these ten bells will chime in a certain order, never heard before, never to be heard again. (Eno has released a CD of “bell studies”, which is on its way to me even as I type.) Hillis invented an algorithm for the ordering of the chimes, so we know what ordering will be played for any day in the next 10,000 years (Eno’s CD plays the sequences for January 7003) and as you’d expect, people have run with this idea. I have an itch to run with it too, probably (predictably) in Haskell.

So where’s the algorithm? I haven’t found it yet, to my great surprise. :) I would have expected it to be fairly easily available, but apparently not. It might be in Stewart Brand’s book about the clock (also on its way to me), but I sure don’t see it anywhere online. However, there are a couple of implementations floating around, so perhaps some reverse engineering is in order…

Sean Burke has created a web page for exploring the bell patterns, with visualisations and MIDI downloads – code in Perl (and a Postscript (!) version here). Not fancying reverse-engineering the Perl too much, I wrote and asked Sean if there was a better source for the algorithm. Apparently Danny’s original version is in Mathematica. I haven’t found it, but Sean says he’ll send me what he can find in a few days. Otherwise, I guess I’ll keep digging, ask around on the Long Now forums, etc.

Sean pointed out that, when reverse engineering, “there’s understanding, and there’s understanding”, and pointed me at this fantastic war story, which sounds like a Daily WTF candidate to me. I’ve been there in the past, and in my case it was spaghetti Perl I was banging my head against – not pleasant. Still, the whole process of reverse engineering, of picking apart the code slowly, gradually and gently teasing the tangled knot open, can be a wonderful thing in itself. Or it might just turn out that the Mathematica code is clean and easily Haskellised. I doubt it, though, from what I’ve seen of Mathematica. :-) I expect it’ll live in a much lower-level domain than I want to work in, which is, of course, more than half the fun. If I can take an esoteric algorithm in a difficult language and translate it into beautiful and readable higher-order code, that’d be something worth writing about. So, watch this space (but don’t, of course, hold your breath).

In the mean time, as I said, the search is turning up all sorts of cool stuff. We present:

Another version of the algorithm, by Joe McMahon, in OS X/AppleScript discussed here, here & here (code via that last link). Interesting mention of ChucK too, which looks quite shiny (though again, maybe a little low-level for my taste).

Prototype chime generator diagram – I would wear a t-shirt with that on it.

An interview with Alan Kay from two weeks ago, which also points at the Kay-says-it’s-a-must-read Doug Engelbart essays (and no, I must confess, I hadn’t heard of Engelbart).

Pop culture lives in the present; it doesn’t really live in the future or want to know about great ideas from the past. I’m saying there’s a lot of useful knowledge and wisdom out there for anybody who is curious, and who takes the time to do something other than just executing on some current plan. Cicero said, “Who knows only his own generation remains always a child.” People who live in the present often wind up exploiting the present to an extent that it starts removing the possibility of having a future.

Stewart Brand meets the Cybernetic Counterculture – whee, the 60s!

(These last two via this del.icio.us page on “admirable people”.)

woof

woof – very handy for occasional one-shot (or multi-shot) file serving. Put it somewhere in your path and every now and then it will make you happy.

Woof (Web Offer One File) tries a different approach. It assumes that everybody has a web-browser or a commandline web-client installed. Woof is a small simple stupid webserver that can easily be invoked on a single file. Your partner can access the file with tools he trusts (e.g. wget). No need to enter passwords on keyboards where you don’t know about keyboard sniffers, no need to start a huge lot of infrastructure, just do a

$ woof filename

and tell the recipient the URL woof spits out. When he got that file, woof will quit and everything is done.

Saving Private Ryan

Bash & I watched Saving Private Ryan this evening. I’ve wanted to see it since it came out, mainly because I’d heard that the first half hour or so was incredible. More generally, I quite like gritty WW2 stuff since I read Cryptonomicon, and I enjoyed what I saw (little) of Band of Brothers, so it seemed likely I’d enjoy this.

Well, yeah, it was all right. Not a life changer, but also not a complete waste of three hours either. The first half hour certainly didn’t disappoint, and overall I was fairly gripped, and impressed by the grittiness, jumpiness, fleshiness of it. I think it would be improved about tenfold by (a) having absolutely no music whatsoever (apart from the Edith Piaf, obviously), and (b) having no Stars ‘n’ Stripes fluttering fore and aft. But hey, shrug, it’s a product of its time and place, I guess (three years before the WTC attacks, btw). I expect it had more impact when it was fresh and the camerawork/lighting techniques were new & exciting, but still, it worked well. Tom Hanks did his Thinking Man’s Regular Joe routine as well as ever, and the supporting cast was gratifyingly full of “ooh look, I know him” faces (Ted Danson – yay). I was surprised to see Matt Damon as Ryan (I didn’t know before I watched it), which of course spawned a slight Team America moment on the couch. Always a pleasure…

(Hey, check the trivia on IMDB: interesting snippet on the Lincoln letter. It figures – historical truth is almost never as stirringly poetic as it is portrayed by propogandists, I suppose.)

Haskell study plan

Haskell study plan [smallcool].

autopushd in zsh – make cd behave like pushd always

I’ve just been exposed to one of those delightful little hacks that are both forehead-slappingly simple/obvious and yet so powerful and clearly “right” that I just know I’m going to be using it for the next n years. Allow me to share.

In every Unix shell I’ve ever seen, one uses the “cd” command to change directory, moving around the directory structure.

cd has no memory of where you were before. Some shells add a stack-based history via the commands “pushd” and “popd” (which I only discovered in 2004). If you pushd to a directory (eg pushd /some/distant/directory), you can subsequently get back to where you were before very easily by just entering popd; furthermore, such calls can be nested arbitrarily deeply (that’s what the stack is for). This is, on occasion, tremendously helpful, and can save a lot of typing/thinking.

The insight: why bother with cd, then? Why not just always pushd around the place? That way, you’ve always got the possibility of using popd to get back to where you were before easily and quickly. There are two possible reasons why not. First the stack takes up some space in memory. Feh, no problem – it’s not that big, and I never have a shell open long enough for it to become a problem. Second, pushd is a few more keystrokes than cd, and my habits have cd hardwired into them.

Thus, the hack: alias cd to pushd. But wait! Don’t do that!

Since I use zsh, I don’t even need to alias it. I can use the zsh config option “autopushd” to make cd always behave like pushd. Hell, yeah! I just add the following to ~/.zshrc:

setopt autopushd

and hey pesto:

# Start by going to some fairly deeply nested directory
[gimbo@mane ~] cd work/teach/cs-238/current 

# Now I cd (but really pushd) to some other, also deeply nested, directory
[gimbo@mane current] cd ~/work/research/mphil/tools/HetCATS/CspCASL
[gimbo@mane current] pwd
/home/gimbo/work/research/mphil/tools/HetCATS/CspCASL

# In the words of Baloo: take me home, daddy!  (This next line doesn't work without the autopushd trickery)
[gimbo@mane current] popd
~/work/teach/cs-238/current ~
[gimbo@mane current] pwd
/home/gimbo/work/teach/cs-238/current

Update 2007-02-24: Will has just pointed out to me that “cd -” will take you back to where you were before, so repeatedly issuing cd - allows you to flip between two places easily – which is of course a common use case. Neato!

[gimbo@orb ~] cd /tmp 
[gimbo@orb /tmp] cd -
~
[gimbo@orb ~] cd -
/tmp
[gimbo@orb /tmp]

Drumming on the streets of Mumbles

Last night, Shiko featuring Gimbo played a gig. Sorry I didn’t mention it sooner – I meant to, but kept forgetting. It was at a fundraiser put on by the University’s Visual Arts student society (jugglers, firebreathers, etc.), at the Chicago Rock cafe off Wind St; they’d asked us to come along and play, and so we did: and it was good. In fact, I’d say it was the best gig we’ve done since I’ve been Shiko’ing (which is, admittedly, less than a year) – quite tight and confident. Speaking personally, it was certainly the most confident I’d ever been when performing to an audience (musically at least). We weren’t playing for long, but what we did was fairly high impact, I think. (We did Kennefoli/Soli, then Kassa, for the record.)

Anyway, fair warning: on Saturday 3rd March 2007, we’ll be playing in a St David’s Day parade in Mumbles. More details than that I do not know, except that we’ll be walking and drumming simultaneously, which will be a new experience for me, at least. Should be fun – hope to see you there!

Ooh look: Shiko a few weeks ago. That’s me seated on the right…

Shiko, 13th February, 2007

(Also, awesome (but 5MB) montage of the Shiko new year party, for which I was sadly absent.)

The Trembling Giant

Introducing Pando, a male Quaking Aspen clonal colony – at six gigagrams the most massive known single organism on the planet, and possibly the oldest (although this is apparently controversial and of course upsets those imaginative people who maintain that the whole planet was flooded by God a mere six thousand years ago).

hpDJ: an automated DJ with floorshow feedback

Here’s a little gem for anyone else out there interested in dance music, DJ’ing, and computers: hpDJ: An automated DJ with floorshow feedback. Basically, it describes a system which allows one to define a “qualitative tempo trajectory” for a DJ set, then chooses tracks to fit the trajectory (the words “partial ordering” occur at this point, but that’s as close as we get to TCS), and automatically beatmatches/timestretches/mixes the tracks accordingly. Towards the end it considers the performance and “collaborative consumption as composition” possibilities, with genetic algorithms and the like, but the bit which impressed me the most was on the mixing, where the opportunities afforded by doing the job in software are recognised and made the most of.

However, because hpDJ operates in the pure software realm of digital signal processing (DSP), it is possible to create as many sweepable band-pass/cut filters as is desired for any particular cross-fade from one track to another. As with traditional hardware mixers, each DSP filter can have variables that control the degree of attenuation or boost, and its center-frequency. In addition to this, the shape of the DSP filter’s transfer function (e.g. the nature and rate of the fall-off or boost) and its bandwidth can also be under automatic control. Recording studios do have filters with these added controls, but such filters (known as a Parametric EQ) are too expensive to be built into each channel of professional mixing desks on a many-per-channel basis.

Thus, it becomes possible to specify hpDJ so that it analyses the audio frequency-time spectrogram for the incoming and outgoing tracks in the cross-fade, and uses a number of heuristics to determine how many DSP Parametric EQ filters are necessary and what their settings should be. This can be used to, for instance, selectively suppress the frequencies for a synthesizer melody-line in one track, attempting to make that melody “disappear” while keeping the bass-guitar and percussion elements in place during the cross-fade. By employing simple heuristics for detecting when one component of one track “clashes” with another component of the other track, such aesthetically unpleasant clashes (which may remain despite perfect beat-matching) could be automatically eliminated by hpDJ.

Note that this is all happening automatically (except maybe setting some initial loose parameters at the start) – very impressive.

Although this was published only in 2005 (I spotted it in this book on this man‘s desk), much of this was apparently done around 2000 or so. Thus, I wonder what goodies they’re cooking up in Bristol even as I type. Dave Cliff’s “inventor profile” page doesn’t really bring us up to speed, I feel… Aha! He’s moved to Southhampton.

Finally, I note that this is the first paper I’ve ever seen which opens with a Sisters of Mercy quote. Rock on.

RMS in lego

I just found two super photos on flickr…

More super for computer people, I guess, but anyway…

(Number 2 in an ongoing Thorsten series.)

Haskell and Category Theory

An introduction to Category Theory from a Haskell point of view. As always, monads are where your brain starts hurting.

Autoattach / automount encrypted home partition in FreeBSD using GBDE

I’ve just set up one of my laptops, running FreeBSD, so that /home is encrypted using GBDE, and is auto-attached/mounted/fsck’d at boot time. The instructions in the FreeBSD handbook aren’t completely clear, so here are some notes on how I did it.

Actually, most of those instructions are very clear, provided you don’t care about attaching at boot time. Indeed, I’ve had an encrypted partition which I attach/detach by hand, for about a year now, for which those instructions were perfectly adequate. Unfortunately, the section on “Automatically Mounting Encrypted Partitions” leaves out two important details which conflict with the rest of the chapter: your lock file’s name needs to end in the text “.lock”, and you need to be careful where you put it. This was confusing, and in some ways what you have to do to get automounting working contradicts the other examples in the chapter.

Anyway, long story short, I had to dig into /etc/rc.d/gbde to work out what had to be done, and here it is… A summary of how to set up an auto-attaching encrypting home partition on FreeBSD using GBDE:

(I did this as pretty much the first thing after a fresh install, before I’d even created any users. Needless to say, then, all of this happens as root.)

Here’s how /etc/fstab looked before I started:

# cat /etc/fstab
# Device        Mountpoint         FStype    Options   Dump Pass#
/dev/ad0s1b     none               swap      sw        0    0
/dev/ad0s1a     /                  ufs       rw        1    1
/dev/ad0s1e     /tmp               ufs       rw        2    2
/dev/ad0s1f     /usr               ufs       rw        2    2
/dev/ad0s1g     /usr/home          ufs       rw        2    2
/dev/ad0s1d     /var               ufs       rw        2    2
/dev/acd0       /cdrom             cd9660    ro,noauto 0    0

Unmount /home because I’m about to blat it completely:

# umount /usr/home

Create a directory to contain the lock files:

# mkdir /etc/gbde

Initialise the partition for GBDE. I used a use sector size of 2048 (which matches the UFS fragment size). Note that the lock file’s name ends in .lock; this is not how the main body of the GBDE instructions in the handbook does it, but it’s necessary to get /etc/rc.d/gbde to attach it properly on boot up:

# gbde init /dev/ad0s1g -i -L /etc/gbde/ad0s1g.lock
Enter new passphrase:
Reenter new passphrase: 

Attach the encrypted partition to the kernel for the first time (entering the passphrase previously specified), and write the filesystem:

# gbde attach /dev/ad0s1g -l /etc/gbde/ad0s1g.lock
Enter passphrase: 
# newfs -U /dev/ad0s1g.bde 
/dev/ad0s1g.bde: 6632.5MB (13583360 sectors) block size 16384,
                 fragment size 2048 using 37 cylinder groups of
                 183.77MB, 11761 blks, 23552 inodes.  with soft
                 updates
super-block backups (for fsck -b #) at:
 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624,

*SNIP*

Test the mount:

# mount /dev/ad0s1g.bde /usr/home

Assuming that looks good (eg in df), unmount it and detach it:

# umount /usr/home
# gbde detach /dev/ad0s1g

Now we’re ready to set it up for auto-attaching. We need to alter /etc/rc.conf and /etc/fs.tab

# tail -n 3 /etc/rc.conf
gbde_autoattach_all="YES"
gbde_lockdir="/etc/gbde"
gbde_devices="ad0s1g"

We need the gbde_lockdir line because otherwise it looks for the lock files just in /etc, I think.

# grep home /etc/fstab
/dev/ad0s1g.bde /usr/home          ufs       rw        2    2

Now when I boot, it asks for the passphrase, attaches the encrypted partition, and mounts it – automatically – then it gets fsck’d with everything else. It doesn’t automatically detach upon halting the system, but I guess that’s no problem. :-)

The only thing I don’t totally like is that if I get the passphrase wrong (3 times), it doesn’t attach or mount the encrypted partition (obviously), but then of course it fails filesystem checks completely and the boot process dumps down to single user mode, messily. Not completely unreasonable, I guess, but still a bit annoying.

Anyway, anyone now stealing my (crappy) laptop will have a much harder time of getting at the data on it (the interesting data, anyway). Whee!

I would be like David Lee Roth!

Currently greatly enjoying reading Questionable Content from start to finish. Bash spotted it linked off xkcd.

I saw Children Of Men last night and it completely broke my brain. That film is completely drenched in fear; every moment of the film, every character, everything that happens is driven by fear. Awful, just absolutely horrible. I mean, a great film (and I really like Clive Owen, I think) but I will be happy never to see it again. The last film I saw which left me feeling like this was Requiem For A Dream back in late 2002, but if anything last night was worse – with Requiem, it was really only the last half hour that was drowning in trauma, whereas last night it was the whole fucking film.

As such, I am spending the day at home nursing my psyche by drinking cups of tea and reading harmless web comics. :-)

Foozles!

Yikes. This is waaaaay too close to home: Foozles – a programming language feature comin’ atcha soon! [wadler]

2013: Francoise Boisbleu proves that under a certain formulation, Foozles are a categorical dual to Aspects, which gets everyone terribly excited.

It certainly would.

Good spam filtering in WP

I’m very impressed with WordPress’ built-in spam protection (well, strictly not built-in – but provided by the Akismet plugin that came bundled with). Since making the switch a couple of weeks ago, it’s caught 354 spam automatically, straight to the spam queue. Today, for the first time, two pieces of spam snuck through to the comment moderation queue. Even better, as soon as I hit “re-check queue for spam”, they were gone – yay, collaborative filtering.

This is such an improvement on my old situation, where I didn’t even have a separate spam queue, and occasionally spam was even published. It was pretty primitive, really. I expect that upgrading my MT version would have helped, but meh. Anyway, as Jo would say: will everyone please stop blogging about blogging, and blog. :-)

Classic comment / code impedance mismatch

Awesome, almost Dadaist WTF:

// MUST be set to 1!
Params->ParentalLevel = 3;  // QA now insist this is set to 2.

Made me laugh out loud, anyway.

The Big Bus

This week, I saw The Big Bus for the first time since I were a lad. It was every bit as fantastically ludicrous as I remember it, except now I could truly appreciates its kitsch 70s retro glory. In short: awesome. God bless mail-order DVD rental.

The film centres on the maiden voyage of `Cylops’, a colossal nuclear powered bus embarking on the first ever non-stop voyage from New York to Denver (clearly an important niche market in the late 70s). Big Oil interests are sabotaging the project, and a bomb-blast has put the pilot and co-pilot out of action. The project designer’s daughter (played by Rizzo – so sexy in her two-tone brown jumpsuit) goes in search of her ex-lover, the legendary Dan Torrance (played by the improbably monikered Joseph Bologna), the only man who can learn to drive such a complicated rig in the two weeks remaining before launch day. But wait! Torrence has a dark past, as we learn in a (to my mind classic) fight scene set in a smoky bus-driver’s dive bar – he was the sole survivor of the disaster of `Mount Diablo’, and is now haunted by accusations that he ate all 110 passengers.

Basically, it goes downhill from there, in glory.

Apparently it was a big influence on the Airplane! movies, Police Squad! and the Naked Gun movies. It was certainly a big influence on the comedy stylings of the Gimblett family. I still have an almost irresistible urge to go “Look out! He’s got a broken milk carton!” every time I see a fight scene. I’m also pleased to report Bash seemed to quite like it too, and I’m fairly sure she wasn’t just humouring me.

You eat one lousy foot and they call you a cannibal. What a world!

Dan Torrence