(With pretext.)

High-level political debate:

Wed, 06 Jan 2010 11:07:14 +0000

High-level political debate:

Soundtracks

Wed, 30 Dec 2009 01:27:43 +0000

A mate of mine — half of Redpoint, my ex-labelmates on Hidden — runs a tribute to John Peel, and his Festive Fifty, every year. He runs a straw poll of his music-geek friends (rules: ten songs, in order, must have been released in the calendar year), compiles the results, and puts it out there ten tracks at a time between Christmas and New Year. You should check it out.

But seeing as it’s list season at the moment, even where you mightn’t otherwise expect it, I thought I’d join in and post my ballot up here. Starting at the top, 2009:

1. Future of the Left, “Arming Eritrea”

2. Telefon Tel Aviv, “You Are The Worst Thing In The World”

3. The Mountain Goats “Ezekiel 7 And The Permanent Efficacy Of Grace”

4. The Horrors, “Primary Colours”

5. The Thermals, “Now We Can See”

6. F—- Buttons, “Olympians”

7. Gui Boratto, “Take My Breath Away”

8. Do Make Say Think, “Make”

9. The Broken Family Band, “Don’t Bury Us”

Couldn’t find these on YouTube or Spotify. (Post-rock from Toronto and Cambridge’s finest, even though they just split up — and yes, I am aware of Pink Floyd — respectively.)

10. The Hold Steady, “Atlantic City”

(A Springsteen cover for a charity record. Trust me, though.)

If this year had a soundtrack, though, it was literate north-American indie rock: The Hold Steady, the Mountain Goats, the Weakerthans, the National, Grizzly Bear, Bon Iver. I’m kicking myself I hadn’t got round to them before. And as a (former) curling skip, here’s one song which’d have made my ballot if it hadn’t come out in, um, 2007…

Shilling in the name

Wed, 23 Dec 2009 18:11:00 +0000

Apparently we will, after all, buy what they sell you:

Rage Against The Machine Have The Christmas Number One

At least, that’s how it seems. The sales figures for the UK Christmas top twenty have been circulating, and here they are.
1	KILLING IN THE NAME	RAGE AGAINST THE MACHINE	502672
2	THE CLIMB		JOE MCELDERRY			450838

sheamus.co.uk, 20th Dec ‘09

Fantastic — everyone loves Cowell getting a kicking, assuming you could hear it over the hand-wringing over the money side of things.

But were we all lining our bete noire’s pockets? All of the RATM sales were digital; in general, 95% of singles sales are, but let’s assume 20% of Joe McElderry’s sales were at retail to allow a nice margin for stocking fillers. Let’s be generous on pricing too; I’m working on assumptions of £1.99 physical, 69p digital (and both singles were going for 29p on Amazon last week). VAT’s 15%; if retailers get a 30% markup, and 15% goes to the artist, mechanicals, songwriting, and so on, then here’s the numbers:

Band	Sales	Artist revenue	Label net	VAT
Rage against the Machine	502672	£29,116.40	£164,992.91	£44,486.47
Joe McElderry	450838	£36,944.98	£209,354.90	£58,834.44

(Check my working here.)

So having both the number one and number two singles, on the highest sales week of the year, with the entire media acting as your hype wing, is only worth £375k to Sony — and only about £65k to the combatants!

If the record industry’s looked weird to you recently, then these numbers explain a lot. Radiohead made way more from In Rainbows than they made from any of their previous records, despite giving it away. Bands are signing 360-degree deals. All the money’s in merch and concerts, and now Ticketmaster and Live Nation are merging.

Bad day to be Guy Hands, then…

"You can do the cleverest, most expensive, most extraordinary bit of programming but put it on a..."

Sun, 08 Nov 2009 19:04:00 +0000

“You can do the cleverest, most expensive, most extraordinary bit of programming but put it on a screen and everyone’ll think they’ve seen it before. And they probably have. In a movie.”

- Russell Davies’s talk from Dconstruct: “Materialising and Dematerialising a Web of Data.
One of the ideas in here is the hyperreality of computer experiences: the CSI effect. There’s a point - we’ve probably reached it already - where everything we do is touched by technology, but right now much of that technology is so in-your-face that it perversely becomes commonplace. Computers are like that; everyone knows screens mean magic. Ubiquitous computing’s extending the tentacles of the surreal into everything else.

So, then. Why Internet economics and big data and cities? There’s a lot of credit — or blame!...

Wed, 04 Nov 2009 00:00:00 +0000

So, then. Why Internet economics and big data and cities? There’s a lot of credit — or blame! — to go round, but a little bit of it has to go to Russell Davies’s conference-planning.

That’s me on the right there. I hadn’t been to any conference quite like Interesting. Still haven’t. More or less every decent idea I’ve had since contains a dash of something I first encountered there.

Here’s Matt Jones, who finished off the day. You know that moment of giddy dislocation you get when one of those Magic Eye drawings kicks in? That’s how I was feeling, and this talk capped it off.

Matt Jones at Interesting2007 from russelldavies on Vimeo.

If that wasn’t enough, not long after Interesting, I took a trip to San Francisco. I wasn’t long out of my graduate studies, and I kept finding myself drifting back to thinking about, well, the kind of things I’m working on now. So that’s one of the ways this story begins.

I trained as a mineral scientist, not as a designer. So when I start to approach these ideas I use what I know, and two of the things mineral physicists think about are ordering and phase transitions. (My thesis even had “disorder” right there in the title.)

Phase transformations happen when symmetries are made and broken. Things melt and recrystallise. How are cities melting and recrystallising?

I can’t really talk about the design issues, but I’m betting that cash and computation will be at the heart of the physics of it. Here’s one thing which which connects the two. Flash memory’s getting seriously cheap very fast. So fast that you’re getting exponentially more storage for your money as time passes — hit the log-scale checkbox on the graph below to see.

Admittedly, cheap storage alone isn’t a story: hard discs are still cheaper per byte, after all. But flash memory has some advantages — it’s lower-power, more robust and smaller. The kicker, though, is that reading data in a random order off flash storage is way, way faster than reading it off a disc. Hard discs aren’t all that far evolved from vinyl records. There’s a motorised read head instead of a stylus, but it hovers over a spinning platter, reading the data off as it hurls past. So if your data doesn’t line up neatly — it’s not contiguous — you’ve got to winch the read head back and forward to read each piece of it in turn, and that’s really slow.

Databases, in particular, have had a lot of logic devoted to laying out data on disc just so. You don’t need to do that at all if you’re writing your data to flash. That lets you focus on different problems instead, like being able to insert data into your database really quickly.

If you’re going to put a sensor in everything, you need to be able to write a lot of data to a database really quickly.

Make faster databases and you can record and query data you couldn’t use before. What’s more, because they’re tiny and can take a beating, flash-backed databases can go wherever environmental sensors go. I’ve got a hunch that smart flash-based storage is the missing part of the whole ubiquitous-computation shtick: cheap flash gives you staging posts for the instrumented street’s data.

Bandwidth, computational power and storage. When you get them in places you haven’t had them before, or when they get a lot faster or a lot cheaper, or when you can combine them in new ways, you break symmetries. That’s where I think the action is.

Winning in the demo

Sun, 01 Nov 2009 23:59:46 +0000

The last big thing I did in student radio (as a postgrad, I spent more time than my supervisors would have liked have in CUR1350’s basement studio) was putting together coverage of the last General Election. It was never going to be a classic; everyone was tired of Labour, but the opposition looked nothing like a government in waiting. It was all cued up for an as-you-were result.

Except in a few seats around the country, that is. Cambridge was one, Cardiff Central, Bristol West, Manchester Withington. All of them had something in common afterwards — new Liberal Democrat MPs — and something in common beforehand: Labour MPs in constituencies with a lot of students.

Students hated the Government in 2005. The Iraq war was the headline, but I reckon the real reason was at least as much student fees. Anne Campbell, the Cambridge incumbent, hadn’t so much swithered as pulled a handbrake turn on the subject; students vote their wallets just as much as anyone else. But there was no way a bunch of eighteen-to-twenty-five-year-olds, who’d become politically aware in the dying years of the Thatcher and Major governments, would even consider voting Tory in numbers.

Enter the Lib Dems.

The Lib Dems had a lock on the ‘05 student vote, and they knew it. And because they knew it, they could exploit their greatest resource. They’ve got the best street team in UK politics. No-one knocks on doors like a Lib Dem activist, and their by-election record proves it. Now, students are a tough audience to canvas. That alone normally gets them beyond the reach of election-day get-out-the-vote efforts, because they’re just not entered in the database.

But if you were a student in Cambridge, in 2005, if you voted then you were probably voting Lib Dem. So if you mobilised a house of six, it was probably three net votes. The Lib Dems could afford to knock on every student door regardless of whether they were certain there were supporters there or not, because they knew their audience and demographic so well that they could trust in the law of averages.

Their candidate, David Howarth, won on a 19% swing.

I’ve been writing a lot about Google this last couple of weeks, so this might seem a bit of a digression. It isn’t, though; Google’s platform strategy now is the Lib Dems’ strategy from 2005. Google know who makes them money — the kind of people who’d use smartphones or netbooks or turn-by-turn navigation. If they sell more of their product by grabbing more time in front of these lovely, high-disposable-income eyeballs, then even if we, of course, would never be influenced by advertising, on average they’re going to win big. Google owns a percentage of the people on the Internet demographic. Anything which mobilises that audience makes them money in the long term, even if it costs them in the short term to do it.

The thing was, I knew I’d seen the tactic before. I just hadn’t worked out where!

The 'less than free' business model

Sat, 31 Oct 2009 12:19:02 +0000

The 'less than free' business model:

Some really sharp observations here about how Google just revalued another information market - this time, turn-by-turn navigation.

Google are so confident that they’re the Internet Inland Revenue that they’re even offering rebates…

Commons people

Fri, 30 Oct 2009 02:39:00 +0000

Earlier today, through an article in the latest US Wired, I discovered Demand Media. It took me the rest of the afternoon to get my breath back. If you’ve not seen them before, they’re really something: maybe half a rung above something like Datapresser on the scale of evil . Driven by sophisticated algorithms which tell them which search terms are worth targetting, they’re generating huge link-farms of low-budget, low-quality articles and video. Bluntly, they’re overfishing the long tail of search queries.

It takes real guts to run such a sketchy business on such an industrial scale: they’ve raised over $300m in funding so far. A part of me almost admires them for it.

He points out that the focus is off: The rippling water is sharp while the kayaking instructor is slightly blurred. But the company he’s working for won’t care, he says, so why should he — especially for $20 a clip? Within a few hours, he has uploaded his work to Demand Media, his employer for the day. It isn’t Scorsese, but it’s fast, cheap, and good enough.

Thousands of other filmmakers and writers around the country are operating with the same loose standards, racing to produce the 4,000 videos and articles that Demand Media publishes every day. The company’s ambitions are so enormous as to be almost surreal: to predict any question anyone might ask and generate an answer that will show up at the top of Google’s search results.

— “The Answer Factory: Fast, Disposable, and Profitable as Hell”, by Daniel Roth, Wired US November 2009

Meanwhile, Google are picking up criticism, and prominent web developers are chucking venom at the SEO industry.

These people are a cancer and must be destroyed.

— “SEO Scumbags”, by Jacob Kaplan-Moss

Search Engine Optimization is not a legitimate form of marketing. It should not be undertaken by people with brains or souls. If someone charges you for SEO, you have been conned.

— “Spammers, Evildoers, and Opportunists”, by Derek Powazek

Everyone ‘knows’ that Google used to be better. But how much can Google really do against turbo-charged content engines like Demand Media? It’s an arms-race. The Bayesian spam filter wars of 2002 are playing out all over again, only this time in Google’s index.

Even so, this might still give rise to an opportunity for someone. Just a thought experiment: what’s going to happen if, for some big high-value niches, Google gets either unpopular or ineffective? The demand’s still going to be there, after all. Who’ll fill it, and how?

After all, Google isn’t approaching smartphones like a normal business. It has made Android open...

Tue, 27 Oct 2009 00:23:17 +0000

After all, Google isn’t approaching smartphones like a normal business. It has made Android open source available freely to any handset maker to use and change at will. Google says its only reason is to expand the use of the Internet on cellphones because it believed this would allow more people to use its search engine and other products on which the company sells ads.

Microsoft, by contrast, charges manufacturers $15 to $25 per Windows Mobile phone sold, an attempt to duplicate the rich business model of Windows for the PC. Google’s a-rising-tide-raises-our-ship approach deeply perplexes Microsoft executives. “If you asked me to go to a venture capitalist and pitch the Android business model, I don’t think I could,” said Robert J. Bach, president of Microsoft’s entertainment and devices division, at a meeting with reporters earlier this month.

Microsoft, Google and the Bear - Bits Blog - NYTimes.com

I’ve heard it said — can’t remember where, I’m afraid! — that governments have absolutely the best way of making money from money funnelled into startups by things like R+D tax credits; don’t get involved, just tax the winners.

With what Google’s doing with Android and Chrome, they must think that AdSense/AdWords/DoubleClick add up to Internet Tax…

Headless NFL players (via cvrt)

Mon, 26 Oct 2009 20:40:05 +0000

Headless NFL players (via cvrt)

Specifically, you have a large group of people who will download and suffer any old shit by the...

Tue, 20 Oct 2009 22:39:00 +0100

Specifically, you have a large group of people who will download and suffer any old shit by the bucketload as long as it is free or extremely cheap. And you have 10% of people who are actually particular about software quality and are willing to pay for it. In other words, you have the Windows market, and the Mac market, but within the app store itself. And you’d better be damn sure which one you’re targeting, and set pricing and development schedule accordingly.

So says Steven Frank on the iTunes App Store. What this reminds me of, a bit, is danah boyd’s thesis on social stratification among teenagers on social networks. In a business context, I guess that you’d call it self-organized market segmentation…

The downfall of newspapers has been caused by a number of things – losing the classifieds business...

Sat, 17 Oct 2009 10:45:49 +0100

The downfall of newspapers has been caused by a number of things – losing the classifieds business was huge – but mainly because when newspapers went online and were no longer able to partition the market geographically, supply in each region went up by orders of magnitudes.

— Chris Dixon: “What’s the relationship between cost and price?”

I hadn’t seen this argument before, and it feels like it makes sense. However, it makes a lot more sense when you talk about the US and its paper-per-big-metro market. All the big British papers are nationals, and they’re getting clobbered too. Sure, now British papers have to compete with the Americans, and the BBC doesn’t really have an American equivalent, but…

Maps and legends and stacks of abstractions

Mon, 12 Oct 2009 23:51:00 +0100

The other day I realised that I still really don’t know London.

Dan and I had to get from Oxford Circus to Clerkenwell the other day. I’d been up there once before, so I was pretty sure how to get there. Farringdon tube, right? Either two changes or two stops north on the Bakerloo line and then along the Circle. Either way, it’d take forever. It was raining, too. And I’d managed to put a hole in my shoes.

Not so fast, says Dan; better to go along the Central Line to Chancery Lane. I thought he was mental, until we checked the maps on our phones.

I had no idea that Chancery Lane was that close to where we were going! I thought it was miles away. After all:

I’m not a Londoner, just a day-tripper, and my internal London is in the shape of the Tube map. The thing is, I know the Tube map’s about topology, not geography. I’ve seen Tom Carden’s stuff on Tube travel times. Still, I just knew Chancery Lane was nowhere near Farringdon, damn it.

In all the writing there’s been on networked urbanism recently, which is terrific and you should read it all, there’s been one thing which has been bothering me by its absence. We’re going to stick sensors in everything and garland cities with wireless, and then we’re going to shove all that data onto the Internet. (In case you wondered, that’s where we come in.)

But once we’ve got all that data, well: networked urbanism means, amongst other things, building visualizations of everything. Abstracted representations of partial recordings of the recent past. And, like the Tube map, if they’re any good than they’re going become part of us.

We’re pushing abstractions of abstractions of abstractions into peoples’ heads.

And, well, here’s what I’ve been worrying about: have we got a moral responsibility here? There’s no Hippocratic oath for informaticists — “First, do no memetic harm”. But if we’re building these cerebral prostheses for ourselves and for each other, then don’t we have a duty of care not to mess with how people perceive the city too badly? And how do we avoid doing that?

Or maybe it’s just me.

"It is by now clear that over the last decade a great number of people on Earth, in the developed and..."

Fri, 09 Oct 2009 02:34:17 +0100

“

It is by now clear that over the last decade a great number of people on Earth, in the developed and the developing world both – certainly the overwhelming majority of those reading these words – have embraced the digital mediation of everyday life, to such a ferocious extent that it can already be difficult to remember how we ever got through our days without the networked things around us.

Without necessarily considering the matter with any particular care, as individuals or societies, we have installed devices in our clothing, our buildings, our vehicles and our tools which register, collect and transmit extraordinary volumes of data, and which share this data with the global network in real time. If some of us once – and recently! – thought of this as the domain of “ubiquitous computing,” the words are already starting to sound obsolescent, as clunky as “horseless carriage.” This is simply the way we do things now.

”

- The kind of program a city is « Adam Greenfield’s Speedbird

“An expert’s someone who can read documentation...

Wed, 07 Oct 2009 13:08:00 +0100

“An expert’s someone who can read documentation faster than you can” - or so the computer-nerd saw goes, anyway.

This all reminds me of another idea I keep meaning to do something with, but never get round to: that usually ‘having ideas’ is a bit like running an import-export operation between different fields of work, and not really all that much about thunderbolts of divine inspiration. Of course, though, that’s not an original notion…

"(Rupert) Murdoch can almost single-handedly take apart and re-assemble a complex printing press, but..."

Mon, 05 Oct 2009 10:59:00 +0100

“(Rupert) Murdoch can almost single-handedly take apart and re-assemble a complex printing press, but his digital-technology acumen and interest is practically zero. Murdoch’s abiding love of newspapers has turned into a personal antipathy to the Internet: for him it’s a place for porn, thievery, and hackers. In 2005, not long after News Corp. bought MySpace, when it still seemed like a brilliant purchase—before its fortunes sank under News Corp.’s inability to keep pace with advances in social-network technology—I congratulated him on the acquisition. “Now,” he said, “we’re in the stalking business.””

- Michael Wolff on Rupert Murdoch | vanityfair.com

By the RFH on the South Bank.

Sun, 04 Oct 2009 15:54:32 +0100

By the RFH on the South Bank.

Hello there.

Sun, 04 Oct 2009 13:22:00 +0100

Hello there.

You spin me round...

Wed, 29 Jul 2009 00:00:00 +0100

There’s about four miles between where I live and where I work. It’s less bad luck or bad planning than pure economics: we rent space in the Computer Science building of the University of Cambridge, which is to the west of the city centre, and I live right over on the eastern edge, by an airport and a floodplain, and where the rents are a bit less arched-eyebrow than in the rest of town.

There are a few advantages though: when I’m crossing the Cambridge compass by bike, I get to cycle along the river – picturesque, as long as you don’t go head-first into a cow, and before you ask that’s not a metaphor – and it gives me a bit of time to listen to things. Recently, it’s been the audiobook version of Free by Chris Anderson. I’ve not finished it yet, so this isn’t anything like a review, but the chapter which I got to on the way home tonight (light drizzle, one close call with a gaggle of yellow-backpacked language students, fairly typical) triggered a few thoughts.

He was talking about 360-degree record deals: record labels, who’ve noticed that moving petrochemical residue and cardboard around the world doesn’t pay like it used to, are trying to get a share of tour income and T-shirt sales from the bands they sign. This is, politely speaking, a bit controversial. Alan McGee, ex-Creation Records:

The record industry’s demand for bands to sign over a portion of their merchandise and tour revenues as part of a recording contract is an admission that selling music is not a sustainable business model.

I’m way below small-time, but the only income I’ve ever seen from music has been the occasional £50 for DJing at a party. Even as a hobby, my little label couldn’t justify selling either CDs or MP3s: CDs are expensive and, unless your decor’s like mine (something out of High Fidelity) you don’t want the clutter. As for MP3s: if your music’s any good it’s going to wind up on BitTorrent or Soulseek. If it isn’t any good, it won’t, but then nobody cares anyway. In either case, you might as well give it away. In this world, what a big record company’s got left to offer is really expensive credit to (theoretically) spend on studio time and producers, their promotional might, and their contacts. If you’re making records by yourself, cheaply, and largely as promos to get people to book you, then all the record company has to offer you is the loan-sharking operation. For that matter, it’s bad news for record producers too: if most records get made on the cheap on laptops in back bedrooms – most dance music, and more and more indie, already is – then their role gets cost-cut right out of existence.

Anyway, after the record business, Chris Anderson moves on to talk about the technical/business book trade. It isn’t looking too clever right now either:

The market has been on a steady decline since mid-2008 and has continued downward right through the first half of 2009. And there are very few signs that the book-buying slump is going to turn around anytime soon… the market performance this year is the worst we’ve seen since the fall of of 2001.

On top of that, technical non-fiction has never made a lot of money for its authors: Chris Anderson states outright that his business model is to use his books as a way to get booked as a consultant and keynote speaker. All in all, with authors beginning to rattle sabres about piracy, the two industries are beginning to seem more and more alike.

Tim O’Reilly has pointed to diversification as one way to survive in the current financial climate, and that includes moving into events and personalised training. In other words: publishers making money from gigs. It’s not quite the same as record-company-style 360, but isn’t there something in common here? Many of the speakers we see at conferences made their names by writing for publishers like O’Reilly: if the same publishers then turn round and book them as speakers, based on the credibility granted by having published…

Of course, there’s nothing even slightly sinister going on here right now. It doesn’t hurt that O’Reilly are beyond reproach ethically! Still, what if one publisher were to break ranks? I can easily imagine someone trading a percentage of their speaking or consulting income for the marketing benefit of being the author of a book on a hot topic from a seemingly-kosher publisher.

Alternatively, as with record labels, maybe traditional publishers will be cut out entirely and books will be written online, turning into something between academic theses – proofs of qualification which hardly anyone really reads – and blogs (the route taken by the Django Book’s subediting-by-commenter). In that world, do publishers evolve into speaker bureaux with a print-on-demand addon, format-shifting books which live online into something you can read in the bath? It’d be a shame if that were to happen, but it was a downer when Too Pure went under too.

Or maybe something entirely new will evolve. It’s difficult to see how the present state of affairs hangs on.

Oxford Geek Night keynote

Thu, 23 Jul 2009 00:00:00 +0100

Not too long ago – a week ago today, in fact, I gave one of the keynotes at J-P Stacey’s Oxford Geek Nights. The team there video all the talks, so here you are — if you’ve got some spare time, check out the others as well.

(Right-click to download.)

If you’d rather read, though, here’s what I was planning to say. The talk melted a bit under the stagelights; this really isn’t an exact transcript. More like a setlist, really, or a business plan, and we all know how long they last in the wild before they become unrecognisable!

Hi there!

You know how it is when you get a phrase in your head and you can’t get it to go away? Kind of like the songs which keep coming back to you, I mean; just a few words that have a good sound, and a few images and impressions which you associate with them, and they become a nucleus for a web of notions and hunches and intuitions: a low-flying flock of ideas.

When J-P asked me if I was interested in coming to speak with you all, this phrase was the one which kept popping back into my head: “accidental journalism”. So I’m going to talk about a few ideas and a story or two: really, I’m trying to work out what the phrase means, because I think it’s important. It’s going to be more philosophical than technical, but I hope you’ll indulge me.

I was at News Innovation: London on Friday, which was an unconference for journalists and people working in these kind-of para-journalism spaces. One of the big themes there was that people are really, really worried about how you fund investigative journalism in the future. Investigative journalism is, in a way, the counterbalancing part of the social contract newspapers have had: they had a pretty-much artificial monopoly on local advertising, but they paid for that, in a hippy karmic kind of sense, by being watchdogs and the trusted information brokers. Now the advertising model’s toast, Craigslist and eBay and Gumtree have seen to that. Maybe Craigslist’ll start sponsoring journalism: they seem pretty socially conscious, and weirder things have happened. But it’s not exactly something you can rely on, is it?

So this really important pillar of society has been a happy accident. There’s that word again; accident. We’ve been relying on serendipity, and our luck’s running out. How do we engineer that kind of good fortune back into our future media?

And you might wonder how we fit into this; my colleagues and I, we’re not exactly traditional journalists. There’s three of us behind Timetric, Dan Wilson, Toby White and me; we all come from science backgrounds, we met doing postdocs designing markup and building data management systems for really absurd volumes of output from quantum mechanical simulations — for us cache is something you find in CPUs, not something you find in dodgy brown envelopes handed to MPs in darkened alleys. One of us isn’t even into the idea of liquid lunches, and that’s definitely a disqualification.

So it does seem kind of unlikely, even given that on top of all the programming you’d expect us to be doing, Dan’s a really really sharp designer (though he’d hate me saying that, it’s true, he’s great) and I’m a full-on obsessive ex-student-radio media-geek, that we’d be the kind of people to do anything about this problem. What we’ve been trying to do is build a really good platform for managing numerical data on the Web. But when you think about that problem, well…

First up, what kind of numbers are really compelling? The ones which give shape to the stories in our lives, is the obvious answer; fuel prices, exchange rates, crime and employment statistics, inflation. And with these, the thing is, their absolute values aren’t as interesting as the trends they represent: whether prices are increasing or decreasing, whether employment’s going up or down. And anything you’re recording is going to make more sense if you can compare it against these big indicators: numbers don’t exist in isolation. If your salary goes up three percent, but inflation’s up four and a half, you just got a pretty substantial pay cut.

So you’ve got to track the history of these numbers over time. You don’t want to build a platform for managing numbers; you want to build a platform for managing, comparing, sharing, recording, and building models on time series, and you want to fill it with data people care about: from the Government, the European Central Bank, the US Federal Reserve, really wherever you can get it. And you want to make it really easy for people to put their own data in.

So we built Timetric. It’s at timetric.com: here’ss a quick video showing you some of what you can do with it.

Another thing: a surprisingly big cultural/zeitgeisty thing right now, and over the last couple of years, is that being into data’s becoming, well, cool. Being into data is big, big entertainment business. One of the biggest online entertainment businesses is nothing much apart from massively multiplayer Excel.

And I don’t mean Warcraft. Never played it, cos I suspect if I had, I wouldn’t be here, I’d be locked in a shady corner of a darkened room babbling about orcs. I’m talking about fantasy football and fantasy baseball. Anyone else here ever played them? The basic mechanic is that they turn stats into points, and we all know points make prizes, right? So you wind up playing a numbers game, trying to work out trends and predict performance. It’s applied statistics gone redneck. It’s great.

And fantasy sports techniques have been bleeding back into real football and real baseball. Here’s a site called FootballOutsiders, which I really, really like: they’ve been breaking down and analyzing the tapes of NFL games so they can get better data to feed into their models. They’ve got some fantastically insightful stuff, and it’s pressuring the mainstream sports journalists to get more in-depth in their analysis: less cliche, more thought.

Baseball’s even further down that road: there are whole teams built around what they call “sabermetric” principles. There’s a book called Moneyball which is meant to be great: I’ve got to admit I haven’t got round to it yet.

And speaking of baseball, one of the guys behind Baseball Prospectus is an economist called Nate Silver. He applied his economics training, and the models he built predicting how baseball pitchers would develop, to predicting how states would vote in the last US election on his site, fivethirtyeight.com. And he went from zero to Colbert Report guest in about two months.

And I haven’t even mentioned Freakonomics yet. Or the Armchair Economist. Or Bad Science. Everywhere you look, this is happening: a lot of the coolest, best new insight is coming from data-analysis. So for all the talk about geek chic, data’s where the action is. Or, at least, that’s what I tell myself at two in the morning when I’m staring at another bloody spreadsheet.

But, seriously: I think there’s something deep here, and that’s that if you can make it fun, make it easy to look at data, people will look through it, and they will find stuff. What you’ve got to do is make it really quick to go from first hypothesis — “I wonder if this is correlated with that?” — to first test. If it takes thirty seconds, you’re in there; if it takes ten minutes, or an hour, only the most obsess… um, dedicated, will get there.

So that’s a kind of quantitative measure of goodness. How much effort is it for people to ask a simple question of the data you’re making available? I used to be a theoretical chemist and a hillwalker, and I’ve seen this kind of thing a lot. The barrier here’s how much effort it is to ask something; make that low and you’ll get lots more questions, some of those will have interesting answers, and hey presto, by accident you’ve engineered a lot more journalism.

There’s those words again. It makes some kind of sense too. Was anyone else at OpenTech? It’s like something Ben “Bad Science” Goldacre said there: he was talking about building a ’shits and giggles’ economy in his ongoing war on the stupid and the venal, and if you can’t pay for journalism with cash, you’d better make it fun for people to get involved and really easy for people to share what they’ve done, so they can get the kudos, the respect of their peers, and the attention of their admirers.

If you can make it playful for geeks too, you’re on a big winner. You need to have good APIs, and we reckon we do; we’re on the RESTy end of things, and you can do things like take these little embedded microcontrollers from ARM, hook them up to sensors, and have them post data at series on Timetric to log it in real time. We think there’s a bunch of interesting potential applications there; if you want to know more, grab me later. Anyway, that was a bit of a digression, so moving onâ€¦

If you’re going to think about the antithesis of fun, though, government websites would be high on the list. Take the Office of National Statistics. Has anyone here actually tried to use their website? A few of you, then. It’s painful. It’s about the opposite of playful, whatever that is. It makes you want to avoid asking questions. Maybe that’s the effect they were going for â€“ it makes it really, really hard to ask questions. It’s weird: I kind of look forward to using that website, because it feels so, so good when you stop and go and look at something, anything else.

Have a look at this. It’s kind of comic, really, how hard it is to get a relatively simple bit of data like headline inflation out of this website; I’ve not counted the number of clicks, but it’s a lot. And once you’ve got there, you get a HTML table, which you’d need to chuck into a spreadsheet, then turn into a graph, then upload that graph somewhere, and only then can you put it into your blog or whatever.

That sucks. That really, really sucks.

So a while back we reverse-engineered the file format beneath all of these things, and uploaded the data to Timetric, and worked out how to break down the titles and glossaries associated with the data to put good tags on it, and once you’ve done all of that, you get a search interface like this.

[video]

And with this it’s really, really quick to grab some data, compare it, ask new questions, generally just mess around and test theories. One thing we did was work out how much lunch costs. We like cheese sandwiches, and the government publishes how much the ingredients — cheese, bread, butter — cost, because they’re part of the basket of staple foods that the government uses to work out inflation. And using Timetric, you can find the data, graph it, compare it, put in a formula which will be kept up to date whenever new inflation data comes in, and here you are; a graph of how much it’d cost you to make yourself a nice cheese sandwich, going back to the early seventies.

That’s fun! And it’s useful, because how much a cheese sandwich costs is something which anyone can relate to. It’s a custom index, like the FTSE 100 is an index. And the implications of a platform where you can aggregate all the models that people build and mine them for the relationships they reveal between concepts is really exciting.

But it turns out that a lot of the playfulness, the secret sauce which makes it fast to find the data you want and do something with it, is in the metadata, and that’s not a phrase I ever expected I’d be able to say straight-facedly.

So we’ve built this thing, and it turns out that our first customers were the Guardian. We like them a lot, and not just because they pay us and sponsor events like this, though gotta say, they do have good taste. We’d been speaking with them for a while, but it was at the Rewired State event at their offices in early March — tagline: The Government isn’t very good at computers. They spend millions to produce mediocre websites, hide away really useful public information and generally get it wrong. Which is a shame. — and it’s hard to argue with that, where things started really moving forward. We’re making the data in the Guardian Data Store, which is essentially a collection of Google spreadsheets filled with data curated by Guardian journalists, more fun and more appealing. So we’ve been doing a few things with them, and it’s been going well.

But, as it happens, the first story we did with them was one we cooked up; no pun intended, but I did promise you drugs. So here’s a story of a real piece of accidental journalism which happened through Timetric.

Part of my job’s getting interesting data into Timetric. We’ve been writing about bits of it, as and when, on our data blog, Byline – that’s at byline.timetric.com. One of the bits of data the Home Office were pushing out looked particularly juicy: drug seizure data. The number of drug busts and the purity of the gear lifted by Customs and by the police. I had to get that into Timetric, alongside some street-drug-price index data from the EMCDDA, the pan-European drug monitoring agency.

So from an email to an Excel spreadsheet to Timetric, and then I wrote a blog post about it;Â here that is. Maybe the most compelling graph in it was showing the precipitous fall in the quality of cocaine on the streets: that was a nice little story buried in the data, and it was much easier to find once I could upload the data and click around in it to see what we might be able to find. Gary Penn, the games journalist, talks about the “toyetics” of systems: that’s an idea I picked up from Matt Jones of Schulze and Webb and latterly Dopplr (http://www.slideshare.net/blackbeltjones/interesting2007?src=embed). By making the data into a toy, making it playful, we let the story emerge from it.

Anyway, we were visiting the Guardian later that week, and Simon Rogers, a journalist there, was rather taken with the tale, and put it up on the Data Blog. We were really excited. A story we’d originated on a national newspaper! Using our software! Really cool. That was a good Friday. Here’s what it looked like on the Guardian.

And then it got weird. Vice Magazine picked the story up. Seriously. And they connected it to a pet theory I’d been carrying around for years: the hidden links between how fast and aggressive dance music is and what the drugs are like…

…and the following Monday, this remarkably similar story appears on the BBC; except they’ve got more recent data there. The journalist there said he hadn’t seen our story, and to be honest I believe him, but that’s not to say that someone else didn’t see it and suggest that it was worth looking into. Even if it was a coincidence, we beat them to press by three, four days, and we were only being journalists by accident!

Accidental journalists. Serendipitous journalism. And that’s where we came in.

So here we are. We’re three erstwhile academics with a thing about data, based in a small office in Cambridge, and through writing systems which make it easier and more fun to play around with datasets, we’re helping national newspapers explain and break news stories.

That’s tremendously exciting. Really motivating. And I reckon everyone in this room’s got stories in them waiting to get out, either by giving them the right tools or you all going out there and building the right tools. Of course, I hope you’re motivated to go and have a play with ours, record and upload some data, beat on our APIs, play around and see what you can find. Stick what you’ve come up with on a blog, tell us about it, and we’ll link to it. And through that, together maybe we can find some interesting things.

Cheers for your time, and if you’ve got any questions, grab me later and I’d love to talk with you all. Thanks very much!