So, then. Why Internet economics and big data and cities? There’s a lot of credit — or blame! — to go round, but a little bit of it has to go to Russell Davies’s conference-planning.

That’s me on the right there. I hadn’t been to any conference quite like Interesting. Still haven’t. More or less every decent idea I’ve had since contains a dash of something I first encountered there.

Here’s Matt Jones, who finished off the day. You know that moment of giddy dislocation you get when one of those Magic Eye drawings kicks in? That’s how I was feeling, and this talk capped it off.

Matt Jones at Interesting2007 from russelldavies on Vimeo.

If that wasn’t enough, not long after Interesting, I took a trip to San Francisco. I wasn’t long out of my graduate studies, and I kept finding myself drifting back to thinking about, well, the kind of things I’m working on now. So that’s one of the ways this story begins.

I trained as a mineral scientist, not as a designer. So when I start to approach these ideas I use what I know, and two of the things mineral physicists think about are ordering and phase transitions. (My thesis even had “disorder” right there in the title.)

Phase transformations happen when symmetries are made and broken. Things melt and recrystallise. How are cities melting and recrystallising?

I can’t really talk about the design issues, but I’m betting that cash and computation will be at the heart of the physics of it. Here’s one thing which which connects the two. Flash memory’s getting seriously cheap very fast. So fast that you’re getting exponentially more storage for your money as time passes — hit the log-scale checkbox on the graph below to see.

Admittedly, cheap storage alone isn’t a story: hard discs are still cheaper per byte, after all. But flash memory has some advantages — it’s lower-power, more robust and smaller. The kicker, though, is that reading data in a random order off flash storage is way, way faster than reading it off a disc. Hard discs aren’t all that far evolved from vinyl records. There’s a motorised read head instead of a stylus, but it hovers over a spinning platter, reading the data off as it hurls past. So if your data doesn’t line up neatly — it’s not contiguous — you’ve got to winch the read head back and forward to read each piece of it in turn, and that’s really slow.

Databases, in particular, have had a lot of logic devoted to laying out data on disc just so. You don’t need to do that at all if you’re writing your data to flash. That lets you focus on different problems instead, like being able to insert data into your database really quickly.

If you’re going to put a sensor in everything, you need to be able to write a lot of data to a database really quickly.

Make faster databases and you can record and query data you couldn’t use before. What’s more, because they’re tiny and can take a beating, flash-backed databases can go wherever environmental sensors go. I’ve got a hunch that smart flash-based storage is the missing part of the whole ubiquitous-computation shtick: cheap flash gives you staging posts for the instrumented street’s data.

Bandwidth, computational power and storage. When you get them in places you haven’t had them before, or when they get a lot faster or a lot cheaper, or when you can combine them in new ways, you break symmetries. That’s where I think the action is.