Up in Ontario

What Wisdom of Crowds?

For Friday, for the Nerds out there, a bit of an amateur tall-forehed post on how this Intermaweb thing is shaking out.

One of the blogs I read on a daily / weekly basis is Bubble Generation run by Umair Haque. It's a great big-ideas, big-pictures view of what's happening in online business strategy and the future directions of the business that generally pays the wages here at Up in Ontario central. Today I read through Umair's posts and came across a short one pointing to another post questioning how much wisdom there is in Digg - an online news / stories aggregator where users get to submit stories and then vote up or down a story's importance (see Digg).

The audience of Digg is mostly made up of online technology geeks and the stories submitted and highly rated reflect this. Some of the stories on Digg also are also derivative stories - the most recent posts on stories that may be actively evolving.

(Author's Note: Actually, stories are always just waiting to be told and to find their kernel of origin is like swimming upstream - you only rarely get to the source, and usually if someone tells you they came up with a story they're either lying or they've forgotten the stories that allowed them to tell their story. Just like I'm doing right now, most stories are retellings of other stories.)

On Digg a few weeks ago a story about stealing a piece of code became very popular. Many people jumped on the story; conclusions were drawn, the story was passed on. Essentially a big game of digital telephone resulted, with the same result that telephone always produces - the story at the ends has changed from the story at the beginning. In fact, more people picked up the story in mid stream, so they never even knew about its preceding versions, some of which contained more information about the source of the code theft. In the end (and of course, this is false too, since it is not the end of the story, it's just a way for me to bracket the story to make it tellable) the wrong person was fingered as the thief of the code. Yawn. But stick with me here.

The most popular method for people to interact with websites is through forms - those check boxes and radio buttons we see all over the place with the Submit or Vote buttons. Forms create machine-readable outputs, essentially numbers that computers can interpret to do things with. Readers of Digg rate stories through forms that Digg computers then interpret and ascribe meaning to - some stories get rated highly, some poorly, and the resulting rankings change the layout and access readers have to the stories. But a problem exists with this method of valuation.

In philosophy this problem is called Argumentum ad populum, or the bandwagon fallacy. Simply put, the volume of votes do not have anything to do with the accuracy or veracity of a claim. (See wikipedia entry on Argumentum ad Populum) I've been thinking about this problem for a long time since the most popular forms of value definition on the web are simple multiple-choice voting, and I've been wondering how to improve it while keeping it machine-readable so it scales up to the huge size required of the web.

So what's better than voting through forms? There's voting on the helpfulness of the votes, such as Amazon does with its reader reviews. But for a big, better way to do it, I don't know yet, but I know there are smart people working on it. Soon enough we may see value judgements derived from natural language analysis (the simple text like this that we all write in) - the same way that we see search engines derive value judgements for relevance from web pages and links. We may also see an open reputation standard evolve similar to eBay reputations, though that again opens the same problem as a fallacy of argumentum ad populum.

So this will become a bigger problem over time. Today it seems to me that we're asking computers to do something that they're not very good at doing, except in dead-simple ways like Digg, so we'll continue to get dead-simple outcomes. Just keep in mind that the outcomes may not be as simple as they appear.