A Million Heads ...

This is one of those days when the blog is about 15 on my to do list, but I'd be unforgivably remiss if I didn't point my readers a post Josh Catone published on crowdsourcing at the Read/Write Web blog yesterday. This in itself isn't unusual; every week or so someone writes a primer on crowdsourcing for his or her respective audience. More power to 'em.

Catone's addition to this growing and, until now largely redundant literature is noteworthy, however. It marks a departure from posts that essentially restate the thesis of my original article using the same old examples. For one, Catone divides and conquers, illustrating how the crowd produces value in a number of different ways. In this it's reminiscent of another excellent post from Sami Viitamäki, a grad student at the University of Helsinki doing his thesis research on crowdsourcing. (I won't adumbrate Sami's in here, but please go read for yourself if you haven't already.) Both posts are evidence that we're beginning to advance our understanding of the phenomenon. Titled "A Million Heads are Better Than One," Catone's essay is an astute examination of a few of the most significant and often overlooked examples of crowdsourcing. Here's what caught my eye:

Catone notes that Google's Pagerank itself is an application of the Wisdom of the Crowds. I haven't dealt with Pagerank in any sort of depth way because it's not crowdsourcing, per se. But it is a case in which a company is deriving an enormous revenue from a function of the crowd—namely, the sites to individuals choose to link on their own blogs. I'm always surprised how rarely it's viewed in this light. Here's Catone:

Perhaps the best exponent so far of Web 'wisdom of crowds' is Google, which organizes websites based on how they link to each other. Google sees links as votes for the relevance of a page. It is of course more complicated than that, but one can make the argument that Google works by utilizing the wisdom of crowds to determine which websites are the most relevant.

I don't think there's an argument to be had; that's wisdom of the crowds by any definition.

Catone divides Crowdsourcing into three sub-categories: Creation, Prediction, and Organization. This is slightly different than the divisions I'm using in my book, but it's close. His creation examples are those readers of CS.com already know well—Cambrian House and CrowdSpirit. Even here Catone ferrets out the overlooked nugget:

It appears possible that people are actually making money from Cambrian House. Their website has a calculator that estimates that with "good" growth, a person with 100 royalty points (the amount you receive for coming up with an original idea) would make $153,600 over a three year period. Not sure exactly what constitutes "good growth" though...

Although Cambrian House crowdsources the conception and creation of its web products, ideas are subject to editorial review by a core team and actual production is subject to a set of quality guidelines. In the case of conflicting code or design contributions, the community decides which is the best.

Cambrian House is always a great case study for examination, not least because CH is always fiddling around with their community production model, and tend to put pragmatism and efficiency ahead of any conceptual consistency. (Which, if it's not clear, I think it great. A theory's just a hypothesis unless it works on the ground as well as on paper.) I haven't looked into CH's books lately, and I love the bit about what 100 royalty points nets you. All eyes will be on CH in the next year or so to see how these products pan out for its community. I think the company gets a bad rap for somehow exploiting the sanctity of open source development. My cautious prediction is, "hogwash," but then the proof will be in the pudding.

My favorite bit in Catone's post are the conclusions he reaches:

1. Crowds should operate within constraints. To harness the collective intelligence of crowds, there need to be rules in place to maintain order.
2. Not everything can be democratic. Sometimes a decision needs to be made, and having a core team (or single person) make the ultimate decision can provide the guidance necessary to get things done and prevent crazy ideas and groupthink from wreaking havoc on your product.
3. Crowds must retain their individuality.  Encourage your group to   disagree, and try not to let any members of the group disproportionately influence the rest.
4. Crowds are better at vetting content than creating it. It is important to note that in most of the above projects, the group merely votes on the final product; they do not actually create it (even at Cambrian House, where the group collaborates to create the product, individuals are still creating each piece on their own and the group votes on whose implementation of an idea is best).

These dovetail neatly with what we're trying to implement on AssignmentZero: A hyrbid (aka, "Pro-Am") system in which the worst instincts of the crowd are mitigated by a layer of checks and balances administered by a skeleton crew of professionals, paid and un-paid.

But the concept I'd like to give the most emphasis to is Catone's third rule, that crowds must retain their individuality. This is Wisdom of Crowds 101, a principle which is clearly stated in James Surowiecki's book of same name.  Ask a classroom of people to guess how many jelly beans are in the jar: If they come up with an answer together, it'll be off by a mile, but have them write their answers individually and the mean will (more often than not) beat any individual answer.

This concept—that a crowd of individuals is smarter than any individual within it—is lost on a subset of crowdsourcing's critics, who persist in believing crowdsourcing is equatable somehow with groupthink.  Hogwash. That danger, as Catone points out, is there, but my prediction is that as crowdsourcing matures we're going to see better and better methods of keeping it in check.