Critiquing Crowdsourcing

Earlier today a college student named Tabish Bhimani asked whether I had "restricted [my] opinion on crowdsourcing to its benefits." Daren C. Brabham, a University of Utah graduate student and occasional Crowdsourcing.com contributor, wrote an interesting comment on the subject and while I started to reply in the comment thread in which Tabish asked the question, I realized the thread might  hold interest for a wider audience. So, without abbreviation (or editing!), I upgraded the thread to a post. (Ah, the pleasures of administrator access!) Here's Daren's comment, followed by my response:

Daren:

@Tabish: Plenty of criticisms out there on crowdsourcing, particularly in its potential to exploit creative labor for little reward. Also some criticism along digital divide lines, in that we can't assume the crowd has a diversity of opinions unless we've measured them and find the crowd is more diverse than internet access data indicate. Otherwise we have to change our theories. Shamless plug: I have an article coming out in the journal Convergence in February which addresses some of these potential criticisms. Email me for a pre-publication copy: daren.brabham utah.edu

@everyone: Another shameless plug--MIT media scholar Henry Jenkins mentioned crowdsourcing in his recent keynote lecture here at the University of Utah on participatory culture. Crowdsourcing mention is at 36:50 in this podcast: http://www2.utah.edu/podcast/category.php?id=6 Jenkins also did an encore of this lecture a few weeks later at the Association of Internet Researchers conference in Vancouver, with the same mention of crowdsourcing.

Jeff:

@Tabish: Funny, I was just discussing this with a friend yesterday. I do think this blog has been a little thin on robust critiques of of crowdsourcing. I wouldn't use the word "restricted," however, as that implies a conscious attempt to ignore any emerging critique. The comment I made to my friend, in essence, was that so much of my labor these past 18 months has revolved around simply gathering and analyzing examples of crowdsourcing in action that it's been difficult to find the time to develop critiques on my own. And I'd disagree with Daren that there's a lot of literature out there, which is exactly what makes his own efforts so valuable. I imagine in the next year or two a lot of critical material will emerge, but so far what we've seen is a lot of Gee Whiz coverage in the mainstream press that essentially just recaps the cases I examined in my original article.

It's important to remember that the migration of peer-production, or open source, models of economic production has just begun. In fact, as I enter the final stages of writing my book, the most prominent critique that comes to mind is that while we can say without any doubt that crowdsourcing thrives in the wild, it's also evident that it's hard to recreate artificially. By that I mean that our purest examples, so to speak--threadless.com, istockphoto.com, topcoder.com--emerged organically from existing communities.

There are exceptions to this: Dell did a wonderful job of soliciting product development ideas and (crucially) acting on the suggestions (It's going to release a Linux laptop in response to consumer demand), but there are also abundant examples of what I would call "limited successes," such as Current TV, which has been disappointed by the number of viewers submitting content to the network, and our own Assignment Zero. While we produced some wonderful work, and we certainly were able to recruit a number of contributors, we underestimated the massive administrative labor that would be required to properly leverage the initial enthusiasm we received. (And those weren't the only mistakes we made! Here's a longer list in a piece I wrote for Wired.com, which counts as a critique of crowdsourcing I guess.)

@Daren, et. al.: I remain, well, let's call it "cautiously skeptical" about the exploitation critique. I dealt in a bit greater depth on this issue in a June blog post, but since writing that I've done a great deal of reporting and an even greater deal of thinking and, if anything, my initial position has hardened: Crowdsourcing is enabled by communities, and communities are held together through shared passion. I just can't square that with any concept of exploitation, per se. iStockers are stoked to get paid anything for their work; ditto Current contributors; ditto Threadless designers.

That said, I still emphatically agree that it's a healthy addition to the discourse around these rapidly developing participatory models. One can envision a day when the cost of various forms of content drops so low that it becomes a buyers market. In photography, that's already the case. But so far it just doesn't resemble anything I think we can call exploitation. Are people like Mark Zuckerberg getting rich off user-generated content? You bet. I personally don't have a problem with that. I really enjoy the five or ten minutes a week I spend futzing with my Facebook page. Zuckerberg created a neat service, and in response people used it. Now he reaps the benefits of his invention. That's the way the market works. But then, I haven't read Daren's paper yet, and he may well have uncovered some cases of which I'm still ignorant.

Regarding diversity, I really couldn't agree more. It's important to draw a distinction here, though. What I would call "group intelligence applications" are really a distinct genre of crowdsourcing. Should we be concerned that Threadless draws a fairly homogenous demographic to its community (I'm guessing here, but I bet it's young, fairly affluent and skews toward whites and Asians). I would say not. But does that same homogeneity reduce the usefulness of Digg? I'd say yes, or at least to someone like me, whose interests include stories on the visual arts and American history. And when it comes to strict group intelligence applications like prediction markets, the lack of diversity is a downright monkeywrench in the works. (A few of you have asked me for a reading list. Here's one recommendation at least: Scott Page does a superb job of examining how diversity fuels models like prediction markets in his book, "The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.") If everyone participating in a prediction market has had similar experiences and training, it won't provide any advantage over just asking one of the participants to predict an outcome.

At any rate, there's much more to say on the subject. Here's hoping all of you will say some of it here.

Comments

[Daren C. Brabham]

On diversity in the crowd: A lack of diversity in the crowd has at least one major potentially negative consequence. My scholarly interests point to theories that seem to argue that the best stuff gets made by diverse sets of people. The problem here is that if we make a slippery application of these theories and claims on crowdsourcing models, we may end up cherishing the greatness of such innovative, diverse, and democratically-produced products and ideas without realizing that those products and ideas were made by wealthy whites. Thus, we could end up replicating white aesthetics and white worldviews, at the expense of other viewpoints, and get away with marketing it in a thin veneer of "diversity and democracy." I guess that's some hybrid critical race/Marxist critique. And if/when (I say "when" because Peer-to-Patent is already testing the waters) crowdsourcing takes hold as a model for efficient and innovative government/public business, this reproduction of dominant interests with the veneer of democracy/diversity becomes troublesome on whole other levels of democratic theory.

On the exploitation of the crowd: I agree, too, that this isn't pure exploitation. We know VERY little about what motivates the crowd to do the work it does (and, since it's Shameless Plug Day, this question is the point of my iStockphoto survey which is concluding soon). But, assuming the data show us that the crowd is motivated by a variety of factors and their attitudes toward their work do not resemble sweatshop laborers, it's still an item of interest for critics. As overhead diminishes, and as "wages" for labor decrease, someone or some corporation gets extremely wealthy. WHERE the locus of this inequity stems from is the object of concern. If people do what they do because they like to do it, we have to investigate the technology/architecture that enables these unequal relationships between producers and profiteers. Anyway I'm off to go teach...see the many existing critiques of the "invisible hand of the market" for a better critique of this part.

[Alan]

Hey Daren how goes it? Your comments prompted a buzzing in my head.

“We know VERY little about what motivates the crowd to do the work it does. But, assuming the data show us that the crowd is motivated by a variety of factors.”

The list of factors is only limited by the ability of the examiners to include not only questions of race or gender, diversity and democracy . . . . . . . but questions of a deeper nature.

Any outcomes arrived at without at least considering larger spans of cultural history, societal shifts and individual biographies are going to be rather one dimensional.

What are the roots of personal motivation, not an easy one to pigeon hole, from whence arise the complicated threads that work in and through us as individuals and guide us to transformation or revelation?

Jeff touched upon a crucial element worthy of some dissection, “crowdsourcing thrives in the wild, it's also evident that it's hard to recreate artificially.” Thriving in the wild verses re-creation has always been a quandary for those who are looking back. Those who create whilst moving forward have no such problems.

As our technological culture matures the manifestation of any particular trend is shaped and defined by such advances and new terms arise to capture and name them!

Just that question alone, the complicated and tangled personal and professional biographical threads that led Jeff to crowdsourcing and the terms creation is worthy of at least some examination. As much as it’s difficult, the individual microcosmic view, can lead to insights that on a macrocosmic level are very difficult to arrive at.

That’s why I like the idea of an ethnographical approach, a phenomenological take on technological, cultural, social phenomena’s and individual destiny as part of the picture/process. Problem is that that points in the direction of a life long study rather than synopsis like characterizations.

Hasn’t there always been (CS?) from hunter gatherer to . . . . a rose by any other name . . . . . . . . . . .

Cheers, Alan

[Alan]

This is new and in my mind an interesting aberration of social networking.

Nicholas Carr’s blog:
Mugshots will no longer be limited to criminals and celebrities.
http://www.roughtype.com/index.php

[Alan]

Here is an interesting look at online collaboration.
Alan

THE PARTICIPATORY CHALLENGE
Trebor Scholz

http://www.collectivate.net/the-participatory-challenge/

[Wouter]

Very interesting discussion in many ways. It's always healthy to take a step back and (allow) critiques on any phenomenon. Three lines of thought from my side:

A critique that seems to refer somewhat to the one of Daren Brabham related to diversity, and in that sense the 'globalizing and flattening' world holds a risk of people being so 'wonderfully understanding' of each other that opinions and ideas converge, hence threatening the very diversity that should be cherished. Maybe this 'risk' seems far-fetched for now on a global scale, but it is a danger that needs ro be acknowledged (see my blog, 12th of October).

Secondly, it could be worthwhile to have some insight into the origins of the 'crowd' that is participating. It has been suggested that this democratic decision making is superior, but this quite depends on who is providing the input and with what motive. Blindly following the crowd's opinion is not necessarily the best way to go, if there is no knowledge about this crowd (blog, Post 7th of January).

Thirdly, it was interesting to observe that when I mentioned the term 'crowd sourcing' in a lecture recently, some students were flabbergasted that such a thing would even exist and severely doubted why any consumer would want to be a part of that. After some explanation, this 'horror' was somewhat mitigated. The concept that consumer input can very well lead to better products and that is the reward is not as common as one might hope.

Leading to the 4th thought: maybe the success and justifiable critique on crowd sourcing may very well depend on the percentage of cases where it has been tried out successfully where the results of the entire process in the form of better (and affordable) products has indeed fed back to the consumers.

[Alan]

Criticisms out there on crowdsourcing:

http://www.wired.com/techbiz/it/news/2007/11/fossett_search#

Cheers, Alan

[Alan]

Here is an example from the other side:
http://thefutureplace.typepad.com/the_future_place/2007/11/red-lake-crowds.html

Cheers, Alan

[Alan]

CrowdFunder: The Power of the Crowd in the Community:

Currently in beta in Boulder, Colorado, though hopefully soon to be rolled out further.

http://www.psfk.com/2007/11/crowdfunder-the-power-of-the-crowd-in-the-community.html

Alan

[Tabish Bhimani]

Dear Jeff and Daren,

I did not know that one simple question would be blown into an entire topic to cover on this blog. So for covering it, I thank the both of you. I look forward to Daren's article. It is indeed funny: The only part where I lost marks on my presentation was not discussing "the downsides for workers - loss of security, benefits, etc." according to my Tutorial Leader.

The part of exploiting creative labor was something that struck me most. As a graphic/web designer, when bidding on projects on RentACoder.com, say I bid $50 for a small project that would take me roughly 20 minutes, I will often get emails saying the buyer chose another bid for $4. That is shocking. But my question is this: Is creative labor being exploited? We need to take into account why the lowest bidder of $4 place such a bid in the first place. Is the individual new to crowdsourcing and thus wants to make a portfolio and a clientele which is more important than money at that particular point in time? Or do the $4 dollars mean to the coder, alot more money in the country he is from, for example, in Pakistan $4 are roughly Rs. 200 which can get you through the day easy. Or from the client side: What is the financial background of the client? Does he really just want a simple solution that takes 20 minutes as opposed to the relatively much more decent solution that I have to offer?

Jeff, you said, "iStockers are stoked to get paid anything for their work; ditto Current contributors; ditto Threadless designers." This comes down to the basic principles of crowdsourcing: the public which may be hobbyists as you said in the article. They do not mind getting paid anything, to them its just an accumulation of some extra disposable income. But can we really limit crowdsourcing to just that and not have professionals or semi-professionals (for lack of better words) get paid what they deserve? Businesses run on essentially the motive to maximize profits and minimize costs. For them, this is just a great way of doing just that: economizing. The public is willingly consenting, however, to this and so it cannot be exploitation. They want to see themselves (or their work) on T.V. and not YouTube (as T.V. has come to be known), but the real, actual, physical T.V.

For the record, I am a student in my second year in York University, Canada studying Communications. How film, information and technology, and advertising effect society are key to my studies. I do web/graphic design on the side (and I am crowdsourced alot). I am from Karachi, Pakistan.

Thank you once again, Jeff and Daren.

[Monica Hamburg]

I have been thinking about the concept of exploitation since I started reading this blog several months ago. I have finally posted my thoughts on the topic:

http://monicahamburg.wordpress.com/2008/02/25/please-sir-can-i-have-another/

[Nursing gowns]

crowdsourcing has really reached almost everyone's ears and i believe it would have positive impact in new technology.

[Elite Fat Burning Guide]

A critique that seems to refer somewhat to the one of Daren Brabham related to diversity, and in that sense the 'globalizing and flattening' world holds a risk of people being so 'wonderfully understanding' of each other that opinions and ideas converge, hence threatening the very diversity that should be cherished. Maybe this 'risk' seems far-fetched for now on a global scale, but it is a danger that needs ro be acknowledged (see my blog, 12th of October).

Secondly, it could be worthwhile to have some insight into the origins of the 'crowd' that is participating. It has been suggested that this democratic decision making is superior, but this quite depends on who is providing the input and with what motive. Blindly following the crowd's opinion is not necessarily the best way to go, if there is no knowledge about this crowd (blog, Post 7th of January).

Thirdly, it was interesting to observe that when I mentioned the term 'crowd sourcing' in a lecture recently, some students were flabbergasted that such a thing would even exist and severely doubted why any consumer would want to be a part of that. After some explanation, this 'horror' was somewhat mitigated. The concept that consumer input can very well lead to better products and that is the reward is not as common as one might hope.

[vancouver internet marketing]

Crowdsourcing is a distributed problem-solving and production model. Problems are broadcast to an unknown group of solvers in the form of an open call for solutions. Users—also known as the crowd—typically form into online communities, and the crowd submits solutions.

[Louis Vuitton New Collection]

The part of exploiting creative labor was something that struck me most. As a graphic/web designer, when bidding on projects on RentACoder.com, say I bid $50 for a small project that would take me roughly 20 minutes, I will often get emails saying the buyer chose another bid for $4. That is shocking. But my question is this: Is creative labor being exploited? We need to take into account why the lowest bidder of $4 place such a bid in the first place. Is the individual new to crowdsourcing and thus wants to make a portfolio and a clientele which is more important than money at that particular point in time? Or do the $4 dollars mean to the coder, alot more money in the country he is from, for example, in Pakistan $4 are roughly Rs. 200 which can get you through the day easy. Or from the client side: What is the financial background of the client? Does he really just want a simple solution that takes 20 minutes as opposed to the relatively much more decent solution that I have to offer?

[Louis Vuitton Outlet]

Very interesting discussion in many ways. It's always healthy to take a step back and (allow) critiques on any phenomenon. Three lines of thought from my side:

A critique that seems to refer somewhat to the one of Daren Brabham related to diversity, and in that sense the 'globalizing and flattening' world holds a risk of people being so 'wonderfully understanding' of each other that opinions and ideas converge, hence threatening the very diversity that should be cherished. Maybe this 'risk' seems far-fetched for now on a global scale, but it is a danger that needs ro be acknowledged (see my blog, 12th of October).

[Navel Rings]

You are so generous! I just love finding beautiful and artistic freebies like this. They really brighten up my day, thank you!

[Free Quick Claim Deed Form]

Great post!the topic is really good,"The 'Living Principles' for Designing Our World".I got inspired by it.

[web designing karachi]

definitely crowd-sourcing is not a good option
it is waste of labour by paying them less than they deserve.

[Web Designing Karachi]

Hi,
A great article indeed and a very detailed, realistic and superb analysis, of these books, very nice write up, Thanks.

[monster headphones]

Beats studio headphones fold up for easy packing. The included touring case features rigid construction for enhanced protection.

[monster headphones]

Beats studio headphones fold up for easy packing. The included touring case features rigid construction for enhanced protection.

[florida cna test]

Is he going for another Pulitzer? A poignant January 12 image is rendered in anticipation of 0bama's well-received address to the nation from University of Arizona, though the hugging eagle seems a bit awkward, if only because it's not clear if he is looking at the viewer or down the back of the personification of Arizona. Either way, something ain't right:

[cna classes in louisville ky]

I take credit for chasing John off.

[cna classes in louisville ky]

close enough. I'm drinking.

No....my statement did not contain "for a magazine called Reason.."!!!

you lose all your cows!

[Web Designing]

I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs up