Chapter 7-What the Crowd Knows: Collective Intelligence in Action, Cont.

I'm in a mad dash for the airport. Palo Alto speaking gig today, then Amsterdam tomorrow, then all the way back to LA next week. This would all be terribly thrilling if I was 27. At 37 it's exhausting. Still fun. But exhausting.

Great comments from the peanut gallery, including a few deserved dressing down (How do I expect to elicit comments when I don't respond to them. I'd blame kids and book, but believe I may have played that card too often already.) At any rate, here's the continuation of Chapter Seven.

Investing in the Future—Prediction Markets

In 1988 the civil rights activist Jesse Jackson shocked the nation by winning the Democratic primary race in Michigan, an outcome that neither polls nor political insiders had anticipated. At the University of Iowa, a handful of political scientists and economists had a very pragmatic reaction to the failure of the polls to predict Jackson’s victory: They decided to build a better system for predictions. Over the course of the next several months, and well in time for the general election between George H. W. Bush and Michael Dukakis, they created the Iowa Political Stock Market. Investors could buy up to $500 in securities that paid out according to the share of the popular vote each candidate received. Shares were priced between $0.00 and $1.00, and were paid out in full. If you snatched up Bush shares when they were running at 55 cents, you’d have made 45 cents for every share you purchased.

By November the results came in. The average error among all the major exit polls in the general presidential election that year was 2.5 percent. Not bad. But the Iowa Political Stock Market did much, much better, predicting the outcome to within one-tenth of a percent. “While the laws of statistic govern opinion polls,” said Robert Forsythe, one of the market’s creators, “the invisible hand of Adam Smith makes political markets work .”

In fact the results were so impressive the university renamed the experiment the Iowa Electronic Markets, or IEM, and began allowing trading in a range of future events, from elections in foreign countries to Google’s market capitalization to the price of Microsoft stock on a certain date to upcoming decisions by the Federal Reserve . In the two decades since trading opened, the IEM outperforms the best polls. But why would this be so? Aren’t polls also tapping into the collective brain? Aren’t they too a form of crowdsourcing? The answer is yes, but that doesn’t make them as fine-tuned a predictive model as an information market like the IEM. Polls are crude instruments. The foolish and the wise alike get a single, equally weighted vote. Prediction markets offer considerable advantages over both polls and surveys as well as prognostications issued by experts, and are becoming increasingly attractive to everyone from the media to private companies to government. While they might differ in scope and methodology, prediction markets are no different than any futures market. Traders bet on the likelihood of outcomes involving presidential elections, say, rather than pork bellies. The price of a security in one of these markets generally reflects the collective prediction of how likely it is to occur.

Crowdcasting networks and prediction markets both utilize collective intelligence, but in very different ways. Prediction markets are essentially just machines used to aggregate information. That doesn’t mean that information markets don’t exhibit their own magical qualities. The same principle we saw at work in our example from Who Wants to Be a Millionaire applies here. Because foolish people can be expected, on average, to vote in random patterns, even a small number of astute observers can create accurate predictions. Unlike a simple information aggregation system like Millionaire, however, not all “votes” are judged to be equal. If an investor possesses inside information, they are likely to invest more money than people working off a hunch. As this book goes to press, Hillary Clinton shares in the Iowa Electronic Market are going for about 13 cents. (Meaning each future would reap 87 cents of profit if Clinton snags the nomination.) If a small collection of people knew that a big scandal was about to ruin Obama’s shot at the Democratic nomination, they would naturally load up on Clinton futures. To put this back into terms F.A. Hayek might use, prediction markets provide an incentive for people to reveal their privately held information. Likewise, the cost of investing in such a market provides an incentive to the ignorant to keep their money in their wallets.

So why do prediction markets beat polls? In a word, greed. Writing on the day before the 2004 presidential election, Salon staff writer Farhad Manjoo noted that while the polls put Democratic contender John Kerry ahead of President Bush by 1 to 7 points, traders on the IEM were favoring Bush. “What accounts for the disparity between the polls and the IEM?” asks Manjoo. The answer is that no one risks money in a poll. “Betting money on an election focuses the mind. … I’m supporting Kerry for the White House, but I’m betting against him on the IEM.” As we know now, that was a smart bet. Even while exit polls were showing a Kerry win, the IEM was predicting a Bush win. By midnight of the same day, the IEM showed Bush with 50.45 percent of the popular vote to Kerry’s 49.55 percent, an uncannily accurate picture of the final outcome.

Diversity too exercises an influence over the results of a prediction market, but it doesn’t trump ability. When it comes to prediction markets, diversity merely equals it. Scott Page again shows how this can be expressed as a mathematical theorem. Page’s “Diversity Prediction Theorem,” says that collective error equals the average individual error minus the diversity of the predictions. Page employs a lot of intimidating-looking formulas to prove this point, but the logic is pretty straightforward: If the variance between predictions is high—I guess 40 and you guess 60, but the answer is 50—the diversity of predictions cancel each other out, much as they do in the Millionaire example. As Page writes, when it comes to prediction markets: “Being different is as important as being good.”

Despite the IEM’s record of accurate predictions, information markets didn’t really become well-known until a political firestorm broke out over the so-called “terrorism futures market.” In May 2001 a project manager at the Defense Advanced Research Projects Agency issued a request for proposals on using markets to predict events such as terrorist attacks, coups and assassinations. One of the resulting $1 million grants was awarded to Robin Hanson, a pioneer in the science of prediction markets. Hanson, known as something of an unorthodox thinker, propounds such a deep belief in the efficacy of prediction markets that he has proposed a form of government—a “futarchy,” in his terms—in which all public policy is based on them. Working with a San Diego firm, Net Exchange, Hanson created the Policy Analysis Market, or PAM. The idea was that a select group of intelligence and policy analysis experts would try to predict the course of foreign affairs by investing up to $100 in such indexes as national stability (Will Kosovo declare independence?), economic growth (Will India’s GDP grow 10 percent this year?) and military readiness (If India invaded Pakistan, could its forces successfully occupy the country?) Unfortunately for Hanson and the fortunes of the prediction market, it also included a terrorism index.

PAM fell under the purview of the Terrorism Information Awareness Office, headed by an already controversial figure, the former National Security Advisor John Poindexter convicted of perjury and other felonies during the Iran-Contra scandal. Given the Congressional scrutiny devoted to Poindexter’s department (to say nothing of the charged environment following 9/11), what happened next shouldn’t have surprised anyone. In July of 2003 the news media caught wind of PAM. Betting on terrorism! In the Defense Department! Soon after the first article appeared, US Senators Ron Wyden (D-OR) and Byron Dorgan (D-ND) held a press conference deriding PAM for encouraging people to profit from terrorist attacks. Before dawn broke the following day the Pentagon had shuttered PAM for good, extinguished as well Hanson’s dream of a Futarchy. 

After a week as a punchline on the late-night talk shows, PAM was effectively swept into the dustbin of history. But many economists and political scientists defended PAM’s underlying premise—that markets could predict future events better than any single expert—and if anything its notoriety only helped accelerate the mainstream adoption of prediction markets. In the last few years the options for futures traders have expanded dramatically. The Hollywood Stock Exchange (HSX) provides a market for trades in everything from box office grosses to Academy Award results. The HSX boasts an impressive track record, having predicted more than 80 percent of all Oscar nominations (and that includes the more obscure categories, like “Best Sound Editing”) and never missing more than one top award since its 1996 launch. Markets have also sprung up around politics (Washington Stock Exchange) and current events (NewsFutures). There is even an academic journal—The Journal of Prediction Markets—devoted to the emerging discipline.

The private sector has been especially warm in its embrace of prediction markets. Companies use them internally in order to crowdsource the decision-making process in such matters as inventory, sales goals and manufacturing capacity. In the mid-1990s Hewlett-Packard and the Caltech economist Charles Plott devised a futures market to predict the sales of a range of HP products. Normally sales forecasts are generated by analysts on the company’s sales staff, but for Plott’s experiment employees were chosen from a variety of departments. The securities represented specific intervals in sales figures. If an “investor” believed the company would sell, say, between 201 and 300 printers in a given month, she would buy shares of that security. If she was right, she would receive one dollar for every share. The market turned out to beat HP’s official forecast for six of the eight products in which HP conducted the experiment. As a result, HP has since set up its own “experimental economics” group to conduct additional research into prediction markets. 

The markets outperform the experts because collectively they have access to far more data. “Quite simply, the central planners don’t have all the information that the dispersed salespeople collectively have,” writes MIT Sloan School of Management professor Thomas Malone in his book, The Future of Work. Google, Microsoft, Eli Lilly, Goldman Sachs and Deutsche Bank have all used prediction markets to help determine corporate strategy, and Malone himself conducted a successful experiment with the computer chip-maker Intel, in which an internal market was used to maximize manufacturing capacity, determining how many chips each manufacturing plant should produce in a given quarter. After some tweaks, Malone achieved a 99 percent efficiency rate, far superior to what Intel had reached by its traditional methods.

Such markets help companies adopt quickly to rapid change, notes Malone. “Because everyone has an incentive to trade as soon as possible to gain an advantage,” vital information is dispersed far more quickly. “Instead of having one group of senior managers sequentially working through a single set of options,” he writes. “Many people can be simultaneously exploring lots of possibilities.” In 2006 Malone founded the MIT Center for Collective Intelligence, which is currently attempting to use prediction markets to crowdsource the viability of solutions to intransigent problems like health care and climate change.

Nothing drives innovation like robust demand, and companies offering ready-made software platforms for prediction markets have recently sprung up. Inkling Markets, a Chicago-based outfit that allows anyone to create their own prediction market, has a client list that includes Cisco, video-game maker Electronic Arts, Chrysler, tech publisher O’Reilly Media, Wells Fargo, the universities of Indiana, Oxford and Stanford and even the Los Alamos National Laboratory.

The problem is that most of these markets lack the key ingredient to a prediction market: the use of real money, which would violate prohibitions against gambling. If a trader doesn’t stand to lose or gain anything, there’s no incentive to reveal his or her “local knowledge,” or private information. While the IEM operates under a special exception from the Commodity Futures Trading Commission, the other prediction markets use virtual dollars. And that, economists agree, is a problem. People are animated to act by a complex welter of motivations, and financial rewards don’t always figure high on that list. And it’s true that participants in markets like the Hollywood Stock Exchange say they are compelled to play because of the competitive aspect. In crowdsourcing terms, we would say it’s because it enhances their reputation. But for a prediction market—which unlike, say, iStockphoto or Threadless.com, doesn’t tend to spawn a rabid, tightly-knit community—the promise of enhanced reputation is a shaky foundation on which to build a house.

Prediction Markets have also shown a tendency to suffer from the same maladies as those that affect all other stock markets—fads, information cascades, and bubbles like the one that saw tech stocks reaching dizzying, and unfounded, highs in the late ‘90s. An even greater obstacle to establishing a prediction market—especially for small companies hoping to use one internally—is that the accuracy of futures trading is in relatively direct proportion to the “thickness” of the market, which simply indicates the number of traders buying and selling shares at any given time. As HP and Google have discovered, it can be difficult to convince a large number of employees to participate in an internal market in exchange for meager payoffs (Google allows people to invest virtual dollars in exchange for prizes like T-Shirts and gift certificates.) “As a result you get thin markets, where there’s not enough trading to effectively predict an outcome,” says Bernardo Huberman, the director of HP’s Social Computing Lab. “Secondly, a thin market can be easily manipulated by just a few trades.” One salesperson might game an outcome in his favor, for instance. Without a crowd, there is no crowdsourcing.

To correct for this bias Huberman has invented a method of counteracting the effects of a thin market, so that even a small number of traders—a corporate board, for example—could generate accurate predictions. Essentially, each participant answers a series of questions meant to evaluate their level of risk aversion. Those who throw caution to the wind are given a high rating, and those on the opposite end of the spectrum score lower. The positions they take in any prediction market are then weighted against their risk index. Huberman believes he’s cracked this particular nut with his system, which HP patented, and he says other companies (he won’t reveal their names) have begun licensing it. “I can imagine this being used in intelligence work,” says Huberman. “A lot of people with imperfect information about, say, Azerbaijan, could come together and make fairly reliable predictions about events that might occur there.”

Advocates of prediction markets are also attempting to change the gambling laws to allow investments in non-profit information markets, as long as they operate for small stakes. (Investment on the IEM is capped at $500 per account.) In May 2007 more than 20 eminent economists sent a letter to Congress and federal regulators urging them to create a “safe harbor” for such operations. “Using these markets as forecasting tools could substantially improve decision making in the private and public sectors and help manage risk more efficiently.”

So far we’ve treated crowdcasting networks and prediction markets as wholly distinct phenomena. And from the perspective of collective intelligence, they are. But theory plays a strange role when it comes to crowdsourcing—a form of economic production that forces us to apply theory to what is already occurring in practice. So it should come as no surprise that in the real world, some applications of collective intelligence are neither information markets nor problem-solving networks, but an intriguing hybrid of the two. Crowdsourcing doesn’t offer a collection of ironclad rules. Sometimes the best strategy is intelligent improvisation.

Comments

[alan booker]

Interesting, the function unintentionally, might be that the blog ultimately serves some other purpose that pure blog blog!

It does appear that intensive bloging is not your cup of tea Jeff. The comment from Joshua is certainly a fair one and if I may say so, well stated.

But as we all know, the signposts might be moved as the book comes to fruition and family dynamics change!

“Palo Alto speaking gig today, then Amsterdam tomorrow, then all the way back to LA next week. This would all be terribly thrilling if I was 27. At 37 it's exhausting. Still fun. But exhausting.” . . . . . . . . . . . . 37, you’re still a spring chicken Jeff. BTW what are the gigs?

The continuation of chapter seven leaves little to be picked at in my opinion other than this:

“So it should come as no surprise that in the real world, some applications of collective intelligence are neither information markets nor problem-solving networks, but an intriguing hybrid of the two. Crowdsourcing doesn’t offer a collection of ironclad rules. Sometimes the best strategy is intelligent improvisation.”

Intuitive improvisation?

The assumption that the hybrid might only consist of two traditional elements might be too narrow a viewpoint. At a future point in time when the present CS phenomena has matured and it’s effects are manifest in other markets one might have to define completely new concept/concepts, or it might end as just a blip on a small screen!

I would bet/predict that if there is ever a format where good money could be made using prediction markets, they will gain weight/thickness and Congress and federal regulators would probably change their tune.

The question of prediction market being weighted against a risk index might be shot if reckless financial involvement driven by greed pushed the concept into a mainstream market.

For what its worth, Alan

[stockpredictions]

Harness the power of collective intelligence for stock price predictions. Cash prizes. There is no charge.
http://www.fusedstockpredictions.com/

[kraloyun]

The question of prediction market being weighted against a risk index might be shot if reckless financial involvement driven by greed pushed the concept into a mainstream market.

[Nike Shox Rivalry]

"Because of the fundamental moral is a bad, though. You have some knowledge and skills, nor very useful."Do you think so?

[50mw laser]

For what? I can not see what i siad here.Don't make me feel dispointed.

[Health News]

Thank you for introducing me the wonderful information.And .....Totally boring.!

[Turbo fire]

Thank you for taking the time and spreading this information with us all. It was indeed very helpful and informative while being straight forward and to the point.

[chaussures femmes]

We have gotten many great comments from our customers and earn a good reputation in foreign makerts, more than 90% customers are satisfied with our products and service, till now our online members are beyond 80,000. As of right now, we currently serve customers from over 18 countries, and we are still growing. We really hope to expand our business through cooperation with individuals and companies from around the world.

[Damen Stiefel]

Ihr viel mehr behandelbar Weg von Pankreas-Melanom-Ansatz weiter verbreitet Art der Erkrankung, bei der, [url=http://www.uggkaufen.net/herren-stiefel-wholesale-26.html]Damen Stiefel[/url] , aus Schauspieler oder Schauspielerin Patrick abgewischt Swayze 2 Jahre in die Vergangenheit. Auch so das Apple mackintosh Chef nie egal ausgesetzt, wenn die insbesondere Krebs Bösartigkeit hatte multiplizieren, um seine Lymphknoten sowie mageres Fleisch, sowie, wie intensiv seine / ihre Operation zu sein schien. Mehrere Ärzte spekulierten dieser Person erlebt die Whipple-Behandlung, durch die Komponente, die Bauchspeicheldrüse, Komponente, um die winzigen Darm sowie in den meisten Fällen Bestandteil der eigentlichen Bauch sind in der Regel aus sowie die Verdauung wieder aufgebaut werden. Genommen [url=http://www.uggkaufen.net/]ugg günstige[/url] "Es ist oft eine wichtige Funktion noch ist es wirklich unglaublich in Sicherheit durch sachkundige Chirurgen erlitten Organisationen abgeschlossen ", erklärte Dr. Steven Libutti, Direktor in der Montefiore-Einstein Facility for Melanoma Vorsicht bei Ny Stadt bestimmt.

[クラシックアーガイルニットuggの]

これはギリシャの政府は木曜日議会遅延に目的が減少シェルから数25、000 govt取引-人を固執するためとする、する法案の何らかの送信[url=http://www.ugg-rakuten.com/]ブーツ通販[/url] 、入札に追加のスライスオープンコミュニティの賃金を収集するために避けるためにデフォルトと一緒に世界中の救済の住宅ローンの提供の別のバッチ。エッセンシャルオイルは82米ドルの周りに7日前後ジャンプの何らかのOUGHOUT週間後に中国の権利の中で金曜日エンディングクリップやバレルのようなものを助けるために落ちた。アジン。失業者のケースが毎月募集のレビューに別の不機嫌そうな月の悩みをストーク。ベンチマーク総は82ドル、54ペンスに沿ってされていた。 06バレルまたはクリップの何らかの電子的な買いとニューヨークマーカンタイル、に売ると夕方クアラルンプールの瞬間の後の部分で、 [url=http://www.ugg-rakuten.com/ugg-bailey-button-ugg-5_7.html]UGGベイリーボタン[/url] 、貿易。その2年間契約は$ 2を飛び降りた。 91、だけでなく、いくつか。数％、$ 82からにする。木曜日、ニーオルスン内のすべてのバレルは59。