Companies seeking to improve the odds of a successful business outcome have a powerful new weapon. They’re turning to employees who play prediction markets
A few years ago, when Microsoft wanted to
answer a question about the date a company product would ship, rather than quiz the manager in charge it decided to ask the
development team how well the project was going. More than this, Todd Proebsting, director of Microsoft’s Center for Software Excellence, used the opportunity to test an
internal prediction market. The 25 members of the project team were given $50 as a stake, which they could use to “bid” on a price for on-time delivery—just as if they were buying shares on the stockmarket.
The product was scheduled for delivery in November 2004. Bidding on the market began in August, and management was in for a shock. While the opening price for on-time delivery was set at 162⁄3 cents, it plummeted almost immediately to just over two cents. No amount of pleading by the project manager could lift that more than a cent, much to the delight of the players. And the product shipped three months late. It had been predicted by the market, but no one in senior management had been told the truth.
That is the beauty of the free market. The crucial difference between the forecasts made by managers, for example, and those made by speculators is that the speculators are much more motivated to predict the correct outcome, rather than the one they consider desirable or politically acceptable.
“Some managers can find it difficult—it’s an anonymous questioning of authority—but there’s no bullshit in these numbers,” says Mat Fogerty, chief executive of Crowdcast, which works with computer games developer Electronic Arts (EA) on a prediction market to forecast shipping dates, sales units and game quality.
“Middle management had trepidation regarding our prediction market,” agrees Chris Ko, EA senior director of NA publishing. “I believe there were a few reasons: not understanding the wisdom of crowds, risk aversion and an element of wanting to control all the views and datapoint related to their product.”
Prediction markets are rooted in familiar technology. Log on and you’ll generally see a trading screen that resembles any Web-based stock trading site. Each outcome, such as the value of second-quarter sales, or whether a product will be delivered on time, has a bid/offer spread and indications of price movements. Users bid on certain outcomes—most often using fake or “play” money, with the price reflecting how likely they believe the outcome to be.
Traders vie with each other to win the most money, and the company running the prediction market often translates the play currency into prizes. They’re not new; the first was set up in 1988 at the University of Iowa to make forecasts on elections and economic indicators.
But a big boost to their profile came from The Wisdom of Crowds, a best-selling 2004 book by James Surowiecki, which uses examples such as the county show contest to guess the weight of a farm animal to illustrate the edge that collective intelligence has over the individual. Often the
average prediction of such contests is highly accurate—much more so than a single participant’s guess. Best Buy, the US electronics retailer, was influenced by The Wisdom of Crowds to create its own internal prediction market in 2006. Using software from Consensus Point, TagTrade predicts the answers to management questions such as future sales figures and project completion dates.
Pioneered by company vice president Jeff Severts, the market is open to all of Best Buy’s 115,000 US employees, although only some 2,100 take part—less than two per cent of the workforce. Each player gets $1m in play money to use over a nine-month period, with a $200 gift certificate up for grabs for the most successful trader.
“The information you need to make a smart decision on any issue is out there, but as you get bigger it gets tougher to find that information and use it,” notes Severts. “The market is not a perfect crystal ball—it’s not going to tell you exactly what’s going to happen all the time. What it does do is roll up the consensus of what the participants in the market are seeing, hearing and feeling.”
EA uses its prediction market to measure three crucial metrics—something called Metacritic, or game quality; competitor release dates; and market forecasts. According to Ko: “The initial results have been promising. For example, our variance on estimating Metacritic greatly improved on our old process by 50 per cent.”
The company’s experience with the accuracy of prediction markets is in line with most users. When Intel trialled prediction markets in 2005, it found accuracy was equal to official figures. This doesn’t mean such markets are perfect, though. Google has been working with a market since 2005 and has found several key biases that can distort results.
According to Google project manager Bo Cowgill, the most important was optimism.
“Outcomes that would be good for Google, such as attracting lots of users, were overpriced. The market gave them a higher probability than it should have,” he noted in a roundtable discussion on prediction markets hosted by consultancy McKinsey. Biases also included underpricing extreme events, so very high or very low sales figures would happen more often than the market expected.
Google found that market accuracy improved over time, as participants grew more experienced. Lloyds TSB’s Innovation Market, which uses an internal stockmarket to rate new ideas from staff, found that while market distortions such as insider trading are anathema to real stockmarkets, they can be beneficial to internal markets.
According to Lloyds TSB’s head of innovation James Gardner, the internal market began to suffer from some of the problems that markets in the wider economy encounter, such as hyperinflation and insider trading.
“We had to put controls in place to avoid what was essentially a system that just printed money,” he says. “Then we had insider trading in ideas. Just as we were about to try to fix that, we realised something interesting: insider trading can be a good thing. It makes people want to be on the inside so they can speculate successfully.”
Others take a different approach. Danish marine coatings specialist Hempel ran a four-week Idea Exchange last September through prediction market supplier Nosco to generate and rate business ideas from 345 staff in departments such as R&D, marketing and sales. Although it offered prizes to winning traders and idea generators, Hempel business developer Lars Risum is cautious about the effect of incentives on player behaviour.
“How do people act when they are influenced by the competitive aspect?” he says. “Do they invest in ideas they believe in, or simply to win?” Hempel believes a better driver of staff engagement is the belief that their ideas matter. To clearly demonstrate management’s commitment to putting the best ideas into practice, it invited participants who submitted the 10 winning ideas, as well as the authors of five suggestions selected by the company, to present a business plan to senior management. As a result, five suggestions have been adopted.
But not all prediction markets last the course. InterContinental Hotels ran one in 2007 to encourage idea generation from its 1,000-strong technology staff. The four-week project was the brainchild of Zubin Dowlaty, vice president of emerging technologies, and was successful enough to spawn two projects that were taken into development. But Dowlaty has since left the the global hotel chain, which now puts its efforts into social media-type websites such as Tripadvisor.com to gather feedback from guests.
“We use our Priority Club Rewards programme to interact with customers and get feedback on ideas,” says communications director Emma Corcoran, who argues that social media’s growing ubiquity on the Web is a key driver. “Why not use the technology people are already comfortable with? If people have to learn another technology they are less likely to take part. Social media is a large area for us. That’s how our guests are doing it already. Seventy per cent of bookings have an online element and many people offer us feedback as part of the process.”
Other internal markets also fall by the wayside. In 2007, the Financial Times ran a prediction market for readers through Intrade. Called FTPredict, it offered a luxury holiday prize to the most successful trader. But although FT.com global head Ien Cheng said at the time “our aim is to extend FTPredict so the exchange is used by the widest possible audience”, the company subsequently let it lapse.
So what can go wrong? Leighton Vaughan Williams, professor of economics and finance at Nottingham Trent University and editor of The Journal of Prediction Markets, says that “the real problem where these markets have not worked as well as management expected is that the companies have simply bought the technology and more or less expected the markets to take care of themselves. I don’t want to stretch the point, but this can be compared to buying a car and expecting it to drive itself. Of course you’d be disappointed with performance.”
Crucially, such markets require a certain minimum participation to function. Without liquidity—that is enough people placing bets, or trades, on different outcomes—the market becomes worthless. Companies have tried a range of strategies to deal with this. Google permits frivolous questions on its market alongside business questions, such as bets on sports outcomes, arguing that participants who are attracted to the fun questions go on to add liquidity in the more serious market.
More radically, EA implemented a new market design in April. Instead of the traditional stockmarket screen, users see a much more intuitive interface that allows them to make more flexible bets, changing the parameters to suit their opinions more easily than before. “We’ve moved away from the stockmarket mechanism. It’s a bit obtuse for making predictions,” says Fogerty.
The argument is that a more user-friendly design will allow non-gamblers or stock speculators the chance to play. It also has the benefit that fewer players are required to provide liquidity for any bet.
Some companies, of course, may be fearful that employees might effectively spend all their time at the bookmaker, with participation in a prediction market overshadowing their real work. “This has been a concern for some clients,” Fogerty points out. “But the EA market doesn’t have too much social networking functionality. People spend only half an hour a month on it.”
Best Buy’s Severts goes even further. He says that taking part in prediction markets has integral value. “By playing the game you are doing your job,” he says. “There’s not much that any of us do in a given day that’s more important than telling the company what we know.”
Following the crowdWhat A prediction market is simply a way of tapping into your employees’ collective knowledge to make predictions about business outcomes. Employees make “bets” on the likelihood of certain events.
Why Because sometimes if you ask a line manager when, say, a particular product will ship, or even how good it is, their answer is affected by the desire to report favourably rather than accurately. Also, the wisdom of crowds often proves more accurate than a single viewpoint.
How Some prediction markets work exactly like stockmarkets with a bid/offer spread and fake money that participants can use to bet on preferred outcomes. Some markets are less complex, incorporating social media tools that forge company relationships as well as share knowledge.
Who Companies offering prediction market tools include Crowdcast, Consensus Point and Nosco.
Where There are still more examples in the US of companies using prediction markets than in the UK, but start-ups such as Hubdub, a B2C news prediction website, are expected to give the concept a public-relations boost.