Test for Prediction Markets: They Say Obama, but Polls Say It’s Tied

Fact: According to the latest Rasmussen poll released Saturday July 12, and promptly headlined by the Drudge Report, “The race for the White House is tied. The Rasmussen Reports daily Presidential Tracking Poll for Saturday shows Barack Obama and John McCain each attract 43% of the vote.” Newsweek is reporting a similar result in its own poll, with Obama moving down and McCain up (“Obama, McCain in Statistical Dead Heat“), and other polls increasingly show a similarly close race.

Analysis: I’ve been tracking the growing divide between two quite different methods purporting to offer statistical predictive analysis for the November presidential election. Polls are saying one thing, but Prediction Markets are saying another. 

Prediction Markets: Research and Limits

Fact: A story in Science Daily this week, “Election Forecasters Preparing For Historic Election,” relates the publication this month of the “assembled insights of prominent election forecasters in a special issue of the International Journal of Forecasting.” 

Analysis:  The journal articles are available here for download.  One of them, “Prediction Market Accuracy in the Long Run” (by Joyce E. Berg, Forrest D. Nelson, and Thomas A. Reitz from the University of Iowa’s Tippie College of Business), compares the presidential election forecasts produced from the granddaddy of them all, the Iowa Electronic Market (IEM), to “forecasts from an exhaustive body of opinion polls.”  Science Daily says they find that the IEM is “usually more accurate than the polls.”

If we extrapolate out, these election markets are special cases of prediction markets, and I’m always interested in those.

Pentagon’s New Program for Innovation, in Context

FACT: According to an article in today’s Washington Post, the Pentagon has announced “the selection of six university professors who will form the first class of the National Security Science and Engineering Faculty Fellows Program. The professors will receive grants of up to $600,000 per year for up to five years to engage in basic research — essentially a bet by the Pentagon that they will make a discovery that proves vital to maintaining the superiority of the U.S. military.”

ANALYSIS: This new program is an innovation from DoD’s Director of Defense Research and Engineering (DDR&E), and since tomorrow I’ll be at Ft. McNair for a two-day conference sponsored by DDR&E on Strategic Communications, I’ll congratulate John Young and his staff for the good idea.

But the Post article falls short in two ways: one immediate (it leaves out key information about next year’s program and the upcoming deadline!) and one longer-term (it ignores the overall context of federal support for R&D).  I’ll fill in the blanks below.

Is It Even Possible to Connect the Dots?

FACT: Among the inspired ideas of polymath Danny Hillis (pioneer of parallel computing) was establishing the Long Now Foundation, whose projects include the millennial Long-Now Clock (“the world’s slowest computer”) and the notion of “Long Bets.”  A Long Bet is an “accountable prediction,” meaning one that has a specified end-date and a testable, wagerable, proposition.  One of the early Long Bets posted wagers $2,000 that “By 2020, no one will have won a Nobel Prize for work on superstring theory, membrane theory, or some other unified theory describing all the forces of nature.”  That particular bet is one of many signs of scientific skepticism about string theory.

ANALYSIS: Even without the ease of hyperlinks, old-fashioned newspapers foster serendipitous connections between articles, particularly if you’re reading a Sunday morning paper with lots of sections. Sunday the Washington Post did me a service by placing in different sections a couple of articles which I connected, about intelligence “failures” and about stock-market prediction, leading me to some web-surfing about the questionable validity of string theory and some related observations about the difficulty of predicting human behavior.

In the Outlook section, the Post has an opinionated and thought-provoking op-ed piece by Mark Lowenthal, one of the most “intelligent” individuals in the recent history of the U.S. intelligence community (after all, he was the 1988 Jeopardy grand champion, as well as a former assistant director of CIA).