So Long, Long Tail?

I’ve been known to disagree with Harvard eggheads before 🙂 

 

And now, perhaps, another opportunity. A new Harvard Business Review article (“Should You Invest in the Long Tail?” by HBS Professor Anita Elberse) throws water on Chris Anderson’s paradigm, arguing that “hit products” are still more valuable than the conglomerated also-rans in the tail; her research is mostly in retail products. Chris has responded on his blog, sparking many comments and debate, and today the Wall Street Journal covered the back-and-forth debate.

I’m interested in the debate mostly because of the interest in the Long Tail way of thinking in some circles of the intelligence community.  I’ve written about the approach and its relevance to some intelligence issues (see “Tradecraft in the Long Tail” and “IARPA and the Virtual Long Tail“).

I’m just not certain that even a total debunking of the retail-oriented paradigm would undercut its value as applied to intelligence analysis. 

For intelligence analysts, obscure “facts” and patterns hidden snugly within the low-scale noise are all important – whether or not they gain numerative bulk in any accumulative way.  The paradoxical “unknown unknowns” are what’s being sought by dogged collection and analysis, and I’m not sure that’s analogous to Elberse’s acknowledged findings. 

Your thoughts welcome, here or by email back to me.

 
Email this post to a friend

 

Pentagon’s New Program for Innovation, in Context

FACT: According to an article in today’s Washington Post, the Pentagon has announced “the selection of six university professors who will form the first class of the National Security Science and Engineering Faculty Fellows Program. The professors will receive grants of up to $600,000 per year for up to five years to engage in basic research — essentially a bet by the Pentagon that they will make a discovery that proves vital to maintaining the superiority of the U.S. military.”

ANALYSIS: This new program is an innovation from DoD’s Director of Defense Research and Engineering (DDR&E), and since tomorrow I’ll be at Ft. McNair for a two-day conference sponsored by DDR&E on Strategic Communications, I’ll congratulate John Young and his staff for the good idea.

But the Post article falls short in two ways: one immediate (it leaves out key information about next year’s program and the upcoming deadline!) and one longer-term (it ignores the overall context of federal support for R&D).  I’ll fill in the blanks below.

Is It Even Possible to Connect the Dots?

FACT: Among the inspired ideas of polymath Danny Hillis (pioneer of parallel computing) was establishing the Long Now Foundation, whose projects include the millennial Long-Now Clock (“the world’s slowest computer”) and the notion of “Long Bets.”  A Long Bet is an “accountable prediction,” meaning one that has a specified end-date and a testable, wagerable, proposition.  One of the early Long Bets posted wagers $2,000 that “By 2020, no one will have won a Nobel Prize for work on superstring theory, membrane theory, or some other unified theory describing all the forces of nature.”  That particular bet is one of many signs of scientific skepticism about string theory.

ANALYSIS: Even without the ease of hyperlinks, old-fashioned newspapers foster serendipitous connections between articles, particularly if you’re reading a Sunday morning paper with lots of sections. Sunday the Washington Post did me a service by placing in different sections a couple of articles which I connected, about intelligence “failures” and about stock-market prediction, leading me to some web-surfing about the questionable validity of string theory and some related observations about the difficulty of predicting human behavior.

In the Outlook section, the Post has an opinionated and thought-provoking op-ed piece by Mark Lowenthal, one of the most “intelligent” individuals in the recent history of the U.S. intelligence community (after all, he was the 1988 Jeopardy grand champion, as well as a former assistant director of CIA).