A Technical Computing revolution

Last week I enjoyed hosting a visit in Redmond from Chris Kemp, NASA’s new Chief Technology Officer for information technology. Our discussions were with folks from the Windows Azure cloud computing team, the high-performance computing and large-data folks, and our Extreme Computing Group. I smiled when Chris said he was a fan of the book Total Recall: How the E-Memory Revolution Will Change Everything, written by Microsoft’s Gordon Bell and colleague Jim Gemmell. (I wrote about their research projects in an earlier post, Total Recall for Public Servants.)

So Long, Long Tail?

I’ve been known to disagree with Harvard eggheads before 🙂 

 

And now, perhaps, another opportunity. A new Harvard Business Review article (“Should You Invest in the Long Tail?” by HBS Professor Anita Elberse) throws water on Chris Anderson’s paradigm, arguing that “hit products” are still more valuable than the conglomerated also-rans in the tail; her research is mostly in retail products. Chris has responded on his blog, sparking many comments and debate, and today the Wall Street Journal covered the back-and-forth debate.

I’m interested in the debate mostly because of the interest in the Long Tail way of thinking in some circles of the intelligence community.  I’ve written about the approach and its relevance to some intelligence issues (see “Tradecraft in the Long Tail” and “IARPA and the Virtual Long Tail“).

I’m just not certain that even a total debunking of the retail-oriented paradigm would undercut its value as applied to intelligence analysis. 

For intelligence analysts, obscure “facts” and patterns hidden snugly within the low-scale noise are all important – whether or not they gain numerative bulk in any accumulative way.  The paradoxical “unknown unknowns” are what’s being sought by dogged collection and analysis, and I’m not sure that’s analogous to Elberse’s acknowledged findings. 

Your thoughts welcome, here or by email back to me.

 
Email this post to a friend

 

Is It Even Possible to Connect the Dots?

FACT: Among the inspired ideas of polymath Danny Hillis (pioneer of parallel computing) was establishing the Long Now Foundation, whose projects include the millennial Long-Now Clock (“the world’s slowest computer”) and the notion of “Long Bets.”  A Long Bet is an “accountable prediction,” meaning one that has a specified end-date and a testable, wagerable, proposition.  One of the early Long Bets posted wagers $2,000 that “By 2020, no one will have won a Nobel Prize for work on superstring theory, membrane theory, or some other unified theory describing all the forces of nature.”  That particular bet is one of many signs of scientific skepticism about string theory.

ANALYSIS: Even without the ease of hyperlinks, old-fashioned newspapers foster serendipitous connections between articles, particularly if you’re reading a Sunday morning paper with lots of sections. Sunday the Washington Post did me a service by placing in different sections a couple of articles which I connected, about intelligence “failures” and about stock-market prediction, leading me to some web-surfing about the questionable validity of string theory and some related observations about the difficulty of predicting human behavior.

In the Outlook section, the Post has an opinionated and thought-provoking op-ed piece by Mark Lowenthal, one of the most “intelligent” individuals in the recent history of the U.S. intelligence community (after all, he was the 1988 Jeopardy grand champion, as well as a former assistant director of CIA).

Social (Network) Science

Fact: The social-networking site LinkedIn claims as users “17,000,000+ Professionals, 500,000+ Senior Executives, Executives from 498 Fortune 500 companies, [and] 65,000 new Professionals every week.”

Analysis: Since I hold the title of “chief technology officer” for my group at Microsoft, I regularly check the widely-read blog CTOvision, written by Bob Gourley, CTO of Crucial Point.

Yesterday he posted a very solid summary of several social networking tools, including my preferred LinkedIn.  If you’re not up to speed on the genre it is a helpful cheatsheet and “buyer’s guide.”

The technology area deserves the attention. There are a dozen or more such sites for each that Gourley covers, and he chooses the ones that have shown growth and potential longevity; why invest any time adding personal data to a site just to watch it disappear? We’ve all had that happen. And yet hockey-stick growth has to be managed – LinkedIn for example has come in for some critical attention for some snafus along the way.

Let’s look at some efforts to understand more about the science behind the software….