Increasing Jointness and Reducing Duplication in DoD Intelligence

Today I’m publishing an important guest-essay, with a brief introduction.  Last month the Wall Street Journal published a 12-part online series about college graduates and their paths to success, featuring surveys and input from job recruiters. One thing caught my eye, at least when blogged by an acquaintance, Prof. Kristan Wheaton of the Mercyhurst College Institute Of Intelligence Studies. The WSJ’s study included a look at recent graduates’ job satisfaction in their new careers, and as Prof. Wheaton strikingly put it in his own blogpost:

“Intelligence Analysts are Insanely Happy.” 

I’m pretty sure that’s not really true by and large; Prof. Wheaton seems slightly dubious as well. Many readers of this blog are intelligence analysts themselves, so I’d love to hear from you (in comments or email) about your degree of giddyness….

We all know that the intelligence-analysis field as currently practiced in U.S. agencies bears many burdens weighing heavily on job satisfaction, and unfortunately weighing on successful performance.  Our youngest and our most experienced intelligence analysts have been battling those burdens. 

One analyst has now put constructive thoughts on paper, most immediately in response to a call by Defense Secretary Bob Gates asking DoD military and civilian employees to submit their ideas to save money, avoid cost, reduce cycle time and increase the agility of the department (see more about the challenge here).  

Your choice, Dataviz as event or book

A friend wrote asking if I could make it to an event happening this week near DC. I can’t make it, but fortunately he also mentioned as consolation that he has a cool new book on the cusp of release – and I’ve now ordered my copy.

The Friend: legendary visualization and HCI guru Ben Shneiderman (Wikipedia entry). Ben is a computer-science professor at the University of Maryland and the founder of its well-known Human-Computer Interaction Laboratory (HCIL), as well as an ACM Fellow and AAAS Fellow.  He has done government a million favors over the years, consulting for agencies, including his recent work on the Recovery.gov site to help that platform of data – from hundreds of thousands of sources – organize, host, and visualize the data for millions of visitors.  I first got to know Ben through his support for better intelligence analysis – he helped invent a longtime intelligence analytics tool, Spotfire (see his article “Dynamic queries, starfield displays, and the path to Spotfire“).  Ben’s also well-known for his award-winning 2002 book Leonardo’s Laptop: Human Needs and the New Computing Technologies, which I enjoyed and still think about when brainstorming new techie toys.

Free Tools for the New Scientific Revolution

Blogs are great for supplementing real-life events, by giving space and time for specific examples and links which can’t be referenced at the time. I was invited to give a talk last week at the first-ever NASA Information Technology Summit in Washington DC, and the topic I chose was “Government and the Revolution in Scientific Computing.” That’s an area that Microsoft Research has been focusing on quite a bit lately, so below I’ll give some examples I didn’t use at my talk.

One groundrule was that invited private-sector speakers were not allowed to give anything resembling a “sales pitch” of their company’s wares. Fair enough – I’m no salesman.  The person who immediately preceded me, keynoter Vint Cerf, slightly bent the rules and talked a bit about his employer Google’s products, but gee whiz, that’s the prerogative of someone who is in large part responsible for the Internet we all use and love today.

I described in my talk the radical new class of super-powerful technologies enabling large-data research and computing on platforms of real-time and archival government data. That revolution is happening now, and I believe government could and should be playing a different and less passive role. I advocated for increased attention to the ongoing predicament of U.S. research and development funding.

Alex Howard at O’Reilly Radar covered the NASA Summit and today published a nice review of both Vint’s talk and mine.  Some excerpts:

Simon Moves On

Jim Simon at the Library of Alexandria, Egypt

One indulgent use of a personal blog is to drop a nod in the direction of a salutary individual, and I’d like to do so for my departing boss, Jim Simon.

Jim has been the founding Director of the Microsoft Institute since 2004, when Bill Gates and Craig Mundie personally decided to establish a small outfit to use the benefits of Microsoft’s advanced research and development activities against intractable problems for the global public sector. They had been talking with Jim for several years, back when he was a senior executive at the Central Intelligence Agency and after, to understand how to improve government’s adoption of modern technologies.

A-Space Past and Future

This week marks the second anniversary of the first live internal demo of the intelligence community’s A-Space project, groundbreaking for the IC in its goal of collaborative use of social media across agency lines. Somewhere in Maryland, a remarkable government employee and friend named Mike Wertheimer should pause and quietly celebrate the fruition of his early evangelism for it.

I was still a government employee then, but wrote about the effort at the time here on Shepherd’s Pi (“A-Space: Top-secret social networking“). It makes me chuckle to remember back to those days when it was still mostly unheard-of for IC employees to blog openly on the public web about current technology projects. Now you can’t shut ’em up! 🙂

It made sense, I thought, to set down a few notes at the time for several reasons:

Once you get past Filter Failure

How do intelligence analysts handle the long-discussed problem of information overload? (The same question goes for information workers and government data of any kind.)

Wordy, by Wordle

Lazy Saturday, so a quick & easy retrospective post at a glance.  Here’s a “Wordle” look at my blog content going back over the past week. Click to enlarge.

You can use http://wordle.net to create a word-cloud of any content – a website (by URL) or a bucket of words you paste in. I realize word-clouds are nothing new… but I think they’re way underutilized in HCI.  Fun for political speech analysis; I can just imagine a newspaper front page that consisted of nothing but word-clouds from yesterday’s speeches by President Bush, Senators McCain and Obama, Biden or Palin, cabinet members, foreign officials, Osama’s latest video, and losing sports-team coaches.   “What’d they say?” Well, take a look.

Gartner Says, Sometimes Hype is Necessary

Fact: Gartner is taking the same approach they often critique with their normally-solid “Hype Cycle” reports – arguing that “a little cloud hype” is beneficial if it “captures the imaginations of a broader audience of decision makers.”

Analysis: With their annual “Hype Cycle” reports, Gartner usually does a solid job of tracking over-optimistic assessments of the “latest and greatest” in technology and calling out overly hyped “hot new tech” and providing realistic assessments of the projected future of trends in software, hardware, and business processes.

Sometimes, Gartner slips up, and falls prey to the error they ascribe to others.  That’s the only interpretation I can make on a curious blog posting on an official Gartner blog designed to promote their new book “Mastering the Hype Cycle: Choosing the Right Innovation at the Right Time.”  Mark Raskino, the book’s co-author with longtime analyst Jackie Fenn, argues that “We have to simplify the business proposition behind this ‘big shift’,  explain it well and socialize it deeply to convince non-tech business leaders to buy-in.” 

Mr. Raskino makes clear that he wants to babytalk these business-side executives into believing “a little cloud hype” because, in his words, IT leaders and CIOs “need help explaining the fundamental change.”

A New Prototype: Research Desktop

Fact: The international conference on Advances in Social Network Analysis and Mining (ASONAM 2009, next July in Athens, Greece) today issued its call for papers on “experimental and theoretical works on social network analysis and mining,” particularly relating to online social Web sites, email logs, phone logs and instant messaging systems “which are widely analyzed using graph theory and machine learning techniques.”  Interested authors are encouraged to submit abstracts of up to 300 words by December 10, 2008; the full papers aren’t due until January 31, 2009.  More info at www.asonam.org.

Analysis: Several Microsoft Research people are preparing papers based on their current research, and I’m considering attending myself (I’ve written before about MSR’s work in analyzing large social networks). There are three Microsoft scientists on the Committee (Dou Shen, Haizheng Zhang, and Rina Panigrahy – check out Rina’s publications on hashing and sketching algorithms).  It should be a top-notch conference, co-hosted by ACM and IEEE.

But that’s way off in the future – what if you want to look at some research stuff right now? Well, I’ve been going through the related “Socio-Digital Systems” work of MSR Cambridge (UK), and they’ve just added more information to their section here of the main MSR site.  That’s some neat stuff, more on the side of the actual social uses of digital data and the effects on our (still-human?) everyday lives.

Using Web 2.0 to Track a Political Crisis

My gut tells me that the latest round in the Georgian-Russian crisis is going to get ugly.

CNN’s “Russian Tanks Rolling into Georgian Breakaway” shows one thing: our old friend Vladimir Putin feels free to call the shots.

Twitter alerts can only get you so far, in following a crisis.  Check out these first attempts: this Twitter search or this one, or this Live News feed.  You can RSS any of these by the way.

I’m working on several Popfly-designed orchestrations of alerts and newsfeeds to keep me updated on the latest news from South Ossetia, incorporating crowd-sourced information as much as I can.