To fix intelligence analysis you have to decide what’s broken

“More and more, Xmas Day failure looks to be wheat v. chaff issue, not info sharing issue.” – Marc Ambinder, politics editor for The Atlantic, on Twitter last night.

Marc Ambinder, a casual friend and solid reporter, has boiled down two likely avenues of intelligence “failure” relevant to the case of Umar Farouk Abdulmutallab and his attempted Christmas Day bombing on Northwest Airlines Flight 253.  In his telling, they’re apparently binary – one is true, not the other, at least for this case.

The two areas were originally signalled by President Obama in his remarks on Tuesday, when he discussed the preliminary findings of “a review of our terrorist watch list system …  so we can find out what went wrong, fix it and prevent future attacks.” 

Let’s examine these two areas of failure briefly – and what can and should be done to address them.

Information-Sharing Issue

This area has received the most attention among pundits and partisan politicians in the past several days. It is redolent with the “stovepipes” language cited properly in critiques of the intelligence community’s performance before September 11, 2001… and continuing in the years since. As President Obama put it on Tuesday:

It’s been widely reported that the father of the suspect in the Christmas incident warned U.S. officials in Africa about his son’s extremist views.  It now appears that weeks ago this information was passed to a component of our intelligence community, but was not effectively distributed so as to get the suspect’s name on a no-fly list… Had this critical information been shared it could have been compiled with other intelligence and a fuller, clearer picture of the suspect would have emerged.  The warning signs would have triggered red flags and the suspect would have never been allowed to board that plane for America… When our government has information on a known extremist and that information is not shared and acted upon as it should have been, so that this extremist boards a plane with dangerous explosives that could cost nearly 300 lives, a systemic failure has occurred.  And I consider that totally unacceptable.” – President Barrack Obama, Tuesday 12/29/2009

The problem isn’t new anymore, so no one can claim that we weren’t aware of it. The 9/11 Commission Report detailed stovepiping and information-sharing problems ad nauseum.  So those responsible need to say they’ve been working on fixing it, and have just fallen short.

I’ll cite two prominent ongoing efforts to “fix” this problem:

(1) The Information Sharing Environment (ISE): Most commentary in the past few days comes from folks who don’t know that the U.S. Government responded to the last big terrorist info-sharing crisis (9/11) by establishing an actual formal program, the official Information Sharing Environment. You can read all about it at www.ise.gov, though I advise against it unless you’ve developed a tolerance for your prescription insomnia cure and need some heavy-duty help getting to sleep. Folks don’t know about the ISE because it has been a massive – but quiet – bureaucratic exercise in disappointment (some say failure). 

The ISE is supposed to be a joint endeavor of, and to support, five communities: Intelligence, Law Enforcement, Defense, Homeland Security, & Foreign Affairs. Why, those are the very ones involved in the run-up to the latest Christmas Day debacle!  And the ISE has been “supporting” them for nearly four years now! 

Back in July 2009, the ISE released its mandatory 2009 Annual Report to the Congress on the ISE. I am the only person I know to have read the darn thing. That’s a shame – not because it’s great reading, but because it could have provoked wider acknowledgment of just how mired in bureaucratic morass this well-funded ISE boondoggle has become. It takes the reader 17 pages just to get to a section titled “Why Information Sharing is Important” (!), which is answered with the bromide that “the proper information, properly controlled, gets to the right people in time to counter terrorist threats to our people and institutions.” O-kay…. Strike one.

Strikes two and three are pages 44 and 45 of the Annual Report, which purport to chart the ISE’s progress on implementing and enforcing “Common Terrorism Information Sharing Standards Program” – I cannot over-emphasize that this should be the heart of the fix for the 9/11 and now Christmas-Day failures. If you can read those two pages without shaking your head with a sad, mournful laugh, then you are a better, more credulous person than I. Enough said. Congress and the media need to take a hard look at the ISE and ask some pointed questions about what has gone wrong with this, our previous large-scale national attempt to fix the info-sharing failures. The ISE could be fixed, but only with a wholesale refocus on smaller scale, shorter term, and much much simpler solutions.

(2) The Presidential Task Force on Controlled Unclassified Information: You may say you’ve never heard of this task force, but taste the irony: it released its work-product just ten days before the Christmas attack, and the Administration garnered quite a bit of press coverage for its two main proposals.

WASHINGTON, Dec. 15 /PRNewswire — Attorney General Eric Holder and Department of Homeland Security (DHS) Secretary Janet Napolitano today announced two major steps in their efforts to implement reforms to enhance information sharing among federal, state, local and tribal law enforcement agencies and safeguard sensitive information used by the government — designed to expand joint capabilities to protect the United States from terrorist activity, violent crime and other threats to the homeland.

The Presidential Interagency Task Force on Controlled Unclassified Information (CUI), led by Attorney General Holder and Secretary Napolitano, today released a report recommending a single, standardized framework for marking, safeguarding and disseminating sensitive but unclassified (SBU) information across the federal government. SBU information refers collectively to the various designations for documents and information that are sufficiently sensitive to warrant some level of protection but that do not meet the standards for classification.

Attorney General Holder and Secretary Napolitano also announced the creation of dual Program Management Offices (PMOs) to coordinate support for state and local Fusion Centers and the Nationwide Suspicious Activity Reporting Initiative (NSI), housed within DHS and the Department of Justice (DOJ), respectively, to work in partnership to enhance information sharing between federal, state, local and tribal agencies and the private sector. Coupled with the CUI framework, these new offices represent a significant milestone toward fully implementing information sharing reforms called for following the terrorist attacks of Sept. 11, 2001.

The first proposal, for a single SBU standards framework, has been long in the making, and its implementation appears to be in the hazy future. The second proposal is one which I predict will however begin taking root right away – because it involves spending money to establish new offices, and a new bureaucratic layer of entities, the PMOs which will “coordinate” (that favored DC word) the work of all the state and local Fusion Centers, which were of course themselves set up to “coordinate” information and intelligence sharing across government levels and bodies.

Might that be valuable, if it worked? Sure – but what’s the timeline? According to the Wall Street Journal’s coverage of the new plan, the new PMO’s will support an expansion of the “program that collects and analyzes potential terrorism tips from local police officers, so that, by 2014, all states will have the capability to analyze that data and share it with other states and the federal government.” Read that again; it will take the next five years to develop that capability.

More bureaucracy has been proposed, therefore, as the answer to a problem of too much bureaucracy. I am not optimistic on the information-sharing front given the current “reform” path.

Wheat vs Chaff Issue

President Obama also reported Tuesday, after citing the information-sharing problem:

There appears to be other deficiencies as well…. [T]here were bits of information available within the intelligence community that could have and should have been pieced together.  We’ve achieved much since 9/11 in terms of collecting information that relates to terrorists and potential terrorist attacks.  But it’s becoming clear that the system that has been in place for years now is not sufficiently up to date to take full advantage of the information we collect and the knowledge we have.”

I read that as a pretty clear assessment of an analytic failure. It’s a harsh assessment particularly because it comes from the president, the primary intelligence consumer and analytic customer of the United States’ $75 billion-per-year intelligence community. The president described the failure as “totally unacceptable.”

So one must turn to the search for improvement, for ways to balance the search for wheat amidst chaff, for insightful threads amongst chaotic data streams. That’s not easy in a world of exploding data sources and a profusion of sensors – the January issue of National Defense magazine carries a story on this topic, entitled “Military ‘Swimming In Sensors and Drowning in Data’ – it’s not an optimistic depiction.

But I’m an optimist by nature, and a technologist by choice, so I tend to see the potential for future progress in addressing the issue. I’ve written before about moving beyond “Filter Failure” – I don’t believe there is “too much information,” but a lack of imagination and technical creativity in designing innovative ways to mine the chaff (as well as the wheat). The solutions aren’t entirely rooted in technology, but new technology certainly helps.

Because I’ve written on this topic often, I’ll just mention one encouraging effort undertaken by the intelligence community’s Intelligence Advanced Research Projects Activity (IARPA). Art Becker, whom I’ve known for quite a while, leads IARPA’s Knowledge Discovery and Dissemination (KDD) Program, and just announced this month that KDD will be accepting proposals until February 10, 2010 for a new KDD system or approach on analysis. The announcment reads as if in direct answer to the President’s identification of analytic shortcomings in the Abdulmutallab case:

Intelligence analysts must gather and analyze information from a wide variety of data sets that include: general references, news, technical journals and reports, geospatial data, entity databases, internal reports and more. The different terminologies, formats, data models, and contexts make it difficult to perform advanced analytic tasks across different data sets… The focus of the KDD program is to develop novel approaches that will enable the intelligence analyst to effectively derive actionable intelligence from multiple, large, disparate sources of information, to include newly available data sets previously unknown to the analyst.

The ability to quickly produce actionable intelligence from unanticipated, multiple, varied data sets require research advances in two key areas: (1) alignment of data models; and (2) advanced analytic algorithms. Making advances in these two research areas, and fully characterizing the performance of the research results using real Intelligence problems, is the focus of the IARPA Knowledge Discovery and Dissemination (KDD) Program. The KDD Program requires a combination of innovative research and the capability to develop robust prototypes.

 

IARPA’s KDD work is squarely aimed at answering the issue President Obama and others have raised about the quality of analytic collation and insights evident in the run-up to the Christmas Day bomb attempt. I’ll revisit this topic again.

What’s the Difference?

Info-sharing or wheat-from-chaff analytic failure – so what? Isn’t it just six of one, half-dozen of the other? Nope. We may not have to “choose” between them as explanations for the Christmas Day failure, as Marc Ambinder has done above, but there is a difference and I’ll explain, with an example.

Another media outlet last night waded into the fray of parcelling blame between these same two vectors, in a curious way.  The New York Times last night published online a well-reported story  but oddly, there was a subtle shift in the story’s emphasis – and its title – within an hour of publication. The original version’s headline: “Spy Agencies Failed to Share Clues on Terror Before Flight.” The revision: “Spy Agencies Failed to Collate Clues on Terror.”  I verified this by going back through my cached html history.

The first version made the rounds heavily on Twitter, as soon as it went live on the Times website. It hits clearly at an information-sharing failure. The second version uses the multiple examples of information being shared from NSA to others, from CIA to NCTC, from NCTC to others, to argue for an analytic, or wheat-from-chaff failure (“collating”).

So what happened here? Just editing? I don’t think so. The Times story is evidently based on heavy leaking from individual agencies, and counter-leaking from within ODNI. (Note to J-School students: When folks from every three-letter agency around the Beltway want to cover their butts and point fingers at another outfit, then a well-sourced story practically writes itself.)

Individual government agencies apparently want to make the case that, “Hey, we shared, but those other analysts just didn’t collate properly” – that way lies protection of your job, and your budget.

So the difference between the two vectors of failure is, generically: if information-sharing was at fault, so were multiple agencies – a network failure, a “systemic failure” to use the president’s phrase.

If poor collation and analysis was to blame, then it’s possible, maybe even likely, that the mistakes can be portrayed as isolated within one agency, or one subset of intelligence officers. In the first case, blame lies higher in the IC hierarchy; in the second, at lower and more expendable rungs.

You don’t have to buy those distinctions – or accept them as “true” – to recognize that they’re being used bureaucratically (through the media, the Congress, and IC internal channels) to apportion blame.

That’s Washington for you 🙂

Share this post on Twitter

Email this post to a friend