Twitter Search as a Government case study

In addition to periodic think-pieces here at Shepherd’s Pi, I also contribute a monthly online column over at SIGNAL Magazine on topics relating to intelligence. This month I keyed off a recent discussion I had onstage at the 2015 AFCEA Spring Intelligence Symposium with Elon Musk, particularly a colloquy we had on implications of the emerging cleavage (post-Edward Snowden) between Silicon Valley technology companies and their erstwhile innovation partners, U.S. intelligence agencies.

That discussion sparked some thinking on the public/private sector divide on tech innovation – and on basic operational performance in building or adopting new technologies. It’s always been a hobbyhorse topic of mine; see previous pieces even from way back in 2007-08 like “Pentagon’s New Program for Innovation in Context,” or “A Roadmap for Innovation – From the Center or the Edge?” or “VC-like Beauty Contests for Government.”

I have an excerpt from my new SIGNAL piece below, but you can read the entire piece here: “The Twitter Hare Versus the Government Turtle.”

Is the public/private divide overstated? Can the government compete? Without going into the classified technology projects and components discussed at the symposium, let’s try a quick proxy comparison, in a different area of government interest: archiving online social media content for public use and research. Specifically, since Twitter data has become so central to many areas of public discourse, it’s important to examine how government and private sector are each addressing that archive/search capability.

First, the government side. More than half a decade ago, the Library of Congress (LoC) announced in April 2010 with fanfare that it was acquiring the “complete digital archives” of Twitter, from its first internal beta tweets. At that time, the LoC noted, the 2006-2010 Twitter archive already consisted of 5 terabytes, so the federal commitment to archiving the data for search and research was significant…

  … Fast forward to today. Unbelievably, after even more years of “work,” there is no progress to report—quite the opposite. A disturbing new report this week in Inside Higher Education entitled “The Archive is Closed” shows LoC at a dead-stop on its Twitter archive search. The publicly funded archive still is not open to scholars or the public, “and won’t be any time soon.”

  … Coincidentally this week, just as the Library of Congress was being castigated for failing in its mission to field a usable archive after five years, Twitter unveiled a new search/analytics platform, Twitter Heron—yes, after just six months [after releasing its previous platform Twitter Storm]. Heron vastly outperforms the original version in semantic throughput and low latency; yet in a dramatic evocation of Moore’s Law, it does so on 3 times less hardware.

Twitter Storm vs Twitter Heron

Oh, and as the link above demonstrates, the company is far more transparent about its project and technology than the Library of Congress has been.

All too often we see government technology projects prove clunky and prone to failure, while industry efforts are better incentivized and managerially optimized for success. There are ways to combat that and proven methods to avoid it. But the Twitter search case is one more cautionary example of the need to reinvigorate public/private partnerships—in this case, directly relevant to big-data practitioners in the intelligence community.

 – Excerpts from SIGNAL Magazine, “The Twitter Hare Versus the Government Turtle.” © 2015 AFCEA International.

Intelligence, Artificial and Existential

"Not to Be or Not to Be?" artwork by Shuwit, http://shuwit.deviantart.com/

“Not to Be or Not to Be?” artwork by Shuwit, http://shuwit.deviantart.com/

I just published a short piece over at SIGNAL Magazine on an increasingly public debate over artificial intelligence, which the editor gave a great Shakespearean title echoing Hamlet’s timeless question “To be, or not to be”:

AI or Not AI?

Caution tempers opportunity as experts ponder artificial intelligence

May 6,2015 – Artificial intelligence, or AI, has been on my mind recently—and yes, that’s something of a sideways pun. But it’s worth exploring the phrase from another double-entendre standpoint by asking whether the nation’s intelligence professionals are paying enough attention to AI.

In the past week I have seen two brand-new movies with AI at their center: the big-budget sequel Avengers: Age of Ultron (I give it one star, for CGI alone), and the more artistically minded Ex Machina (three stars, for its lyrical dialogue expressed in a long-running Turing Test of sorts). With Hollywood’s efforts, the uptick in public attention to AI is mimicking the increasing capabilities of real-world AI systems. And the dystopian plot elements of both Ultron and Ex Machina also are mirroring a heightened sense of impending danger or doom among many of the world’s most advanced thinkers….

…continues at “AI or Not AI?”

Besides the Hollywood attention, mainstream publications are exploring the topic. On a flight from London yesterday I read EconomistThe Economist’s new cover story, “Rise of the Machines: Artificial ntelligence scares people—excessively so,” and recommend it as an up-to-the-moment backgrounder on the economic and social questions being posed with the increasing levels of AI investment and research by the likes of Google, Facebook, Microsoft, and Baidu.

My interests in the topic include the applicability, potential benefits, and any unanticipated risks of AI use in national security, including defense and intelligence systems. Last month I helped lead the National Reconnaissance Office’s 2015 Industry Day, which laid out in a classified setting the mission architectures and implementation of advanced research efforts. While those briefings were classified, NRO Director Betty Sapp has been quoted describing NRO’s Sentient Project:

[Director Sapp] cites an experiment now in limited operations known as Sentient. It is demonstrating the power of using the full architecture against a problem set by doing automated tipping and cueing from one sensor to another—acting at machine speeds, not at the pace of humans. “I can see the strength of that [complete ground system approach] when I look at Sentient in even the way it is behaving in operations,” Sapp states. Saying Sentient is doing a very good job of getting new capability out of existing assets, she allows that more people from the defense and intelligence communities have come to the NRO to view the system’s demonstrations than for any other capability since the beginning of the organization’s history. “It is demonstrating the capabilities we want throughout our Future Ground Architecture,” she offers, adding that these capabilities probably will become operational in the year 2020 or beyond.

If the overall AI topic tickles your fancy, as I point out in the SIGNAL piece there are only a few seats left for the May 20/21 Spring Intelligence Symposium where I’ll be discussing the topic with Elon Musk, in a broader discussion of the future of Research & Development. If you have a TS clearance, please join me and register here.

Meet the Future-Makers

Question: Why did Elon Musk just change his Twitter profile photo? I notice he’s now seeming to evoke James Bond or Dr. Evil:

twitter photos, Elon v Elon

I’m not certain, but I think I know the answer why. Read on…

________________________________________

“Prediction is very difficult, especially about the future.”

      – Niels Bohr, winner of the 1922 Nobel Prize for Physics

“History will be kind to me for I intend to write it.”

      – Winston Churchill

If you take those two quotations to heart, you might decide to forego the difficulty of predicting the future, instead aiming to bend the future’s story arc yourself. In a nutshell, that’s what R&D is all about: making the future.

Who makes the future for the intelligence community? Who has more influence on the future technologies which intelligence professionals will use: government R&D specialists, or private-sector industry?

On the one hand, commercial industry’s R&D efforts are pulled by billions of invisible consumer hands around the globe, driving rapid innovation and ensuring that bold bets can be rewarded in the marketplace. Recent examples are things like Web search, mobile phones and tablets, and SpaceX launches.

To be fair, though, the US IC and DoD have the ability to focus intently on specific needs, with billions of dollars if necessary, and to drive exotic game-changing R&D for esoteric mission use. During my time in government I saw great recent successes which are of course classified, but they exist.

If you want to explore both sides and you have a Top Secret clearance, you’re in luck, because you can attend what I expect will be an extraordinary gathering of Future-Makers from inside and outside the IC, at next month’s AFCEA Spring Intelligence Symposium.

Spring IntellLast fall, the organizing committee for this annual classified Symposium began our planning on topics and participants. We decided that this year’s overall theme had to be “IC Research & Development” – and we decided to depart from tradition and bring together an unprecedented array of senior leaders from inside and outside, to explore the path forward for IC innovation and change.

The May 20-21 Symposium, held at NGA’s Headquarters, will be a one-of-a-kind event designed to set the tone and agenda for billions of dollars in IC investment.  On the government front, attendees will witness the roll-out of the new (classified) Science & Technology 2015-2019 Roadmap; see this article for some background on that. Attendees will also meet and hear R&D leaders from all major IC agencies, including:

  • Dr. David Honey, Director of Science and Technology, ODNI
  • Dr. Peter Highnam, Director, Intelligence Advanced Research Projects Activity (IARPA)
  • Glenn Gaffney, Deputy Director for Science & Technology, CIA
  • Stephanie O’Sullivan, Principal Deputy Director of National Intelligence
  • Dr. Greg Treverton, Chairman, National Intelligence Council
  • The IC’s functional managers for SIGINT, MASINT, GEOINT, HUMINT, OSINT, and Space

Meanwhile from the private sector, we’ll have:

  • Elon Musk, CEO/CTO of SpaceX, CEO/Chief Product Architect of Tesla Motors, CEO of SolarCity, Co-founder of PayPal
  • Gilman Louie, Partner at Alsop Louie Venture Capital, former CEO of In-Q-Tel
  • Bill Kiczuk, Raytheon VP, CTO, and Senior Principal Engineering Fellow
  • Zach Lemnios, IBM VP for Research Strategy and Worldwide Operations
  • Pres Winter, Oracle VP, National Security Group

When I first proposed that we invite an array of “outside” future-makers to balance the government discussion with a different perspective, I said to my colleagues on the planning committee, “Wouldn’t it be awesome to get someone like Elon Musk…”

Well, we did, and next month I’ll be welcoming him on stage.

These are dark and challenging times in international security, but for scientists, technologists, and engineers, there’s never been a more exciting time – and like them, intelligence professionals should stretch their horizons.

I’m looking forward to the conference… and here’s your link to register to join us.

PS: Just to whet your appetite: new video of this week’s SpaceX revolutionary Falcon9 first-stage landing attempt on a drone barge at sea – nearly made it, very exciting:

Young Americans and the Intelligence Community

IC CAE conferenceA few days ago I travelled down to Orlando – just escaping the last days of the DC winter. I was invited to participate in a conference hosted by the Intelligence Community’s Center of Academic Excellence (IC CAE) at the University of Central Florida.  The title of my speech was “The Internet, 2015-2025: Business and Policy Challenges for the Private Sector.” But I actually learned as much as I taught, maybe more.

First, several surprises I learned while preparing for my presentation. UCF is now the nation’s second-largest university, with over 60,000 students (second only to Arizona State University). The size of the undergraduate/graduate student population obviously translates into a robustly diverse set of student activities. Among those we met with were several leaders and members in the Collegiate Cyber Defense Club (see their site at HackUCF.org), which has hundreds of members, weekly meetings, and is fresh off winning the national 2014 Collegiate Cybersecurity Championship Cup, beating out more than 200 schools based on performance in competitions throughout the year.

LS speaking at IC CAE conference

IC CAE conference, photo by Mike Macedonia

Another surprise was the full extent of the umbrella activity within which the conference was organized.  The UCF IC-CAE is one of several such centers (overall info page here) established over the past decade, since a 2005 congressionally-mandated mission “to increase intelligence community job applicants who are multi-disciplinary, as well as culturally and ethnically diverse [via] grants to competitively accredited U.S. four-year colleges and universities to support the design and development of intelligence-related curricula.” There are some two dozen colleges now participating, including Duke, Penn State, and Virginia Tech.

One more significant thing I learned: young Americans today are not hostile to the nation’s intelligence mission and those who perform it. In fact, I published a short piece today at SIGNAL Magazine on some startling recent poll results, which I reviewed while preparing for the UCF visit. From my piece:

NSA’s negative coverage [over the past two years] was driven by a long series of front-page stories covering Edward Snowden’s leaked documents, including their impact on technology giants such as Facebook, Apple, Google and others. Many pundits have opined that young American millennials are horrified by the revelations and angry at the NSA for “domestic spying.”

Yet the 2015 national survey by the respected Pew Research Center asked specifically about the NSA, and it reveals that, “Young people are more likely than older Americans to view the intelligence agency positively. About six in 10 (61 percent) of those under 30 view the NSA favorably, compared with 40 percent of those 65 and older.”

 – excerpt from “NSA’s Biggest Fans are Young Americans

I didn’t know much about the IC-CAE program before my campus visit, but it strikes me overall as a valuable channel for the Intelligence Community to remain in close sync with the nation’s values and societal changes.  And, as I wrote in SIGNAL, I learned that “students at the nation’s second-largest university reflect the Pew poll findings, and on balance hold a positive view of the intelligence community and its efforts in the national interest. They admire our nation’s intelligence professionals, and they’re supportive of a robust foreign-intelligence collection program.”

I’ve posted the slides that accompanied my talk below, though of course much of the discussion isn’t reflected in the slides themselves. Smart audience, insightful questions.

    [space]

    [space] 

I must mention that the other presenters at the conference were great as well. In sum it was an enjoyable and enlightening experience, and it was reassuring to observe that America’s next great generation will be eager and expert recipients of the reins of national security.

    [space]

Intelligence Technology, Waiting for Superman

…or Superwoman.

Amid the continuing controversies sparked by Edward Snowden’s whistleblowing defection revelations, and their burgeoning effects on American technology companies and the tech industry worldwide, the afflicted U.S. intelligence community has quietly released a job advertisement for a premier position: the DNI’s National Intelligence Officer for Technology.

You can view  the job posting at the USAJOBS site (I first noticed it on ODNI’s anodyne Twitter feed @ODNI_NIC), and naturally I encourage any interested and qualified individuals to apply. Keep reading after this “editorial-comment-via-photo”:

How you'll often feel if you take this job...

How you’ll often feel if you take this job…

Whether you find the NSA revelations to be infuriating or unsurprising (or even heartening), most will acknowledge that it is in the nation’s interest to have a smart, au courant technologist advising the IC’s leadership on trends and directions in the world of evolving technical capabilities.

In the interest of wider exposure I excerpt below some of the notable elements in the job-posting and description…. and I add a particular observation at the bottom.

Job Title: National Intelligence Officer for Technology – 28259

Agency: Office of the Director of National Intelligence

Job Announcement Number: 28259

Salary Range: $118,932.00  to  $170,000.00

Major Duties and Responsibilities:

Oversees and integrates all aspects of the IC’s collection and analytic efforts, as well as the mid- and long-term strategic analysis on technology.

Serves as the single focal point within the ODNI for all activities related to technology and serves as the DNI’s personal representative on this issue.

Maintains senior-level contacts within the intelligence, policymaking, and defense communities to ensure that the full range of informational needs related to emerging technologies are met on a daily basis, while setting strategic guidance to enhance the quality of IC collection and analysis over the long term.

Direct and oversee national intelligence related to technology areas of responsibility; set collection, analysis, and intelligence operations priorities on behalf of the ODNI, in consonance with the National Intelligence Priorities Framework and direction from the National Security Staff.

In concert with the National Intelligence Managers/NIOs for Science and Technology and Economic Issues, determine the state of collection, analysis, or intelligence operations resource gaps; develop and publish an UIS which identifies and formulates strategies to mitigate gaps; advise the Integration Management Council and Integration Management Board of the gaps, mitigation strategies, progress against the strategies, and assessment of the effectiveness of both the strategies and the closing of the intelligence gaps.

Direct and oversee Community-wide mid- and long-term strategic analysis on technology. Serve as subject matter expert and support the DNI’s role as the principal intelligence adviser to the President.

Oversee IC-wide production and coordination of NIEs and other community papers (National Intelligence Council (NIC) Assessments, NIC Memorandums, and Sense of the Community Memorandums) concerning technology.

Liaise and collaborate with senior policymakers in order to articulate substantive intelligence priorities to guide national-level intelligence collection and analysis. Regularly author personal assessments of critical emerging technologies for the President, DNI, and other senior policymakers.

Develop and sustain a professional network with outside experts and IC analysts, analytic managers, and collection managers to ensure timely and appropriate intelligence support to policy customers.

Brief senior IC members, policymakers, military decisionmakers, and other major stakeholders.

Review and preside over the research and production plans on technology by the Community’s analytic components; identify redundancies and gaps, direct strategies to address gaps, and advise the DNI on gaps and shortfalls in analytic capabilities across the IC.

Determine the state of collection on technology, identify gaps, and support integrated Community-wide strategies to mitigate any gaps.

Administer National Intelligence Officer-Technology resource allocations, budget processes and activities, to include the establishment of controls to ensure equities remain within budget.

Lead, manage, and direct a professional level staff, evaluate performance, collaborate on goal setting, and provide feedback and guidance regarding personal and professional development opportunities.

Establish and manage liaison relationships with academia, the business community, and other non-government subject matter experts to ensure the IC has a comprehensive understanding of technology and its intersection with global military, security, economic, financial, and/or energy issues.

Technical Qualifications:

Recognized expertise in major technology trends and knowledge of analytic and collection issues sufficient to lead the IC.

Superior capability to direct interagency, interdisciplinary IC teams against a range of functional and/or regional analytical issues.

Superior interpersonal, organizational, and management skills to conceptualize and effectively lead complex analytic projects with limited supervision.

Superior ability to work with and fairly represent the IC when analytic views differ among agencies.

Superior communication skills, including ability to exert influence with senior leadership and communicate effectively with people at all staff levels, both internal and external to the organization, to give oral presentations and to otherwise represent the NIC in interagency meetings.

Expert leadership and managerial capabilities, including the ability to effectively direct taskings, assess and manage performance, and support personal and professional development of all levels of personnel.

Superior critical thinking skills and the ability to prepare finished intelligence assessments and other written products with an emphasis on clear organization, concise, and logical presentation.

Executive Core Qualifications (ECQs):

Leading People: This core qualification involves the ability to lead people toward meeting the organization’s vision, mission, and goals. Inherent to this ECQ is the ability to provide an inclusive workplace that fosters the development of others, facilitates cooperation and teamwork, and supports constructive resolution of conflicts. Competencies: Conflict Management, Leveraging Diversity, Developing Others, and Team Building

Leading Change: This core qualification involves the ability to bring about strategic change, both within and outside the organization, to meet organizational goals. Inherent to this ECQ is the ability to establish an organizational vision and to implement it in a continuously changing environment. Competencies: Creativity and Innovation, External Awareness, Flexibility, Resilience, Strategic Thinking, and Vision.

HOW YOU WILL BE EVALUATED:

You will be evaluated based upon the responses you provide to each required Technical Qualifications (TQ’s) and Executive Core Qualifications (ECQ’s). When describing your Technical Qualifications (TQ’s) and Executive Core Qualifications (ECQ’s), please be sure to give examples and explain how often you used these skills, the complexity of the knowledge you possessed, the level of the people you interacted with, the sensitivity of the issues you handled, etc. Your responses should describe the experience; education; and accomplishments which have provided you with the skills and knowledge required for this position. Current IC senior officers are not required to submit ECQs, but must address the TQs.

Only one note on the entire description, and it’s about that last line: “Current IC senior officers are not required to submit Executive Core Qualifications, but must address the Technical Qualifications.”  This is perhaps the most important element in the entire description; it is assumed that “current IC senior officers” know how to lead bureaucratically, how to manage a staff – but in my experience it cannot be assumed that they are necessarily current on actual trends and advances in the larger world of technology. In fact, some might say the presumption would be against that currency. Yet they must be, for a variety of reasons never more salient than in today’s chaotically-evolving world.

Good luck to applicants.

[note: my title is of course a nod to the impressive education-reform documentary “Waiting for Superman“]

 

Debating Big Data for Intelligence

I’m always afraid of engaging in a “battle of wits” only half-armed.  So I usually choose my debate opponents judiciously.

Unfortunately, I recently had a contest thrust upon me with a superior foe: my friend Mark Lowenthal, Ph.D. from Harvard, an intelligence community graybeard (literally!) and former Assistant Director of Central Intelligence (ADCI) for Analysis and Production, Vice Chairman of the National Intelligence Council – and as if that weren’t enough, a past national Jeopardy! “Tournament of Champions” winner.

As we both sit on the AFCEA Intelligence Committee and have also collaborated on a few small projects, Mark and I have had occasion to explore one another’s biases and beliefs about the role of technology in the business of intelligence. We’ve had several voluble but collegial debates about that topic, in long-winded email threads and over grubby lunches. Now, the debate has spilled onto the pages of SIGNAL Magazine, which serves as something of a house journal for the defense and intelligence extended communities.

SIGNAL Editor Bob Ackerman suggested a “Point/Counterpoint” short debate on the topic: “Is Big Data the Way Ahead for Intelligence?” Our pieces are side-by-side in the new October issue, and are available here on the magazine’s site.

Mark did an excellent job of marshalling the skeptic’s view on Big Data, under the not-so-equivocal title, Another Overhyped Fad.”  Below you will find an early draft of my own piece, an edited version of which is published under the title A Longtime Tool of the Community”:

Visit the National Cryptologic Museum in Ft. Meade, Maryland, and you’ll see three large-machine displays, labeled HARVEST and TRACTOR, TELLMAN and RISSMAN, and the mighty Cray XMP-24. They’re credited with helping win the Cold War, from the 1950s through the end of the 1980s. In fact, they are pioneering big-data computers.

Here’s a secret: the Intelligence Community has necessarily been a pioneer in “big data” since inception – both our modern IC and the science of big data were conceived during the decade after the Second World War. The IC and big-data science have always intertwined because of their shared goal: producing and refining information describing the world around us, for important and utilitarian purposes

What do modern intelligence agencies run on? They are internal combustion engines burning pipelines of data, and the more fuel they burn the better their mileage. Analysts and decisionmakers are the drivers of these vast engines, but to keep them from hoofing it, we need big data.

Let’s stipulate that today’s big-data mantra is overhyped. Too many technology vendors are busily rebranding storage or analytics as “big data systems” under the gun from their marketing departments. That caricature is, rightly, derided by both IT cognoscenti and non-techie analysts.

I personally get the disdain for machines, as I had the archetypal humanities background and was once a leather-elbow-patched tweed-jacketed Kremlinologist, reading newspapers and HUMINT for my data. I stared into space a lot, pondering the Chernenko-Gorbachev transition. Yet as Silicon Valley’s information revolution transformed modern business, media, and social behavior across the globe, I learned to keep up – and so has the IC. 

Twitter may be new, but the IC is no Johnny-come-lately in big data on foreign targets.  US Government funding of computing research in the 1940s and ‘50s stretched from World War II’s radar/countermeasures battles to the elemental ELINT and SIGINT research at Stanford and MIT, leading to the U-2 and OXCART (ELINT/IMINT platforms) and the Sunnyvale roots of NRO.

In all this effort to analyze massive observational traces and electronic signatures, big data was the goal and the bounty.

War planning and peacetime collection were built on collection of ever-more-massive amounts of foreign data from technical platforms – telling the US what the Soviets could and couldn’t do, and therefore where we should and shouldn’t fly, or aim, or collect. And all along, the development of analog and then digital computers to answer those questions, from Vannevar Bush through George Bush, was fortified by massive government investment in big-data technology for military and intelligence applications.

In today’s parlance big data typically encompasses just three linked computerized tasks: storing collected foreign data (think Amazon’s cloud), finding and retrieving relevant foreign data (Bing or Google), and analyzing connections or patterns among the relevant foreign data (powerful web-analytic tools).

Word Cloud Big Data for IntelligenceThose three Ft. Meade museum displays demonstrate how NSA and the IC pioneered those “modern” big data tasks.  Storage is represented by TELLMAN/RISSMAN, running from the 1960’s throughout the Cold War using innovation from Intel. Search/retrieval were the hallmark of HARVEST/TRACTOR, built by IBM and StorageTek in the late 1950s. Repetitive what-if analytic runs boomed in 1983 when Cray delivered a supercomputer to a customer site for the first time ever.

The benefit of IC early adoption of big data wasn’t only to cryptology – although decrypting enemy secrets would be impossible without it. More broadly, computational big-data horsepower was in use constantly during the Cold War and after, producing intelligence that guided US defense policy and treaty negotiations or verification. Individual analysts formulated requirements for tasked big-data collection with the same intent as when they tasked HUMINT collection: to fill gaps in our knowledge of hidden or emerging patterns of adversary activities.

That’s the sense-making pattern that leads from data to information, to intelligence and knowledge. Humans are good at it, one by one. Murray Feshbach, a little-known Census Bureau demographic researcher, made astonishing contributions to the IC’s understanding of the crumbling Soviet economy and its sociopolitical implications by studying reams of infant-mortality statistics, and noticing patterns of missing data. Humans can provide that insight, brilliantly, but at the speed of hand-eye coordination.

Machines make a passable rote attempt, but at blistering speed, and they don’t balk at repetitive mindnumbing data volume. Amid the data, patterns emerge. Today’s Feshbachs want an Excel spreadsheet or Hadoop table at hand, so they’re not limited to the data they can reasonably carry in their mind’s eye.

To cite a recent joint research paper from Microsoft Research and MIT, “Big Data is notable not because of its size, but because of its relationality to other data.  Due to efforts to mine and aggregate data, Big Data is fundamentally networked.  Its value comes from the patterns that can be derived by making connections between pieces of data, about an individual, about individuals in relation to others, about groups of people, or simply about the structure of information itself.” That reads like a subset of core requirements for IC analysis, whether social or military, tactical or strategic.

The synergy of human and machine for knowledge work is much like modern agricultural advances – why would a farmer today want to trudge behind an ox-pulled plow? There’s no zero-sum choice to be made between technology and analysts, and the relationship between CIOs and managers of analysts needs to be nurtured, not cleaved apart.

What’s the return for big-data spending? Outside the IC, I challenge humanities researchers to go a day without a search engine. The IC record’s just as clear. ISR, targeting and warning are better because of big data; data-enabled machine translation of foreign sources opens the world; correlation of anomalies amid large-scale financial data pinpoint otherwise unseen hands behind global events. Why, in retrospect, the Iraq WMD conclusion was a result of remarkably-small-data manipulation.

Humans will never lose their edge in analyses requiring creativity, smart hunches, and understanding of unique individuals or groups. If that’s all we need to understand the 21st century, then put down your smartphone. But as long as humans learn by observation, and by counting or categorizing those observations, I say crank the machines for all their robotic worth.

Make sure to read both sides, and feel free to argue your own perspective in a comment on the SIGNAL site.

Petraeus as Ozymandias

I only met David Petraeus once before he came to CIA, in 2006 at U.S. Central Command while he was winding up his tour as commander of the Multi-National Security Transition Command Iraq (acronymically pronounced “minsticky”), and before he took command of MNF-I or CENTCOM, or the war in Afghanistan for that matter. I briefed him on something topical going on (I was still working at DIA at the time) and we certainly didn’t talk long. In fact I came away with only one impression: not so much about him, but about his already-well-commented-on entourage of “Petraeus guys.” He had a reputation as a fast-moving reformer, but it was an outsized group of admirers, I thought, who showed not respect for him, but devotion – even awe.

They weren’t alone; the man’s been compared as a military leader to “Ulysses S. Grant, John J. Pershing, George Marshall and Dwight D. Eisenhower” – and that was by his own boss! (That’s the comparison made by Chairman of the Joint Chiefs of Staff Adm. Mike Mullen last year when Petraeus retired from the military to join CIA.)

So, yes, news that the Director of the CIA had resigned because of an extramarital affair hit DC like a thunderclap yesterday.  Check out the volume of this twitter search for the prevailing phrase people uttered when they heard the news: “Holy shit.” It was almost comic that the news broke the same day that the new James Bond film opened in DC. Its plot features an intelligence agency director under personal assault and its title mirrors the mood of many in Langley today: “Skyfall.”

I’m not surprised by the fact that a powerful man was having an affair – heck, I did marry a divorce lawyer after all.  The news won’t affect intelligence operations immediately; the professionals at CIA and the intelligence community are still going about their business and tend to look forward to the horizon, not backward. Meanwhile journalists are already delving into the particulars of this peculiar turn of events. Pundits (and the Congressional intelligence oversight committees) will be exploring any linkages or ramifications of this scandal for the Benghazi investigations, and the candidates for Petraeus’s replacement are already making their direct or whisper campaigns known, in emails already bcc’ing around the Beltway. More on that in due time.

I only have two observations now, one larger in scope and one quite small, at human scale. The first is the question of what the scandal says about the intelligence security practices in our modern national security state. Petraeus held the highest security clearances. He earned the confidence of the President, the trust of his silent warrior employees, the endorsement of the U.S. Senate (94-0!) and the faith of a nation that had cheered his battlefield successes in the Iraq surge and in Afghanistan. Yet the CIA’s confidence in its director was undergirded not only by the Petraeus resume, but by our national security infrastructure of clearances, polygraphs, and professional investigators. Forget the question of one man’s integrity – he was living a lie, big-time, and we missed it. Completely. There will be many questions asked about what that means for other high government clearance-holders, but for now there’s a feeling prevalent in DC akin to what happens when a law-enforcement crime lab discovers shoddy mistakes: all previous convictions are under suspicion and, sometimes, verdicts are reversed. Something to ponder about CIA institutional analytic or operational judgment over the past year….

Secondly, I’m struck by the ironies in the personal side of this affair. David Petraeus grew up as a literature-loving son of a New England village librarian. I know this because I read his biography – yes, the hagiographic book All In: The Education of David Petraeus written by the woman at the center of the affair. Now I may be one of the few in DC who actually read the whole book when it came out – as in, I didn’t just flip through the index looking for the “good parts.”

The book has the literature-loving Petraeus actually quoting poetry at a pivotal point in his life. At his change-of-command ceremony, giving up his praetorian position in Afghanistan, Petraeus gave a thoughtful set of remarks and then chose to quote several lines from an obscure poem by young British soldier John Bailey, serving in Afghanistan in 2008. I say “obscure,” because until today the poem itself appears in only one spot on the Internet: a small U.K. site devoted to British war poetry.  Did poetry-lover Petraeus find the poem there himself, or was it simply good staff/speechwriter work? These are the words Petraeus used, in his “emotional” farewell to the wars he had led, and to his chosen career as a military leader:

And what is asked for the service we give?

No high praise or riches if we should live,

Just silence from friends, our name on a wall,

If this time around, it is I that fall.

– from “The Volunteer” by John Bailey

When Petraeus read out that poem, he was standing like Caesar astride a narrow world, a four-star general having “won” two wars in distant ancient lands and commanded USCENTCOM, whose mission area sprawls across Africa, the Middle East and Central Asia.

Perhaps this poetry lover knows Percy Bysshe Shelley well; perhaps like me in school Petraeus read Shelley’s Ozymandias, based on the ironic life of Ramesses II, mighty Egyptian pharaoh. One account writes, “Ramesses could have filled an ancient edition of the Guinness Book of Records all by himself: he built more temples, obelisks and monuments; took more wives (eight, not counting concubines) and claimed to have sired more children (as many as 162, by some accounts) than any other pharaoh in history. And he presided over an empire that stretched from present-day Libya to Iraq in the east, as far north as Turkey and southward into the Sudan.”

Yet Ramesses is mostly forgotten now, and Shelley’s poem about him captures the fall of great men in a short, powerful sonnet. When I first heard the news about Petraeus from my wife, this is the poem I thought of, and I believe its irony pairs with the lines Petraeus quoted quite sadly.

I met a traveller from an antique land

Who said: “Two vast and trunkless legs of stone

Stand in the desert. Near them on the sand,

Half sunk, a shattered visage lies, whose frown

And wrinkled lip and sneer of cold command

Tell that its sculptor well those passions read

Which yet survive, stamped on these lifeless things,

The hand that mocked them and the heart that fed.

And on the pedestal these words appear:

`My name is Ozymandias, King of Kings:

Look on my works, ye mighty, and despair!’

Nothing beside remains. Round the decay

Of that colossal wreck, boundless and bare,

The lone and level sands stretch far away”.

Follow

Get every new post delivered to your Inbox.

Join 6,779 other followers

%d bloggers like this: