Problem Number One, Watching for Superintelligence

Two years ago, the AFCEA Intelligence Committee (I’m a member) invited Elon Musk for a special off-the-record session at our annual classified Spring Intelligence Symposium. The Committee assigned me the task of conducting a wide-ranging on-stage conversation with him, going through a variety of topics, but we spent much of our time on artificial intelligence (AI) – and particularly artificial general intelligence (AGI, or “superintelligence”).

I mention that the session was off-the-record. In my own post back in 2015 about the session, I didn’t characterize Elon’s side of the conversation or his answers to my questions – but for flavor I did include the text of one particular question on AI which I posed to him. I thought it was the most important question I asked…

(Our audience that day: the 600 attendees included a top-heavy representation of the Intelligence Community’s leadership, its foremost scientists and technologists, and executives from the nation’s defense and national-security private-sector partners.)

Here’s that one particular AI question I asked, quoted from my blogpost of 7/28/2015:

“AI thinkers like Vernor Vinge talk about the likelihood of a “Soft takeoff” of superhuman intelligence, when we might not even notice and would simply be adapting along; vs a Hard takeoff, which would be a much more dramatic explosion – akin to the introduction of Humans into the animal kingdom. Arguably, watching for indicators of that type of takeoff (soft or especially hard) should be in the job-jar of the Intelligence Community. Your thoughts?”

Months after that AFCEA session, in December 2015 Elon worked with Greg Brockman, Sam Altman, Peter Thiel and several others to establish and fund OpenAI, “a non-profit AI research company, discovering and enacting the path to safe artificial general intelligence (AGI).” OpenAI says it has a full-time staff of 60 researchers and engineers, working “to build safe AGI, and ensure AGI’s benefits are as widely and evenly distributed as possible.”

Fast-forward to today. Over the weekend I was reading through a variety of AI research and sources, keeping current in general for some of my ongoing consulting work for Deloitte’s Mission Analytics group. I noticed something interesting on the OpenAI website, specifically on a page it posted several months ago labelled “Special Projects.”

There are four such projects listed, described as “problems which are not just interesting, but whose solutions matter.” Interested researchers are invited to apply for a position at OpenAI to work on the problem – and they’re all interesting, and could lead to consequential work.

But the first Special Project problem caught my eye, because of my question to Musk the year before:

1. Detect if someone is using a covert breakthrough AI system in the world. As the number of organizations and resources allocated to AI research increases, the probability increases that an organization will make an undisclosed AI breakthrough and use the system for potentially malicious ends. It seems important to detect this. We can imagine a lot of ways to do this — looking at the news, financial markets, online games, etc.”

That reads to me like a classic “Indications & Warning” problem statement from the “other” non-AI world of intelligence.

I&W (in the parlance of the business) is a process used by defense intelligence and the IC to detect indicators of potential threats while sufficient time still exists to counter those efforts. The doctrine of seeking advantage through warning is as old as the art of war; Sun Tzu called it “foreknowledge.” There are many I&W examples from the Cold War, from the overall analytic challenge (see a classic thesis  “Anticipating Surprise“), and from specific domain challenge (see for example this 1978 CIA study, Top Secret but since declassified, on “Indications and Warning of Soviet Intentions to Use Chemical Weapons during a NATO-Warsaw Pact War“).

The I&W concept has sequentially been transferred to new domains of intelligence like Space/Counter-Space (see the 2013 DoD “Joint Publication on Space Operations Doctrine,” which describes the “unique characteristics” of the space environment for conducting I&W, whether from orbit or in other forms), and of course since 9/11 the I&W approach has been applied intensely in counter-terrorist realms in defense and homeland security.

It’s obvious Elon Musk and his OpenAI cohort believe that superintelligence is a problem worth watching. Elon’s newest company, the brain-machine-interface startup Neuralink, sets its core motivation as avoiding a future in which AGI outpaces simple human intelligence. So I’m staying abreast of indications of AGI progress.

For the AGI domain I am tracking many sources through citations and published research (see OpenAI’s interesting list here), and watching for any mention of I&W monitoring attempts or results by others which meet the challenge of what OpenAI cites as solving “Problem #1.” So far, nothing of note.

But I’ll keep a look out, so to speak.

 

 

Coming to DC, One-Day Delivery from Jeff Bezos

If you read this blog you care about government and technology. And whether you’re a technologist or not, you can see the tech forces shaping and sharpening the uses of digital capabilities in accomplishing the ends of government, whether that’s citizen-service delivery and local law enforcement, or global diplomacy and nation-state combat. I’ve worked on and written about them all – from intelligence to space to AI, or the quantification of Supreme Court humor, even “Punk Rock and Moore’s Law.”

Understanding and forecasting that radical pace of external change is difficult for government professionals, and they need help doing that. Let’s say you wanted to tap someone to offer insight. Who’d be on your dream list? At the top of my dream list – my absolute “if-only-I-could-ask” list – would be Jeff Bezos, founder of Amazon and Blue Origin.

So I’m going to sit down with Jeff Bezos on stage later this month, at the annual AFCEA Intelligence Symposium, for a conversation about areas where technology critically intersects with the nation’s response to enduring challenges and opportunities, such as artificial intelligence, digital innovation, the revolution in cloud computing, and commercial space operations. (I serve as national Vice Chairman of the Intelligence Committee at AFCEA, the 35,000-member Armed Forces Communications & Electronics Association.)

The 2017 Symposium features, as usual, a stellar line-up of top leaders in national security, with panels on Advanced Conventional Threats, the Contested Environment of Space, Terrorism, Cyber Threats from Nation-States and Non-State Entities, and Gray-Zone Conflicts/Hybrid Warfare (topic of last year’s Defense Science Board study on which I sat). All sessions feature senior thought-leaders from government and industry.

Jeff Bezos might be new in that particular mix, but you can understand why we invited him. He has been TIME Magazine’s Person of the Year (early in his career in 1999), Fortune Magazine’s Businessperson of the Year, topped the Forbes annual list of “World’s Greatest Leaders,” and our rationale for this conversation is his long track record of revolutionary contributions to international technological/economic advance, as well as to US national security. AWS is now of central importance to the public sector (including intelligence), and the broader contributions of Amazon and Blue Origin to the nation’s economic future and success are incalculable.

 

If you have a Top Secret/SI/TK clearance, you can attend the Symposium – register. [Update: sorry, 2017 Symposium = sold out]

There’s a longstanding meme that government should be “run like a business.” I typically don’t think in precisely those terms, having been on both sides and recognizing the significant differences in intent and stakeholders.

Panel on Defense Innovation and DIUx

I’ve been more interested in helping each sector understand the unique contributions of the other, and the complexities inherent in their relationship. (See for example my recent post on DoD Innovation and DIUx in Silicon Valley.)

But the “run government like a business” impetus is understandable here in the United States as a reflection of dissatisfaction with government performance in meeting its own goals, and the expectations of the citizens it serves. President Trump recently assigned Jared Kushner to lead a new White House Office of American Innovation, and Kushner told the Washington Post “We should have excellence in government. The government should be run like a great American company.” The Washington Post (coincidentally owned by, yes, Jeff Bezos) ran a piece exploring the history of that thinking, dating its surge in popularity to the early 1980s under President Reagan – a timeline borne out by running the phrase through Google’s Ngram Viewer (see chart).

The last time I invited a smart young billionaire to come speak to Intelligence Community leaders, it worked out pretty well for the audience (see “Burning Man and AI: What Elon Musk told me and the role of Art“). So I’m aiming even higher this year…

If you don’t have a Top Secret clearance, you can’t get into the Symposium, and won’t be able to hear Bezos firsthand on April 27. But here’s a substitute, nearly as good: this week Bezos published his annual Letter to Shareholders of Amazon. Most people in the business world know about his legendary 1997 “first annual letter to shareholders” in which he laid out an extraordinary long-term vision for his company. The 2017 version is also extraordinary, and I urge you to read it in full. My friend Jeff Jonas, former IBM Chief Scientist for Context Computing and now founder/Chief Scientist at Senzing, calls it “the most impressive annual letter to shareholders I’ve ever read; this line of thinking leads to greatness.”

– – – – –

For some parting eye-candy, here’s video from last week’s annual Space Symposium in Colorado Springs, where attendees got a first-hand look at the historic Blue Origin New Shepard rocket booster (first to land vertically after spaceflight, first to relaunch again, and now a five-time-reuse trophy), and an inside tour of the crew capsule with “the largest windows in space travel.”

 

 

Through the Afghan Looking Glass

The news today that the United States government will be paying $367 million dollars to Russia, for 21 Russian Mi-17 “Hip” helicopters for use by Afghanistan’s military, for some reason made me recall something I heard Monday.  I was talking about the Libya crisis to an E-Ring friend and former colleague in the Pentagon who told me, “the difficulty in Libya is that this is all new territory for us, new because it’s more complex, and so we have to figure it out as each new complication comes along.”

That’s one way of looking at modern life, as if drowning in too much data. Perhaps there’s another, driven more by longer memory, and analysis “à la recherche du temps perdu.”  Let’s set down some facts, past and present, and see if any lessons emerge. With apologies to Mark Twain whose forward to Adventures of Huckleberry Finn reads:

“Persons attempting to find a motive in this narrative will be prosecuted; persons attempting to find a moral in it will be banished; persons attempting to find a plot in it will be shot. By Order of the Author”

Once upon a time, not so long ago (the 1980s), the United States armed Afghan “rebels” against an oppressive central government and its foreign puppetmaster patron, the Soviet Union. The rebels pre-existed the foreign involvment; in fact there is difficulty finding a historical point in the region’s history when there weren’t “rebels” against anyone claiming to be “the government.” (If it’s easier for you, imagine the residents of the hills of Kentucky and West Virginia.)

Contributing to Intelligence Innovation

Below are two ways to contribute to innovation in government, and specifically in intelligence matters. One is for you to consider, the other is a fun new path for me.

DARPA crowd guru gets a new lab

It’s been a little over two years since I came back to the tech private sector from my government service, and it’s great when we have other folks take the same path, for it improves the knowledge of each side about the other. Today we’re announcing that Peter Lee, currently the leader of the Defense Advanced Research Projects Activity’s innovative Transformational Convergence Technology Office (TCTO), is joining Microsoft to run the mighty flagship Redmond labs of Microsoft Research.

An Afghanistan Echo from 1986

In all the hubbub over Gen. Stanley McChrystal’s disastrous Rolling Stone profile which sparked an international furor today, I notice there hasn’t been time yet for most Beltway armchair analysts to focus on the article’s actual depiction of the state of American policy in Afghanistan.

To sum up: grim.  The quotes from McChrystal’s team reinforce the assessment – there’s little confidence on display. (Here’s the full article in pdf, it’s worth the read.)  As the RS article’s last lines put it: “There is a reason that President Obama studiously avoids using the word “victory” when he talks about Afghanistan. Winning, it would seem, is not really possible. Not even with Stanley McChrystal in charge.”

Is that an unfair assessment, too bleak? I’ve been a fairly consistent supporter of the Afghanistan war since the inception, but even I was struck that a “senior military official in Kabul” is quoted in the article saying: “There’s a possibility we could ask for another surge of U.S. forces next summer if we see success here.”  So even hypothesized success of McChrystal’s current surge would result in more troops, not less, heading for the fight – a decade in. 

Net-clever personal PSYOP targeting

In a way I’ve been studying Information Operations (info-ops, or IO) and Psychological Operations (PSYOP) all my life. We all have – particularly if you grew up in the marketing-saturated post-World War II United States. But I also started reading intently about those practices when I first worked at the Cold War Pentagon in the mid-1980s.

Those two terms have specific meanings in a military and international-relations context. The Pentagon’s official doctrinal definitions can be found in Joint Publication 3-13, “Information Operations” (updated in 2006) and in Joint Pub 3-53 “Doctrine for Joint Psychological Operations” (dating back to 2003). They make plain that PSYOP is one of “five core IO capabilities: electronic warfare, computer network operations, psychological operations, operations security, and military deception.”  As the latter manual states, “The overall function of PSYOP is to cause selected foreign audiences to take actions favorable to the objectives of the United States and its allies or coalition partners” (page xii).

Follow the USS Carl Vinson to Haiti

As I write on Wednesday afternoon (EST), the scenes of chaos, death, and destruction in Haiti are only now beginning to be visible to the outside world through media. As horrific and heart-rending as those scenes are, they serve a purpose in letting other nations comprehend the magnitude of the crisis and the urgency required in lending direct aid. The U.S. military is uniquely positioned to contribute.

Flight Deck of the USS Carl Vinson

What a difference a day makes: barely 24 hours ago, several hours before the earthquake struck, the Nimitz-class supercarrier USS Carl Vinson (CVN 70) was cranking up its nuclear engines and setting a peaceful course out of Hampton Roads at the base of the Chesapeake Bay, Virginia. At long last after completing a complex overhaul and new sea trials, she was heading south, to make the South America turn and return to homeport in San Diego as part of the Pacific Fleet.

To fix intelligence analysis you have to decide what’s broken

“More and more, Xmas Day failure looks to be wheat v. chaff issue, not info sharing issue.” – Marc Ambinder, politics editor for The Atlantic, on Twitter last night.

Marc Ambinder, a casual friend and solid reporter, has boiled down two likely avenues of intelligence “failure” relevant to the case of Umar Farouk Abdulmutallab and his attempted Christmas Day bombing on Northwest Airlines Flight 253.  In his telling, they’re apparently binary – one is true, not the other, at least for this case.

The two areas were originally signalled by President Obama in his remarks on Tuesday, when he discussed the preliminary findings of “a review of our terrorist watch list system …  so we can find out what went wrong, fix it and prevent future attacks.” 

Let’s examine these two areas of failure briefly – and what can and should be done to address them.

The Purple History of Intelink

When I first began talking with DIA CIO Mike Pflueger and Deputy CIO Mark Greer in the fall of 2003 about the work I’d be doing with them inside government, most of the ideas were big ones: let’s re-architect the DoDIIS enterprise, let’s find and deploy revolutionary new analytical software. One of our thoughts was a little one, but for me personally it turned out to be a most valuable project. They let me pull together a panel for the upcoming 2004 DoDIIS Conference called “Geeks and Geezers,” featuring some of the grand old names of intelligence technology. The panel was a success, and in organizing it, I spent quite a bit of time talking to those giants, or should I say listening to them. I learned an enormous amount about “the early days.” This post describes the important work of one of those fellows.