Problem Number One, Watching for Superintelligence

Two years ago, the AFCEA Intelligence Committee (I’m a member) invited Elon Musk for a special off-the-record session at our annual classified Spring Intelligence Symposium. The Committee assigned me the task of conducting a wide-ranging on-stage conversation with him, going through a variety of topics, but we spent much of our time on artificial intelligence (AI) – and particularly artificial general intelligence (AGI, or “superintelligence”).

I mention that the session was off-the-record. In my own post back in 2015 about the session, I didn’t NGA Photo: Lewis Shepherd, Elon Musk 2015characterize Elon’s side of the conversation or his answers to my questions – but for flavor I did include the text of one particular question on AI which I posed to him. I thought it was the most important question I asked…

(Our audience that day: the 600 attendees included a top-heavy representation of the Intelligence Community’s leadership, its foremost scientists and technologists, and executives from the nation’s defense and national-security private-sector partners.)

Here’s that one particular AI question I asked, quoted from my blogpost of 7/28/2015:

“AI thinkers like Vernor Vinge talk about the likelihood of a “Soft takeoff” of superhuman intelligence, when we might not even notice and would simply be adapting along; vs a Hard takeoff, which would be a much more dramatic explosion – akin to the introduction of Humans into the animal kingdom. Arguably, watching for indicators of that type of takeoff (soft or especially hard) should be in the job-jar of the Intelligence Community. Your thoughts?”

Months after that AFCEA session, in December 2015 Elon worked with Greg Brockman, Sam Altman, Peter Thiel and several others to establish and fund OpenAI, “a non-profit AI research company, discovering and enacting the path to safe artificial general intelligence (AGI).” OpenAI says it has a full-time staff of 60 researchers and engineers, working “to build safe AGI, and ensure AGI’s benefits are as widely and evenly distributed as possible.”

Fast-forward to today. Over the weekend I was reading through a variety of AI research and sources, keeping SpecialProjectscurrent in general for some of my ongoing consulting work for Deloitte’s Mission Analytics group. I noticed something interesting on the OpenAI website, specifically on a page it posted several months ago labelled Special Projects.”

There are four such projects listed, described as “problems which are not just interesting, but whose solutions matter.” Interested researchers are invited to apply for a position at OpenAI to work on the problem – and they’re all interesting, and could lead to consequential work.

But the first Special Project problem caught my eye, because of my question to Musk the year before:

  1. Detect if someone is using a covert breakthrough AI system in the world. As the number of organizations and resources allocated to AI research increases, the probability increases that an organization will make an undisclosed AI breakthrough and use the system for potentially malicious ends. It seems important to detect this. We can imagine a lot of ways to do this — looking at the news, financial markets, online games, etc.”

That reads to me like a classic “Indications & Warning” problem statement from the “other” non-AI world of intelligence.

I&W (in the parlance of the business) is a process used by defense intelligence and the IC to detect indicators of potential threats while sufficient time still exists to counter those efforts. The doctrine of seeking advantage through warning is as old as the art of war; Sun Tzu called it “foreknowledge.” There are many I&W examples from the Cold War, from the overall analytic challenge (see a classic thesis  Anticipating Surprise“), and from specific domain challenge (see for example this 1978 CIA study, Top Secret but since declassified, on “Indications and Warning of Soviet Intentions to Use Chemical Weapons during a NATO-Warsaw Pact War“).

The I&W concept has sequentially been transferred to new domains of intelligence like Space/Counter-Space (see the 2013 DoD “Joint Publication on Space Operations Doctrine,” which describes the “unique characteristics” of the space environment for conducting I&W, whether from orbit or in other forms), and of course since 9/11 the I&W approach has been applied intensely in counter-terrorist realms in defense and homeland security.

It’s obvious Elon Musk and his OpenAI cohort believe that superintelligence is a problem worth watching. Elon’s newest company, the brain-machine-interface startup Neuralink, sets its core motivation as avoiding a future in which AGI outpaces simple human intelligence. So I’m staying abreast of indications of AGI progress.

For the AGI domain I am tracking many sources through citations and published research (see OpenAI’s interesting list here), and watching for any mention of I&W monitoring attempts or results by others which meet the challenge of what OpenAI cites as solving “Problem #1.” So far, nothing of note.

But I’ll keep a look out, so to speak.

 

 

Video of DoD Innovation Discussion at Cybersecurity Summit

Earlier this week I wrote (“Beware the Double Cyber Gap“) about an upcoming Cybersecurity Summit, arranged by AFCEA-DC, for which I would be a panelist on innovation and emerging technologies for defense.

The Summit was a big success, and in particular I was impressed with the level and quality of interaction between the government participants and their private-sector counterparts, both on stage and off. Most of the sessions were filmed, and are now available at http://www.cybersecuritytv.net.

You can watch our panel’s video, “Partnering with Industry for Innovation,” and it will provide an up-to-the-moment view of how US Cyber Command and the Department of Defense as a whole are attacking the innovation challenge, featuring leadership from the USCYBERCOM Capabilities Development Group, and the Defense Innovation Unit-Experimental. Solarflare CEO Russ Stern (a serial entrepreneur from California) and I offered some historical, technical, market, and regulatory context for the challenge those two groups face in finding the best technologies for national security. Most of my remarks are after the 16:00 minute mark; click the photo below to view the video:

photo: Lewis Shepherd; Gen. “Wheels” Wheeler (Ret.) of DIUx; Russell Stern, CEO Solarflare

From my remarks:

“I’m here to provide context. I’ve been in both these worlds – I came from Silicon Valley; I came to the Defense Intelligence Agency after 9/11, and found all of these broken processes, all of these discontinuities between American innovation & ingenuity on one hand, and the Defense Department & the IC & government at large…
Silicon was a development of government R&D money through Bell Labs, the original semiconductor; so we have to realize the context that there’s been a massive disruption in the divorcing of American industry and the technology industry, from the government and the pull of defense and defense needs. That divorcing has been extremely dramatic just in the past couple of years post-Snowden, emblematically exemplified with Apple telling the FBI, “No thanks, we don’t think we’ll help you on that national security case.”
So these kinds of efforts like DIUx are absolutely essential, but you see the dynamic here, the dynamic now is the dog chasing the tail – the Defense Department chasing what has become a massive globally disruptive and globally responsive technology industry…  This morning we had the keynote from Gen. Touhill, the new federal Chief Information Security Officer, and Greg told us that what’s driving information security, the entire industry and the government’s response to it is the Internet – through all its expressions, now Internet of Things and everything else – so let’s think about the massive disruption in the Internet just over the last five years.
Five years ago, the top ten Internet companies measured by eyeballs, by numbers of users, the Top 10 were all American companies, and it’s all the ones you can name: Amazon, Google, Microsoft, Facebook, Wikipedia, Yahoo… Guess what, three years ago the first crack into that Top 10, only six of those companies were American companies, and four – Alibaba, Baidu, Tencent, and Sohu – were Chinese companies. And guess what, today only five are American companies, and those five – Google, Amazon, Microsoft, Facebook, Yahoo – eighty percent or more of their users are non-U.S. Not one of those American internet companies has even twenty percent of their user-base being U.S. persons, U.S. citizens. Their market, four out of five of their users are global.
So when [DoD] goes to one of these CEOs and says, “Hey c’mon, you’re an American” – well, maybe, maybe not. That’s a tough case to sell. Thank God we have these people, with the guts and drive and the intellect to be able to try and make this case, that technological innovation can and must serve our national interest, but that’s an increasingly difficult case to make when [internet] companies are now globally mindsetted, globally incentivized, globally prioritizing constantly…”

Kudos to my fellow panelists for their insights, and their ongoing efforts, and to AFCEA for continuing its role in facilitating important government/industry partnerships.

Through the Afghan Looking Glass

The news today that the United States government will be paying $367 million dollars to Russia, for 21 Russian Mi-17 “Hip” helicopters for use by Afghanistan’s military, for some reason made me recall something I heard Monday.  I was talking about the Libya crisis to an E-Ring friend and former colleague in the Pentagon who told me, “the difficulty in Libya is that this is all new territory for us, new because it’s more complex, and so we have to figure it out as each new complication comes along.”

That’s one way of looking at modern life, as if drowning in too much data. Perhaps there’s another, driven more by longer memory, and analysis “à la recherche du temps perdu.”  Let’s set down some facts, past and present, and see if any lessons emerge. With apologies to Mark Twain whose forward to Adventures of Huckleberry Finn reads:

“Persons attempting to find a motive in this narrative will be prosecuted; persons attempting to find a moral in it will be banished; persons attempting to find a plot in it will be shot. By Order of the Author”

Once upon a time, not so long ago (the 1980s), the United States armed Afghan “rebels” against an oppressive central government and its foreign puppetmaster patron, the Soviet Union. The rebels pre-existed the foreign involvment; in fact there is difficulty finding a historical point in the region’s history when there weren’t “rebels” against anyone claiming to be “the government.” (If it’s easier for you, imagine the residents of the hills of Kentucky and West Virginia.)

Continue reading

Increasing Jointness and Reducing Duplication in DoD Intelligence

Today I’m publishing an important guest-essay, with a brief introduction.  Last month the Wall Street Journal published a 12-part online series about college graduates and their paths to success, featuring surveys and input from job recruiters. One thing caught my eye, at least when blogged by an acquaintance, Prof. Kristan Wheaton of the Mercyhurst College Institute Of Intelligence Studies. The WSJ’s study included a look at recent graduates’ job satisfaction in their new careers, and as Prof. Wheaton strikingly put it in his own blogpost:

Intelligence Analysts are Insanely Happy.” 

I’m pretty sure that’s not really true by and large; Prof. Wheaton seems slightly dubious as well. Many readers of this blog are intelligence analysts themselves, so I’d love to hear from you (in comments or email) about your degree of giddyness….

We all know that the intelligence-analysis field as currently practiced in U.S. agencies bears many burdens weighing heavily on job satisfaction, and unfortunately weighing on successful performance.  Our youngest and our most experienced intelligence analysts have been battling those burdens. 

One analyst has now put constructive thoughts on paper, most immediately in response to a call by Defense Secretary Bob Gates asking DoD military and civilian employees to submit their ideas to save money, avoid cost, reduce cycle time and increase the agility of the department (see more about the challenge here).  

Continue reading

DARPA crowd guru gets a new lab

It’s been a little over two years since I came back to the tech private sector from my government service, and it’s great when we have other folks take the same path, for it improves the knowledge of each side about the other. Today we’re announcing that Peter Lee, currently the leader of the Defense Advanced Research Projects Activity’s innovative Transformational Convergence Technology Office (TCTO), is joining Microsoft to run the mighty flagship Redmond labs of Microsoft Research.

Continue reading

An Afghanistan Echo from 1986

In all the hubbub over Gen. Stanley McChrystal’s disastrous Rolling Stone profile which sparked an international furor today, I notice there hasn’t been time yet for most Beltway armchair analysts to focus on the article’s actual depiction of the state of American policy in Afghanistan.

To sum up: grim.  The quotes from McChrystal’s team reinforce the assessment – there’s little confidence on display. (Here’s the full article in pdf, it’s worth the read.)  As the RS article’s last lines put it: “There is a reason that President Obama studiously avoids using the word “victory” when he talks about Afghanistan. Winning, it would seem, is not really possible. Not even with Stanley McChrystal in charge.”

Is that an unfair assessment, too bleak? I’ve been a fairly consistent supporter of the Afghanistan war since the inception, but even I was struck that a “senior military official in Kabul” is quoted in the article saying: “There’s a possibility we could ask for another surge of U.S. forces next summer if we see success here.”  So even hypothesized success of McChrystal’s current surge would result in more troops, not less, heading for the fight – a decade in. 

Continue reading

Net-clever personal PSYOP targeting

In a way I’ve been studying Information Operations (info-ops, or IO) and Psychological Operations (PSYOP) all my life. We all have – particularly if you grew up in the marketing-saturated post-World War II United States. But I also started reading intently about those practices when I first worked at the Cold War Pentagon in the mid-1980s.

Those two terms have specific meanings in a military and international-relations context. The Pentagon’s official doctrinal definitions can be found in Joint Publication 3-13, “Information Operations” (updated in 2006) and in Joint Pub 3-53 “Doctrine for Joint Psychological Operations” (dating back to 2003). They make plain that PSYOP is one of “five core IO capabilities: electronic warfare, computer network operations, psychological operations, operations security, and military deception.”  As the latter manual states, “The overall function of PSYOP is to cause selected foreign audiences to take actions favorable to the objectives of the United States and its allies or coalition partners” (page xii).

Continue reading

%d bloggers like this: