Bullshit Detector Prototype Goes Live

I like writing about cool applications of technology that are so pregnant with the promise of the future, that they have to be seen to be believed, and here’s another one that’s almost ready for prime time.

TruthTeller PrototypeThe Washington Post today launched an exciting new technology prototype invoking powerful new technologies for journalism and democratic accountability in politics and government. As you can see from the screenshot (left), it runs an automated fact-checking algorithm against the streaming video of politicians or other talking heads and displays in real time a “True” or “False” label as they’re speaking.

Called “Truth Teller,” the system uses technologies from Microsoft Research and Windows Azure cloud-computing services (I have included some of the technical details below).

But first, a digression on motivation. Back in the late 1970s I was living in Europe and was very taken with punk rock. Among my favorite bands were the UK’s anarcho-punk collective Crass, and in 1980 I bought their compilation LP “Bullshit Detector,” whose title certainly appealed to me because of my equally avid interest in politics :)

Today, my driving interests are in the use of novel or increasingly powerful technologies for the public good, by government agencies or in the effort to improve the performance of government functions. Because of my Jeffersonian tendencies (I did after all take a degree in Government at Mr. Jefferson’s University of Virginia), I am even more interested in improving government accountability and popular control over the political process itself, and I’ve written or spoken often about the “Government 2.0″ movement.

In an interview with GovFresh several years ago, I was asked: “What’s the killer app that will make Gov 2.0 the norm instead of the exception?”

My answer then looked to systems that might “maintain the representative aspect (the elected official, exercising his or her judgment) while incorporating real-time, structured, unfiltered but managed visualizations of popular opinion and advice… I’m also a big proponent of semantic computing – called Web 3.0 by some – and that should lead the worlds of crowdsourcing, prediction markets, and open government data movements to unfold in dramatic, previously unexpected ways. We’re working on cool stuff like that.”

The Truth Teller prototype is an attempt to construct a rudimentary automated “Political Bullshit Detector, and addresses each of those factors I mentioned in GovFresh - recognizing the importance of political leadership and its public communication, incorporating iterative aspects of public opinion and crowd wisdom, all while imbuing automated systems with semantic sense-making technology to operate at the speed of today’s real world.

Real-time politics? Real-time truth detection.  Or at least that’s the goal; this is just a budding prototype, built in three months.

Cory Haik, who is the Post’s Executive Producer for Digital News, says it “aims to fact-check speeches in as close to real time as possible” in speeches, TV ads, or interviews. Here’s how it works:

The Truth Teller prototype was built and runs with a combination of several technologies — some new, some very familiar. We’ve combined video and audio extraction with a speech-to-text technology to search a database of facts and fact checks. We are effectively taking in video, converting the audio to text (the rough transcript below the video), matching that text to our database, and then displaying, in real time, what’s true and what’s false.

We are transcribing videos using Microsoft Audio Video indexing service (MAVIS) technology. MAVIS is a Windows Azure application which uses State of the Art of Deep Neural Net (DNN) based speech recognition technology to convert audio signals into words. Using this service, we are extracting audio from videos and saving the information in our Lucene search index as a transcript. We are then looking for the facts in the transcription. Finding distinct phrases to match is difficult. That’s why we are focusing on patterns instead.

We are using approximate string matching or a fuzzy string searching algorithm. We are implementing a modified version Rabin-Karp using Levenshtein distance algorithm as our first implementation. This will be modified to recognize paraphrasing, negative connotations in the future.

What you see in the prototype is actual live fact checking — each time the video is played the fact checking starts anew.

 - Washington Post, “Debuting Truth Teller

The prototype was built with funding from a Knight Foundation’s Prototype Fund grant, and you can read more about the motivation and future plans over on the Knight Blog, and you can read TechCrunch discussing some of the political ramifications of the prototype based on the fact-checking movement in recent campaigns.

Even better, you can actually give Truth Teller a try here, in its infancy.

What other uses could be made of semantic “truth detection” or fact-checking, in other aspects of the relationship between the government and the governed?

Could the justice system use something like Truth Teller, or will human judges and  juries always have a preeminent role in determining the veracity of testimony? Will police officers and detectives be able to use cloud-based mobile services like Truth Teller in real time during criminal investigations as they’re evaluating witness accounts? Should the Intelligence Community be running intercepts of foreign terrorist suspects’ communications through a massive look-up system like Truth Teller?

Perhaps, and time will tell how valuable – or error-prone – these systems can be. But in the next couple of years we will be developing (and be able to assess the adoption of) increasingly powerful semantic systems against big-data collections, using faster and faster cloud-based computing architectures.

In the meantime, watch for further refinements and innovation from The Washington Post’s prototyping efforts; after all, we just had a big national U.S.  election but congressional elections in 2014 and the presidential race in 2016 are just around the corner. Like my fellow citizens, I will be grateful for any help in keeping candidates accountable to something resembling “the truth.”

Kinecting Communities

On April 16 I will be speaking at the Mobile Citizen Summit in Washington DC (registration still open), which brings together “practitioners across the  government, nonprofit, advocacy, and political spaces—the kinds of  people who develop the strategy and the tools to reach, engage, educate,  and enable citizens across the country and around the world.”

But I’m going to be talking about “mobile” in a different way than others still use the term, i.e. they focus on a handheld device, while I will be focusing on the mobile citizen. As I have said before I don’t believe our future involves experiencing “augmented reality” by always holding up little 3-inch plastic screens in front of our faces. Natural user interfaces and immersive computing offer much more to how we access computational resources – and how technology will help us interact with one another. Here’s an example, in a story from the past week.

Continue reading

Through the Afghan Looking Glass

The news today that the United States government will be paying $367 million dollars to Russia, for 21 Russian Mi-17 “Hip” helicopters for use by Afghanistan’s military, for some reason made me recall something I heard Monday.  I was talking about the Libya crisis to an E-Ring friend and former colleague in the Pentagon who told me, “the difficulty in Libya is that this is all new territory for us, new because it’s more complex, and so we have to figure it out as each new complication comes along.”

That’s one way of looking at modern life, as if drowning in too much data. Perhaps there’s another, driven more by longer memory, and analysis “à la recherche du temps perdu.”  Let’s set down some facts, past and present, and see if any lessons emerge. With apologies to Mark Twain whose forward to Adventures of Huckleberry Finn reads:

“Persons attempting to find a motive in this narrative will be prosecuted; persons attempting to find a moral in it will be banished; persons attempting to find a plot in it will be shot. By Order of the Author”

Once upon a time, not so long ago (the 1980s), the United States armed Afghan “rebels” against an oppressive central government and its foreign puppetmaster patron, the Soviet Union. The rebels pre-existed the foreign involvment; in fact there is difficulty finding a historical point in the region’s history when there weren’t “rebels” against anyone claiming to be “the government.” (If it’s easier for you, imagine the residents of the hills of Kentucky and West Virginia.)

Continue reading

The wikileaks label ticks off Wikipedia cofounder

Note from Lewis: I have commented on the latest Wikileaks outrage elsewhere (Facebook, Twitter), making clear my thoughts for what they’re worth.  Briefly, they summarize in pointing out that the U.S. Government has now allowed a dynamic to emerge without challenge: an “acceptable” intermediary between Traitor and Public. The original insider-threat individual who ripped the 251,000 cables and all the other previously leaked Iraq war data would likely not have been able to simply provide that to the New York Times personally and have it immediately published; they might have turned him in themselves. But the miraculous creation of a self-appointed, self-sanctified group like Wikileaks has allowed motivated groups like the Times & the UK’s Guardian to proclaim that their hands are clean. I find it outrageous. But the government did not press the point after the first major release (Iraq war data) with any forceful intent, so now we’re simply going to see this continue – until an Administration gets serious with criminal charges including treason for anyone involved, right up the chain of those stealing/mediating/publishing classified information.

 An online friend, Larry Sanger, today posted some very thoughtful remarks from a unique perspective – as a cofounder of Wikipedia who obviously is offended among other things by the misleading use of “Wiki-” in the Wikileaks name.  But he makes some other profound points as well. He offered to have them reposted, which I have done below. Reader comments are welcome, either below or as always by email.

 Continue reading

Increasing Jointness and Reducing Duplication in DoD Intelligence

Today I’m publishing an important guest-essay, with a brief introduction.  Last month the Wall Street Journal published a 12-part online series about college graduates and their paths to success, featuring surveys and input from job recruiters. One thing caught my eye, at least when blogged by an acquaintance, Prof. Kristan Wheaton of the Mercyhurst College Institute Of Intelligence Studies. The WSJ’s study included a look at recent graduates’ job satisfaction in their new careers, and as Prof. Wheaton strikingly put it in his own blogpost:

Intelligence Analysts are Insanely Happy.” 

I’m pretty sure that’s not really true by and large; Prof. Wheaton seems slightly dubious as well. Many readers of this blog are intelligence analysts themselves, so I’d love to hear from you (in comments or email) about your degree of giddyness….

We all know that the intelligence-analysis field as currently practiced in U.S. agencies bears many burdens weighing heavily on job satisfaction, and unfortunately weighing on successful performance.  Our youngest and our most experienced intelligence analysts have been battling those burdens. 

One analyst has now put constructive thoughts on paper, most immediately in response to a call by Defense Secretary Bob Gates asking DoD military and civilian employees to submit their ideas to save money, avoid cost, reduce cycle time and increase the agility of the department (see more about the challenge here).  

Continue reading

An Afghanistan Echo from 1986

In all the hubbub over Gen. Stanley McChrystal’s disastrous Rolling Stone profile which sparked an international furor today, I notice there hasn’t been time yet for most Beltway armchair analysts to focus on the article’s actual depiction of the state of American policy in Afghanistan.

To sum up: grim.  The quotes from McChrystal’s team reinforce the assessment – there’s little confidence on display. (Here’s the full article in pdf, it’s worth the read.)  As the RS article’s last lines put it: “There is a reason that President Obama studiously avoids using the word “victory” when he talks about Afghanistan. Winning, it would seem, is not really possible. Not even with Stanley McChrystal in charge.”

Is that an unfair assessment, too bleak? I’ve been a fairly consistent supporter of the Afghanistan war since the inception, but even I was struck that a “senior military official in Kabul” is quoted in the article saying: “There’s a possibility we could ask for another surge of U.S. forces next summer if we see success here.”  So even hypothesized success of McChrystal’s current surge would result in more troops, not less, heading for the fight – a decade in. 

Continue reading

To fix intelligence analysis you have to decide what’s broken

“More and more, Xmas Day failure looks to be wheat v. chaff issue, not info sharing issue.” – Marc Ambinder, politics editor for The Atlantic, on Twitter last night.

Marc Ambinder, a casual friend and solid reporter, has boiled down two likely avenues of intelligence “failure” relevant to the case of Umar Farouk Abdulmutallab and his attempted Christmas Day bombing on Northwest Airlines Flight 253.  In his telling, they’re apparently binary – one is true, not the other, at least for this case.

The two areas were originally signalled by President Obama in his remarks on Tuesday, when he discussed the preliminary findings of “a review of our terrorist watch list system …  so we can find out what went wrong, fix it and prevent future attacks.” 

Let’s examine these two areas of failure briefly – and what can and should be done to address them.

Continue reading

Total Recall for Public Servants

MyLifeBits is a Microsoft Research project led by the legendary Gordon Bell, designed to put “all of his atom- and electron-based bits in his local Cyberspace….MyLifeBits includes everything he has accumulated, written, photographed, presented, and owns (e.g. CDs).” 

SenseCam - Click to enlarge

Among other technical means, Bell uses the SenseCam, a remarkable prototype from Microsoft Research.  It’s a nifty little wearable device that combines high-capacity memory, a fisheye lens passively capturing 3,000 images a day, along with an infrared sensor, temperature sensor, light sensor, accelerometer, and USB interface. My group has played with SenseCam a bit, and shared it with quite a few interested government parties and partners. More info on SenseCam here, and more on its parent Sensors and Devices Group in MSR.  

Continue reading

Inside Cyber Warfare

One year ago, the buzz across the government/technology nexus was focused on a pair of political guessing games. Neophytes mostly engaged in debating over whom the newly-elected President would name to be the nation’s first Chief Technology Officer. Grizzled Pentagon veterans and the more sober Silicon Valley types wondered instead who would get the nod as President Obama’s “Cyber Czar.”

Continue reading

Cyber Deterrence Symposium webcast

As I type this, I’m sitting in a seventh-floor conference area at George Washington University’s Elliott School of International Affairs, listening to the keynote speaker for the second of five panels today in the “Cyber Deterrence Symposium,” a joint production of INSA (the Intelligence and National Security Alliance), and the Homeland Security Policy Institute.

If you’re reading this on the day of the symposium (Monday November 2, 2009), you can tune in to the live webcast of the speakers and panels. It is a stellar line-up, see the roster below.

Continue reading

Follow

Get every new post delivered to your Inbox.

Join 6,241 other followers

%d bloggers like this: