Tearing the Roof off a 2-Terabyte House

I was home last night playing with the new Kinect, integrating it with Twitter, Facebook, and Zune. Particularly because of the last service, I was glad that I got the Xbox 360 model with the 250-gigabyte (gb) hard disk drive. It holds a lot more music, or photos, and of course primarily games and game data.

So we wind up with goofy scenes like my wife zooming along yesterday in Kinect Adventures’ River Rush – not only my photo (right) but in-game photos taken by the Kinect Sensor, sitting there below the TV monitor.

Later as I was waving my hands at the TV screen, swiping magically through the air to sweep through Zune’s albums and songs as if pawing through a shelf of actual LP’s, I absent-mindedly started totting up the data-storage capacity of devices and drives in my household.  Here’s a rough accounting:

• One Zune music-player, 120gb;
• 2 old iPods 30gb + 80gb;
• an iPad 3G at 16gb;
• one HP netbook 160gb;
• an aging iMac G5 with 160gb;
• three Windows laptops of 60gb, 150gb, and 250gb;
• a DirecTV DVR with a 360gb disk;
• a single Seagate 750gb external HDD;
• a few 1gb, 2gb, and a single 32gb SD cards for cameras;
• a handful of 2gb, 4gb, and one 16gb USB flash drives;
• and most recently a 250gb Xbox 360, for Kinect. 

All told, I’d estimate that my household data storage capacity totals 2.5 terabytes. A terabyte, you’ll recall, is 10¹² bytes, or 1,000,000,000,000 (1 trillion) bytes, or alternately a thousand gigabytes.

Mix, Rip, Burn Your Research

You’ve done research; you’ve collected and sifted through mounds of links, papers, articles, notes and raw data. Shouldn’t there be a way to manage all that material that’s as easy and intuitive as, say, iTunes or Zune – helping you manage and share your snippets and research the way you share and enjoy your music?

Your choice, Dataviz as event or book

A friend wrote asking if I could make it to an event happening this week near DC. I can’t make it, but fortunately he also mentioned as consolation that he has a cool new book on the cusp of release – and I’ve now ordered my copy.

The Friend: legendary visualization and HCI guru Ben Shneiderman (Wikipedia entry). Ben is a computer-science professor at the University of Maryland and the founder of its well-known Human-Computer Interaction Laboratory (HCIL), as well as an ACM Fellow and AAAS Fellow.  He has done government a million favors over the years, consulting for agencies, including his recent work on the Recovery.gov site to help that platform of data – from hundreds of thousands of sources – organize, host, and visualize the data for millions of visitors.  I first got to know Ben through his support for better intelligence analysis – he helped invent a longtime intelligence analytics tool, Spotfire (see his article “Dynamic queries, starfield displays, and the path to Spotfire“).  Ben’s also well-known for his award-winning 2002 book Leonardo’s Laptop: Human Needs and the New Computing Technologies, which I enjoyed and still think about when brainstorming new techie toys.

Free Tools for the New Scientific Revolution

Blogs are great for supplementing real-life events, by giving space and time for specific examples and links which can’t be referenced at the time. I was invited to give a talk last week at the first-ever NASA Information Technology Summit in Washington DC, and the topic I chose was “Government and the Revolution in Scientific Computing.” That’s an area that Microsoft Research has been focusing on quite a bit lately, so below I’ll give some examples I didn’t use at my talk.

One groundrule was that invited private-sector speakers were not allowed to give anything resembling a “sales pitch” of their company’s wares. Fair enough – I’m no salesman.  The person who immediately preceded me, keynoter Vint Cerf, slightly bent the rules and talked a bit about his employer Google’s products, but gee whiz, that’s the prerogative of someone who is in large part responsible for the Internet we all use and love today.

I described in my talk the radical new class of super-powerful technologies enabling large-data research and computing on platforms of real-time and archival government data. That revolution is happening now, and I believe government could and should be playing a different and less passive role. I advocated for increased attention to the ongoing predicament of U.S. research and development funding.

Alex Howard at O’Reilly Radar covered the NASA Summit and today published a nice review of both Vint’s talk and mine.  Some excerpts:

Enabling Eureka via Citeability

The story of Archimedes resonates with everyone, because we all regularly feel that rush of excitement that he famously felt when discovering the principle of water displacement: “Eureka!” he shouted, “I have found it!”

Whether it’s car keys or the perfect birthday present for a loved one, we know that feeling. But how often do you feel like shouting “Eureka” when you’re surfing the web looking for a particular piece of government information?

To fix intelligence analysis you have to decide what’s broken

“More and more, Xmas Day failure looks to be wheat v. chaff issue, not info sharing issue.” – Marc Ambinder, politics editor for The Atlantic, on Twitter last night.

Marc Ambinder, a casual friend and solid reporter, has boiled down two likely avenues of intelligence “failure” relevant to the case of Umar Farouk Abdulmutallab and his attempted Christmas Day bombing on Northwest Airlines Flight 253.  In his telling, they’re apparently binary – one is true, not the other, at least for this case.

The two areas were originally signalled by President Obama in his remarks on Tuesday, when he discussed the preliminary findings of “a review of our terrorist watch list system …  so we can find out what went wrong, fix it and prevent future attacks.” 

Let’s examine these two areas of failure briefly – and what can and should be done to address them.

Total Recall for Public Servants

MyLifeBits is a Microsoft Research project led by the legendary Gordon Bell, designed to put “all of his atom- and electron-based bits in his local Cyberspace….MyLifeBits includes everything he has accumulated, written, photographed, presented, and owns (e.g. CDs).” 

SenseCam - Click to enlarge

Among other technical means, Bell uses the SenseCam, a remarkable prototype from Microsoft Research.  It’s a nifty little wearable device that combines high-capacity memory, a fisheye lens passively capturing 3,000 images a day, along with an infrared sensor, temperature sensor, light sensor, accelerometer, and USB interface. My group has played with SenseCam a bit, and shared it with quite a few interested government parties and partners. More info on SenseCam here, and more on its parent Sensors and Devices Group in MSR.  

Education for Information Security in a Connected World

Much of what I work on involves technologies which address information security and cyber security. So I have to ask, Who is training our next generation of technologists? And are those educators doing enough to focus on the dynamically changing demands of Information Security?

Those fundamental questions took me to Chicago recently, to take part in a roundtable discussion sponsored by DeVry University, “The Demand for Information Security in a Connected World.”

Gunning the Microsoft Semantic Engine

New Bing Maps Beta with embedded data layers from Twitter and other social feeds, click to enlarge screenshot

There’s a lot of information on the Internet already. Every day, more is added – a lot more. And while there are a concomitant number of new analytic or sense-making tools on the web, they butt up against the fact that the data – the all-important data – is held in multiple places, formats, and platforms.

How are we going to deal with all this? One approach is almost mechanical: ensuring that datasets can be accessed commonly, as in our new Microsoft Dallas platform associated with the Windows Azure cloud platform.  In the government realm, the anticipated reliance on “government-as-a-platform” (a meme popularized by Tim O’Reilly) holds promise in allowing somewhat aggregated datasets, openly accessible.

Data in the Cloud from Dallas to Mars

There’s a lot going on at this week’s Microsoft Professional Developers Conference (PDC 09); it’s a traditional launchpad for cool new stuff. I thought I’d point out several of the government-relevant announcements and technology roll-outs.

I specifically want to spotlight something called Codename Dallas, and how NASA and others have begun using it. In the keynote this morning Microsoft’s Chief Software Architect Ray Ozzie told PDC attendees (and his streaming-video audience) that a landslide of new sensors and observational systems are changing the world by recording “unimaginable volumes of data… But this data does no good unless we turn the potential into the kenetic, unless we unlock it and innovate in the realm of applications and solutions that’s wrapped around that data.”

Here’s how we’re addressing that, with a bit of step-by-step context on the overall cloud-computing platform enabling it.  The steps are: 1. Azure, 2. Pinpoint, and 3. Dallas.