Twitter Search as a Government case study

In addition to periodic think-pieces here at Shepherd’s Pi, I also contribute a monthly online column over at SIGNAL Magazine on topics relating to intelligence. This month I keyed off a recent discussion I had onstage at the 2015 AFCEA Spring Intelligence Symposium with Elon Musk, particularly a colloquy we had on implications of the emerging cleavage (post-Edward Snowden) between Silicon Valley technology companies and their erstwhile innovation partners, U.S. intelligence agencies.

Your choice, Dataviz as event or book

A friend wrote asking if I could make it to an event happening this week near DC. I can’t make it, but fortunately he also mentioned as consolation that he has a cool new book on the cusp of release – and I’ve now ordered my copy.

The Friend: legendary visualization and HCI guru Ben Shneiderman (Wikipedia entry). Ben is a computer-science professor at the University of Maryland and the founder of its well-known Human-Computer Interaction Laboratory (HCIL), as well as an ACM Fellow and AAAS Fellow.  He has done government a million favors over the years, consulting for agencies, including his recent work on the Recovery.gov site to help that platform of data – from hundreds of thousands of sources – organize, host, and visualize the data for millions of visitors.  I first got to know Ben through his support for better intelligence analysis – he helped invent a longtime intelligence analytics tool, Spotfire (see his article “Dynamic queries, starfield displays, and the path to Spotfire“).  Ben’s also well-known for his award-winning 2002 book Leonardo’s Laptop: Human Needs and the New Computing Technologies, which I enjoyed and still think about when brainstorming new techie toys.

Free Tools for the New Scientific Revolution

Blogs are great for supplementing real-life events, by giving space and time for specific examples and links which can’t be referenced at the time. I was invited to give a talk last week at the first-ever NASA Information Technology Summit in Washington DC, and the topic I chose was “Government and the Revolution in Scientific Computing.” That’s an area that Microsoft Research has been focusing on quite a bit lately, so below I’ll give some examples I didn’t use at my talk.

One groundrule was that invited private-sector speakers were not allowed to give anything resembling a “sales pitch” of their company’s wares. Fair enough – I’m no salesman.  The person who immediately preceded me, keynoter Vint Cerf, slightly bent the rules and talked a bit about his employer Google’s products, but gee whiz, that’s the prerogative of someone who is in large part responsible for the Internet we all use and love today.

I described in my talk the radical new class of super-powerful technologies enabling large-data research and computing on platforms of real-time and archival government data. That revolution is happening now, and I believe government could and should be playing a different and less passive role. I advocated for increased attention to the ongoing predicament of U.S. research and development funding.

Alex Howard at O’Reilly Radar covered the NASA Summit and today published a nice review of both Vint’s talk and mine.  Some excerpts:

A New Prototype: Research Desktop

Fact: The international conference on Advances in Social Network Analysis and Mining (ASONAM 2009, next July in Athens, Greece) today issued its call for papers on “experimental and theoretical works on social network analysis and mining,” particularly relating to online social Web sites, email logs, phone logs and instant messaging systems “which are widely analyzed using graph theory and machine learning techniques.”  Interested authors are encouraged to submit abstracts of up to 300 words by December 10, 2008; the full papers aren’t due until January 31, 2009.  More info at www.asonam.org.

Analysis: Several Microsoft Research people are preparing papers based on their current research, and I’m considering attending myself (I’ve written before about MSR’s work in analyzing large social networks). There are three Microsoft scientists on the Committee (Dou Shen, Haizheng Zhang, and Rina Panigrahy – check out Rina’s publications on hashing and sketching algorithms).  It should be a top-notch conference, co-hosted by ACM and IEEE.

But that’s way off in the future – what if you want to look at some research stuff right now? Well, I’ve been going through the related “Socio-Digital Systems” work of MSR Cambridge (UK), and they’ve just added more information to their section here of the main MSR site.  That’s some neat stuff, more on the side of the actual social uses of digital data and the effects on our (still-human?) everyday lives.

How to Keep Up with the Semantic Space

Fact: Reshma Sohoni, chief executive of Seedcamp (the VC accelerator focusing on European tech startups) said in an interview this week that “In the past few months, we’ve seen a lot more developments around the semantic web,” and talked about the coming value of semantic web services.  

Analysis:  There’s never-ending buzz about the semantic space. Just this week John Conroy writing at the Content Matters blog had a long and interesting piece examining “three ingredients needed to enable and popularize the semantic Web,” with the aim of answering “if we really are on the cusp of a content revolution.”

Also, I admit it, I can’t stop playing with the semantic-space startups, whether it’s Sensebot or the newly launched Evri, or Powerset (just acquired by Microsoft, as I wrote about previously).