Virtual recipe stirs in Apple iPad, Microsoft Kinect

Who says Apple and Microsoft can’t work together?  They certainly do, at least when it involves the ingenuity of their users, the more inventive of whom use technologies from both companies (and others).

Here’s a neat example, “a just-for-fun experiment from the guys at Laan Labs” where they whip up a neat Augmented Reality recipe: take one iPad, one Kinect, and stir:

Some technical detail from the Brothers Laan, the engineers who did the work:

We used the String Augmented Reality SDK to display real-time 3d video+audio recorded from the Kinect. Libfreenect from http://openkinect.org/ project was used for recording the data coming from the Kinect. A textured mesh was created from the calibrated depth+rgb data for each frame and played back in real-time. A simple depth cutoff allowed us isolate the person in the video from the walls and other objects. Using the String SDK, we projected it back onto a printed image marker in the real world.” – source, Laan Labs blog.

As always, check out http://www.kinecthacks.com/ for the latest and greatest Kinect hacks – or more accurately now, the latest cool uses of the openly released free Kinect SDK, available here.

There are several quiet projects underway around the DC Beltway to make use of the SDK, testing non-commercial but government-relevant deployments – more detail and examples at the appropriate time. We will eventually release a commercial SDK with even more functionality and higher-level programming controls, which will directly benefit government early adopters.

In the meantime, I may report on some of the new advances being made by our research group on Computational User Experiences, who “apply expertise in machine learning, visualization, mobile computing, sensors and devices, and quantitative and qualitative evaluation techniques to improve the state of the art in physiological computing, healthcare, home technologies, computer-assisted creativity, and entertainment.” That’s a rich agenda, and the group is in the very forefront of defining how Natural User Interaction (NUI) will enhance our personal and professional lives….

Share this post on Twitter

Email this post to a friend

Air Everything

Like many people, I was very impressed by a video over the weekend of the Word Lens real-time translation app for iPhone.  It struck with a viral bang, and within a few days racked up over 2 million YouTube views. What particularly made me smile was digging backwards through the twitter stream of a key Word Lens developer whom I follow, John DeWeese, and finding this pearl of a tweet (right) from several months ago, as he was banging out the app out in my old stomping grounds of the San Francisco Bay Area. That’s a hacker mentality for you 🙂

But one thought I had in watching the video was, why do I need to be holding the little device in front of me, to get the benefit of its computational resources and display? I’ve seen the studies and predictions that “everything’s going mobile,” but I believe that’s taking too literally the device itself, the form-factor of a little handheld box of magic.

Playing with virtual data in spatial reality

For the past few months, when I’ve had visitors to Microsoft Research on the Redmond campus one of the things I’ve enjoyed demonstrating is the technology behind the new system for Xbox 360 – the controller-free gaming and immersive entertainment system that Microsoft is releasing for the holiday market in a month or so. In particular, I’ve enjoyed having Andy Wilson of MSR talk with visitors about some of the future implications in non-gaming scenarios, including general information work, and how immersive augmented-reality (AR) could transform our capabilities for working with information, virtual objects, and how we all share and use knowledge among ourselves.

We’re further along in this area than I thought we’d be five years ago, and I suspect we’ll be similarly surprised by 2015.

In particular, there is great interest (both in and out of the government circles I travel in) in the “device-less” or environmental potential of new AR technologies. Not everyone will have a fancy smartphone on them at all times, or want to stare at a wall-monitor while also wearing glasses or holding a cellphone in front of them in order to access other planes of information. The really exciting premise of these new approaches is the fully immersive aspect of  “spatial AR,” and the promise of controlling a live 3D environment of realtime data.

Contributing to Intelligence Innovation

Below are two ways to contribute to innovation in government, and specifically in intelligence matters. One is for you to consider, the other is a fun new path for me.

The promise of mobile augmented reality

My intention with this blog is always to write medium-length “think-pieces,” about technology, government, or preferably both. I’m working on several (the Jefferson Gov 2.0 piece, the Evil Twin 2.0 piece, and one on “whither the multilingual web”), but they do truly require thought and some free time, so they percolate a bit.

In the meantime, readers like the latest cool demo videos, so for Friday fun here’s another one (watch below or on youTube), which was featured on TechCrunch last night (“Bing comes to the iPhone via Robotvision”), with an augmented reality app for the iPhone which uses Bing Maps and Bing’s real-time data (website here). The company describes itself this way:

Immersed in Augmented Reality

Here’s a quick post, about a talk I gave this week – but as an excuse to link to a much more compelling presentation given at the TED Talks recently. Yesterday I had the good fortune to deliver the “Technology Keynote” address at the annual International Field Directors and Technology Conference, in Delray Beach, Florida. The IFD&TC is a well-known group in its field – no pun – of the world’s leading academic and government researchers, conducting large-scale and longitudinal social-scientific research studies.

As an example, think of the U.S. Census – and indeed I had the opportunity to spend some time with Cheryl Landman, Chief of the U.S. Census Bureau’s Demographic Surveys Division. You know those large studies drawn from every decade’s data? She runs them. That division’s work should feature prominently (I hope) in the forthcoming U.S. government’s “Data.gov” set of services.

Hiding new technology in plain sight

If I’m giving a private demo of an advanced technology or piece of software that Microsoft is cooking up, I often encounter the response, “Yeah, but we’ll probably never see this released, will we?”

War is Virtual Hell

FACT: According to market research compiled by Microsoft, the global market for Modeling and Simulation (M&S) software/hardware platforms across all industries, including the defense industry, has hit $18 billion per year; the cumulative growth rate is estimated at 9.6% annually.

ANALYSIS:  The lovely wife and I have been lackadaisically house-hunting down in Virginia’s Northern Neck, the Athens of America and the cradle of our democracy. The Neck is the birthplace of George Washington, the Lee brothers (the revolutionary patriots about whom John Adams used the phrase, “This Band of Brothers,” among them Declaration of Independence signers Richard Henry Lee and Francis Lightfoot Lee, not to mention their later nephew Robert E. Lee), James Monroe, John Ballentine, etc. etc. 

Anyway, recently we toured the historic 1859 house at Braehead, an 18-acre estate actually located within the Civil War battlefield in Fredericksburg, Virginia. The house is for sale, and while it’s likely overpriced (like everything else on the market these days) [this observation has been energetically and somewhat persuasively disputed by the listing agent, who read the post], but we enjoyed the tour.  I’ve posted many (too many) photos of our little tour here.  My interest in the house is the historic angle: it’s actually where Robert E. Lee visited and took breakfast on the morning of the Battle of Fredericksburg in 1862, one of his successful efforts against U.S. forces.  Here’s an article about Braehead’s history and historic preservation.

It was in fact at Fredericksburg that Lee spoke the words which would sum up the entire war, nay all wars, as he witnessed thousands of Union soldiers falling in battle to Confederate guns on the hills above the Rappahannock River: “It is well that war is so terrible, lest we grow too fond of it.”