Stop being so touchy

Remember Tom Cruise’s virtual 3D in Minority Report? Wouldn’t it be cool if, instead of an iPhone-like touch screen, we could manipulate data and 3D images on screens by simply moving our hands – interacting virtually without touching a keyboard or screen?

Fact: The market for touch-screen devices has been expected to grow at a dramatic rate — one industry group estimated earlier this year that “the touch-panel market is on track to grow to $3.3 billion and 660 million units by 2015,” according to DisplaySearch’s Touch Panel Market Analysis report.  Touch-enabled mobile devices like the iPhone or my Samsung i760 account for only 34 percent of touch-panel shipments.

Analysis: Everybody liked what Tom Cruise could do in “Minority Report,” and for a few years after the movie came out, a lot of people in the national security community wondered, “when will we able to use technology like that?” Pentagon generals asked tech companies to build it; Pentagon critics assumed it was already being used deep within Ft. Meade 🙂 And technologists inside and outside the Beltway just wanted to explore the advantages of new human-computer interfaces. As one blogger wrote in 2006, “There is work going on in this area already, from the U.S. military and others. The rest of us, for now, can only dream.”

Well, Microsoft Office Labs has just released “Touchless,” a webcam-driven multi-touch interface SDK that enables “touch without touching,” producing some of the same kind of magic as seen in the movie. 

 

The video features a quick demo by Mike Wasserman, a software development engineer at Microsoft, who only graduated from Columbia University a year ago, and began working on the Touchless project as an undergrad and finished it at Microsoft in his “spare time!”

Using the SDK lets developers offer users “a new and cheap way of experiencing multi-touch capabilities, without the need of expensive hardware or software. All the user needs is a camera,” to track the multi-colored objects as defined by the developer.  Just about any webcam will work.

Oh, and surprise: the Touchless SDK is open source.  It is released free and open-source to the world under the Microsoft Public License (Ms-PL) on CodePlex.  (Also, we announced further open-source support today with funding for advanced Silverlight development capabilities with Eclipse.) 

From the Touchless release notes:

What is the Touchless SDK?
The Touchless SDK is an open source SDK for .NET applications. It enables developers to create multi-touch based applications using a webcam for input. Color based markers defined by the user are tracked and their information is published through events to clients of the SDK. In a nutshell, the Touchless SDK enables touch without touching.

What is the Touchless Demo?
The Touchless Demo is an open source application that anyone with a webcam can use to experience multi-touch, no geekiness required. The demo was created using the Touchless SDK and Windows Forms with C#. There are 4 fun demos: Snake – where you control a snake with a marker, Defender – up to 4 player version of a pong-like game, Map – where you can rotate, zoom, and move a map using 2 markers, and Draw – the marker is used to guess what….  draw!

 

Mike demonstrated Touchless at a recent Office Labs’ Productivity Science Fair where it was voted by attendees as “most interesting project.” Office Labs has a number of interesting projects around “future knowledge work” scenarios, and you can try several others at the Office Labs Projects page.

If you wind up using the SDK, I’d love to hear what use you make of it!

Email this post to a friend