Microsoft Research’s Pinch-the-Sky Dome takes multi-touch interactivity to 3D space

[flv:pinchthesky.f4v 670 377]

Minority Report and Iron Man 2 may have set the benchmark for multi-touch interaction for years to come but Microsoft Research is never far behind. At the CHI (Computer-Human Interaction) 2010 conference last month, MSR publically showed off a technology it’s been working on for at least a year now, conservatively dubbed the “Pinch-the-Sky Dome”.

Unlike mainstream 2D multi-touch systems or even Microsoft’s Spherical Surface, this system projects omni-directional imagery spanning 360 degrees onto the dome immersing the user. It allows them to navigate the virtual space using multi-touch gestures anywhere in 3D space. Since the system utilizes infrared camera that see wherever the image is projected, the user does not have to wear gloves, use special tracking devices and is not limited to gestures on a flat (physical or virtual) surface.

As the name might suggest, an obvious use for this technology is viewing astronomical imagery from WorldWide Telescope. In addition, naturally omni-directional imagery including 3D panoramic photos and 3D visualizations also work very well. Could you imagine this with Natal for a first person shooter?

Although in the demos the controller conducts his gestures fairly close to the projector/camera casting a fairly large shadow, it is explained they are looking to enable much more natural gestures like pointing to something in the virtual space to navigate there.

If you find the official video demo above a little too slow-paced, the following CNN video report gives a better overview of the system.

48 insightful thoughts

  1. I don’t know. The 3D gesturing is interesting technology, but in the end you’re still looking at a 2D image. The dome doesn’t make the image 3D, so you’re still just gesturing in the sky and watching a flat image elsewhere move around. It doesn’t have that sense of actually touching and manipulating the content the way you get with Surface. If they managed to make the images 3D though, that’d be amazing.

Leave a Reply