If you’ve ever had the chance to try out Kinect Fun Labs‘ “Build a Buddy” or “Googly Eyes” games, then you would have had the chance to witness the magic of Kinect’s object digitization technology to build animated and texture-mapped 3D model of almost anything. It turns out being able to do this in under two seconds required some help from Microsoft Research Asia.
In the video, researchers Minmin Gong and Xin Sun demonstrates that the raw data from Kinect’s depth sensors is actually much rougher than as one might have imagined. To produce a smoother and “water-tight” model, they use a process called “Poisson Surface Reconstruction” to process the spatial points into more realistic surfaces.
To make the algorithm practical for a gaming platform like the XBOX 360 with limited computing power and memory amongst other game code and resources, they were able to optimize the algorithm to reduce 80% of triangles in the 3D mesh with minor change to the resulting shape which cut the process down from 20 seconds to just 2.
Although they didn’t go into more detail, one can only imagine cryptic math equations like those drawn on a glass wall in the video were involved.
Just as they say at the end of the video, “in the future, it will become faster, more accurate and more realistic”, I can only hope these guys are having lunch every day with the Microsoft Research people working on KinectFusion. Only good things can come from that.