If you think you’ve seen what’s possible with Photosynth, then you’ve seen nothing yet. The collaborative research team from the University of Washington and Microsoft Research who only two years ago in 2006 published their paper “Photo Tourism” and their technology demonstration “Photosynth” have again pushed the boundaries of what can be achieved by intuitively processing the abundance of digital images shared on the web.
This week at SIGGRAPH 2008 they’re sharing with the world some even better technology they’ve been working on which they call “Finding Paths through the World’s Photos“. Don’t let the name fool you, it’s damn cool. If you’re not much of a reading person like me, take a look at this video demonstration. (Watch it till the end)
This technology is much better than Photosynth simply because instead of just presenting individual photographs in a cool 3D environment, it actually manipulates the photo to give you a seamless and more lifelike experience. It’s one thing to click around different photos taken at a particular museum, it’s a whole other story to “walk through” the museum.
Now if you want to know exactly how they did it, and you’re a rocket scientist, take a look at their conference paper. For the rest of us, just take it for granted.
52 Comments
deeper2k
Nice like most of Microsoft Research projects, but will we see this technology in commercial product anytime soon?
Avatar
MS tech in photography is now the best around. just from what they presented at the Photo Pro event. it is truly insane how many photo tech they got now.
Chustar
This is cool. What they need to do is buy flickr and build this into the website. Suddenly, going to flickr becomes useful to me.
Imran
Nice
tinwheeler
Put enough of these frames together and you could have a 360 degree 3D movie sans glasses.Speaking of Microsoft Research projects; go to the MS Research project page and pick up their super cool astronomy project.
Albert
Wow…great implications in slideshows and movie video fx. I would love to use it for a scene showing time progression and at the same time with the object being rotated.
MJ
it’s unusual for you to use YouTube Long!
Edootjuh
Sounds, and looks, cool
I’d just prefer if they would show multiple photos so that the entire view is filled with photos. It’s just a bit too slow to appear like really walking through the place.
Yert
Now if only they could eliminate the people in these pictures with enough of them; assuming high enough quality we could get the best 3D models with just pictures… I think I sound random, but the possibilities of this technology are almost endless! I would like to see this implemented into Live Maps very soon!
Evan
Wonderful, wonderful stuff.
I’m guessing that, as someone else mentioned, this amazing and wondrous technology will never actually be used anywhere, but will fill blog posts with wonderment.
That makes me sad
Why won’ t Microsoft actually yet people use this stuff?
Himanshu
This is great.. but
is it scalable? What i mean is, I haven’t seen anything new coming up at Photosynth. there are so many point of interests around the world but they are still not ‘photosynthed’.
ArMyZ
Great work. I wonder about workstation insane processing power..
mrmckeb
I was wondering when they’d do this… they mentioned a technology like this when they launched Photosynth.
The question I want answered is why display photos at all? Why not just make a fully textured 3D object?
dexotaku
Impressive stuff. It’s a shame that Microsoft is involved.
Fred
@mrmckeb: Where would you get the 3D data?
Steve
@fred: its pretty evident that they are deriving 3d data from the photos. It’s clear they are working on presenting the 3d data. perhaps a problem is one of isolating the object in question from the background.
@himanshu: If you try the photosynth demo, you’ll see that it is a viewer only. The viewer works pretty well, but works on images that have already been analyzed. But microsoft haven’t released the software to analyze the images, find common keypoints and assemble them into some kind of 3d data set, and its not clear if they will ever do that.
@chustar: flickr is owned by yahoo. microsoft was trying to buy yahoo recently, but the deal fell through.
Tim
@Fred: that’s the point of the algorithm, picking out key features and mapping them into 3D space. It’s not a complete model, but with more data (more, better pictures) it could be.
@dexotaku: I disagree, Microsoft funds a lot of research; and after the manipulative behavior they used to corner the market, and the damage they’ve done to software development’s progress for the last 30 years, it’s really the least they can do.
Tom
I know http://www.everyscape.com does something similar so that you can walk around the exteriors a la google street view and interiors as well.
Elgato
I cant help but wonder what an organization like the NSA or CIA could do with technology like this. Imagine including satalite data, IR data, Landsat data, LIDAR, blueprints, etc all on a large multi-touch screen! That would be fun.
Scott L
The computation required to correlate images is high, but not too high for multi-core desktop computers, never mind the computers of 2013. One strong implication of this technology is that it can be applied – with enough correlating input – towards facial recognition, or to combine multiple images of a given car model, or similar-looking dogs… all kinds of applications as a heuristic algorithm useful to streamlining image analysis of all kinds. Implications for intelligence gathering are clear, and I shudder to think what will happen when this technology is combined with porn.
Scott L
Oh, I forgot to include this demo of what’s going to move it to the next level…
http://www.youtube.com/watch?v=nice6NYb_WA
Howard
@steve: Actually, the software to bundle the images is available.. not only that, but the source code is available.
http://phototour.cs.washington.edu/bundler/
That is the tool that calculates the 3d data points. I do not know, given that it is actually from the Phototour – Univ. of Washington side of the aisle, whether it produces data that can be fed directly to Photosynth or not.
needy
Can please someone compile it for Windows as well? Microsoft is involved but the compiled version is for linux
Otto Von Bismark
You know that the military and Homeland Security are going to eat this shit up. Imagine this with video, and using all the cameras that are being installed everywhere. Panopticon.
dj_cityboy
well it seems interesting but not really mind blowing to me, i would like to see a view with complete photo views that white fuzzy background is horrible, security is going to be a huge issue as multiple groups of people would and could utilize this for the wrong reasons as well…Hell! you can already walk the streets in google maps…
i wish there was a way to enable much more cleaner transitions, it looks as though they had some good ideas on removing that sort of strobing effect from night and day pics being together, but it didnt really seem like it did much other then removing all the night pics and left day ones in there place, creating a less animated effect…
i dunno maybe um stuck on older concepts like phototriage, phodeo and timequilt concepts for browsing my person photo collection not sumthing mashed together with someone elses pics as well…
i would still like to check it out though, cause like i said, it does look interesting!!
peas
cityboy
Matt
@Chuster…. yeah, that was probably a large part of their desire to buy Yahoo. Flickr comes with Yahoo.
Mirta Khashtur
This software is way ahead of the time.
I think the results would be even more realistic, if multiple videos would be used instead of the images.
Sam Powers
I think this is going to be on sites like ebay within 2-3 years, and Silverlight is going to put it there.
Frank
“Fred: that’s the point of the algorithm, picking out key features and mapping them into 3D space. It’s not a complete model, but with more data (more, better pictures) it could be”
….which is why I don’t mrmckeb’s question. Why not map the photo’s to a fully textured 3D object? This -is- the best 3D object they can get from these pictures.
Frank
And for some reason, now my name is Frank instead of Fred.
mrmckeb
It would be complicated Fred/Frank, but I can’t see why they can’t work out the object’s 3D parameters from a large collection of photos of the object… they seem to be almost there as it is,
Some CADD geek
I honestly think that Google’s freeware program SketchUp is much more useful for viewing real 3D objects. With the ability to create complex 3D shapes quickly and easily using a diverse range of extruding options, people have made accurate, scale models of Ships, Buildings, city streets, and even animals. I’ve personally made accurate 3D models of some of my products, as well as a few real buildings – buildings that look almost identical when I walk in them.
This program uses a lot of creative technologies that make 3D imaging easy, but it’s not useful 3D imaging. Google SketchUp, Rhino 3D, AutoCAD, SolidWorks, and other Computer Aided Drafting programs allow any degree of detail to exist, and the context of the image (which is actually an environment consisting of an infinite number of images) is far more valuable for any professional application.
matt
That’s cool, but wasn’t quicktime VR doing stuff like that 10 years ago?
maybe not that good, but it seems like it was going down that path.
joe
no matt – quicktime VR is used to view a panorama (or object) which is composed of photos made under very carefully controlled conditions.
this stuff takes a random collection of pictures of a scene, recovers the 3d geometry of the scene from all the pictures, and then from that geometry can place where the pictures were taken. then you can do panoramas or flybys of objects using the original, uncontrolled shots.
its really awesome and steve and his team have had 2 (or maybe 3) consecutive best of siggraph papers based on this work, which is unprecedented.
Steve
Wow… cool technology. They need to merge this with GIS, particularly the work of Microsoft virtual earth or ESRI based mapping systems.
Maybe this is the foundation for next gen video games??
Cheers
Steve
Jay
Nice, but is this actually technology … or a presentation of what’s possible. More vaporware? Please release a demo rather than just spread the fact there’s a good idea! Because these are very old ideas!
joe
man you guys are jaded. its not vaporware. its just research quality code. check out noah’s home page:
http://www.cs.washington.edu/homes/snavely/
note this related work:
http://www.vimeo.com/1513129
and the comments from “pro” who is the author. he says he’s going to release his source code.
joe
also, jay, you pretty much have no idea how university-level graduate computer science research works. people don’t get PhD’s for just coming up with an idea. they have to implement it as well. everything you see in that video is the result of a computer algorithm running on a PC – its not a powerpoint presentation or something like that. furthermore the predecessor to this, photosynth, IS actually available as a demo program from microsoft labs.
Jay
Thank you for the reality check, Joe, and thank you for your patience. Alas, you are correct, I am not a CS researcher. Is there a URL I from which I can download this product?
joe
http://labs.live.com/photosynth/
Jay
Only let’s me download if I have “Windows XP SP2 and Windows Vista” … I’ll see if I can find a Windows machine.
joe
you also need a pretty recent video card… since microsoft co-sponsors this research i wouldnt hold my breath for a linux or mac port…
Jay
Thank you, Joe. I’m disappointed, but understand. As a non-technical user, I’m hoping for cross-platform applications … Again, your patience is appreciated.
Jay
BTW, I used to work on videos about research at MIT’s Media Lab, and I’m as excited as the next gal about research. I’ve seen a lot of exciting demos and code over the years. Now, however, my primary interest is how intellectual property is transferred, over years, to the consumer market, and how it can change the world.
Gilbert Cassy
Well, that quickly went from “meh” to “holy crap”. Bravo, UW/Seadragon team, bravo.
Frank
@Some CADD geek: Comparing this to Sketchup or other 3D modeling tools is pointless. It’s not about 3D modeling. It’s about displaying a large image library in such a way that you can navigate through them in such a way that you can tell where each picture was taken.
theone
Looks great, but these day is it not easier just to video the bloody thing ?
Schlomo
Mind-blowing O_O
Nathanael
I just wanted to point out that the path-finding work has finally begun to be integrated into Photosynth.
Since 2009 April 21, when you use the Highlights list (on the right hand side of any photosynth whose author has added highlights) you will be flown along a path calculated through any photos between your current position in the synth and the highlight you have just selected, just as shown in the Photo Tourism work above.
Also of note is that this new feature coincides with the switch to the Photosynth viewer being written in Silverlight 2.0 which means that all synths should be viewable on all Intel Macs. If all goes well with Moonlight, Linux users will be able to view synths by September 2009.
joe
Nathanael:
awesome! i can’t wait to try this stuff out… mac-only shop here.
Nathanael
Joe,
Just a clarification and full disclosure:
Synths are still only *calculated* in Windows, but you should be able to *view* a whole world of synths now.
http://photosynth.net/geoexplore.aspx
As always, there’s Boot Camp, Parallels, or VMWare Fusion if you’re dying to create a few.
joe
bogus
yeah i can run fusion i guess.
Leave a Reply
www.vistablog.at
Microsoft Research: Finding Paths…
Das ist einfach nur cool. Georg fragt: "Wann hat MS zuletzt etwas wirklich cooles gemacht?" Na da gibts schon immer wieder richtig gute Dinge, und das was jetzt auf der Siggraph präsentiert wurde, gehört sicher auch dazu: Downloa…
mindblowing 3D photo viewer - Surpass Web Hosting Forums
[...] If you think you’ve seen what’s possible with Photosynth, then you’ve seen nothing yet. The collaborative research team from the University of Washington and Microsoft Research who only two years ago in 2006 published their paper “Photo Tourism” and their technology demonstration “Photosynth” have again pushed the boundaries of what can be achieved by intuitively processing the abundance of digital images shared on the web. full story: Uni. Washington and Microsoft Research collaborates on (yet another) mindblowing 3D photo viewer – i… [...]
CrunchGear » Archive » Really damn cool: Photosynth team demos new version of compositing app
[...] Deep respect to Microsoft Research and UW for putting this together. Check out the higher-res version at their site. [via Reddit and I started something] [...]
Receiver Gallery - A San Francisco Art Gallery
[...] stuff for the interwebs everyday, and taking much inspiration from amazingly mindblowing user interface / experience projects like this! Imagine this hooked into Google [...]
» Photo Tourism - MicroMiel
[...] details at the project site. Kudos: Long Zhen | [...]
Freebase的超震撼演示
[...] Photosynthæ›´åŠ ç‰›B的技术，今年SIGGRAPH [...]
Some interesting videos « Vijay Chidambaram
[...] Check out the last video! Its amazing! Link to post about it is here [...]
Mind boggling 3D photo viewer | Good Times & Happy Days
[...] University of Washington and Microsoft have teamed up to create a very cool 3D photo viewer. This isn’t the PhotoSynth project from months ago either, it’s a whole new, cooler [...]
tdaxp » Blog Archive » 3D Visualization of Interesting Places from Publicly Available Photo Sources
[...] first alerted me of Photosynyth. The post about orbital navigaton is courtesy of Slashdot. « U.S. Defense Secretary: Russia’s actions are [...]
The Daily Bizarre » Blog Archive » Photo Frenzy
[...] saw a link yesterday to the second-generation of this technology, and damn, but it’s cool. The new version basically allows you to take 3-D virtual tours of [...]
Microsoft shows off updated Photosynth tech at SIGGRAPH | MostReviews.com
[...] Read more at istartedsomething.com [...]
Mitch Butler - Mindblowing 3D Photo Viewer
[...] 3D photo viewer from University of [...]
danshust.com » Blog Archive » New 3D Photo Viewer From Microsoft Research
[...] you thought Photosynth was cool, you haven’t seen anything yet! On his blog, Long Zheng shares with us an amazing video out of Microsoft Research that focuses on a new immersive 3D [...]
Foto Turismo em 3D | TECNOSH | Tecnologia, Gadgets e Design.
[...] todo aqui a tentar descrever-lhe como aquilo funciona, mas isto é mesmo uma daquelas coisas em que um vÃdeo vale mil palavras. Bastará uma olhadela para ficar imediatamente convencido de que o futuro será [...]
Microsoft libera Photosynth | lobo_tuerto
[...] Uni. Washington and Microsoft Research collaborates on (yet another) mindblowing 3D photo viewer. [...]
Spyware Confidential mobile edition
[...] feel people mention enough. My friend and wider-blogging symposium colleague, Long Zheng, often throws in an MSR mention with the things they are coming out with. It’s funny how these “products” have [...]
Photo Tourism: ¿El Futuro de la FotografÃa? | Mapas del mundo
[...] (tours) virtuales automáticamente entre fotos, en un entorno de tipo 3D. La noticia la dio primero Long Zheng en su blog I Started Something, y se resume todo al siguiente [...]
WEngineering Blog » Uni. Washington and Microsoft Research collaborates on (yet another) mindblowing 3D photo viewer
[...] Uni. Washington and Microsoft Research collaborates on (yet another) mindblowing 3D photo viewer [...]