Thursday, August 21, 2008

State of the Art: From Snapshots a 3-D View

(from http://www.nytimes.com/)
By DAVID POGUE
Published: August 21, 2008

How do you describe a place? How do you express its essence to people who aren’t there?
Photosynth technology from Microsoft turns Parthenon photos into a three-dimensional world. You use the best technology you have available.

In the beginning, there was the printed word (“Dear Queen Isabella: You gotta see this place!”). Then there was audio (“My God, it’s full of stars!”). Eventually, photos (“That’s us in Hawaii. Or is it Cape Cod?”).

Wednesday, Microsoft introduced yet another way to represent a place: Photosynth. This sophisticated technology does a simple thing. It turns a bunch of overlapping photos into a 3-D panorama.

The result, called a photosynth, is a little bit like a virtual world. You can move sideways, up or down into neighboring photos. You can turn around to look behind you. And at any time, you can zoom forward incredibly far into a photo, since it retains all of its original, multi-megapixel resolution.

Creating a photosynth is free and automatic. You visit www.photosynth.com, click to install the necessary Web browser plug-in, and start uploading your photos. (You need a Windows PC and either Internet Explorer or Firefox 3. A Mac version is under development.)

Photosynth is brought to you by a 15-person team at Microsoft Research, a division of Microsoft whose projects aren’t expected to generate income. This team hopes that the popularity of Photosynth will explode and that it will actually become a new medium. In fact, someday Microsoft hopes to design a way to connect these photosynths, eventually building a complete photographic, navigable, digital version of our real world.

In the meantime, Photosynth is a great way to visit a place before you go there, to remember a place after you’ve been there, or to show your inner sanctum to the whole world.

It’s also great for museums, galleries and anyplace else where extremely close-up inspection by the masses would be desirable, but not otherwise practical. And it doesn’t take much imagination to see how Photosynth might appeal to real-estate agents. In a Photosynth demonstration at the annual TED conference last year, the presenter blew the crowd’s mind with a photosynth of the Notre Dame cathedral in Paris, made up of photos mined from Flickr.com. In other words, Microsoft had created a seamless, successful photosynth using hundreds of existing photos, taken by different people at different times using all kinds of cameras.

That, frankly, is a far more amazing prospect than what Photosynth is today. If using existing Web photos were an option, you could have a substantial amount of the world photosynthed already.

But for technical and legal reasons, Microsoft has backed away from that approach. Now you have to take the pictures yourself, expressly for the purpose of photosynthing.

You have to take lots and lots of overlapping pictures. Ideally, every inch of the room or space should be included in at least three photos. You stand at each corner of the room and sweep the interior, snapping away; then repeat while standing against the wall. Stand on chairs, walk to a new spot, walk up close to interesting objects.

Eventually, your photosynth is given a “synthiness” score, meaning the degree of recognizable overlap. The synthiness determines how seamlessly your 3-D panorama flows from view to view.

You can also walk completely around a stationary object, taking at least 24 photos during your circumnavigation. Later, you can rotate that object online using a doughnut-shaped scroll bar.

Once you’ve transferred the photos to your PC, you sign in to Photosynth; bizarrely, you need both a Microsoft Passport account and a Photosynth account, both free. Then you click the Create Photosynth button. Technically, your PC, not the Web site, does the analysis to knit all of those photos together, so the speed of the analysis-and-uploading process depends on your computer’s horsepower.

When it’s all over, your photosynth is live on the Web, visible for all to see. (In this version, you can’t create private photosynths.)

Amazingly, the software figures out how your photos relate, even though they’re taken at different angles, zoom amounts, exposures and so on.

It’s also amazing how, when you’re exploring the resulting photosynth, the full resolution of the original photos is available. You can zoom in, and in, and in, revealing more and more detail along the way, without ever waiting for even the biggest photo files to “load.” In one example on the Photosynth site, you start in the marbled gallery of the National Archives building in Washington — and you can zoom all the way in to the words “We the People” on the Constitution in its display case.

(This is one crucial difference between Photosynth and Apple’s QuickTime VR, which has been around for years. You can zoom in slightly in a QuickTime scene, but you can’t fly nearly infinitely into one of the component photos. Other differences: Making a QuickTime VR scene requires a special, 360-degree tripod apparatus and commercial software. And a QuickTime VR scene is rooted to one spot; you can’t freely walk through a space, as you can with Photosynth.)

So yes, it’s amazing. But in its current version (“call it beta, call it 1.0, call it whatever you want,” as the Web site puts it), Photosynth is also very frustrating.

First, synthing takes practice and patience. There’s no easy way to track whether you’ve taken enough photos to cover your space. You don’t know that you’ve failed until you’ve uploaded all the pictures and discovered blank spots or clumps of detached photos.

It’s also incredibly frustrating to navigate a photosynth. There are three ways to move around in your 3-D space: the mouse, the arrow keys or the letter keys. But what you press may not be what you get. If you press the up-arrow key, hoping to look upward, you may suddenly be teleported to a totally different part of the room. That can happen, Microsoft explains, if that other photo is the only one showing a patch of ceiling. Well, fine, but it’s totally disorienting.

At other times, you might want to scroll onto an adjoining photo — but all you get is black space and a bunch of white, dusty pixels known as the “point cloud.” That’s because Microsoft has decided that once you’ve magnified a photo enough to fill the screen, you can’t scroll to any other photo until you zoom back out. Frustrating.

These annoyances add up to the most frustrating thing of all: Photosynth’s photo-centric nature. I’ve probably made it sound like a photosynth gives you a seamless virtual environment, but it’s really not that; the component photos very much retain their “photoness.” You see their outlines flicker into view as you move your mouse around a scene. When you scroll between photos, Photosynth makes no effort to make them appear as a single image; one just fades into the next.

Finally, only one photo is ever in sharp focus at a time; neighboring photos are blurred and dimmed, which destroys the illusion of being in a virtual space.

In other words, Microsoft has designed Photosynth to be less a virtual-reality tool than a glorified slideshow, a clever way to arrange a bunch of discrete photos in space. That’s fine, but it does make photosynths less magical than they could be. Compare Photosynth’s one-photo-at-a-time focus with, say, the seamless views of a QuickTime VR scene (Google “QuickTime VR gallery” to see some), where everything is in focus as you look up, down, left, right, forward or back, as if you’re inside a giant wraparound photo.

Even so, Photosynth is wicked cool, and it will find all kinds of new uses. At the very least, it represents another milestone in the evolution of place-description technologies. Until someone comes up with brain-to-brain image sharing, that will have to do.
NOTE FROM THE BLOGGER: Phrasal verbs are highlighted in red. You can find their definitons and examples in the "Phrasal Verbs" section of this blog.

No comments: