A tour of MS Virtual Earth, on TED.com

Stephen Lawler, from Microsoft, takes us on a tour through the company’s new Virtual Earth project, which is, basically, an attempt to turn the entire planet into an interface to the web. Collecting and synthesizing massive amounts of data — bird’s-eye views, street-level photos, 3D wireframes — the Virtual Earth team are building a world of possibilities. Bonus: This talk may explain the mysterious vans with cameras on top that you might have seen cruising up and down your street. (Recorded March 2007 in Monterey, CA. Duration: 06:55) Read Stephen Lawler’s profile on TED.com

NEW: Read the transcript >>

Watch this talk on TED.com where you can download it, rate it, comment on it and find other talks and performances.Stephen Lawler — Look! Up in the sky! It’s Virtual Earth!

What I want to talk to you about today is virtual worlds, digital globes, the 3D web, the metaverse. What does this all mean for us? What it means is the web is going to become an exciting place again.

(video — 3D virtual reality fly-through of cityscape)

It’s going to become super exciting as we transform to this highly immersive and interactive world. With graphics, computing power, low latencies, these types of applications and possibilities is going to stream rich data into your lives. So, the Virtual Earth initiative, and other types of these initiatives, are all about extending, you know, our current search metaphor.

When you think about it, we’re so constrained by browsing the web, remembering URLs, saving favorites. As we move to search, we rely on the relevance rankings, the web matching, the index crawling, but — we want to use our brain! We want to navigate, explore. Discover information. In order to do that, we have to put you as a user back in the driver’s seat. We need a cooperation between you and the computing network and the computer.

So what better way to put you back in the driver’s seat than to put you in the real world that you interact in every day? Why not leverage the learnings that you’ve been learning your entire life?

So Virtual Earth is about, you know, starting off, creating the first digital representation, comprehensive, of the entire world.

(flythrough slows at representation of football stadium, descends to 50 yard line)

What we want to do is mix in all types of data. Tag it, attribute it. Metadata. Get the community to add local depth — Global perspective, local knowledge. So when you think about this problem, what an enormous undertaking, you know, where do you begin?

Well, we collect data from satellites, from airplanes, from ground vehicles, from people. This process is, you know, it’s both an engineering problem, a mechanical problem, a logistical problem, an operationals problem.

(slide of surveillance camera)

Here is an example of our aerial camera. This is panchromatic, it’s actually four color cones. In addition, it’s multi-spectral. We collect 4 gigabits per second of data. If you can imagine that kind of data stream coming down. That’s equivalent to a constellation of 12 satellites at highest res capacity. We fly these airplanes at 5,000 feet in the air, you can see the camera on the front-

(photo of virtual earth plane flying above SF)

We collect multiple view points, vantage points, angles, textures, we bring all that data back in.

(slide showing Virtual Earth surveillance vehicles)

We sit here — you know, the thing about the ground vehicles — the human scale — what do you see in person? We need to capture that up close to establish that ‘what it’s like’ type experience.

I bet many of you have seen the Apple commercials, kind of poking at the PC for of course their brilliance and simplicity. (laughs) So, a little unknown secret is — did you see the one with the guy, he’s got the web cam, the poor PC guy, they’re duct taping his head, they’re just wrapping it on him. Well, a little unknown secret is his brother actually works on the Virtual Earth team.

(shot of guy wearing helmet with camera)(laughter)

So! They’ve got a little bit of a sibling rivalry thing going on here. But let me tell you — it doesn’t affect his day job.

We think a lot of good can come from this technology. This was after Katrina. We were the first commercial fleet of airplanes to be cleared into the disaster impact zone. We flew the area, we imaged it, we sent in people, we took pictures of interiors, disaster areas. We helped with the first responders, the search and rescue. Often the first time anyone saw what happened to their house was on Virtual Earth. We made it all freely available on the web, just to — you know, it was obviously our chance of helping out with the cause.

When we think about, you know, how all this comes together, it’s all about software, algorithms, and math. You know, we capture this imagery-

(image of car driving down street, driver perspective; then other driving perspectives become overlaid in a moving mosaic; then transforming to linked fish-eye perspectives, photos giving way to software renderings)

-but to build the 3D models, we need to do geo-positioning. We need to do geo-registering of the images. We have to bundle adjust them. Find tie points. Extract geometry from the images. This process is a very calculated process. In fact, it was always done manual. Hollywood would spend millions of dollars to do a small urban corridor for a movie, ’cause they’d have to do it manually. They’d drive the streets with lasers, called LIDAR. They’d collected information with photos, they’d manually build each building. We do this all through software, algorithms, and math. A highly automated pipeline creating these cities. We took a decimal point off what it cost to build these cities, and that’s how we’re going to be able to scale this out, and make this reality a dream.

(view switches to car driving on rotating stylized globe, bristling with spatially distorted cities)

We think about the user interface. What does it mean to look at it from multiple perspectives? An ortho-view, a nadir-view. How do you keep the precision of the fidelity of the imagery while maintaining the fluidity of the model?

I’ll wrap up by showing you the — this is a brand new peek I haven’t really shown, kind of into the lab area of Virtual Earth.

(overhead shot of virtual city, downtown skyline, with street names overlaid map-style)

What we’re doing is — people like this a lot, this bird’s eye imagery we work with. It’s this high resolution data. But what we’ve found is they like the fluidity of the 3D model. A child can navigate with an X-box controller, or a game controller.

(shows pointer isolating parts of the view, changing overlays, switching from graphics to photos, etc; zooms in to street level shot)

So here what we’re trying to do is we bring the picture and project it into the 3D model space. You can see all types of resolution. From here, I can slowly pan the image over-

(drags image view with pointer, next view blends seamlessly from graphic to photo)

-I can get the next image, I can blend and transition. By doing this, I don’t lose the original detail. In fact, I might be recording history. The freshness, the capacity. I can turn this image. I can look at it from multiple viewpoints and angles.

(rotates whole cityscape)

What we’re trying to do is build a virtual world. We hope that we can make computing a user model you’re familiar with, and really drive insights from you, from all different directions. I thank you very much for your time.