Shared mixed reality at planet scale

In his last novel -Rainbows End- science fiction author Vernor Vinge, depicts a world in which reality and virtuality are seamlessly merged. In this world, people send their avatars across the planet so you can be talking with someone and after some moment notice it is indeed an avatar of a distant person.
Now as everyone (almost) carries an ever connected and geolocated computer this vision is about to become a reality.
So what's missing ? We already have augmented reality (layar, ...) virtual representations of the planet (google earth, streetview, ...) live video streaming from your geolocated smartphone (bambuser, ...) etc...
Mixed reality is there, but we loosely share it. When I open layar or streetview I do not see any people around, no moving avatars in sight.
The problem is again the same: how to put millions of people and avatars to interact in a huge common space.
And yes, you know the story: twinverse know how to do it.

I've made a proposal for a panel on the subject at sxsw: "mixed reality: the next frontier in geolocation".
Votes are welcomed.



Note: I found another fictional share mixed reality in this video clip (sound off is better)