Shared mixed reality at planet scale

In his last novel -Rainbows End- science fiction author Vernor Vinge, depicts a world in which reality and virtuality are seamlessly merged. In this world, people send their avatars across the planet so you can be talking with someone and after some moment notice it is indeed an avatar of a distant person.
Now as everyone (almost) carries an ever connected and geolocated computer this vision is about to become a reality.
So what's missing ? We already have augmented reality (layar, ...) virtual representations of the planet (google earth, streetview, ...) live video streaming from your geolocated smartphone (bambuser, ...) etc...
Mixed reality is there, but we loosely share it. When I open layar or streetview I do not see any people around, no moving avatars in sight.
The problem is again the same: how to put millions of people and avatars to interact in a huge common space.
And yes, you know the story: twinverse know how to do it.

I've made a proposal for a panel on the subject at sxsw: "mixed reality: the next frontier in geolocation".
Votes are welcomed.



Note: I found another fictional share mixed reality in this video clip (sound off is better)

Scalability issues. A VW based on Google Earth. Part 2.

We know that Second Life and others face scalability issues. Indeed, virtual worlds barely scale: recently Eve Online announced they can now support 1000+ users playing together and this was seen as a great achievement. Imagine, msn or facebook teams announcing they can now support 1000+ users chatting simultaneously. This would be laughable, but for virtual worlds a so poor scalability level seems normal. Definitively, there is something hard here.

Let's try to understand the problem (and how Twinverse solves it):

Take, for example, 1 million avatars evolving inside a huge virtual world, say Google Earth, and imagine you are in control of avatar XYZ.

Suppose you move the right arm of your avatar.

The naive solution is to send "avatar XYZ has moved his right arm" to the whole million running instances of GE.

Then each GE instance will see if it's concerned by the event (if avatar XYZ is in sight). If so, the event will be transformed into an animation of XYZ's local representation otherwise it will be dropped.

This solution is clearly impractical and do not scale.

In region-based systems (like Second Life) the solution is the following:
There is a server for each region of the world, and avatar XYZ is connected to the region where it seats.

So "avatar XYZ has moved his right arm" is sent to the region. And since all the concerned avatars are connected to this region, the region send them all the "avatar XYZ has moved his right arm" message and all the receivers are more or less concerned.

Among other problems (the well known “too crowded region” problem), this solution is also impractical for huge virtual worlds, and for instance the earth is too vast. If your virtual world is vast, thousands (millions?) of servers are needed to cover the world -even before the first users arrive.


Twinverse solution:
Each avatar is connected to the avatars concerned by its actions. These connections are the red lines of the doughnut picture (link).

So when avatar XYZ move its arm, the "avatar XYZ has moved his right arm" is sent directly to the concerned avatars.

The point is how to keep the "Each avatar is connected to the avatars concerned by its actions", how to keep the "red network" when avatars move.

The trick: the avatars collaborate to maintain these connections.

Let's put an example:

Avatar alice is connected to avatar bob and avatar bob is connected to avatar carol. but alice is not connected to carol since they are not in sight.

Then avatar carol starts moving toward alice, and since bob is connected to carol, bob sees carol moving and tells alice and carol to connect together.

So when alice and carol come into sight they are already connected. Thanks to bob.

More precisely, there are many connections so alice and carol do not rely only on bob to make the connection, there is enough redundancy.

You can find more on that in this short document:
http://www.twinverse.com/TwinverseTechnology.pdf

A Virtual World based on Google Earth ?

How to populate Google Earth with millions of avatars...

Some might argue that Google Earth is already a Virtual World. But what I mean here is a different thing: I want to see and interact with the people that are in the same place I am. When I stand in Hyde Park London, I want to hear the conversation and participate.

Fig. 1: Twinverse distributed algorithm running on a torus-shaped world.
Black dots are users' computers positioned at their virtual location.

The Metaverse Roadmap group envisions a Google Earth -250 millions of downloads- populated by millions of avatars as a probable future of the web and in a article called Second Earth, MIT's Technology Review depicts a virtual world, mix of Second Life and Google Earth:

But within 10 to 20 years [...] something much bigger [...] may emerge: a true Metaverse. In Neal Stephenson's 1992 novel [...] the Metaverse was a planet-size virtual city that could hold up to 120 million avatars, each representing someone in search of entertainment, trade, or social contact. The Metaverse that's really on the way, some experts believe, will resemble Stephenson's vision, but with many alterations. It will look like the real earth, and it will support even more users than the Snow Crash cyberworld [...]. It will be accessible both in its immersive, virtual-reality form and through peepholes like the screen of your cell phone as you make your way through the real world.

Why wait 10-20 years ?
This is a very long time to wait in internet dog's years and -inevitably- the omnipotent Google has been rumored several times to be near of putting a end to the waiting but Google finally came up with a deceiving Lively.
In the same article, the explanation of why we should wait that long comes:

This architecture is what makes it next to impossible to imagine re-creating a full-scale earth within Second Life, even at a low level of detail. At one region per server, simulating just the 29.2 percent of the planet's surface that's dry land would require 2.3 billion servers and 150 dedicated nuclear power plants to keep them running. It's the kind of system that "doesn't scale well," to use the jargon of information technology.

Sure, if you think the only possible technology is the one-server-per-region system, it's not possible to populated Google Earth just now and even 20 years might not be enough. But, Twinverse, taking the problem upside down, have found a solution. See previous post.

Fig 2: Is the world flat or doughnut shaped ?

It's like no one wants to discard the one-server-per-region rule. Even the so called peer-to-peer systems Metaverse Project, Outback, Croquet, etc... just stick to the one-server-per-region (the peers host the servers).

Twinverse paradigm shift: one-computer-per-avatar that's the new rule, that's the eureka. After all, why an empty virtual region -with nobody actually seeing it- would ever need computing power?
Avatars meeting in someplace can collaborate to “give life” to their surrounding and make the needed computations.
One computer per avatar seems a lot, but indeed only today ~1% of a computer is needed and also just in case you forgot: when connecting to a virtual world, a user already has a computer.
Few weeks ago, Twinverse has launched beta a virtual world on top (inside?) Google Maps. This has good points since it runs inside a web browser and integrates seamlessly with the web. But 3D is still missing.
How far are we from the “true Metaverse” depicted by MIT's Technology Review ?
Not very far. If the teams in charge of Google Earth or Microsoft Virtual Earth open up a little bit further their APIs this will be ready in months, not years.

Fig 3: Twinverse screenshot of the day

Virtual worlds and memory

Last week, Heidi Ballinger invited me to a metanomics meeting. It was my first time inside Second Life doing something real and not just observing how Second Life is (yes, I've been an SL newbie for years). Indeed I was not really immersed in the world, learning on what to click, being annoyed with the lag, the somehow noisy VoIP and all other imperfections of SL. Surely, if I get regularly into SL, at some point my brain will get rewired and I will feel immersed in SL in the same way when using a mouse I “feel” my hand is “inside” the screen dragging documents and touching things.

The strange thing was the day after when I wake up I had the feeling to have been in a real meeting “somewhere”. My memory had build a sort of fake souvenir. I remembered things in a way that look more real than what really happened. For example, Zha Ewry who works for IBM and talked about opensim is in my memory a forty something guy with a white shirt while in SL he is a twenty something girl.

After this probably common experience I got puzzled and spend some time reading SL related blogs. I do not find much related to the topic but a blogger as puzzled as I am with a short post about memory and virtual worlds.

An incomplete conclusion is that virtual worlds are not only about being efficient and useful communication systems. They have a “trick”, a mental trick that hooks the users. At first sight, 3D does not seems very efficient and useful. 3D is not easy to deal with and it do not convey a lot of information since the gestures are stereotypes. But it gives the user a feeling of something that really happened, a feeling of shared human warmth. And that helps the communication.

Are virtual worlds necessarily small ?

Virtual worlds are much smaller than real world. The largest one, Second Life, is a few thousands square kilometers. Real world is roughly million-fold larger than that.
In the case of WoW, we can argue it is because it's expensive in terms of man power to build such a 3D world. But in Second Life ? There are people willing to build wonderful 3D spaces. But they cannot, because the terrain is too expensive (compare to webpages or blogs that are free). They have to pay a monthly fee to Linden Lab. So the scarcity of content is not the real reason and with a free (like beer) terrain, virtual worlds would be larger.
Is Second Life making good money out of closing its doors to these potential contributors? Probably not. This problem have its roots in the technical architecture of Second Life. There is a server in charge of each 65536 square meters parcel of terrain and this is weird. Most islands and sims are empty (see previous post) and need no computing power while some are desperately crying for more power as people come in. This is a really bad situation, most Second Life servers (70-90% ? SL crew must know) are running empty, sipping electrical power just in case someone decides to have a look at this portion of the world. And very often, this visitor do not stay and leaves the place short after entering, see previous post.
So while most SL servers are sleeping, totally useless, users are asking for more computing power, either to have they own piece of land or to be able to enter a place hosting an interesting -and therefore popular- event.
For Twinverse virtual world architecture, we have reversed the equation: there is a computer (more precisely a portion of computer) in charge of each user. And this is pretty convenient since every user have a computer at her hand when accessing a virtual world. OK, maybe you do no like this way of presenting things and just want to call that a peer-to-peer system. But keep in mind that this computer in charge of the user can be her own computer or otherwise hosted in Twinverse server farms, that's not the point. The important point to keep in mind is : the land in Twinverse is costless. No matter how big the world is, the cost is per active user.
So the world can be as vast as needed... As vast as Google Maps or Google Earth, as vast as the real world.

Computronium

Is there a limit for Moore's Law ? Sure, an exponential growth is not sustainable and soon the limit will be reached. Soon ?
Science fiction writer Charles Stross, made the calculations and it stands that at one atom per bit all data we have recorded so far can be stored in few milligrams of diamond.
May be it is even possible to store data and make computations at proton level, or even at a lower scale. After all, today's hard disks use spins to convey bits. We do not know for sure how far we can go and research in physics will not stop to progress just now.
wikipedia:

computronium refers to a hypothetical material engineered to maximize its use as a computing substrate.

OK, we got a limit: when all the matter on earth will be transformed in computronium, Moore's law will end. Or should we say, all the matter in the solar system ? all the matter in the universe ?
Physical limits for Moore's Law are far and the exponential growth will not be exhausted by lack of substrate. Not in this century.

Virtual worlds and chatrooms

Chatting with previously unknown people is a common experience... in the real world. You can go to a cocktail party or to a mass demonstration, or simply walk on the street, meet people and engage conversation with anyone.
Naturally the idea arose (in science fiction first) to mimic the real world and allow this kind of conversation to happen online. Shared virtual reality is the telecommunication system enabling this.
In shared VR systems - also called virtual worlds when the decorum is persistent- a user is represented by an avatar and only sees a small portion of the world: the immediate surrounding of her avatar. Theoretically, there is no limit on how many people could chat inside a virtual world: user's perceptions are not saturated by the total amount of people talking since she is only aware of a reasonable part of the talk stream. Hence, one of the major problem of the chatrooms is solved: A virtual world never gives the feeling of being too crowded.
There are 1 billion connected to the internet. Imagine, that people spend in average 15 minutes a day chatting online. Anytime, that's 10 millions people connected and chatting. Mostly, they use instant messaging. But only a very small portion chat inside virtual worlds. I understand that chatrooms are not very compelling (see my previous post). But what's the problem of virtual worlds?
A virtual world can also become locally too crowded. It is not that the user feels it too crowded: it is a technical problem. Indeed, a virtual world is made of “rooms”. Each portion or zone of the world is supported by a server (see this) and when there are too many people getting inside a given zone the corresponding server is overloaded and the zone becomes lagged, unusable: too crowded. The actual technical limit for a zone is around 300 people and for most implementations even less.
Zones are like chatrooms. And hence the same rules apply.
Most zones are empty and remain empty (you do not go inside a VW to be alone) and non empty zones are soon overcrowded (their server are overloaded).
If you are not convinced, you can go and watch popular virtual worlds. Last time I went inside Google's Lively, I found thousands of rooms: sure, it denotes an attraction to virtual worlds. But roughly 20 rooms were non empty and the other were empty.
Inside Second Life the same is true. I do not have statistics but you can see that most islands are empty. There is perpetual struggle to attract people (with sex or whatever) and organize events to try to fill up islands. When somehow, somewhere, some event or place finally succeeds in attracting people, it immediately starts in repelling people because only 80-90 can fit in an island.
I am convinced that eventually virtual worlds will succeed as major form of real time communication and even replace or absorb instant messaging and telephone. But not if they are a collection of rooms stitched together.

Note: I should have said, instead, most virtual world are made of rooms. Twinverse is a virtual world that is not made of zones, it's a contiguous world. You would say I am advertising my company and I am.