Meta is augmented reality

There’s a lot of criticism and mockery of Facebook’s rebranding to Meta. The timing, coming so soon after revelations that Facebook was profiting off of disinformation and outrage, is certainly suspect. However, there is a key idea here that is lost in the meta-discussion about Facebook itself – the idea of augmented reality, or AR for short, which if is to be brought to reality, deserves genuine admiration.

I have previously written about how I thought Google Glass was the first real effort at AR, despite many skeptics and its eventual failure. The problem with Google Glass was that it was positioned as an interface to search and take photos, i.e., content acquisition rather than an augmentation of reality:

Note that Google describes Glass as having a primarily voice-directed interface, for initiating search queries, taking a picture, or real-time language transcription. The main function of Google Glass is to record video and take pictures (not content creation, but content acquisition), to facilitate access to information, and most importantly to overlay data onto the visual field, such as maps or translations. It’s the latter that is the “augmentation” of reality part, and is very, very crude.

This was eight years ago, well before smart assistants like Alexa and Google Home were mature, and of course, suffering from Moore’s Law limitations. Natural Language Processing has come a long way since then, with graph processing tools and other incredible advancements in neural networks.

Facebook’s Meta has the advantage of building on that foundation and learning from those pioneering failures. The mistake that I think Meta is making now is that it is distracted by visions of the Metaverse from Snowcrash or The Oasis from Ready Player One (the latter probably being more accurate, as an ultimate escapist repository for all our nostalgia and cultural baggage). You can see those influences in how Mark Zuckerberg himself describes it:

The next platform will be even more immersive — an embodied internet where you’re in the experience, not just looking at it. We call this the metaverse, and it will touch every product we build.

The defining quality of the metaverse will be a feeling of presence — like you are right there with another person or in another place. Feeling truly present with another person is the ultimate dream of social technology. That is why we are focused on building this.

Immersive. You’re IN the experience. In ANOTHER place. This language describes an alternate reality, a virtual reality. It does not describe a meta-reality or an augmented reality.

To be fair, in the same letter, Zuckerberg does allude to augmented reality to stay present in the real world while you interact with the virtual. But the framework here is an Otherworld where you can decide on a use-case basis to what degree you are present. Think of it as an opacity, with 0% being fully in the real world and 100% being fully immersed (using, say Oculus).

The difference between this and truly augmented reality is that there is no “other” world. You are always in the present reality – but there are added layers. I’ve evangelized the truly groundbreaking anime, Dennou Coil, as a visionary example of a truly AR future. One screenshot alone is sufficient to convey what the true potential is.

Screenshot from Dennou Coil

I don’t know if Facebook/Meta will ever be able to be AR, but I do know that succeed or fail, Facebook/Meta will definitely bring us closer.

Check out my earlier posts on Dennou Coil here.

a semi-skeptical view of Google Glass

Dave Winer takes a semi-luddite view about Google Glass (which he refers to as Google glasses, minus branding and capital G). He writes,

I think they will make an excellent display device for the obvious reason that they’re mounted in front of your eyes, the organ we use for vision. The idea of moving your fingers to the side of your head, of winking to take a picture, well I don’t like that so much. I admit I might be a luddite here, and am going to keep my eyes and ears open for indications that I’m wrong. It happens, quite a bit when it comes to brand-new tech.

I think they could be a great part of a mobile computing platform. With more computing power and UI in my pocket, in the form of my smart phone, or in a big pocket, in the form of a tablet. They communicate over Bluetooth, and together form a more useful reading and communication device, but probably still not a very good writing tool.

I totally agree with Dave that a mouse/keyboard will be a requirement for any serious content creation, which is why I still prefer a Blackberry (lusting after the Q10, to be precise). But Google Glass is not going to be a content creation device so much as the initial, baby step towards true Augmented Reality. Note that Google describes Glass as having a primarily voice-directed interface, for initiating search queries, taking a picture, or real-time language transcription. The main function of Google Glass is to record video and take pictures (not content creation, but content acquisition), to facilitate access to information, and most importantly to overlay data onto the visual field, such as maps or translations. It’s the latter that is the “augmentation” of reality part, and is very, very crude.

denmo coil 1

A much more sophisticated vision of Augmented Reality is the one in the anime series, Dennou Coil. I’ve written a number of posts reviewing the series, including a review of my favorite episode where digital, virtual lifeforms colonize a character’s bald head (not unlike the Futurama episode Godfellas) and my closing thoughts on the series as a whole. The screenshot at right is from the first episode, which clearly lays out the technology paradigm: people wear special glasses that let them see virtual realities overlaid onto our real, physical world. Sound familiar?

But it’s cooler than that. In the screencap, the main character is using a cell phone that she draws in the air. There’s no need for physical technology anymore like cell phones or PDAs or even ipods or tablets. Literally, the entire world is your canvas and you consume your content through your regular senses. This is a vision that transcends mere augmentation of reality and becomes more akin to and extension of reality itself.

And it’s not limited to tech gadgetry – the concept extends to virtual pets, to virtual homes, even ultimately to evolution of virtual lifeforms that inhabit the same geographic space as we do but are invisible unless your glasses reveal them. I will be astonished if at least someone on the Google Glass team has not seen this series.

So, Google Glass really is a tentative step towards something new, and there is enormous potential in where it might lead. But as a device itself, Glass won’t be very transformative, because as Dave points out it will be an adjunct to our existing devices. And the content that people pay to consume won’t be created on Glass any more than it is created on iPads or Galaxy phones. Every single major technological advance of the past ten years has been in content consumption devices, not creation. Glass will be no different in that regard.

But content creation vs consumption is the old paradigm. The new one has less to do with “content” which is passively consumed and more with “information” which is a dynamic, contextual flow of information.

let the world be your PC

ok, anyone who watched Dennou Coil will immediately recognize where this is headed:

Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user’s hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

The thing is a project at the MIT Media Lab and can be built for $350 in off-the-shelf hardware. And just to make the obviousness of it all even more so, compare the following:

sixthsense08denmo coil 1

I posited in this slideshare presentation on the future of the Web that mapping a virtual layer on top of reality would be “web 4.0”. I think i may have been more right than I realized.

unchi! (うんち)

In Dennou Coil, Kyoko (age 3) has a penchant for running about, pointing at things, and exclaiming, “unchi!” (poop):

dennou coil 2

This is fairly accurate as far as a characterization of 3-year old humor goes. By that logic, the Himeji City Museum of Literature, in Himeji, Japan (near Osaka) was until May of this year the funniest place in the universe. This is because the museum, inexplicably, had an exhibit devoted to poop:

translation:

Everybody come and play! Come and look! We have poop books!

Rabbit: It’s poop time!

Gorilla: Come and see my poop too!

Elephant: Animal poop is here yo!

See it. Touch it! Smell it! Explore!

Can you guess what animals made this poop? (3 pictures)

Himeji Museum Of Literature, Special Exhibit. April 1st-May 18th

Head over to thomas’s post at Babelhut and see for yourself. It is in fact not only exactly what it seems to be, but in fact even more so than you think it would be. Fear the Japanese, indeed. Though it must be admitted that were this exhibit to come to any children’s museum in the United States, it would make more money than the mind can comfortably comprehend.

(It should be noted that I have blogged about poop before. I also have on occasion been fascinated with toilets. Insert bad joke about straining too hard here…)

Closing thoughts on Dennou Coil

I think Nick says it best:

Sometimes, I wonder how the Japanese are able to show what are considered “children’s anime” which can deal with very serious topics. How do they allow shows that put their viewers through such an emotional roller coaster that are said to be aimed at elementary school students? And why don’t I see or hear about this in many modern American animations, unlike the American classics (recall something like Bambi, for example)? I don’t hear of anything that runs like Full Moon wo Sagashite in America today, but I have found another example of an emotionally tiring series in Denno Coil.

Emotionally tiring, precisely because it’s a serious children’s anime. There is a lack of pretense and a willingness to treat children seriously that is the hallmark of the kind of anime I am drawn to. There are few to none of examples of this in American animation, though I think Pixar makes an effort. However, the closest analouge in our media would be children’s books, like Bridge to Terabithia or Holes (both of which have since made the transition to movie form; I’ve read both but only seen the former).

Dennou Coil is definitely targeted at children, but it seemed a blend of a lot of more adult-level anime. The Haibane Renmei parallel is obvious, with a Reki-Rakka empath/loner pair joined by an emotional bond. There’s also the Serial Experiments Lain parallel, toying with the nature of reality and the implications of granting too much “reality” to cyberspace (I would argue in fact that Dennou Coil did a superior job of this than Lain). The magical girl concept is neatly transposed to the technological here, you could even argue for a hint of Someday’s Dreamers in there. But the point about Dennou Coil is that it manages to straddle all these genres of anime and yet establish it’s own unique identity on its own terms.

I have to agree, this is one of the best anime I have ever seen. I have to rank it slightly more highly than Shingu, since it was equally enjoyable to watch but simply had greater heft, its backstory formly grounded in the rich human experience instead of needing to import aliens wholesale.

I don’t have much to say about the ending, spoilerwise or otherwise. Some random screen shots below the fold… Continue reading “Closing thoughts on Dennou Coil”

Dennou Coil 12: A Space Odyssey

This episode was largely tangential, but it was a blast. I mean that literally:

denno coil 12

In a nutshell, Yasako becomes a God, literally. She communicates to her People, but they do not understand…

denno coil 12denno coil 12

and they reject her counsel and engage in ultimate folly:

denno coil 12

But ultimately they realize the error of their ways and embark upon a new age of enlightenment:

denno coil 12

This episode paid homage to grand epic science fiction and philosophical themes rather than trying to actively explore them in any depth. It was a lot of fun as an interlude – not much in way of the larger plot was advanced. I just had a big goofy grin on my face the entire time.

Dennou Coil 2

Some screenshots from episode 2.

dennou coil 2

Sort of a cute and fuzzy Maximillian. well, shiny, not fuzzy. I love that smiley face.

dennou coil 2

Reki! already my favorite character.

dennou coil 2

Tribbles! Man this is the best series ever.

dennou coil 2

There are exactly two anime I’ve seen where children aged 3-4 were portrayed with any accuracy. Totoro was one, and this is another. Poop.

dennou coil 2

I use tis face all the time. And I got my fair share of it as a kid.

I am having a blast. It’s like watching Shingu.

Dennou Coil OP

I’m watching the Ureshi sub, which provides translit/translate of the song. I am transcribing both below by hand:

daremo ga sagashiteru
dareka wo sagashiteru
te wo nobaseba itsudemo
aru hazu no nukumori wa
osonai hi no maboroshi
mabushisugite mitsumeru koto mo
dekinai taiyo
hikari ga afureru kono michi ni
itsumo kage wa hitotsu
asu wa soko ni aru no daro

everyone is looking for…
…looking for someone
whenever you stretch your hands
the warmth that should be there
is an illusion of your childhood
a sun that’s shining so brightly
to be looked upon
on this street, where the light is overflowing
there’s always one shadow…
The future is probably there.

Dennou Coil 1

I got a little bored with Sayonara Zetsubu Sensei, so I watched the first episode of Denmo Coil, and was wowed. I love it. It’s like a wierd mix of Totoro, Serial Experiments Lain, and Haibane Renmei. What a fun show! The OP was also particularly good. I can see this one holding my interest.

And I need to get me one of these phones when they come out:

denmo coil 1