You don’t have to look very far to find inspiration about what future meetings look like. Decades of Sci-Fi (Science Fiction), whether in cinematographic or written form, have explored the question. It’s interesting to see that it used to be the case for technologies that are nowadays omnipresent in the meeting room. Videoconferencing was introduced in movies in the 50s before being a reality in the 90s, the most famous sequence probably being the videophone sequence in “2001: a space odyssey”, that become an iconic view of the future.
A more recent trend explored in many pieces of Si-Fi (Science Fiction) is the concept of holographic communication, communication with a full-scale person as if you were face-to-face. The most obvious reference here is probably Star Wars with the blue transparent hologram introduced in “A New Hope” and consistently used in the rest of the episodes.
Apparently there’s a technical trade-off between transmitting 3D data from a different galaxy and having a full colour palette.
Many other works have used what has become a Sci-Fi cliché, especially when they come with glitches and visual static. It has become a very visual and easy way to display futuristic-looking conversations.
What kind of sorcery is this? Well, a combination of three different technological advances allowed Rémi Rousseau to build the software behind this:
Volumetric capture: the “3D” aspect is possible thanks to the data we get from a depth camera (here the Microsoft Kinect v2). Depth sensors are promised to a fantastic future because they will be soon the center of a big competition race in the mobile world. Prepare for such a sensor in your next iPhone. Yep, this is exactly what we’re building: a 3D motion capture format
Live streaming: the internet has turned into a great infrastructure for streaming high resolution 2D content. We use that, yeah.
AR and VR headsets: The most important brick ; don’t expect hologram technology in the next years (though we’re already compatible with one of the most promising ones). Our holograms aren’t holograms, there are 3D meshes projected through AR (Augmented Reality) or VR (Virtual Relaity) devices. Incorporating real people in virtual environments (VR) or in real ones (AR) becomes then much more feasible than having to deal with physical constraints. On the video, Davy is looking at me through a VR headset, the Vive. The VR headsets such as the Vive Pre or the Oculus Rift are already at a very high performance stage, whereas we’re not sure yet of the performances of AR headsets such as Magic Leap and Hololens.
Mimesys promise is the following: tomorrow you’ll have holographic communications while being recorded by a depth camera and while wearing a VR headset/AR glasses or lenses. That is what we’re exploring with Mimesys. We’re closer than ever to perform remote face-to-face meetings.
VR remote meetings
The problem with VR meetings is obvious: you have a headset on, so you’ll look like this to your colleagues. Sexy.
But for a lot of usecases, this will be good enough! There are still thousands of applications that value spatial positioning, body langage more than facial emotions (think of training applications for instance).
AR Remote meetings
Where remote meetings will thrive is with AR. With AR lenses, you’ll be able to see your partners face and integrate their bodies in your current world. This has been more than well described in the book Rainbows End, where every meeting is an AR one.
Prototyping the future
In parallel to our development, Mimesys's exploring and prototyping what could Augmented Reality meetings look like, with an advanced platform with 68 cameras. Mimesys will share the results very soon.
There are indeed still a lot of stuff to explore around this concept that Hollywood hasn’t, especially :
How to adapt the space and furniture to get a perfect experience? Holographic communication is complicated to achieve. As in the early days of videoconferencing, you’ll need a fixed setup and perfect conditions. The limited FOV on the current AR headsets will also imply some additional constraints : don’t be too close to the person you’re talking with. This setup could be an interesting challenge for furniture companies: how do you design a chair with discrete markers for careful hologram positioning? how do you design a chair for virtual people only? Could you reconfigure a room to host different meetings at the same time?
What’s the User Experience for holographic communication? The Star Wars experience is just the beginning. When you bring the virtual in the real world, there’s much more you can do in terms of inputs and commands. How would you share a virtual document with a holographic colleague? With real sheets of paper with markers on? Is it polite to change the environment during the meeting like in Heavy Rain? Is it polite to turn into an avatar during a conversation?
There are still a few years before requesting Obi Wan’s Kenobi’s help in 3D, but count on Mimesys to be among the first to try.
Author: Rémi Rousseau | Founder/CEO at Mimesys
Cover Source: Medium