The mixed reality — the future of computing?

Alex Kipman knows what hardware. Joining Microsoft 16 years ago, he was the chief inventor of hundreds of patents, including innovative technology motion detection XboxKinect, which paved the way to some of the features in his latest creation — holographic 3D headset called HoloLens.

But today, sitting in his office at Microsoft headquarters in Redmond, Washington, United Kipman not talking about hardware. He discusses the relationship between people and machines with a wider philosophical point of view. Regardless of whether we interact with machines through screens or things that sit on our heads, for it is all just "point in time".

Born in Brazil, Kipman as a technical specialist in WindowsandDevicesGroup at Microsoft, enthusiastically explains that a key advantage of technology is its ability to displace time and space. He cites the example of "mixed reality" (MR, mixedreality), the term Microsoft, which represent a mixture of the real world with computer generated graphics. According to Kipman, one day she will unite invisibly augmented and virtual reality. He said that among the most striking special MR — the potential to unleash a "super substitution" in the real world.

People attach special value to the feeling you experience when you physically share space with another person. That is why Alice Bonasio from FastCompany decided to interview Chipman face-to-face. "But if you had the opportunity to implement this kind of interaction without spending the time needed to travel, says Kipman, life would be so much more interesting." Further, in the first person.

"My daughter can communicate with their brethren in Brazil every weekend, and my employees don't need to travel the world to do my job," he continues. "With the advent of artificial intelligence we could continue the conversation, but I wouldn't be here. One day you and I will talk, you will be on Mars and I'll be a hundred years dead. Our work technicians to hasten the future and constantly wonder how to do it."

Microsoft is making a bet on mixed reality to help us get to the future. And here we are back to hardware. The availability of the right device at the right price will be the factor that will determine whether consumers take mixed reality (although some of the devices by themselves are unlikely to start a revolution MR, as shown by the example VR). Although HoloLens is the only standalone holographic computer on the market (unlike OculusRift or HTCVive if you do not want to attach cables to the external device), smart glasses cost $ 3000 are more proof of concept than a consumer product.

Now Microsoft wants to change that. This fall the company is launching the headset Windows Mixed Reality Headsets, which will be the first serious attempt to sell the concept to the General public. Although this device is still closer to a perfect hybrid of augmented and virtual reality, it already embodied the main features of HoloLens — like advanced tracking capabilities and mapping, and suggested a more affordable price of 300-500 dollars. The headset will be available in different forms from different partners hardware, including Dell, HP and Samsung, and will allow users to create three-dimensional spaces that can be personalized through media, applications, browser Windows, and other.

Microsoft believes that the introduction of the platform that allows anyone create their own digital world, is the first step in achieving the leap into the world of tomorrow. "If you believe, as we that mixed reality is the inevitable next trend in computing, you have to connect productivity, creativity, education and a whole new range of entertainment, from casual to hardcore games," says Kipman.

Improving mixed reality

Kipman not the only one optimistic about a mixed reality. California startup Avegant is working on a platform that provides a detailed three-dimensional images, layering a lot of focal planes, which the company calls the technology "light field". "Endless applications," says Avegant CEO Joerg Tewes. "From designers and engineers to directly manipulate 3D models with their hands, to professors of medicine, illustrating various heart diseases, for example, is almost a living model to my students. At home users can surround the virtual shelves of their favorite products. Mixed reality allows people to interact directly with their ideas instead of screens and keyboards".

To do all of this, mixed reality device should support virtual image, which will seem indistinguishable from the real world and seamlessly interact with it. According to Professor Gregory Welch, a computer scientist from the University of Central Florida, most of the technologies developed to date, has not yet reached that equilibrium. "Mixed reality is particularly difficult because there is no hiding of imperfections in the virtual, nor the striking purity of the real."

Together with colleagues, he found that in some cases a relatively wide field of view of the real world, which provides the HoloLens, can harm important sense of presence. While a healthy person sees 210 degrees, the display HoloLens increases the centre of your field of vision to 30 degrees or so. In experiments conducted by Welch and his team, the gap between the real and extended landscape has reduced the sense of immersion and presence.

"This means that if you look at the virtual human in front of him (as was the case in our experiment), you will see only part of it floating in space in front of you," says Welch. "You will need to move your head up and down to "draw" the perception of him, because you can't see the whole person at once, unless we look at it from afar (it would seem less). The problem is that your brain always sees "normal" world around them, and it "overwrites" the many types of perception that you would otherwise have."

Further, Welch explains that the demonstrations that we see today with HoloLens and Apple ARKit, for example, virtual objects can be fixed on a flat surface, but in addition to the basic shape and visual appearance of the software usually does not recognize many important physical characteristics of the object such as weight, center of mass, and behavior, or the surface on which it is situated, not to mention any activity in the real world, which occurs around objects.

"If I accidentally lose a couple of cubes from the virtual table, they don't "fall", reaching the edge, and will not bounce, as would be expected, based on their type and floor material", he explains.

In the work that Welch co-wrote with Professor Jeremy Beilinson, Director of the Virtual Human Interaction Lab (VHIL) at Stanford University, they presented some of their findings, which show that the virtual content has a much higher value when it demonstrates the behavior that we expect from physical objects in the real world.

"In the laboratory we will be using HoloLens to understand the relationship between augmented reality experiences and subsequent psychological attitude relative to the physical space," says Beilinson. For example, he explains that his experiments show that virtual people who "are like ghosts" using real objects, and not bypass them or try to avoid, are perceived as less "real" than those who obey the laws of physics.

Advances in mixed reality, most likely, will lead to the fact that headsets will become more accessible and easier, but it is also possible that at least some of our future interactions with this technology will enable wearable electronics at all. "Spatial augmented reality" (SAR), for example, Welch developed many years ago, allows the use of projectors to change the appearance of the physical objects around you, such as a material color of a table or the couch with no points.

"of Course, SAR will not work in all situations, but when it is, it is clearly and easily," says Welch. "If something is magical is when the world around you is changing and you don't have to do with this — no headset, no phone, nothing. You simply exist in the physical world that changes around you."

Virtual collaborative tool in the real world

Nonnie de La peña, founder and chief Executive officer of the company Emblematic, helped to bring the use of virtual reality as a tool for reporting and telling stories. It is called "the godmother of virtual reality", and she believes that technology immersion is the closest you can imagine the kind of audience — that is, to put it in place of the narrator. She believes that HoloLens has the potential to increase the quality and depth of our understanding of the world, partly due to the method of volumetric capture, which creates a three-dimensional model of the objects through multiple cameras and a green screen. "Microsoft has started to offer a high level of realism by using the volume capture, and he was immediately picked up by journalists," says de La peña. Own creation, Emblematic, After Solitary, this award-winning documentary film, created in partnership with PBS and the KnightFoundation, which used this technique, ...