VR/AR (XAR) Makes its Way through a Fragmented Landscape
VR/AR developers continue to strive for a fully immersive experience and additional and innovative content. But why is consumer VR/AR (XAR) adoption so slow?
“XAR” is a term that industry experts use to refer to some mixture of Augmented Reality (AR) and Virtual Reality (VR). VR is at one end of the spectrum and AR is at the other extreme. In between you have some virtual and some augmented reality, but it is a mixture, which is XAR. VR is a stepping stone to something even better; eventually there will be more complex devices than what we have today. XAR devices are made up of three main areas: visual, audio, and interaction with the virtual environment. First, developers want to create an immersive 360° experience equivalent to reality. To do that, the visuals must be of very high density with a lot of pixels to make sure that whatever the user sees in a virtual world is equivalent to what they would see in real life. However, the VR industry is not there yet. Second, the user needs immersive, realistic sound such that the experience is intuitive and natural, exactly emulating a situational environment in every sense. Again, the industry is not there yet. The third area for any XAR business is interaction that is intuitive, or like a natural interruption. Many children experiencing VR for the first time want to touch or push what they are seeing. A natural response to a virtual entity is confirmation that the virtual environment is doing something right. Ideally, the most immersive experience would have all three areas: visual, spatial audio, and interaction through touch. The industry is seeing products like UltraHaptics for interaction, but touching virtual objects in mid-air using ultrasonic vibration is nascent technology.
XAR can bridge gaps and connect people to experiences where they can access places that are otherwise inaccessible. With XAR, people can immerse themselves in a new environment that they couldn’t otherwise experience. The future of XAR is an immersive medium where people can experience something directly through their senses, as opposed to having that experience interrupted by a headset that blocks the view. How will XAR technology evolve? Visually, the industry is making progress towards the desired pixel density. The goal is a very high resolution that equals the resolution of the human eye while also delivering a seamless sensation with a high number of frames per second (fps), although many would say that a 1K by 1K pixel quality per eye is good enough. Processors that can deliver a high rate of fps, a high resolution in the density of pixels, and very low latency are the holy grail for VR. Frame rates less than 90 fps tend to cause simulation sickness in most people. Eliminating the headset requires the same performance delivered to wall-sized displays. Rendering data that meet such demanding requirements provides a quality immersive experience but implies the need for potent processors.
Overcoming Platform Fragmentation
VR gaming, unlike traditional gaming platforms, is taking a long time to make its way into the mainstream. One difficulty in attracting interest is that it’s difficult to portray the richness of VR in a two-dimensional video when advertising to the uninitiated. Another challenge facing VR is that technology is changing fast in a very fragmented VR platform market. For example, in VR gaming there are no standards for a User Interface (UI). Each platform creates a different way for users to navigate. Without standards, a fragmented realm of VR gaming platforms means that VR games and other content must be ported to several different platforms. Yet much of what drives the gaming industry is the content, not the platform.
Developers want to maximize the content experience on every form factor, yet they have a lower performance margin to work with when altering games and user experiences in VR because they cannot risk dropping framerate. It’s difficult for VR game developers to ensure maximum saturation, the maximum amount of immersive or detailed content, while also accounting for all of the user devices that games or applications inhabit.
Augmented Reality (AR) on mobile devices like smartphones and tablets is experiencing uptake, ever since Pokémon Go was released in mid-2016. For example, with an AR app on a smartphone, a customer can hold their phone up to a shelf and see a product information overlay. However, people do not want to hold their phone at arm’s length for a long time.
Why is VR so slow to be taken up?
VR has not hit the tipping point yet. One analogy to consumer VR adoption is the PC market of the 80s and 90s, when at first PCs were too expensive while also having too little memory, storage, and processor speed to be good enough for the market that we have in gaming today. Simple gaming platforms like Atari were the limit of the gaming experience. Today we expect fast-moving, immersive gaming experiences that can also carry multi-player communities with a social aspect.
XAR does open up new experiences. One can watch video on a personal screen and avoid disturbing others because the headset is immersive and confines the screen to the user’s eyes only. For training purposes, XAR cannot be beat, since trainees are not as easily distracted as they would be watching a typical 2D training video. The headset forces the wearer to be in the training environment. Surreptitiously looking at one’s phone is not possible without removing the headset. Future consumer-oriented devices using VR would include cameras that will allow families to take stereoscopic videos of children or pets that can be shared with family members on another continent. The family VR video would be an immersive captured memory of an event, such as sharing the kids opening Christmas presents with a parent stationed in the military overseas.
As a solitary experience, VR for the moment is still, for the most part, socially isolating. Currently, the price for a decent VR system is as high as a laptop. A genuinely good system for a top-of-the-line VR experience costs in the range of a thousand dollars or more, which is out of reach of the mainstream consumer. Three areas need to improve before VR enters the mainstream consumer market: an affordable price for quality VR, the ability for people to share or socialize with the VR experience, and additional and innovative content.
The social aspect of VR is improving. Often, parents will buy a gaming platform with the excuse of kids or families playing it together. But unlike bowling with a Wii console at a party, VR has been slow to create socially engaging experiences. Presently, VR is only experienced by the individual who is wearing a headset. When PCs first enabled chat facilities on the internet, the social aspect of engaging with real people made services like AOL take off. VR has a capability similar to chat in that an immersive conversation, albeit using an avatar (one’s physical representation to others in VR), seems more real than text flowing across a screen. Supported on Oculus Rift and HTC Vive, Facebook Spaces (still in beta) offers a VR experience much like a chat room but with the physical experience of seeing other VR users in the room, also gesturing and talking. Facebook recently updated the capability for user’s avatars to look more like the user, although they still resemble cartoon characters.
Bigscreen is a VR room (or “lounge”) for sharing experiences with others around the world. According to the Bigscreen site, “Use cases of Bigscreen include both entertainment and productivity. It’s used as a virtual living room to watch movies, play video games, browse the web, and hang out with friends. It’s also used for productivity as a tool for remote teams to collaborate together in virtual offices.” This use case for XAR enables collaborating with physical movements possible, such as arranging dance choreography or rehearsing a theater act. Bigscreen requires a quality VR headset as well as a PC with a minimum of 16 GB RAM, a powerful graphics card, and an Intel Core i5/i7 processor or AMD Ryzen (or higher) core for a comfortable performance. Another such VR lounge is the Oculus Rec Room, where users can play games together with virtual others. Many users claim that in a VR meeting space, they immediately feel like the person they’re talking to is right next to them. The audio experience correlates to the visual experience. Users on a quality system like the Oculus can hear a colleague on the left and another on the right in these virtual meeting rooms, which makes all the difference versus a flat-screen video conference in terms of meeting others.
XAR is a growing yet challenging industry. However, rapidly improving technology is enabling XAR to incrementally improve quality for a more immersive experience.
Lynnette Reese is Editor-in-Chief, Embedded Intel Solutions and Embedded Systems Engineering, and has been working in various roles as an electrical engineer for over two decades. She is interested in open source software and hardware, the maker movement, and in increasing the number of women working in STEM so she has a greater chance of talking about something other than football at the water cooler.