Virtual and Augmented Reality: Hot or Hype?
If people can get past the goofy headset, VR will open markets for entertainment, education, and marketing to ride a wave of technology with augmented reality and virtual reality (AR/VR).
Virtual Reality (VR) and Augmented Reality (AR) will take life to a new level of entertainment, gaming, education, marketing, and more. MTV was a whole new experience in the early 80s as cable television was expanding rapidly; now the music industry will have a new venue that goes far beyond music videos by delivering immersive personal concerts through VR. Prospects include a more immersive gaming experience with 360° views with heavy graphics processing and greater computing power than existing gaming platforms, further boosting the gaming platform market for growth as hardware tries to keep up with demanding graphics. Additional demands on platforms are imposed by the high throughput and low latency that cloud gaming requires.
Some see AR/VR as the “fourth wave of technology” due to the vast potential of applications. IDC forecasts the global augmented and virtual reality headset market to grow at a compound annual rate of 58%, with 99.4 million units sold and shipped in 2021. One of the largest barriers to sales of VR headsets, also known as head mounted displays (HMDs), is the demanding performance requirements for computers to run HMDs, which are essentially dual high-definition displays. Nevertheless, growth in HMDs feeds growth in computers and games to run on this relatively new head-bound platform. The gaming market is not waiting until VR is a proven technology to get on the adoption curve. According to Goldman-Sachs, 200,000 developers registered at Oculus by September 2015. Samsung announced at CES 2017 that the Gear VR HMD sales had hit 5 million units since first sales began in 2015.
The sophistication of AR/VR hardware leads to prices from $600 – $1,000 or more for a good-to-great HMD. Some available VR HMDs are the Samsung Gear VR, HTC Vive, Oculus Rift, and OSVR-compatible HMDs. (The oh-so-affordable Google Cardboard may qualify as a VR tool, however, it is not an HMD.) The Razer HMD is based on Open Source Virtual Reality (OSVR™). OSVR offers “an open standard for Virtual Reality input devices, games and output with the sole goal of providing the best possible game experience in the Virtual Reality space.” An AR/VR developer’s kit that combines a VR device and open-source software is available as the OSVR Hacker Dev Kit (HDK2). The HDK2 operates at 1080 x 1200p per eye at 90Hz (90fps) and supports over 300 games on Steam (a cloud-based game streaming and delivery platform).
So aside from gaming, what is AR/VR being used for today? One pragmatic application is Sotheby’s use of AR/VR to market multi-million-dollar luxury homes using 3D and VR tours. Not the same as in-person tours, nevertheless, Sotheby’s VR tours can help to significantly narrow down choices in distant locations without the cost of travel or a carbon footprint. According to Goldman-Sachs, “VR/AR technology could also reduce the need for in-store display inventory and potentially accelerate the erosion in value of physical stores to the extent that the viewing experience can be deployed in the home and via mobile devices. Finally, doctors and medical professionals are experimenting with AR as a hands-free medical tool, playing into a $16 billion patient monitoring devices market.”
In the workplace, AR can take us to a higher level of productivity by providing a kind of instant expertise. As technology improves, the complexity of related tasks can increase. Boeing, in collaboration with the University of Iowa, completed a study in airplane wing assembly that showed a dramatic reduction in the number of mistakes made when first-time assemblers used AR. In the study, 50 different steps and 30 parts were offered with three different sets of instructions. Three groups were tasked with assembling the wing using either a PDF on a desktop computer, a handheld tablet with the PDF, or a handheld tablet with AR software that included graphical overlays along with instructions. The three separate groups were each given two attempts to assemble the wing, and results were tallied with respect to mistakes made. Results in the first attempt (Figure 4) were significantly better using the tablet with AR.
Consumers (and retailers) can also benefit from AR. For example, a large U.S. home improvement store is experimenting with an AR application called Tango, which uses a smartphone to help customers find specific items quickly. The smartphone is held up by the user to capture surroundings with the smartphone’s camera, and the Tango app combines camera input with the user’s location within the store. Tango, having located the product via the use of a cloud database, overlays directional information onto the smartphone’s display. Tango-enabled smartphones use a depth-perception camera, 3D mapping, and area learning to accomplish the task of guiding customers to specific products. The same technology can be used for museums and other large indoor areas with information overlays and navigational assistance, reducing overall labor cost and customer frustration.
Developed by Google, Tango is a mobile phone platform that allows developers to overlay visual objects and/or directions onto the user’s surroundings as viewed through the user’s smartphone screen. Google’s Tango developer site states, “To implement Depth Perception, manufacturers of Tango devices can choose among common depth technologies, including Structured Light, Time of Flight, and Stereo. Structured Light and Time of Flight require the use of an infrared (IR) projector and IR sensor; Stereo does not.” The structured light method for depth perception casts a pattern onto an object; the deformation of the light striking surfaces is used to calculate depth information. With the stereo method, depth perception is determined using more than one image of the same scene, taken at slightly different angles.
Originally, Time-of-Flight (TOF) was a scientific term describing the time it takes to measure the distance that something travels through a medium. The term has been borrowed to describe depth perception for machines and their applications. Example applications for 3D TOF come in measuring the gestures people make in a live-action game, or to help an autonomous drone navigate around objects by itself using onboard TOF-assisted intelligence. TOF provides 3D camera imaging using pulsed light; for example, from an infrared LED (operating at about 850nm), which humans cannot see. A fast photo-detecting sensor picks up the reflected light and starts a counter using fast electronics, since achieving accuracy within a millimeter requires pulses that are just a few picoseconds long. The photo-sensing transducer converts spectral energy into an electronic signal. It should be noted that not all depth perception technologies are TOF-based. Intel® RealSense™ technology is another AR/VR-enabling technology with depth perception capability. The Intel® Core™ i7 69xx/68xx processor family, with up to 10 cores and 20 threads, is suggested for AR/VR, and Merged Reality (MR) use. Gamers can play a demanding game while simultaneously communicating with game teammates and live-streaming ongoing gameplay progress in full HD graphics on a 10-core machine.
A hunger for content will follow both AR and VR platforms, as it was with the explosion in cable television in the 1980s. Software programming has become more complex and involved due to portable, highly capable devices like smartphones. The term “programmer” has been replaced with “developer” in many application spaces. Technical degrees in game development also apply to practical purposes such as apps that use AR to help us find products or locate specific paintings in museums. Pokémon Go, a wildly popular AR game that hit smartphones in July 2016, did not use Tango. However, an interesting note is that the maker of Pokémon Go, Niantic Labs, spun-off Google about a year before the Pokémon Go launch.
The future for productivity applications and game developers looks bright. As multiple technologies converge in virtual reality, augmented reality, and with capabilities such as depth of field perception, we may see the world in a whole new way; or at least initially with our smartphones held out at selfie-length while we try to find an obscure item at the home improvement store.
- AR/VR Is Fourth Wave of Technology: Digi-Capital Founder. (2016, November 17). Retrieved March 26, 2017, from https://www.bloomberg.com/news/videos/2016-11-17/ar-vr-is-fourth-wave-of-technology-digi-capital-founder
- Shirer, Michael. “Worldwide Augmented and Virtual Reality Headset Market Expected to Grow at a Compound Annual Rate of 58%, Reaching 99.4 Million Units in 2021, According to IDC.” Www.idc.com. N.p., 16 Mar. 2017. Web. 26 Mar. 2017.
- “Virtual and Augmented Reality: Understanding the Race for the Next Computing Platform.” Profiles in Innovation. Goldman Sachs, 13 Jan. 2016. Web. 25 Mar. 2017.
- Jagneaux, David. “Samsung Confirms 5 Million Gear VR Mobile Headsets Sold To Date.” UploadVR. UploadVR, 05 Jan. 2017. Web. 26 Mar. 2017.
- Gaudiosi, John. “Now You Can Shop for Luxury Homes in Virtual Reality.” Fortune. Fortune, 9 Sept. 2015. Web. 26 Mar. 2017.
- “Tango.” Google. Web. Accessed 26 Mar. 2017. <https://get.google.com/tango/>.
- “Tango.” Google. Web. Accessed 26 Mar. 2017 <https://developers.google.com/tango/overview/depth-perception>
Lynnette Reese is Editor-in-Chief, Embedded Intel Solutions and Embedded Systems Engineering, and has been working in various roles as an electrical engineer for over two decades. She is interested in open source software and hardware, the maker movement, and in increasing the number of women working in STEM so she has a greater chance of talking about something other than football at the water cooler.