Augmenting Reality and Transforming Industries

Integrating Data into Life

Augmented Reality (AR) is a transformative technology that we believe has the potential for a significant impact across many different sectors. AR is often discussed in the same context as Virtual Reality (VR), but we see them differently and have chosen to focus our efforts on AR technologies. VR also has the potential to be extremely important and is the source of far more venture capital investment than AR, but the AR applications that impact the work, retail, and communications sectors are more intriguing to us than the disruption that VR will bring to the entertainment industry.

AR is a hybrid visualization technology that combines the real world and the digital world. Rather than being fully immersive (i.e., a blacked-out headset where all you see are digital representations), AR systems overlay digital information onto the user’s normal field of view. The technology comes in various forms: glasses, headsets, heads-up displays (HUDs) in vehicles and helmets, and, in increasing volume, mobile applications and gaming. The 2016 Pokémon Go game craze was the first true mainstream AR boom and is illustrative of how the technology works. In this game, you hold a mobile phone as you walk around real places and hunt for digital cartoon characters. When viewed on the phone, the characters appear to be live in the real-world environment. While we admit to not fully appreciating the appeal, Pokémon Go had 45 million daily users at one point in 2016, and several hundred million installs.

AR gaming applications will undoubtedly proliferate as a result of the recent introduction of Apple’s ARKit, which opens AR app development to the wider iOS developer community. However, we are focused on the professional applications of AR technologies. Using smart glasses empowered with AR software, companies have already begun increasing the productivity and quality of skilled workers in engineering, mechanics, manufacturing, and logistics.

Professional AR Applications

Upskill, a company based out of the Washington, DC, metro area, has built its business on developing industry-specific AR software for smart glasses with a huge range of industry applications. Using Upskill’s AR platform, end-user companies can customize real-time worker enhancements for their labor forces. For instance, Boeing uses smart glasses powered by AR to guide technicians as they install wiring in planes. Once the AR-powered devices were implemented, Boeing was able to cut production time by 25 percent and lower its error rates to functionally zero. Another example is in warehouse management. GE Healthcare implemented AR software in smart glasses for its warehouse workers and immediately saw a 46 percent improvement in completing pick-list order fulfillment.

The fact that companies have already seen such vast improvements in worker skill, precision, and work quality in this first wave of AR-enabled industrial devices is evidence that AR will impact the workforce massively in the decade to come. Companies have the latitude to not only think about how AR can benefit their workers within current workflows, but also rethink the very scope of capabilities assumed of their workers in the first place. Jobs that were previously thought to be impossible can now be made possible.

AR will also change the ways that we interact with information on a personal level and the way that businesses interact with their customers. The earliest computers were based on a model of fetching data in response to commands; in order to retrieve or use the data, you were required to sit down at a terminal. This evolved into our current model of mobile and instantaneous data-fetching that we keep in our hands and pockets at all times.

Augmented reality is the next iteration in that cycle—now the information we seek will be integrated into the very environment we inhabit. Instead of looking at a terminal or a mobile device, we will look through the device and watch data interact with our world, right in front of our eyes. How will it change our interactions when you look at someone and see their LinkedIn profile, Facebook, and Twitter feeds digitally superimposed next to their faces? Or how about their criminal record? What will it mean to have the entirety of the internet packaged and visualized so that we no longer have to look facts up on Wikipedia on our phones, but see them superimposed when we look at the object in question?

The retail shopping experience is another area of broad impact. In the retail space, stores can supply customers with AR-powered wearables to use as they shop. Prices, inventory, and accessory products can be automatically shown as an overlay on each item a customer picks up or looks at. IKEA has already provided customers with a tool that allows them to place furniture they are considering within an image of their living room with the touch of a button. In the home-buying market, realtors have created in-depth AR home tours of properties they showcase in open house events.

Consider the virtual showroom that could enable a better shopping experience without the inventory costs of having to maintain every type of furniture, television, or dishware at every retail location. Consider also the diminishing importance of the retail showroom, as you could use an AR system at home to see exactly what that couch would look like in your living room. These tools are available now and will only increase in capability over time. How will real estate markets be impacted if many retail chains decide that investments in AR technology for in-home shopping are better uses of capital than large facility and inventory costs? What will that mean for the local tax base when these technologies enable online shopping for more and more categories of goods if an in-state presence is required in order to apply sales tax?

These technologies can also change how we interact with our work colleagues, especially those in remote locations. There are a number of challenges associated with managing a dispersed or remote workforce, but it has simply become a necessary part of the operating environment, especially for companies with multi-location or international operations. Conference call and video teleconference technologies have improved significantly over the past 15 years. While inexpensive video teleconferences have been a great addition to the corporate communications toolset, some argue that they provide only marginal benefits over the telephone, and that they often fail to create the feeling that you are interacting with colleagues or customers.

AR technologies could provide a significantly new environment for remote collaboration. Imagine looking around a conference room table and seeing digital representations of your colleagues sitting at the table with you. A manager of a remote team of employees located all over the world can create virtual rooms for real-time meetings, with crucial interactions like brainstorming and whiteboard planning—historically difficult to do on a conference or video call—visible as an overlay on the virtual environment, accessible and editable by all participants. Training employees will also become more immersive, effective, and most importantly, expedient. Complex instructions can be conveyed through a series of prompts and tutorials visible over the very work the employee is performing, giving “on the job training” a new and more precise meaning. Or imagine walking through a tour of a facility with an inspector or a potential client where they can actually see what you are seeing. The hologram avatars of science fiction could be made reality by AR systems, which could finally bridge the gap between onsite and remote collaboration.

This will certainly drive the companies that operate video teleconference systems to make adjustments and investments in order to remain competitive, but the impact will be far broader, affecting not only where businesses choose to operate but also where people live. And any forces that broadly affect where and how people live will impact all of the different types of companies and public sector institutions that provide products and services.

Security and Privacy

There are also significant applications of these technologies for military and law enforcement professionals. Imagine a world in which law enforcement has the ability to monitor populations not only with street cameras, but with augmented reality devices equipped on patrolling officers as well. Combining that with simple facial recognition (the same thing Facebook uses to automatically tag your friends) and iris scans, correct identification of persons of interest will be efficient, rapid, thorough, and accurate. Paired with machine-learning-enabled software, there is a potential for identifying suicide bombers before they ever make it into a crowd, or notifying people in the crowd that something is amiss.

These systems could also vastly outperform body cameras as reliable tools for helping officers provide complete real-world context behind why and how specific decisions were made in stressful situations. On the other hand, it raises questions of personal security and the role of the government. How would search and seizure rules change in this landscape? Would people be required to don certain types of wearables? How much information should citizens be required to share?

It is a common practice to place a red sticker in the window of a child’s bedroom so that firemen know where to go first. Augmented reality for emergency responders could send them into buildings with this information already in their field of view, as well as a full blueprint of gas lines, bedrooms, and access and egress points.

The military applications for AR technology are widespread and growing fast. VR and AR are currently being used for battlefield simulation, vehicle simulation, and virtual boot camp, and immersive heads-up displays will soon be standard issue for ground personnel. Troops will be able to view a real-time overlay of maps, mission parameters, target identification, temperature warnings, weapon sights, topographical data—the list goes on and on. AR is much better suited for this type of environment than VR, as the transparent material of AR wearables—as opposed to occlusive and immersive VR headsets—allows the wearer to maintain situational awareness within their environment. The reality they see is enhanced, not virtual.

Small unit commanders could have information around targeting, supporting weapon systems, adjacent forces, and commands overlaid on a HUD that allows for real tactical and decision-making support from anywhere in the world, without forcing an operator to stop and read a computer screen in battle. This will serve to significantly increase the precision and effectiveness of military units, allowing more and more operations to be conducted by smaller and smaller deployed forces.

Military vehicles will also receive the AR treatment in upcoming implementations. AR HUDs are being developed with the objective of turning tanks, planes, and ships into essentially transparent boxes, allowing the occupants to be fully immersed and aware of the environment outside the vehicle as they travel through it, while still protected by full defensive capabilities. No longer will a gunner be required to physically put his head into the turret to look around for targets; an AR headset will allow him to have a 360-degree view of the battlefield from inside the vehicle. That view will then be enhanced by target identification, friend-or-foe recognition, mapping, and other AR overlays.