What Is Augmented Reality?

The basic idea of augmented reality is to superimpose graphics, audio and other sensory enhancements over our existing real-world environment in real time. Sounds pretty simple. Besides, haven't television networks been doing that with graphics for decades? However, augmented reality is more advanced than any technology you've seen in television broadcasts, although some new TV effects come close, such as RACEf/x and the super-imposed first down line on televised U.S. football games, both created by Sportvision. But these systems display graphics for only one point of view. Next-generation augmented-reality systems will display graphics for each viewer's perspective.

Some of the most exciting augmented-reality work began taking place in research labs at universities around the world. In February 2009, technophiles at the TED conference were all atwitter because Pattie Maes and Pranav Mistry presented a groundbreaking augmented-reality system, which they developed as part of MIT Media Lab's Fluid Interfaces Group. They called it SixthSense, and although the project is stalled, it's a good overview of how you'll find basic components that are found in many augmented reality systems:

These components were strung together in a lanyard-like apparatus that the user wore around his neck. The user also wore four colored caps on the fingers, and these caps were used to manipulate the images that the projector emitted [source: TED 2009].

SixthSense was remarkable because it used these simple, off-the-shelf components that cost around $350. It was also notable because the projector essentially turned any surface into an interactive screen. Essentially, the device worked by using the camera and mirror to examine the surrounding world, feeding that image to the phone (which processed the image, gathered GPS coordinates and pulled data from the Internet), and then projected information from the projector onto the surface in front of the user, whether a wrist, a wall, or even a person. Because the user was wearing the camera on his chest, SixthSense augmented whatever he looked at; for example, if he picked up a can of soup in a grocery store, SixthSense found and projected onto the soup information about its ingredients, price, nutritional value — even customer reviews.

By using his capped fingers — Pattie Maes said even fingers with different colors of nail polish would work — a user could perform actions on the projected information, which were then picked up by the camera and processed by the phone. If he wanted to know more about that can of soup than was projected on it, he could use his fingers to interact with the projected image and learn about, say, competing brands. Sadly, the SixthSense project went into a years-long hiatus and will probably never reach markets. But there are many other products stepping into the AR fray [source: Vulcan Post].

In the Netherlands, cell phone owners can download an application called Layar that uses the phone's camera and GPS capabilities to gather information about the surrounding area. Layar then shows information about restaurants or other sites in the area, overlaying this information on the phone's screen. You can even point the phone at a building, and Layar will tell you if any companies in that building are hiring, or it might be able to find photos of the building on Flickr or to locate its history on Wikipedia [source: Ensha]

Layar isn't the only application of its type. In October 2018, an organization called Mural Arts Philadelphia created a gigantic interactive outdoors mural. Viewers pointed their smartphones at parts of the mural and then viewed various holograms and listened to matching music for a fully immersive art experience. And in a science-fiction dream come true, fans of the movies can now play "Star Wars" Holochess right on their phones, complete with futuristic graphics and sound [sources: Dickinson, Matney].

Health care professionals may soon rely on smartphone-enabled AR. A company called Tissue Analytics is honing an app that helps doctors and nurses use their phones to quickly identify specific types of wounds for faster diagnosis and more efficient care [source: Comstock].

An app that's simply called Augment will project just about any new product into a real-world environment. Whether it's a new recliner or a lamp, potential buyers can "see" the product in their own homes and read all information and reviews before making a purchase. AR Compass Map 3D is like a mapping app on steroids. It combines compass and map overlays with your camera to create a fully-immersive 3D map that guides you wherever you want to go [source: NewGenApps].

Similarly, a company called Total Immersion makes a wide variety of applications for business and fun. Want to know what those new glasses frames will look like perched on your face before you buy them online? Use the app to slide those frames onto your virtual face and you'll instantly know that horned rims really are not your style [source: Total Immersion].

Then there are apps like the previously mentioned Pokemon Go, a game that was insanely popular in 2016, and allowed players to hunt on their smartphone or tablet virtual creatures scattered in public spaces.

Militaries were some of the early adopters of gaming technology, seeing the possibilities for training soldiers for warfare realistically but in safe settings. And militaries are likely to do the same with AR.

Soldiers will plunge into an evermore immersive battlefield environment accentuated with helmet-mounted displays, smart glasses and much more. It's a market that may be worth $1.4 billion in 2018 [source: VisionGain].

Arcane Technologies, a Canadian company, has sold augmented-reality devices to the U.S. military. The company produces a head-mounted display — the sort of device that was supposed to bring us virtual reality — that superimposes information on your world. Consider a squad of soldiers in Afghanistan, performing reconnaissance on an opposition hideout. An AR-enabled head-mounted display could overlay blueprints or a view from a satellite or overheard drone directly onto the soldiers' field of vision.

And in an odd bit of fiction becoming reality, a lot of these military applications are just now becoming possible, in part because of the amazingly advanced physics calculations and programming made possible by ... the first-person-shooter gaming industry [source: Nichols].

Now that we've established some of the many current and burgeoning uses of augmented reality, let's take a look at the technology's limitations and what the future holds.

We could go on and on with examples of smartphone-driven AR apps, but these programs come and fade rather quickly. It's perhaps more notable to point out that manufacturers are so certain of AR's success that they're actually building devices meant to accentuate the AR experience. ASUS, for example, made the Zenfone AR in hopes of making an early splash in the AR-specific market [source: ASUS].

Two powerful tech leaders – Apple and Google – continue to tweak their mobile devices to handle the demands of AR-specific software. With ever-faster processors for the iPhone, iPad, and the entire galaxy of Android-driven smartphones, these pocket-sized computers are now powerful enough to run data-intensive apps of all kinds, including those that feature AR. Paired with the inevitable rollout of faster 5G data networks, those devices will be able to send and receive mind-boggling amounts of data that will make AR faster and better than ever before, and it will work whether you're in an office building or rolling down a country highway [source: Tardiloli].

You can bet that social media networks will try to capitalize on the coming AR trends. Facebook has its own AR Studio, which helps developers create AR apps meant specifically to work within Facebook's framework, and it is working on AR glasses, too. In the meantime, Google is touting its Tango AR platform, which sports visual search capabilities through Google Lens, a range of camera-enabled AR tools. Apple is in the middle of the race, too, with an ARKit that arms programmers with the code they need to dive into AR app creation for iPhones and other iOS-driven devices [source: Kahney].

The start-stop-start history of the famous Google Glass project might be a fitting way to sum up the nature of AR these days. In 2013, Google unveiled its AR-capable Google Glasses, which essentially provided an AR-type heads-up display of the world around the wearer. But the project lost momentum and stalled in 2015, only to find new hope in 2017, as the company resurrected the Glasses for business purposes. And in 2018, a company called Brain Power began selling the Google Glass as part of a program to help people with autism, improving their social skills and rewarding them for slowly but surely learning to interact in positive ways with the world around them [source: Kronk].

Augmented reality still has some challenges to overcome. For instance, people may not want to rely on their smartphones, which often have small screens on which to superimpose information. For that reason, wearable devices like augmented-reality capable contact lenses and glasses will provide users with more convenient, expansive views of the world around them. Screen real estate will no longer be an issue. In the near future, you may be able to play a real-time strategy game on your computer, or you can invite a friend over, put on your AR glasses, and play on the tabletop in front of you.

There is such a thing as too much information. Just as smartphone and internet addictions are concerns, an overreliance on augmented reality could mean that people are missing out on what's right in front of them. Some people may prefer to use their AR iPhone applications rather than an experienced tour guide, even though a tour guide may be able to offer a level of interaction, an experience and a personal touch unavailable in a computer program. And there are times when a real plaque on a building is preferable to a virtual one, which would be accessible only by people with certain technologies.

There are also privacy concerns. Image-recognition software coupled with AR will, quite soon, allow us to point our phones at people, even strangers, and instantly see information from their Facebook, Twitter, Amazon, LinkedIn or other online profiles. With most of these services people willingly put information about themselves online, but it may be an unwelcome shock to meet someone, only to have him instantly know so much about your life and background.

Despite these concerns, imagine the possibilities: You may learn things about the city you've lived in for years just by pointing your AR-enabled phone at a nearby park or building. If you work in construction, you can save on materials by using virtual markers to designate where a beam should go or which structural support to inspect. Paleontologists working in shifts to assemble a dinosaur skeleton could leave virtual "notes" to team members on the bones themselves, artists could produce virtual graffiti and doctors could overlay a digital image of a patient's X-rays onto a mannequin for added realism.

And we'll continue to see a mix of AR-enabled applications. For instance, companies will undoubtedly continue to unleash basic tools like an AR toothbrushing game from Dixie Cups, meant to teach kids basic brushing skills. And in the same breath, manufacturers, research facilities and more will find new ways to make people more and more productive in the face of an aging populace. All of these tools will help nudge the acceptance and capabilities of AR a bit further along this uncharted technological path [sources: Odell, Abraham and Annunziata].

Over the next few years, we'll likely see jumps in the evolution of the AR concept, in terms of both software, hardware, and a plethora of new applications. You might say that the future of augmented reality is so bright, you're going to need shades – and AR-glasses, too.