Augmented reality (AR) technology was publicly popularised by the viral Pokémon Go mobile game in 2016. Remember when everyone was walking around their neighbourhoods in search of Bulbasaur?
But there’s more uses for AR than video games. The technology also drives Snapchat Lenses, measuring apps, an Ikea furniture app, and even Google Translate.
It’s also making its way into the car, with brands such as Mercedes-Benz and Genesis adopting the technology to create smarter, safer satellite navigation systems.
Will the technology spread? And should we care?
What is augmented reality?
Augmented reality, as the name suggests, places computer-generated objects or environments on top of images from the real world.
This gives the illusion the holographic content is in the actual physical world, essentially superimposing objects into the view being displayed through a mobile device’s camera.
Depending on the application, AR experiences can place objects that move, are adaptable, or play video and audio.
Take an app such as Google Translate. Users can point the smartphone camera at a menu with foreign words and in a matter of seconds, Google will then place translated words onto that surface in real time.
If the camera moves around, the app tries to keep the translated text in its original place to make the image seem as natural as possible.
Ditto, Pokémon Go which places an animated character on the ground.
Social media app Snapchat features AR Lenses capable of detecting faces and imposing an animated filter exactly matching the user’s facial shape and size. It’s also adaptable – opening your mouth can trigger the Lens to change what’s shown on screen.
AR is seen as a useful tool for a plethora of sectors, including retail, healthcare, education, construction, real estate – and now the automotive space.
Google (among others) has attempted to make AR smart glasses where information such as time, directions, and notifications overlaid reality, but has been unsuccessful thus far.
It’s worth noting, AR is distinctly different from virtual reality (VR) and mixed reality (MR), and is not a form of artificial intelligence.
How does AR work?
Without diving into technicalities, AR uses computer vision, localisation, mapping, and depth tracking to more accurately place virtual objects.
It’s predominantly software-driven on a smartphone or tablet, processing real-world elements like the ground and table surfaces by analysing colours, textures, lines, and shapes.
This allows the software to memorise real-world elements and place a virtual object that appears to be aware of its relationship to its surroundings.
Hardware advancements like the time-of-flight (ToF) lidar sensor has allowed AR to analyse faster and be more precise. Devices such as the Samsung Galaxy S20 Ultra, Honor View 20, and iPad Pro (2020) tablet already house a ToF sensor.
Of course, this constant software processing is very power-hungry. Mobile tech gadgets quickly heat up, rapidly drain their batteries, and require powerful computing processors when users dive into the world of AR.
But how will AR benefit driving?
In the context of cars, Mercedes-Benz was one of the first carmakers to implement AR in the Mercedes-Benz User Experience (MBUX) infotainment system, debuted on the 2018 W177 series A-Class.
The AR navigation system in the A-Class uses the front-facing camera (usually used for the surround-view camera system) to project AR labels and arrow graphics onto a live image of the road when a destination is set.
Ever wondered if the next turn is the upcoming intersection or the one immediately after? Mercedes-Benz alleviates this worry – and possible safety hazard – by displaying the front-camera view overlaid with a live arrow on the touchscreen as you approach a junction or roundabout.
Hyundai’s luxury brand, Genesis, is employing a similar feature on its latest G80 sedan and GV80 SUV models. Expect augmented reality navigation to spread into more cars.
AR is capable of more than Need for Speed-style navigation overlays, though. There are plenty of potential applications that improve safety, and make tricky tasks slightly easier.
In the USA, the 2020 GMC Sierra 1500 pickup truck has an optional ProGrade Trailering System that includes a transparent trailer view.
Using the rear-facing tailgate camera and an accessory camera mounted at the back of a trailer, the Sierra can virtually merge both camera views to essentially eliminate a trailer or caravan for a clear view behind.
The system projects the trailer-mounted camera view onto the trailer and constantly adjusts it to ensure the view of what’s behind is accurate. This form of augmented reality is shown on the infotainment display to make the trailer virtually ‘transparent’.
Similarly, the 2018 Range Rover Evoque debuted Land Rover’s new Clearsight Ground View feature. It stitches together three forward-facing camera views to give a 180-degree view of what’s ahead, eliminating blind spots caused by the bonnet.
It places AR graphics of the front wheels and bonnet outlines for reference, while giving a clear view of what’s in front on the infotainment screen. This implementation is ideal for off-roading, where one wrong move could see the front of your expensive British SUV ripped off by an errant rock.
Constantly looking down at a touchscreen isn’t exactly ideal – it could be a serious distraction, dragging the driver’s eyes off the road. Enter the head-up display (HUD).
Head-up displays have been used for a while now to overlay important driving information such as speed, rudimentary navigation details, and safety information, but how the information is displayed has generally been limited.
With head-up displays increasing in size, the proliferation of augmented reality is the next step, potentially projecting three-dimensional graphics onto the road ahead in real time.
Mercedes-Benz has previewed the W223 S-Class, which will employ an augmented reality head-up display to display direction arrows, labels, and safety assist symbols safely in the driver’s line of sight.
Similarly, the Volkswagen ID.3 electric hatchback will feature the technology in its head-up display. Expect AR to gradually be offered in more cars (with more affordable price tags) over time.
How will AR evolve the car industry?
AR has already been on the minds of manufacturers for almost a decade.
BMW was the first carmaker to preview an augmented reality head-up display concept back in 2011.
Three years later, Jaguar Land Rover showed-off the ‘Follow-Me Ghost Car Navigation’ concept. It utilises AR by showing a virtual vehicle to follow on the head-up display for safer driving, rather than relying on navigation directions.
The British brand also proposed the HUD could project AR warnings for pedestrians near the vehicle, points of interest, or a countdown to indicate when the traffic lights will turn green as part of vehicle-to-everything (V2X) communication.
Outside the car, AR is transforming the way consumers buy cars. Companies such as Hyundai, Skoda and Porsche have launched AR mobile apps where users can project a life-sized car model into real-life environments.
Especially in the stay-at-home era of COVID-19, AR has been helpful to see how a car fits in someone’s garage or driveway. AR mobile apps have also been used as an alternative to the traditional manual.
Meanwhile, it’s transforming the car repair sector by guiding owners or mechanics through the physical complexities of each vehicle model.
It’s also opening opportunities to improve the design and car manufacturing processes using AR on smartphones, tablets and smart glasses – alongside VR and MR technologies.
BMW unveiling its landmark Vision Next 100 concept in 2016 with an AR HUD spanning the entire windscreen is a sign that AR will continue to grow in cars.
AR is simply an additional feature to add to the tech equipment list arsenal.
By making driving more distraction-free and aiding the car design, production and repair stages, it could be transformative to the automotive industry in years to come.
