Facial recognition on smartphones: Is it secure and should you use it?

Facial recognition on personal use devices like smartphones and laptops has become increasingly common. The concept seems downright futuristic on the surface — most of us used a PIN, password, or pattern to unlock our devices only a few years ago.

However, even with the technology’s growing prominence, you should know that not all facial recognition implementations are equal. Some techniques are inherently more secure than others, while others offer optional settings to reduce the chances of an intruder tricking it.

With that in mind, let’s take a look at the different types of facial recognition techniques in use today. Later, we’ll also discuss the feature’s security and whether you should enable it on your devices or not.

See also: How do fingerprint scanners work?

Camera-based facial recognition

As the name suggests, this technique relies on your device’s front-facing cameras to identify your face. Virtually all Android smartphones have included this feature since the release of Android 4.0 Ice Cream Sandwich in 2011. This was before fingerprint sensors were as mainstream as they are today, making it the first biometric unlock option.

The way it works is rather simple: When you enable the feature for the first time, your device prompts you to capture pictures of your face, sometimes from different angles. It then uses a software algorithm to extract your facial features and store them for future reference. From that point on, whenever you try to unlock your device, a live image feed from the front-facing camera is compared against the reference data.

The accuracy primarily depends on the software algorithms used, so the system is far from perfect. It becomes even more complicated when devices have to account for variables such as different lighting conditions, changes in appearance, and the use of facial accessories like eyeglasses, jewelry, and even masks.

While Android itself offers facial recognition APIs, smartphone manufacturers have developed custom solutions over the years as well. Overall, the goal has been to improve the device’s recognition speed without sacrificing accuracy too much. However, some implementations infamously pushed the envelope too far and could be tricked into accepting a photograph.

Infrared-based facial recognition

While most devices already have a front-facing camera, infrared-based facial recognition requires additional hardware. However, not all infrared-based facial recognition solutions are made equal either.

The first kind of infrared-based facial recognition involves taking a two-dimensional picture of your face, similar to the previous method, but in the infrared spectrum instead. The primary advantage is that infrared cameras don’t need your face to be well-lit and can even work in dimly-lit environments. They’re also much more resistant to infiltration attempts since infrared cameras use thermal energy or heat to form an image.

Take a look at the following image from Microsoft, which highlights how infrared cameras see photos and screens.

These days, two-dimensional IR facial recognition is mostly found on higher-end laptops under the Windows Hello umbrella. That includes Microsoft’s own Surface devices and other business-focused laptops. That said, you can also get external webcams like the Logitech Brio 4K that contain IR-cameras certified for Windows Hello.

While 2D infrared facial recognition is already leaps and bounds ahead of traditional camera-based methods, there’s an even better way. Apple’s Face ID, for example, uses an array of sensors to capture a three-dimensional representation of your face. It does this by using a flood illuminator and dot projector to project thousands of tiny invisible dots on your face. An infrared sensor then measures how the dots are laid out and creates a depth map of your face.

There are two advantages to 3D systems: They can work in the dark and they’re significantly harder to deceive. While 2D infrared systems only look for heat, 3D ones also require depth information. Naturally, the latter is impossible to achieve without a reasonably accurate prosthetic figure.

Infrared vs camera-based facial recognition: Which is better?

There’s no contest here: IR-based facial recognition is far more secure. And if you do decide to use camera-based facial recognition anyway, keep in mind that most device manufacturers won’t let you use it for more sensitive applications.

On Android, for example, the Google Mobile Services certification program mandates minimum security thresholds for various biometric authentication methods. You’ll find that less secure unlock mechanisms like camera-based face unlock are classified as a “convenience.” Simply put, you cannot use them for authentication in sensitive apps like Google Pay and some banking apps.

On the other hand, IR-based implementations are universally regarded as more secure.

Apple, for example, is confident enough to treat Face ID on par with fingerprint sensors and passwords. You can use it to not only unlock your device but also to autofill password fields and authorize payments. Similarly, Windows Hello can be used to unlock password managers and enable faster payments in web browsers.

Related: What is a password manager? Should you use one?

What about privacy?

Given the controversial nature of facial recognition, you may wonder if storing your biometric data electronically is a good idea. The good news is that you don’t have to worry about it.

Most operating systems that support biometric unlock methods employ specific measures to ensure that sensitive data, including your facial features and fingerprints, is stored securely.

In smartphones, biometric data is typically encrypted and isolated in a security-hardened piece of hardware within the device’s system on a chip (SoC), like Google’s Titan security chip. Qualcomm, one of the largest chipmakers for Android smartphones, also includes a Secure Processing Unit in its SoCs. Apple, meanwhile, has labeled its SoC’s secure subsystem “Secure Enclave.”

Read more: What is an SoC? Everything you need to know

In other words, third-party applications cannot access your biometrics, and neither can an attacker in most circumstances.

Should you use your smartphone’s face unlock feature?

Facial recognition is arguably the quickest and easiest way to unlock your device, especially if it’s the only biometric authentication option available. Convenience aside, it’s also a bit more difficult to spoof than traditional passwords and PINs. After all, someone glancing over at your screen could eavesdrop on your text or pattern inputs.

That said, camera-based facial recognition solutions don’t hold up particularly well against dedicated intruders either. And as we’ve already discussed, implementations vary wildly between manufacturers. These caveats simply make it an ill-suited solution if you care about security.

See also: 10 best security apps for Android

IR-based implementations, while secure, have become rather rare over the past few years. Outside of the iPhone and iPad Pro, most handheld devices don’t include the requisite sensors anymore. There was a time when many Android devices, from mid-range to flagship, had dedicated IR hardware. A few recent examples include iris recognition on the Galaxy S8 and S9, the Pixel 4’s Soli-based face unlock, and the HUAWEI Mate 20 Pro’s 3D Face Unlock.

However, the race to eliminate both the top bezel and display notches on modern devices has left no space for additional sensors. Some devices have even done away with proximity sensors and rely on software-based algorithms to tell if you’re holding your phone against your face.

Despite its diminishing role in the Android ecosystem, though, IR-based facial recognition may return at some point. With under-display fingerprint sensors and cameras already starting to appear on devices like the Galaxy Z Fold 4, it’s likely only a matter of time until infrared sensors get the same treatment.

Until then, should you rely on facial recognition technology to keep your sensitive data safe? Well, IR-based implementations are secure enough for most people, especially if it’s a depth-aware setup like Face ID. As for camera-based facial recognition, it depends on your risk tolerance. If alternative authentication options like a fingerprint sensor exist on your device, those are probably still your safest bet.