Facial recognition technology explained

Facial recognition technology is now a staple of smartphone security, along with the trust old PIN and increasingly elaborate fingerprint scanners. While not necessarily more secure than a fingerprint scanner, biometric ideas like facial recognition tend to be faster and more convenient to use. So let’s explore what options are out there, how they work, and what they mean for security.

Android’s basic facial recognition

While specific OEMs may be talking up their own fancy security technologies, did you know that Android has been able to unlock your phone with your face since Ice Cream Sandwich? Almost all smartphones implement this technology today, as an alternative to unlocking your phone with a PIN or fingerprint.

Unfortunately, the way that this standard facial recognition works is not very secure. It only relies on your front facing camera and a 2D facial recognition algorithm, which makes it cheap and easy to implement. These two features are all Android needs to create a picture of your face and features. However, as this is just a 2D image a simple photograph of you is enough for a theif to fool the sytem and unlock your phone.

The speed level of security provided by this technique varies a lot, and many Android OEMs have worked to improve on it over the years. The quality of the front camera is a determining factor, as is the complexity of the algorithm used to extract facial details. The use of neural network hardware can also accelerate more secure algorithms on high-end smartphones. See HUAWEI’s 360 Face Unlock that shipped with its P20 series and OnePlus’ speedy unlocky technologies as examples. Sadly, lower cost models are seldom as snappy.

Samsung Intelligent Scan

Samsung was the first to pack advanced facial recognition technologies into a top-tier flagship with iris scanning technology inside the ill-fated Galaxy Note 7. The technology stuck around inside the Galaxy S8 and new Note 8, which forms part of Samsung’s security suite alongside a broader face recognition system and fingerprint options.

Samsung’s iris scanning technology works by identifying the patterns in your irises. Just like fingerprints, these are unique to each person, making them very difficult to replicate. To do this, Samsung’s latest flagships are equipped with an infrared diode that illuminates your eyes regardless of the surrounding lighting conditions. This light wavelength can’t be detected by a regular front-facing camera, so a special infrared narrow focus camera then captures the detailed iris information. This image is then stored and processed locally on the device, nothing is sent via the internet.

In more recent times, Samsung introduced its Intelligent Scan facial recognition system inside the Galaxy S9. Intelligent Scan is also included in the Galaxy Note 9. This promises to be harder to fool than the Galaxy S8’s technology, which could be tricked with photographs and contact lenses.

The technology combines iris and facial scanning techniques together to improve accuracy and security. It even works better in low light. The facial scanning part of technology simply builds up a 2D image map of your face, which is common to all Android phones. The key is to combine the infrared iris scanning part with this 2D image to double up on the security layers.

Ultimately, Samsung’s technology is only so secure. The company doesn’t allow facial scanning biometrics to be used for highly sensitive security concerns like making payments. Instead, these can still only be made using the more secure fingerprint scanner.

Infrared assisted face unlock

Another more secure option is to use infrared cameras to detect faces, rather than just a regular camera that operates in the visible spectrum. This requires extra hardware in the form of an IR emitter and a camera capable of detecting the IR light, but this doesn’t have to be massively expensive.

This method is just like taking a 2D picture but in the IR spectrum. This makes it much harder to dupe with a simple picture. The use of IR also ensures that the feature works consistently regardless of the lighting conditions. It’s a similar idea to Samsung’s Iris Scanner but involves looking at the user’s whole face. This is certainly faster, but not necessarily more secure.

2D IR facial recognition isn’t hugely common, but it is a less expensive alternative to high-end 3D face unlock technologies. You can find this in technology inside the POCO F1 and the regular Xiaomi Mi 8.

Apple Face ID and 3D scanning

Apple unveiled its new Face ID technology as part of its iPhone X launch, the first 3D face scanning tech in a smartphone. Unlike the basic IR technology mentioned previously, 3D scanning is designed to map out a user’s entire face in a highly secure manner. It doesn’t just rely on the phone’s familiar front-facing camera, there are actually lots of sensors crammed onto that strip at the top.

The iPhone X comes equipped with an array of sensors designed to capture details of your face. For starters, it uses an infrared flood light to illuminate your face, which will work regardless of your surrounding lighting conditions as it’s outside of the visible spectrum. A secondary 30,000-point infrared laser matrix is then beamed out, which reflects off the flood light. Rather than snapping a picture of this infrared light, a special infrared camera detects subtle changes in the matrix point reflections as your face makes minute movements, which allows the camera to capture very accurate 3D depth data.

Apple isn’t the only company making use of an infrared sensor for 3D face mapping though. Similar technologies can be found inside the Xiaomi Mi Explorer Edition, the OPPO Find X, and Huawei’s Mate 20 Pro.

The technology on offer from all of these companies is very similar. All three utilize an infrared light array to create a detailed depth map of your face. The resolution of the map depends on the size of the infrared matrix array. HUAWEI uses 30,000-points just like Apple, but OPPO halves this with 15,000 dots. vivo has another solution that uses just up to 1,000 points.

While there are accuracy differences between implementations, the only way to consistently fool this technology would be to make a very accurate prosthetic. Or have an identical twin who wants to snoop around in your phone.

ZTE Eyeprint ID and Hawkeye

Although it never made it to market, ZTE’s Project CSX (or Hawkeye) was an intriguing prospect, as it boasted some interesting biometric security and software technologies. The iris scanning technology was likely based on the company’s earlier Eyeprint ID, developed by EyeVerify, which featured in the company’s older Grand S3, Blade S6, and original Axon smartphone.

Unlike Samsung’s infrared technology, Eyeprint ID used the phone’s high-resolution front camera to scan the user’s eye, identifying blood vessel patterns that are unique to each individual. While that’s a great way save costs, it’s more prone to being fooled than modern infrared implementations.

On top of the security angle, the ZTE Hawkeye’s technology would also have allowed users to control Android using eye motions. Compatible pieces of Android software could have been made to scroll up, down, left, and right using nothing more than the movement of your eyes.

That’s a novel idea, but probably won’t ever make its way into a smartphone now that IR technology is increasingly commonplace.

The most secure is …

3D IR depth mapping is by far the most secure method of facial recognition technology currently on the market. Unlike basic camera-based solutions, 3D IR technologies can’t be fooled by photographs and are faster and more reliable than ideas like iris scanning. Any IR implementation is better than a regular camera unlock, which we wouldn’t recommend using if you’re after tight security.

Currently, 3D face unlocking is the only face scanning technology currently on the market secure enough to use as authentication for mobile payments. That being said, a trusty password or PIN is still arguably the more secure option here, as these are the security features that are the hardest to break.