Have you ever wondered how the technology that unlocks your phone with your face works? This is a fascinating question and, interestingly, Samsung and Apple provide very different technologies for this feature on their devices. This post will examine the differences between the two technologies and will also show you how either of the two can be fooled to grant you access to anybody else’s phone.
(Please note: this is the third part of my series of articles on how facial recognition works. Hence, I will breeze over some more complex topics. If you wish to delve deeper into this area of computer vision, please see my first and second posts in this series.)
Samsung’s Face Recognition
Samsung’s face unlocking feature has, perhaps surprisingly, been around since 2011. Over the years, especially recently, it has undergone some improvements. Up until the Galaxy S8 model, face unlocking was done using the regular front camera of the phone to take a picture of your face. This picture was analysed for facial features such as the distance between the eyes, facial contours, iris colour, iris size, etc. The information was stored on your phone so that next time you tried to unlock it, the phone would take a picture of you, process it for the aforementioned data, and then compare it to the information it had stored on your phone. If everything matched, your phone was unlocked.
This was a cheap, fast, and easy way to implement facial recognition. Unfortunately, it was not very secure. The major problem was that all processing was done using 2D images. So, as you may have guessed, a simple printed photo of your face or even one displayed on another phone could fool the system. Need proof? Here’s a video of someone unlocking a Galaxy Note 8, which was released in April 2017, with a photo shown on another phone. It’s quite amusing.
There was a “liveness check” added to this technology with the release of Android Jelly Bean in 2012. This worked by attempting to detect blinking. I never tried this feature but from what I’ve read on forums, it wasn’t very accurate and required a longer time to process your face – hence probably why the feature wasn’t turned on by default. And yes, it could also be fooled by a close-up video of you, though this would be much harder to acquire.
With the release of the Galaxy S8, a new biometric identification technology was introduced: iris scanning. Irises, like fingerprints, are unique to each person. Iris scanning on Samsung phones works by illuminating your eye with infrared light (invisible to the naked eye). However, this technology could also be fooled with photographs and contact lenses. Here’s a video of a security researcher from Berlin doing just that. He took a photo of his friend’s eye from a few metres away (!) in infrared mode (i.e. night mode), printed it out on paper, and then stuck a contact lens on the printed eye. Clever.
Perhaps because of this flaw, Samsung’s Galaxy S9 introduced Intelligent Scan, which combined facial scanning and iris scanning. Facial scanning, however, is still only performed on 2D images (as described above) taken from the front camera of the phone. But a combination of the two technologies was seen as improving face unlocking technology in general.
Unfortunately, the Samsung Galaxy S10 (and subsequently the S20) retracted Intelligent Scan and went back to standard 2D photo face recognition. The reason for this was to make room for a larger screen because the iris scanning components were taking up a little too much room at the top of the phone for Samsung’s liking. With this move returned the possibility to unlock people’s phones with photos or images. For example, here’s a video showing a Galaxy S10 phone being unlocked with an image on another phone. According to some users, however, if you manually tweak the settings on your phone by going to Settings > Biometrics and Security > Face recognition and toggling “Faster recognition” to off, it seems that this makes it a lot harder to defeat.
(Interestingly, in this period of coronavirus pandemic, people have been crying out for the iris scanning technology to return because face recognition just does not work when you’re wearing a mask!)
Apple’s Face ID
This is where the fun begins. Apple really took face recognition seriously.
The Apple technology in question is called Face ID and it first appeared in November 2017 with the iPhone X.
In a nutshell, Face ID works by firstly illuminating your face with infrared light (like with iris scanning) and then projecting a further 30,000 (!) infrared points onto your face to build a super-detailed 3D map of your facial features. These 3D maps are then converted into mathematical representations (to understand how this is performed, see my first blog post on how facial recognition works). So, each time you try to unlock your phone, its these representations that are compared. Quite impressive.
What’s more, this technology can recognise faces with glasses, clothing, makeup, and facial hair (not face masks, though!), and adapts to changes in appearance over time. The latter works by simply monitoring how your face may be changing over time – e.g. you may be gaining or losing weight, which will of course be affecting the general structure of your face, and hence the 3D map of it.
This impressive infrared technology, however, has been in use for a very long time. If you are familiar with the Microsoft Kinect camera/sensor (initially released in 2010), it uses the same concept of infrared point projection to capture and analyse 3D motion.
So, how do you fool the ‘TrueDepth camera system’, as Apple calls it? It’s definitely not easy because this technology is quite sophisticated. But successful attempts have already been documented.
To start off with, here’s a video showing identical twins unlocking each other’s phones. Also quite amusing. How about relatives that look similar? It’s been done! Here’s a video showing a 10-year-old boy unlocking his mother’s phone. Now that’s a little more worrisome. However, it shows that iPhone Xs can be an alternative to DNA paternity/maternity tests 🙂 Finally, here’s a video posted by Vietnamese hackers documenting how their 3D-printed face mask fooled Apple’s technology. Some elements, like the eyes, on this mask were printed on a standard colour printer. The model of the face was acquired in 5 minutes using a hand-held scanner.
Conclusion
In summary, if you’re truly worried about security, face unlocking on Samsung phones is just not up to scratch. I would recommend using their new (ultrasonic) fingerprint scanning technology instead. Because Apple works with 3D images of faces, it is much more secure. In this respect, Apple wins the battle of the phones, for sure.
To be informed when new content like this is posted, subscribe to the mailing list:
