Lie Detection with Thermal Imaging – A Task for Computer Vision

Can thermal imaging detect if you’re lying or not? It sure can! And are there frightening prospects with respect to this technology? Yes, there are! Read on to find out what scientists have recently done in this area – and all using image processing techniques.

Thermal Imaging

Thermal imaging (aka infrared thermography, thermographic imaging, and infrared imaging) is the science of analysing images captured from thermal (infrared) cameras. The images returned by these cameras capture infrared radiation not visible to the naked eye that are emitted by objects. All objects above absolute zero (-273.15 °C or −459.67°F) emit such radiation. And the general rule is that the hotter an object is, the more infrared radiation it emits.

There has been some amazing work done recently by scientists with respect to thermal imaging and deception detection. These scientists have managed to construct sophisticated lie detectors with their thermal cameras. And what is interesting for us is that these lie detectors work by using many standard Computer Vision techniques. In fact, thermal imaging is a beautiful example of where Computer Vision techniques can be used on images that do not come from “traditional” cameras.

This post is going to analyse how these lie detectors work and underline the Computer Vision techniques that are being used by them. The latter part of the post will extrapolate how thermal imaging might impact us in the future – and these predictions are quite frightening/exciting, to say the least.

Lie Detectors

The idea behind lie detectors is to detect minor physiological changes, such as an increase in blood pressure, pulse or respiration, that can occur when we experience a certain anxiety, shame or nervousness when dropping a fib (aka telling a porky, taking artistic license, being Tony Blair, etc.). These are such slight physiological changes in us that instruments that measure them need to be very precise.

We’ve all seen polygraphs being used in films to detect deception. An expert sits behind a machine and looks at readings from sensors that are connected to the person being interrogated. Although accuracy is said to be at around 90% in detecting lies (according to a few academic papers I studied), the problem is that highly trained experts are required to interpret results – and this interpretation can take hours in post-interview analysis. Moreover, polygraph tests require participants’ cooperation in that they need to be physically connected to these sensors. They’re what’s called ‘invasive’ procedures.

Thermal imagery attempts to alleviate these problems. The idea behind them is to detect changes in the surface temperature of the skin caused by the effects of lying. Since all one needs is a thermal camera to observe the participant, they’re non-invasive procedures. And because of this, the person being interrogated can be oblivious to the fact that he’s being scrutinised for lying. Moreover, the process can be automated with image/video processing algorithms – no experts required for analysis!

Computer Vision in Thermal Imaging

Note: Although I gloss over a lot of technical details in this section, I still assume here that you have a little bit of computer vision knowledge. If you’re here for the discussion on how thermal imagery could be used in the future, skip to the next section. 

There are some interesting journal papers on deception detection using thermal imagery and computer vision algorithms. For example, “Thermal Facial Analysis for Deception Detection” (Rajoub, Bashar A., and Reyer Zwiggelaar. IEEE transactions on information forensics and security 9, no. 6 (2014): 1015-1023) reports results of 87% on 492 responses (249 lies and 243 truths). Techniques such as machine learning were used to build a model of deceptive/non-deceptive responses. The models were then utilised to classify responses.

But I want to look at a paper published internally by the Faculty of Engineering at the Pedagogical and Technological University of Colombia in South America. It’s not a very “sophisticated” publication (e.g. axes are not labelled, statistical significance of results is not presented, the face detection algorithm is a little dubious, etc.) but the computer vision techniques used are much more interesting to analyse.

The paper in question is entitled “Detection of lies by facial thermal imagery analysis” published this year (2017) by Bedoya-Echeverry et al. The authors used a fairly low-resolution thermal camera (320×240 pixels) to obtain comparable results to polygraph tests: 75% success rate in detecting lies and 100% success rate in detecting truths.

I am going to work through a simplified version of the algorithm presented by the authors. The full version involves a few extra calculations but what I present here is the general framework of what was done. I’ll give you enough to show you that implementing a thermal-based lie detector is a trivial task (once you can afford to purchase the $3,000 thermal camera).

This simplified algorithm can be divided into two stages:

1. Face detection and segmentation
2. Periorbital area detection and tracking

Let’s work through these two stages one-by-one and see what a fully-automated lie detector based on Computer Vision techniques can look like.

1. Face detection and segmentation

For face detection the authors first used Otsu’s method on their black and white thermal images. Otsu’s method takes a greylevel (intensity) image and reduces it to a binary image, i.e. an image containing only two colours: white and black. A pixel is coloured either white or black depending on whether it falls below or above a certain dynamically calculated threshold. The threshold is chosen such that it minimises the intra-class variance of the intensity values. See this page for a clear explanation on how this is done exactly.

When the binary image has been produced, the face is detected by calculating the largest connected region in the image.

Note: functions for Otsu’s method and finding the largest connected components are all available in OpenCV and are easy to use.

2. Periorbital area detection and tracking

Once the face has been detected and segmented out, the next step is to locate the periorbital region, which is the horizontal, rectangular region containing your eyes and top of your nose. This region has a high concentration of veins and arteries and is therefore ideal for scrutinising for micro-temperature changes.

The periorbital region can be found by dividing the face (detected in step 1) into 4 equally-spaced horizontal strips and then selecting the second region from the top. To save having to perform steps one and two for each frame, the KLT algorithm is used to track the periorbital area between frames. See this OpenCV tutorial page for a decent explanation of how this tracking algorithm works. It’s a little maths intensive – sorry! But you can at least see that it’s also easy to implement.

Temperature readings are then made from the detected region and an average calculated per frame. When the average temperature (i.e. pixel intensity) peaks during the answering of a question, the algorithm can deduce that the person is lying through their teeth!

That’s not that complicated, right? Even though I simplified the algorithm a little (a few additional calculations are performed to assist tracking in step 2), the gist of it is there! Here you have a fully-automated, non-invasive lie detector that uses Computer Vision techniques to get results comparable to polygraph tests.

What the Future Holds in Lie Detection and Thermal Imagery

Now, let’s have a think about what a non-invasive lie detector could potentially achieve in the future.

Lie detectors are all about noticing micro-changes in blood flow, right? Can you imagine such a system tracking you around the store to gauge which products get you a little bit excited? Results can be instantly forwarded to a seller who now has the upper hand over you. Nobody will be safe from second-hand car dealers any more. To confirm anything, all he has to do is ask if you like a certain car or not.

What about business meetings? You can put in a cheeky thermal camera in the corner of a meeting room and get live reports on how clients are really feeling about your pitch. Haggling will be a lot easier to deal with in this way if you know what your opponent is truly thinking.

And what about poker? You will be able to beat (unethically?) that one friend who always cleans up at your “friendly” weekend poker nights.

The potential is endless, really. And who knows?! Maybe we’ll have thermal cameras in our phones one day, too? Computer vision will definitely be a powerful tool in the future 🙂

What other uses of deception detection using thermography can you think of?

Summary

Traditionally, lie detection has been performed using a polygraph test. This test, however, is invasive and needs an expert to painstakingly analyse results from the various sensors that are used. Digital thermography is looking like a viable alternative. Scientists have shown that using standard computer vision techniques, deception detection can be non-invasive, automated, and get results comparable to polygraph tests. Non-invasive lie detectors are a scary prospect considering that they could track our every move and analyse all our emotions in real time.

To be informed when new content like this is posted, subscribe to the mailing list:

Leave this field empty if you're human: