In last week’s post I talked about plotting tracked customers or staff from video footage onto a 2D floor plan. This is an example of video analytics and data mining that can be performed on standard CCTV footage that can give you insightful information such as common movement patterns or common places of congestion at particular times of the day.
There is, however, another thing that can be done with these extracted 2D coordinates of tracked people: generation of heatmaps.
A heatmap is a visual representation or summary of data that uses colour to represent data values. Generally speaking, the more congested data is at a particular location, the hotter will be the colour used to represent this data.
The diagram at the top of this post shows an example heatmap for eye-tracking data (I did my PhD in eye-tracking, so this brings back memories :P) on a Wikipedia page. There, the hotter regions denote where more time was spent gazing by viewers.
There are many ways to create these heatmaps. In this post I will present you one way with some supporting code at the end.
I’m going to assume that you have a list of coordinates in a file denoting the location of people on a 2D floor plan (see my previous post for how to obtain such a file from CCTV footage). Each line in the file is a coordinate at a specific point in time. For example, you might have something like this in a file called “coords.txt”:
x_coords,y_coords
200,301
205,300
208,300
210,300
210,300
210,300
Update: Note the ‘301’ in the first row of coordinates in the y-axis column. After an update to one of the libraries I use below, the variance in a column cannot be 0.
In this example we have somebody moving horizontally 5 pixels for two time intervals and then standing still for 3 time intervals. If we were to generate a heatmap here, you would expect there to be hot colours around (210, 300) and cooler colours at (200, 300) through to (210, 300).
But how do we get these hot and cold colours around our points and make the heatmap look smooth and beautiful? Well, some of you may have heard of a thing called a Gaussian kernel. That’s just a fancy name for a particular type of curve. Let me show you a 2D image of one:
That curve can also be drawn in 3D, like so (notice the hot and cold colours here!):
Now, I’m not going to go into too much detail on Gaussian kernels because it would involve venturing into university mathematics. If you would like to read up on them, this pdf goes into a lot of explanatory detail and this page explains nicely why it is so commonly used in the trade. For this post, all you need to know is that it’s a specific type of curve.
With respect to our heatmaps, then, the idea is to place one of these Gaussian kernels at each coordinate location that we have in our “coords.txt” file. The trick here is to notice that when Gaussian kernels overlap, their values are added together (where the overlapping occurs). So, for example, with the 2D kernel image above, if we were to put another kernel at the exact same location, the peak of the kernel would reach 0.8 (0.4 + 0.4 = 0.8).
If you have clusters of points at a similar location, the Gaussian kernels at these locations would all push each other up.
The following image shows this idea well. There are 6 coordinates (the black marks on the x-axis) and a kernel placed at each of these (red dashed lines). The resulting curve is depicted in blue. The three congested kernels on the left push (by addition) the resulting curve up the highest.
This final plot of Gaussian kernels is actually called a kernel density estimation (KDE). It’s just a fancy name for a concept that really, in it’s core, isn’t too hard to understand!
A kernel density estimation can be performed in 3D as well and this is exactly what can be done with the coordinates in your “coords.txt” file. Take a look at the 3D picture of a single Gaussian kernel above and picture looking down at that curve from above. You would be looking at a heatmap!
Here’s a top-down view example but with more kernels (at the locations of the white points). Notice the hot colours at the more congested locations. This is where the kernels have pushed the resulting KDE up the highest:
And that, ladies and gentlemen is how you create a heatmap from a file containing coordinate locations.
And what about some accompanying code? For the project that I worked on, I used the seaborn Python visualisation library. That library has a kernel density estimator function called kdeplot:
# import the required packages
import pandas as pd
import seaborn as sns
import numpy as np
from matplotlib import pyplot as plt
# Library versions used in this code:
# Python: 3.7.3
# Pandas: 1.3.5
# Numpy: 1.21.6
# Seaborn: 0.12.1
# Matplotlib: 3.5.3
# load the coordinates file into a dataframe
coords = pd.read_csv('coords2.txt')
# call the kernel density estimator function
ax = sns.kdeplot(data = coords, x="x_coords", y="y_coords", fill=True, thresh=0, levels=100, cmap="mako")
# the function has additional paramter, e.g. to change the colour palette,
# so if you need things customised, there are plenty of options
# plot your KDE
# once again, there are plenty of customisations available to you in pyplot
plt.show()
# save your KDE to disk
fig = ax.get_figure()
fig.savefig('kde.png', transparent=True, bbox_inches='tight', pad_inches=0)
It’s amazing what you can do with basic CCTV footage, computer vision, and a little bit of mathematical knowledge, isn’t it?
To be informed when new content like this is posted, subscribe to the mailing list (or subscribe to my YouTube channel!):
