As a surveillance technique it’s ubiquitous, but is still a political, legal and ethical conundrum.
Facial recognition technology has spread prodigiously. It’s there on Facebook, tagging photos from the class reunion, your cousin’s wedding and the office summer party. Google, Microsoft, Apple and others have built it into apps to compile albums of people who hang out together.
It verifies who you are at airports and is the latest biometric to unlock your mobile phone where facial recognition apps abound. Need to confirm your identity for a £1 000 bank transfer? Just look into the camera.
New applications crop up all the time. Want to know who’s at the door? A video doorbell with facial recognition will tell you, provided you’ve uploaded a photo of the person’s face.
Other systems are used to spot missing persons and catch slackers who lie about the hours they spend in the office. Advertisers, of course, are in on the act.
Thanks to facial recognition, billboards can now serve up ads based on an estimate of your sex, age and mood.
That sounds quite Big Brother. Is it a surveillance tool?
Sometimes, yes. China uses facial recognition for racial profiling and its tracking and control of the Uighur muslims has been roundly condemned as a shameful first for a government. Its cameras also spot and fine jaywalkers, verify students at school gates, and monitor their expressions in lessons to ensure they are paying attention.
Russia has embraced the technology too. In Moscow, video cameras scan the streets for “people of interest” and plans have been mooted to equip the police with glasses that work the same way.
There have been reports that Israel is using facial recognition for covert tracking of Palestinians deep inside the West Bank.
Meanwhile, in Britain, the Metropolitan and South Wales police forces have trialled facial recognition to find people in football and rugby crowds, on city streets, and at commemorations and music festivals. Taylor Swift even installed the tech at a gig in California to weed out stalkers.
Shops are increasingly installing the technology to deter and catch thieves. Next year, it will make its Olympic debut in Tokyo.
How did it get everywhere?
Advances in three technical fields have played a major part: big data, deep convolutional neural networks and powerful graphics processing units or GPUs.
Thanks to Flickr, Instagram, Facebook, Google and others, the internet holds billions of photos of people’s faces which have been scraped together into massive image datasets.
They are used to train deep neural networks, a mainstay of modern artificial intelligence, to detect and recognise faces.
The computational grunt work is often done on GPUs, the superfast chips that are dedicated to graphics processing. But there is more to it than flashy new technology.
Over the past decade in particular, facial recognition systems have been deployed all over the place, and the data gathered from them has helped companies hone their technology.
How does it work?
First off, the computer has to learn what a face is. This can be done by training an algorithm, usually a deep neural network, on a vast number of photos that have faces in known positions. Each time the algorithm is presented with an image, it estimates where the face is. The network will be rubbish at first, like a drunk playing pin the tail on the donkey. But if this is done multiple times, the algorithm improves and eventually masters the art of spotting a face. This is the face detection step.
Next up is the recognition part. This is done in various ways, but it’s common to use a second neural network. It is fed a series of face pictures and learns — over many rounds — how best to tell one from another.
Some algorithms explicitly map the face, measuring the distances between the eyes, nose and mouth and so on. Others map the face using more abstract features. Either way, the network outputs a vector for each face — a string of numbers that uniquely identifies the person among all the others in the training set.
In live deployments, the software goes to work on video footage in real time. The computer scans frames of video usually captured at crowd pinch points, such as entrances to football stadiums. It first detects the faces in a frame, then churns out vectors for each one.
The face vectors are then checked against those for people on a watchlist. Any matches that clear a preset threshold are then ranked and displayed.
In the British police trials a typical threshold for a match was 60 percent, but the bar can be set higher to reduce false positives.
It’s not the only way the police use facial recognition. If a suspect has been picked up, officers can upload their mugshot and search CCTV footage to potentially trace the suspect’s movements back to the scene of a crime.
How accurate is it?
The best systems are impressive. Independent tests by the US National Institute of Standards and Technology (Nist) found that between 2014 and 2018 facial recognition systems got 20 times better at finding a match in a database of 12m portrait photos. The failure rate fell from 4 percent to 0,2 percent over the period, a massive gain in accuracy that was largely down to deep neural networks. Nist says the networks have driven “an industrial revolution” in facial recognition.
But such great performance depends on ideal conditions: where a crisp and clear mugshot of an unknown person is checked against a database of other high quality photos. In the real world, images can be blurred or poorly lit, people look away from the camera, or may have a scarf over their face, or be much older than in their reference photo. All tend to reduce the accuracy. — The Guardian.