Amazon is trying to sell its Rekognition facial recognition technology to law enforcment, but the American Civil Liberties Union doesn't think that's a very good idea. And today, the ACLU provided some seemingly compelling evidence — by using Amazon's own tool to compare 2,500 criminal mugshots to members of Congress.
Sure enough, Amazon's tool thought 28 different members of Congress looked like people who've been arrested.
Here's the full list, according to the ACLU:
Senate
- Johnny Isakson (R-Georgia)
- Ed Markey (D-Massachusetts)
- Pat Roberts (R-Kansas)
House
- Sanford Bishop (D-Georgia)
- G. K. Butterfield (D-North Carolina)
- Lacy Clay (D-Missouri)
- Mark DeSaulnier (D-California)
- Adriano Espaillat (D-New York)
- Ruben Gallego (D-Arizona)
- Tom Garrett (R-Virginia)
- Greg Gianforte (R-Montana)
- Jimmy Gomez (D-California)
- Raúl Grijalva (D-Arizona)
- Luis Gutiérrez (D-Illinois)
- Steve Knight (R-California)
- Leonard Lance (R-New Jersey)
- John Lewis (D-Georgia)
- Frank LoBiondo (R-New Jersey)
- Dave Loebsack (D-Iowa)
- David McKinley (R-West Virginia)
- John Moolenaar (R-Michigan)
- Tom Reed (R-New York)
- Bobby Rush (D-Illinois)
- Norma Torres (D-California)
- Marc Veasey (D-Texas)
- Brad Wenstrup (R-Ohio)
- Steve Womack (R-Arkansas)
- Lee Zeldin (R-New York)
That's a lot of Congresspeople who may soon have some very valid questions about facial recognition and its potential to be abused — particularly since Amazon thinks the ACLU didn't use it properly to begin with!
It turns out that the ACLU got its mugshot matches by using the Rekognition software at its default 80-percent confidence threshold setting, rather than the 95-percent plus confidence level that Amazon recommends for law enforcement agencies.
"While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn't be appropriate for identifying individuals with a reasonable level of certainty. When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95 percent or higher," an Amazon spokesperson told CNET by email.
But an ACLU lawyer tells CNET that Amazon doesn't necessarily steer law enforcement agencies toward that higher threshold — if a police department uses the software, it'll be set to the same 80-percent threshold by default and won't ask them to change it even if they intend to use it to identify criminals. "Amazon makes no effort to ask users what they are using Rekognition for," says ACLU attorney Jacob Snow.
The ACLU says that even when it comes to facial recognition for security purposes, Amazon's website suggests that the 80-percent confidence threshold is sufficent.
Screenshot by ACLU
A source close to the matter tells CNET that when Amazon works with law enforcment agencies directly, like the Orlando Police Department, it teaches them how to reduce false positives and avoid human bias. But there's nothing to necessarily keep other agencies from simply using the tool the same way the ACLU did, instead of developing a relationship with Amazon.
It's worth noting that false positives are (currently!) an accepted part of facial recognition technology. Nobody — including the ACLU — is saying police would arrest or (God forbid) shoot someone based on a false match alone. Facial recognition narrows down the list of suspects, and then humans take over. Recently, facial recognition helped ID the Russian assassins who poisoned a spy in the UK, as well as the Capital Gazette shooter.
And Amazon didn't actually create that many false positives even at the 80 percent confidence ratio, compared to, say, the UK Metropolitan Police's facial recognition tech.
But the ACLU worries that Amazon's false positives might bias a police officer or government agent to search, question or potentially draw a weapon when they shouldn't — and we've all seen how those encounters can turn deadly.
The ACLU also provided CNET this statement:
Amazon seems to have missed, or refuses to acknowledge, the broader point: facial recognition technology in the hands of government is primed for abuse and raises significant civil rights concerns. It could allow – and in some cases has already enabled – police to determine who attends protests, ICE to continuously monitor immigrants, and cities to routinely track their own residents, whether they have reason to suspect criminal activity or not. Changing the threshold from 80 to 95 percent doesn't change that. In fact, it could exacerbate it.
Should Congress regulate facial recognition? Microsoft thinks so, and now 28 members of Congress have some very personal food for thought — 95-percent confidence threshold or no.
Update, 11:07 a.m. PT: Two of the Congressmen who were mismatched with mugshots, Reps. Jimmy Gomez and John Lewis, have requested an immediate meeting with Amazon CEO Jeff Bezos to "address the defects" of its facial recognition technology, according to an official letter obtained by BuzzFeed News.
