An entire community fears AI facial recognition is being deployed "by design" to target them
“A disturbing and dystopian expansion of live facial recognition.”
Image by Cybernews
- WA Police's NEC NeoFace M40 trial has scanned 130,000+ faces in Perth and Fremantle since June, yielding 33 alerts, 18 arrests, and one false ID against a 4,000-person watchlist in week one.
- Experts note facial recognition algorithms are trained mainly on white male faces, making them less accurate for people of color and women – and WA Police won't confirm if the system was tested on First Nations faces.
- Aboriginal Legal Service WA got just 20 minutes' notice before launch, and the state's privacy watchdog wasn't consulted either, fueling claims the rollout deliberately targets First Nation populated areas – which police deny.
- The same tech is now live at London's Victoria Station, with Big Brother Watch calling it a "dystopian expansion" unfit for a democracy.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
A mass AI surveillance pilot run by police in Western Australia has been criticized for targeting areas with significant Aboriginal populations.
The trial, which launched in June across Perth and Fremantle, has seen police use NEC's NeoFace M40 facial recognition software to scan more than 130,000 people, comparing their faces against a police watchlist in real time.
According to police data from the project's first week, the technology triggered 33 alerts and led to 18 arrests. The algorithm also falsely identified one person.
The project is being closely watched by law enforcement agencies across the country, but it has drawn criticism from privacy experts and legal advocates, who say police failed to properly consult with Aboriginal communities before the rollout.
Questions have also been raised about an opaque watchlist that critics say disproportionately targets First Nations people.
Faces checked against the watchlist
Police said the cameras check faces against a watchlist of about 4,000 people, including those accused of serious offenses, missing persons, and people who may pose a risk to themselves or others.
But Melbourne Law School associate professor Jake Goldenfein told ABC news that the algorithms powering these systems were trained predominantly on white male faces, leaving other demographic groups underrepresented – and, as a result, less accurately identified.
Historically, these systems have performed worse for people of color and for women, and especially worse for women of color.Melbourne Law School associate professor Jake Goldenfein
WA Police did not respond to a question about whether NEC's technology had been tested or trained on First Nations faces prior to the trial.
Aboriginal Legal Service of Western Australia legal services director Peter Collins told the ABC that very little time was set aside to discuss the pilot before it launched – he received an email offering just 20 minutes of consultation just 2 days before the trial began.
Surveillance by design?
Collins said the trial was one of a growing number of ways the WA Aboriginal community was being over-surveilled by police.
It can only have been by design, not chance, that facial recognition technology has been rolled out in those suburbs and places where everyone is aware that significant numbers of Aboriginal people either live or frequent.Peter Collins, Aboriginal Legal Service of Western Australia legal services director
WA's privacy watchdog, the Office of the Information Commissioner, said the state police force did not invite it to take part in any "meaningful" consultation process for the trial.
Defending the rollout, a WA Police spokesperson told the ABC that "the locations are identified ... based on policing needs and policing objectives. Locations do not target any group of people or demographic."
London’s NeoFace pilot expands to the Underground
NEC’s NeoFace M40 algorithm facial recognition technology is also currently being trialed by the British Transport Police
Stay updated with our latest stories and follow us on social media
Be the first to discover new stories, ideas, and updates from our team.
The pilot began in February this year at selected key transport hubs in London and expanded to Victoria Station on the London Underground on Tuesday.
While Transport for London claimed that the move would increase the security of passengers – especially female passengers – on the underground, privacy advocates argued that, in a democracy, it is wrong to use live facial recognition routinely in a way that captures the images of innocent individuals.
Silkie Carlo, the director of Big Brother Watch, told the BBC: "This is a disturbing and dystopian expansion of live facial recognition…in the sort of pervasive way you might expect in China but not in a democracy."
