Firm that verifies mugshots for ChatGPT and Roblox feeds US surveillance apparatus with 269 distinct checks

Every selfie or ID you upload to ChatGPT, Roblox, LinkedIn, and many other sites for verification is handled by a San Francisco firm called Persona. A massive leak has exposed its other side – a platform capable of feeding the US government with 269 sophisticated surveillance checks on millions of users worldwide.

A security researcher at vmfunc.re, who goes by the alias Celeste, discovered exposed infrastructure belonging to Persona, the identity verification company used by ChatGPT and other major services.

Persona also runs a platform authorized by FedRAMP (Federal Risk and Authorization Management Program), offering federal agencies “to verify users’ identities” in over 200 countries, detect fraud, and ensure regulatory compliance.”

Celeste claims they obtained the entire dashboard codebase from the ONYX government deployment “app.onyx.withpersona-gov.com,” which was left unprotected and publicly exposed.

“Every internal model, every API call, every permission check, every workflow,” the report on “identity surveillance machine that files reports on you to the feds” reads.

“On a FedRAMP-authorized government endpoint, it’s CATASTROPHIC.”

The blog claims that 2,456 source files expose 269 verification checks offered to government customers, including checks for whether a face looks “suspicious,” and two parallel systems for politically exposed persons (PEPs).

The Persona’s platform is capable of screening users against global watchlists, flagging potential money launderers or terrorist financiers. Built-in functionality includes filing direct reports to the US Treasury’s financial intelligence unit, FinCEN, and Canada’s equivalent, FINTRAC (the Financial Transactions and Reports Analysis Centre of Canada).

However, deeper concerns arise about the source of the data. When users submit their IDs and selfies for verification on popular platforms, the data likely ends up being analyzed and resold for many other purposes.

The conversation on this topic is live. Join in the discussion.

The company acknowledged “an oversight by the team,” and said that the discovered instance is currently under development and is not actively used. However, Persona’s CEO also accuses the investigators of bias and said they were not contacted before the findings were released.

“We do not work with any federal agency today,” Rick Song, co-founder and CEO of Persona, said in a correspondence with vmfunc.re researcher.

“We do not want our technology to be used by ICE or the government for any surveillance purposes.”

ChatGPT users analyzed for national security?

The vmfunc.re researchers initially sought to develop a tool to bypass Persona’s age verification system. They probed the internet-facing company’s infrastructure and said they never breached any system or used any credentials.

However, one IP address led to “openai-watchlistdb” endpoints, suggesting databases containing data belonging to ChatGPT platform users. Servers were not protected behind Cloudflare, and led to Google Cloud.

Certificates revealed that the service had been running since November 2023, nearly two years before OpenAI actually started requiring ID verification for advanced models.

Persona’s own case study confirms that OpenAI screens 99% – millions of users – each month with zero friction “behind the screens.” The firm’s website lists many other customers, including Coursera, Wilio, Square, Roblox, LinkedIn, and others.

“The same company that takes your passport photo when you sign up for ChatGPT also operates a government platform that files Suspicious Activity Reports with FinCEN and tags them with intelligence program codenames. Same codebase. Confirmed by matching git commit hashes across deployments,” the researchers at mfunc.re allege.

Alon Gal, a Co-founder and CTO at Hudson Rock, explains that the findings reveal how standard “age verification” selfies end up being processed “through a system that links facial biometrics to financial records and law enforcement databases.”

A simple login step for many popular apps turns out to be “a live feed for a national surveillance apparatus.”

“Routine ID checks for OpenAI users are being fed into a dedicated ‘watchlist’ database that has been operational since 2023,” Gal said in a LinkedIn post.

The leak also revealed that Persona itself uses OpenAI integration for its operation as a productivity tool.

However, the researchers admit that the findings do not prove the direct bidirectional data pipeline between ChatGPT users and screenings performed by the government.

“The code does prove that Persona operates both systems, that both run the same software, and that both are live right now.”

Broad capabilities

The leaked code allegedly exposes broad surveillance and reporting capabilities. The system can check selfies, look up government databases, such as the US driver’s licence database, and detect tampering or synthetic content.

Vmfunc.re lists the following features of the exposed Persona’s governmental system:

• Files Suspicious Activity Reports directly with FinCEN and Suspicious Transaction Reports with FINTRAC
• Tags Suspicious Transaction Report (STRs) with intelligence program codenames, such as Project SHADOW, Project LEGION, and others
• Maintains biometric face databases with 3-year retention
• 269 distinct verification checks against every user
• Comparing selfies to political figures with facial similarity scoring
• Flagging users as a “suspicious entity” based on face alone
• Classifying selfie spoof risk with hardcoded rejection thresholds
• Screening against 14 categories of adverse media, from terrorism to espionage
• Functionality for uploading and executing custom FinCEN screening lists against the entire user base
• Continuous re-screenings on configurable intervals
• Tracking users across 13 types of lists, ranging from browser fingerprints to geolocations
• Crypto wallet screening against sanctioned addresses through integration with Chainalysis
• Experimental unnamed ML models analyzing biometric data
• Data encryption with shared symmetric keys and obfuscation
• Two parallel PEP screening systems with known incompatibilities

“You hand over your passport to use a chatbot, and somewhere in a datacenter in Iowa, a facial recognition algorithm is checking whether you look like a politically exposed person. Your selfie gets a similarity score. Your name hits a watchlist. A cron job re-screens you every few weeks just to make sure you haven’t become a terrorist since the last time you asked GPT to write a cover letter,” the report alleges.

The leak allegedly contains 53MB of original source code on a public IP address.

What does the company say?

Persona’s CEO Song reacted to the allegations on X, accusing the investigators of bias and a lack of good journalism for not contacting the company before releasing their findings. He also shared the correspondence with the vmfunc.re researchers behind the report.

However, Song did not address the findings directly. The shared correspondence only confirms the cybersecurity incident itself.

“Unfortunately, providing the sourcemaps was an oversight by the team working on this, and we haven’t started an in-depth security review/pen test on the project yet, given it’s still in early development,” the response reads.

It also appears that the Vmfunc’s report initially exposed the names of engineers who were working on the Persona’s platform. The author removed the names after Song’s intervention.

The researchers noticed that the subdomain “onyx.withpersona-gov.com” bears the same name as ICE's $4.2 million AI surveillance tool ONYX, though Song assured the resemblance is coincidental, and the name was taken from a coworker's favorite Pokémon.

Cybernews has reached out to Persona for official statements or comments and will update the story with its response.

Unlock more exclusive Cybernews content on YouTube.