AI vs hate speech in gaming: a Call of Duty or a slippery slope?

Updated on: 15 November 2023

Neil C. Hughes
Contributor

Activision, the publisher of the Call of Duty franchise, recently revealed that it was addressing hate speech during online gaming sessions. Utilizing machine learning technology, they are rolling out a new tool called ToxMod, developed by Modulate, to identify discriminatory language and harassment in real-time.

ToxMod has been incorporated into Call of Duty's Modern Warfare II and Warzone titles, exclusively within the U.S. The full-scale deployment is slated for November 10th, coinciding with the next chapter's release, Modern Warfare III. With nearly 90 million people playing each month, the scale of the problem and solution is enormous. The tool promises to categorize toxic behavior based on severity before a human takes action, allowing for scalability in moderation efforts.

Activision's CTO, Michael Vance, emphasized that the aim is to make the gaming environment "a fun, fair, and welcoming experience for all players," a move that could set a precedent for how online communities handle toxicity. But the tool doesn't offer an opt-out feature for gamers; they either have to be part of this system or turn off in-game voice chat altogether. This raises questions about privacy and surveillance in digital spaces.

The double-edged sword of AI surveillance in online gaming

Introducing AI moderation tools in games may seem like a step in the right direction for building more inclusive online communities. But this innovation also throws open the doors to some big ethical questions. Just imagine if this technology ended up in the wrong hands. What started as a tool to make gaming more inclusive could become a sweeping surveillance apparatus, going far beyond its intended use and posing a real threat to our freedoms.

The rising incidence of data breaches further exacerbates this tension, introducing a very real layer of risk surrounding the storage and potential exploitation of voice recordings. This isn't just a hypothetical concern – it's a pressing issue that demands immediate and thoughtful attention.

Machine learning algorithms' technical limitations and ethical complexities also cannot be ignored.

Despite big promises, algorithms are susceptible to false positives and negatives, putting players at risk of unjust punishments or allowing genuine violators to go undetected. Equally troubling is the prospect of algorithmic bias, where an improperly trained machine learning model could inadvertently perpetuate existing societal prejudices, such as racial or gender biases.

There is no avoiding that poorly executed AI could transform a moderator into a propagator of inequality, contradicting the ethos of inclusivity it aims to uphold. The conversation around these AI solutions needs to transition from their capabilities to their limitations and the ethical ramifications that come with them. But this is not the only story this month about big tech flirting with privacy violations.

Big technology and the quiet intrusion into children's lives

Before accepting AI moderation listening tools, it's crucial to remember that Amazon is paying more than $30m in fines for multiple privacy violations, including illegally keeping Alexa recordings of children's voices. Despite being sold to families under the label of convenience, it finds itself at the crossroads of a disturbing violation of the Children's Online Privacy Protection Act (COPPA).

While the technology giant assures us of Alexa's privacy features, evidence suggests a blatant retention of children's voice recordings – even after parents have explicitly requested their deletion. This data hoarding not only contravenes the law but also imperils the privacy and safety of minors, risking unauthorized access and nefarious uses of sensitive personal information.

The FTC's allegations paint a chilling portrait wherein Amazon has used these voice recordings for algorithmic training, thereby commoditizing children's data without parental consent or knowledge. This is not just an issue of failing to delete data; it is a calculated oversight that thrusts children into an opaque data ecosystem with potentially lifelong ramifications.

As we evaluate the practical benefits of digital assistants and AI listening moderation tools, we must also scrutinize the ethical and legal boundaries they should respect. This incident serves as a cautionary tale of the indiscriminate data practices of big tech and signals an urgent need for more stringent oversight and consumer vigilance.

Confronting the unseen risks of voice technology

Voice recognition may seem less invasive than a facial scan, but its rising popularity also demands ethical and legal scrutiny. It's a field of biometric data collection that is subtle, often going unnoticed as it blends into our daily interactions with smart assistants and voice-activated devices. Yet, this quiet assimilation makes it equally, if not more, dangerous.

With calls to ban predictive policing algorithms and the increasing ubiquity of other biometric surveillance methods, it's crucial to expand the conversation to include voice technology. As we embrace the allure of hands-free commands and voice-activated convenience, we must also confront the reality that our speech could be used against us. For instance, tech expert Rob Williams recently produced an AI-generated voice recording so convincing that it even fooled his own wife.

We're entering a groundbreaking era where AI not only replicates our voices with uncanny accuracy but also eavesdrops on our conversations and exerts control over our digital accounts based on what it hears. These capabilities aren't distant science fiction – they're today's reality, underscoring the urgent need to navigate the imminent ethical and security challenges that lie ahead.

AI moderation: a slippery slope from online games to smart homes

The Call of Duty online experience has for too long involved the reflex action of muting disruptive players who enjoy hurling abuse while blasting annoying music into your ears. The advent of AI moderation across the franchise promises to replace negativity with a more harmonious community. But we should be under no illusion that this also opens a Pandora's box that extends beyond the virtual battlegrounds.

As we approach the closing of 2023, the question arises: what's stopping such technology from migrating to the digital assistants that share our living spaces – like Alexa, Google, and Siri? It's one thing for an AI to police an online game; it's another to monitor our homes. The line between a secure digital environment and an Orwellian surveillance state could become increasingly blurred.

The stakes are significantly higher in a future where your smart devices assist and "moderate" you. It's a chilling thought – what if an algorithmic misjudgment locks you out of your smartphone or, even worse, your own home? The balancing act here isn't merely about constructing smarter technology but critically examining the ethical landscape it helps to shape.

Welcoming AI into online gaming communities may be seen as a necessary evil to tackle toxicity. However, extending that invitation to our personal spaces must be approached with extreme caution and ethical rigor. The conversation around AI's role in our lives needs to include not just its capabilities but its boundaries as well. As these technologies converge, this is no longer a theoretical debate for a far-off future; now is the time to address these concerns.