Don’t ignore data protection when developing AI, ICO warns

Following an investigation into Snap’s ‘My AI’ chatbot, the UK’s Information Commissioner’s Office (ICO) has warned all organizations to protect privacy rights before releasing AIs to the market.

“We warn that organizations must not ignore data protection risks as we conclude the Snap ‘My AI’ chatbot investigation,” the ICOs statement reads.

The watchdog says that early actions into investigating generative AI deployment highlight the importance of privacy rights protection.

The ICO has recently concluded its investigation into Snap’s launch of the ‘My AI’ chatbot, which raised concerns that data protection risks may not have been adequately assessed.

“Our investigation resulted in Snap taking significant steps to carry out a more thorough review of the risks posed by ‘My AI’ and demonstrate to us that it had implemented appropriate mitigations. We are satisfied that Snap has now undertaken a risk assessment relating to ‘My AI’ that is compliant with data protection law. We will continue to monitor the rollout of ‘My AI’ and how emerging risks are addressed,” ICO says.

The Snap tool is powered by OpenAI’s GPT technology and is designed to learn about users over time and make users feel as though they’re chatting with a friend.

On October 6th, 2023, Snap received a preliminary enforcement notice over its potential failure to properly assess the privacy risks posed by the chatbot. The provisional findings suggested “a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching My AI.”

“Our investigation into ‘My AI’ should act as a warning shot for industry. Organizations developing or using generative AI must consider data protection from the outset, including rigorously assessing and mitigating risks to people’s rights and freedoms before bringing products to market,” said Stephen Almond, ICO Executive Director of Regulatory Risk.

He assures that the ICO will continue monitoring organizations and using “the full range of enforcement powers – including fines – to protect the public from harm.”

The ICO plans to publish its final decision in Snap’s case in the coming weeks.