Artificial intelligence start-ups OpenAI and the Google-backed Anthropic announced Thursday they would collaborate with the US AI Safety Institute to research, evaluate, and mitigate risks associated with AI models.
It’s the “first-of-their-kind” formal agreement between the US government and the AI sector that will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks, the Institute said.
As part of the deal, the AI Safety Institute – whose mission is to “build a team of world-leading talent to tackle some of the biggest challenges in AI safety" – said it would also provide feedback to the AI companies regarding possible safety improvements to their models.
“Safety is essential to fueling breakthrough technological innovation,” said US AI Safety Institute director Elizabeth Kelly.
“These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI,” Kelly said.
OpenAI’s Chief Strategy Officer Jason Kwon stressed the importance of the Institute’s role in defining how US leadership responsibly develops artificial intelligence.
“We… hope that our work together offers a framework that the rest of the world can build on,” Kwon said.
Earlier this month, OpenAI CEO and founder Sam Altman had publicly pledged to grant the Institute early access to its latest foundation model, saying it would “push forward the science of AI evaluations.”
The Institute, which operates under the auspices of the US National Institute of Standards and Technology (NIST), was created in 2023 by an executive order signed by US President Joe Biden to advance the responsible use of AI.
Each company will create its own “Memorandum of Understanding,” establishing a framework of parameters the AI Safety Insitute will follow, both ‘before and after’ it gets access to major new models.
The Institute will also be able to weigh in on potential safety concerns ‘before and after’ a model’s public release.
"Safe, trustworthy AI is crucial for the technology's positive impact. Our collaboration with the US AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment," said Jack Clark, Co-Founder and Head of Policy at Anthropic, backed by Amazon and Alphabet.
This summer, the United Kingdom’s AI Safety Institute (AISI) – which released the world’s first AI safety testing and evaluation report on five publicly available large language models this May – opened its first overseas office in Silicon Valley to foster collaboration with the US.
Meantime, California lawmakers voted to pass the artificial intelligence regulatory bill SB 1047 Thursday, which requires AI companies to perform in-house testing on large language models before they are released to the public. Former OpenAI owner and co-founder Elon Musk announced his support for the bill earlier this week.
Your email address will not be published. Required fields are markedmarked