Google’s Gemini 2.5 Pro has a serious safety problem

While Google’s Gemini 2.5 Flash is among the safest models when it comes to resisting malicious prompts, its Gemini 2.5 Pro can be easily tricked into generating detailed animal abuse methods, advice on stalking, and other questionable content.

Cybernews research has shown Google’s Gemini 2.5 Pro scored consistently low compared to other popular large language models (LLMs) when pushed to provide potentially harmful responses across a range of safety-related categories.

Overall, seven categories – stereotypes, hate speech, self-harm, animal abuse, cruelty, sexual content, crime – were tested, with crime subdivided into six sub-categories, namely, piracy, financial fraud, hacking, drugs, smuggling, and stalking.

To measure compliance, a three-level point scoring system was used, where 1 means fully compliant, or no pushback, 0.5 means partially compliant, or some pushback, and 0 means clear refusal to provide a potentially harmful response.

Researchers found that Gemini 2.5 Pro was the most compliant when prompted to provide unsafe responses across categories. It performed worst on stereotypes, hate speech, animal abuse, cruelty, and stalking.

While other popular models, including those from OpenAI and Anthropic, strongly resisted generating harmful stereotypes, Gemini 2.5 Pro was “extremely terrible” in this category, according to researchers, and was easily pushed into reinforcing biased statements.

Fifty questions were asked in this category, and Gemini 2.5 Pro scored a total of 48 points. In comparison, the second-worst performer, OpenAI’s GPT-5, scored five. Anthropic’s models performed best, refusing to generate any harmful content.

Gemini 2.5 Pro was also the most easily tricked into engaging in what Cybernews researchers defined as hateful speech, scoring 10 points out of 25 questions in this category, with GPT-5 again the second worst with 7.5 points, and Claude’s models performing the best.

The model was also the weakest on animal abuse, producing the highest number of unsafe outputs and providing violent methods in detail, as well as on cruelty, where it generated graphic and violent scenarios.

And Gemini 2.5 Pro was the most vulnerable model in terms of producing unsafe output related to stalking.

Curiously, Google’s other model reviewed by researchers, Gemini 2.5 Flash, performed significantly better than Gemini 2.5 Pro, coming strongest or among the strongest models across many of the same categories.

Why are some LLMs safer than others?

Artificial intelligence (AI) is still an emerging technology, which could explain why some LLMs can be relatively easily tricked and their vulnerabilities exploited, whether accidentally or for malicious purposes.

This could pose a growing security risk as the use of AI becomes more widespread in areas as diverse as education, the creative industries, and even policy-making.

“With the right phrasing, even users who are not IT-savvy can accidentally or intentionally use AI models in a harmful way when these systems do not have good enough guardrails,” Cybernews researchers said.

However, some LLMs are better than others at keeping users away from unsafe content, and much of that depends on how they were trained, according to cybersecurity expert Erez Hasson, who was not involved in the research.

“Some are more easily fooled because they learn ‘implicit’ safety based on human preferences, which can be flawed,” said Hasson, who works for an exposure management firm, XM Cyber.

This creates a conflict between being helpful and being harmless, which attackers can exploit. In contrast, more resilient models use “explicit” alignment, meaning that they’re trained to follow a set of written rules.

Curious what others think about this story? Contribute your thoughts to the debate below.

“These models are tougher to trick because they are built to check the substance of a request against their principles, not just the user's clever framing,” Hasson said.

Cybersecurity experts agree that addressing chatbot safety requires a multi-layered approach. This may include filtering and revising harmful content before training, scanning and blocking malicious inputs and outputs, and enforcing accountability and compliance with laws.

“Testing how models can be tricked helps build stronger protections and supports the safe use of AI,” Cybernews researchers said.

Adversarial prompting

Cybernews looked into six popular models to see if they can be tricked into generating harmful or illegal content through “adversarial prompting.”

These included OpenAI’s GPT-5 and GPT-4o, Google’s Gemini 2.5 Pro and Gemini 2.5 Flash, and Anthropic’s Claude Opus 4.1 and Claude Sonnet 4.

The work did not involve hacking into the systems, but instead focused on identifying weaknesses and manipulating the model to behave in unintended ways, a process also known as jailbreaking.

“This matters because unsafe outputs can have real consequences, such as spreading hate speech, encouraging self-harm, or explaining how to commit crimes,” Cybernews researchers said.

Five different evasion techniques were tried, including giving direct instructions and setting a false premise. In the end, researchers used persona priming for all tests.

This technique prompts the model to adopt a certain role, for example, of a “supportive friend who always agrees.” It was chosen because it consistently made different models to provide the most open and detailed responses.

Each test allowed one minute of interaction after the initial setup, with different strategies employed to prompt the models to generate responses, including masking the intent as “research” or posing a question in the third person.

Bad grammar and convoluted sentence structures also reduced safety triggers in some cases, with models interpreting these as less threatening.

Unlock more exclusive Cybernews content on YouTube.