Eurostar AI chatbot flaws exposed after “painful” disclosure process

A series of vulnerabilities in Eurostar’s AI-powered customer support chatbot have been uncovered, raising questions about both the security of customer-facing bots and how penetration test reports are handled.

The issues were disclosed on Monday by Pen Test Partners in a blog post authored by senior consultant Ross Donald, following what the firm describes as a lengthy and difficult process.

Donald first encountered the chatbot when he was planning a trip, where he noticed a disclosure that stated :”The answers in this chatbot are generated by AI” which piqued the researcher’s interest.

He reports in his blog: “I could ask slightly less structured or less predictable questions and see the chatbot respond in a way that clearly went beyond a simple scripted flow. That was the first sign that this was likely backed by a modern LLM rather than a fixed rules-based bot.

Donald then took the opportunity to test the chatbot “within the bounds of their [Eurostar’s] vulnerability disclosure program."

A chatbot built for general queries

Donald notes that the chatbot was originally designed as a general customer enquiry tool, sitting on top of an LLM that was not intended to handle sensitive data or security critical functions.

This design choice, the researcher notes, is key to understanding both the nature of the flaws and their impact.

At the time of testing, the chatbot was not connected to booking systems or customer records, limiting the immediate risk – but its underlying architecture still allowed for manipulation.

How the vulnerabilities worked

One of the central issues was how the chatbot handled conversation history. According to the blog, the server validated only the most recent message, while trusting all earlier messages supplied by the customer.

“The server would happily accept the entire conversation history from the client, without any validation,” Donald wrote.

This meant that an attacker could alter the earlier images and feed them back to the AI, effectively rewriting what the chatbot believed had already happened.

“The server only verified the signature on the latest message. It never re-validated or re-signed the rest of the history. Any older message in the array, even one that had previously failed the guard, would be accepted as-is and passed into the model as trusted context,”

Ross Donald, Pen Test Partners

Donald explains that this flaw enabled prompt injection, a technique that can cause AI systems to reveal internal instructions or behave unpredictably.

The researchers also identified a HTML injection (self-XSS) issue, where unfiltered HTML could be rendered in chatbot responses, potentially allowing attacker-controlled code to run in a user’s browser.

Additional weaknesses included insufficient validation of conversation and message IDs, which could increase risk if chatbot sessions were shared and stored.

Disclosure process: “slow and painful”

Pen Test Partners say they first reported issues on June 11th, but received no response for weeks, despite repeated follow-ups. The disclosure was later said to have been lost during a transition to an outsourced vulnerability disclosure provider.

Donald described the process as “quite painful” and reported at one point Eurostar staff implied that researchers were attempting blackmail.

“At one point, the communication implied that we were trying to blackmail Eurostar” he wrote.

Donald rejected this suggestion, adding “The definition of blackmail requires a threat to be made, and there was, of course, no threat. We don’t work like that.”

The blog does not allege malicious intent by Eurostar, but criticises what it describes as poor coordination and limited familiarity with AI specific security reporting.

Issues resolved

Pen Test Partners say all reported vulnerabilities have since been fixed and that publication followed responsible disclosure timelines.

Douglas says that the core lesson is that old web and API weaknesses still apply even when an LLM is in the loop.

He advises firms using customer-facing chatbots to have a “simple incident response plan that covers AI features as well as the rest of the site, and give yourself an emergency kill switch so you can disable the chatbot or specific tools quickly if things go wrong.”

He also said that it was important for users and support teams to understand that AI answers are not authoritative and may be manipulated.

Commenting on the vulnerabilities and the disclosure process a Eurostar spokesperson said:

“Eurostar takes cyber security very seriously. The customer-facing AI chatbot referenced is an experimental service and does not provide access to internal systems or sensitive customer data. Any issues identified during early testing were addressed promptly, and we continue to monitor and strengthen our security controls.”

Eurostar

Unlock more exclusive Cybernews content on YouTube.