Waymo is testing Gemini AI integration in its robotaxis

Alphabet-owned Waymo is reportedly testing a new feature: an integration of Google’s Gemini AI chatbot as an in-car assistant in its robotaxis, which will answer questions and support riders on the road.

According to researcher Jane Manchun Wong, Waymo has a document internally titled “Waymo Ride Assistant Meta-Prompt”, which explains how an AI assistant would behave inside the vehicles.

“While digging through Waymo’s mobile app code, I discovered the complete system prompt for its unreleased Gemini integration. The document, internally titled ‘Waymo Ride Assistant Meta-Prompt’, is a 1,200+ line specification that defines exactly how the AI assistant is expected to behave inside a Waymo vehicle,” Wong writes.

Wong explains that the system prompt implies this is “more than a simple chatbot.” She explains that the assistant fully shapes the rider’s experience and is able “to answer questions, manage select in-cabin functions like climate control, and provide reassurance to riders.”

Additionally, the chatbot appears to have a clear personality (a friendly and helpful AI companion) and purpose (to enhance the rider's experience by providing useful information and assistance in a safe, reassuring, and unobtrusive manner).

A rule cited by Wong orders the chatbot to maintain a clear distinction between its identity (Gemini, the conversational AI) and the autonomous driving technology (the Waymo Driver). The chatbot is explicitly prohibited from attributing the act of driving or the vehicle's perception capabilities to itself.

When a driver activates the assistant via the in-car screen button, the chatbot should randomly select one of the pre-approved greetings, personalized with the rider's first name. The assistant can also access contextual data about the rider at runtime, such as the number of Waymo trips the rider has been on.

The assistant is also required to first see if a rider’s request maps directly to one of its defined “ai_control_types” rather than falling back to search. These function calls include controls of temperature, lighting, and music. Wong notes that some features are notably absent: volume control, window control, seat adjustment, and route changes, meaning that the assistant should deflect these to the in-car screen or Waymo app.

The system prompts allow the assistant to manage various interactions, using different tactics depending on the query. For example, for unclear requests, it uses a "Guess and Confirm" strategy, while for comfort-related requests, it opts for an action hierarchy: Clarify → Best Guess → Deflect.

While the assistant can answer general knowledge questions and even make dad jokes, it’s not allowed to handle financial requests, place orders for the rider, provide emergency assistance, or make phone calls.

In August, Waymo received its first permit to begin testing its autonomous vehicles in New York City in two of the city's five boroughs. New York City regulators have extended the company’s testing permit until the end of the year, with a requirement to have a human safety operator behind the wheel.

Yet, according to a Waymo spokesperson, the permit exempts their drivers from keeping one hand on the wheel at all times.