Researchers recently observed a team of call center agents at a regional power grid company in China and interviewed them about their experiences with a newly introduced (artificial intelligence) AI assistant. The study revealed that AI in call centers can create more problems than it solves.
The tool was designed to handle transcriptions and form-filling, while also providing emotion recognition. What they found was a mix of convenience, frustration, and subtle pressure that never quite makes it into marketing decks. This wasn't just a test of AI performance. It offers a close-up look at how people interact with technology that often feels unfinished, even when it functions properly.
A digital notepad that sometimes keeps up
One of the most appreciated features for agents was real-time transcription. When calls became chaotic or customers rambled, the AI helped agents follow the thread. It allows them to scroll back and retrieve an address, a meter ID, or other details they may have missed the first time.
In a call center where speed and accuracy matter, that kind of backup support has obvious value. One agent described using the AI transcript as an on-screen notepad, especially when customers rattled off long strings of numbers. In some cases, it even stepped in as a communication lifeline when headsets malfunctioned. That meant fewer delays and fewer customer callbacks, which matters when every second counts. But it wasn't flawless.
Agents reported that the tool flagged customer frustration based solely on volume, missing the actual tone or intent of the customer.
Geoffrey Hinton says AI agents may need emotions to be effective in human roles
undefined Haider. (@slow_developer) June 16, 2025
For example, In a call center, if someone wants to chat all day -- the AI agent acts bored or irritated to end the call
undefinedif it behaves like it’s annoyed, i’d say it has an emotionundefined pic.twitter.com/O54tPtRm8c
Templates that save time until they don't
After each call, the AI attempted to draft a service summary based on the conversation. Despite promising to save time and help agents complete post-call documentation more efficiently, in practice, the drafts often needed heavy editing.
Templates were too long, too vague, or inconsistent with internal terminology. Some agents found it quicker to type the summary from scratch than to clean up the auto-filled version. The system also required extra steps that prefixed content, slowing them down. Editing address fields or power supply details required navigating through pop-up windows and rigid interface layers.
One agent explained that the template gave a starting point, but by the time you've trimmed the wording and rewritten the key parts, you're back to square one. Another noted that even when the tool pre-filled an address, they still had to double-check the power unit data to avoid triggering downstream errors. That sense of working around the tool instead of with it came up repeatedly in the study.
The study also categorized the new challenges. Each point represents a different kind of load the AI places on human workers. The first was what the researchers referred to as the learning burden. This encompasses the mental effort required to utilize the AI tool effectively.
Agents also had to learn how to navigate the interface, interpret the output, and compensate for errors. When the AI suggested incorrect phrasing or omitted required fields, agents had to revisit and correct them. Over time, that manual clean-up added to their workload. It also added stress, since errors could trigger quality checks or require callbacks to customers.
When the AI provided cluttered or incorrect information during a busy shift, it felt more like interference than help. Agents described frustration when trying to extract the relevant information from a wall of automated text.
What's being missed in AI rollouts?
Clearly, AI transcribes audio, captures useful snippets, and even steps in during hardware glitches. However, for call center agents, it never felt fully integrated into the way people worked.
Tools that function perfectly in test environments often fail to perform as expected in complex, real-world settings. People talk over one another, and systems freeze or lag. Sometimes, AI doesn't just need to keep up, it needs to read the room and stay out of the way.
Several agents said they would rather have fewer features that work consistently than a system packed with extras that underperform. The problem is that many leaders become distracted by what the technology can do rather than how it fits into existing workflows, what kind of training is required, or the mental effort it entails.
Despite early expectations that AI would significantly shrink support teams, many organizations are beginning to rethink that plan.
According to Gartner, by 2027, half of the companies that once aimed to reduce their customer service workforce through automation will abandon those goals. The findings also suggest that despite the shiny promises and hype surrounding agentic AI, 40% of projects will be canceled within the next two years.
40% of agentic AI projects will be canceled by the end of 2027 according to @Gartner_inc. 🤔
undefined Brian Solis (@briansolis) June 27, 2025
Why? Escalating costs, unclear business value, or inadequate risk controls. I'd add also because they're not rethinking the flow of work and reimagining outcomes. 💸
To date, 19% of… pic.twitter.com/vNJSWjcHUE
The future of AI will be measured by what it gets wrong first
When tech gets something wrong, it can quickly go viral for all the wrong reasons.
A great example is Air Canada's virtual assistant, which attempted to use empathy when it advised a flyer to book a full-fare ticket for their grandmother's funeral and apply for a bereavement discount after the trip.
When he later tried to claim the discount, the airline denied the request, stating that the chatbot had provided incorrect information and claiming its chatbot was "responsible for its actions."
The tribunal dismissed the argument and ordered the airline to pay over $800 in damages and fees. Consumer rights advocates say this ruling sent a clear message that if a business chooses to let AI speak on its behalf, it must also accept the consequences when that technology makes mistakes.
Releasing swarms of AI agents into customer-facing roles risks opening the door to reputational damage, regulatory scrutiny, and viral public backlash that even a hybrid communications team cannot contain.
Maybe it's time for organizations to consider whether their AI systems are equipped to handle the complex and unpredictable nature of human interaction, given that the cost of getting it wrong is not just operational, but also reputational.
Your email address will not be published. Required fields are markedmarked