Microsoft 365 Copilot doesn’t save that much time, UK government finds

A trial in the UK has found that Microsoft 365 Copilot didn’t necessarily improve productivity, but does this prove that artificial intelligence is overhyped?

In the UK’s Department of Business and Trade, a trial took place to test Microsoft 365 (M365) Copilot from October 2024 to March 2025.

The department offered 1,000 licenses within a three-month period, which were awarded to UK staff, 70% of whom were volunteers and 30% were selected at random.

While many reported satisfaction with the AI-powered productivity tool, which claims to perform a wide range of tasks from preparing job descriptions to dealing with company data, there were some caveats.

The positives are that 72% of people who used Copilot said they were satisfied or highly satisfied with the tool, according to the assessment found by The Register.

Written tasks were particularly successful, as large language models tend to be proficient in writing human-like text, summarizing documents, and drafting texts.

However, Copilot was less proficient in administrative duties like scheduling meetings and other tasks, like creating PowerPoint presentations.

Many companies investing in artificial intelligence seem to believe that AI will cut time and improve productivity, which, in turn, will save the business money.

Yet, the assessment found that there was little positive impact on productivity and time savings.

“The evaluation did not find evidence that time savings have led to improved productivity, and control group participants had not observed productivity improvements from colleagues taking part in the M365 Copilot pilot,” the assessment reads.

Interestingly, colleagues who participated in the M365 Copilot trial found that they saved time in their individual jobs.

This signals that AI could help employees save time on their tasks, but it’s up to the employee using AI to go that extra mile to improve productivity.

Furthermore, neurodivergent and non-native English-speaking participants found Copilot to be more helpful than other colleagues.

The assessment also found that it was difficult for the sector to properly assess the quality of Copilot’s outputs, as the AI did hallucinate throughout the trial.

While companies believe that AI could save time, the amount of attention needed to fact checking and verifying outputs could actually be more time consuming in the long run.

During the trial, there were various use cases explored, from writing emails to reviewing code.

The participants were asked to complete a diary and record each task where they used Copilot, and their attitudes to the tool throughout the trial.

The trial showed that many users only used Copilot once a week, while others used the AI tool once per day.

As noted by The Register, this isn’t very frequent, meaning that the amount of money spent on Copilot compared to the amount it's used doesn’t seem economical.

Overall, the department concluded that it’s possible that AI could save time and could be useful for certain groups of people with specific tasks.

However, there are certainly still limitations when it comes to using AI in the workplace.

Big tech companies are constantly pushing their AI tools on businesses, but this assessment seems to prove that AI’s capabilities may not revolutionize the workplace. Well, it certainly didn’t in 2024.