Trust generative AI too much, and your critical thinking will suffer

Artificial intelligence (AI), like almost any other new technology, is bound to change the human brain. Researchers say that overreliance on AI at work can cause us to lose critical thinking skills.

With generative AI growing in capabilities and popularity, humans are outsourcing more of their brainpower to computers. What do we actually know without Google Search these days?

Think about it – if you can still remember the question, that is, because the modern internet has also massively hurt our attention spans.

This time, though, researchers from Carnegie Mellon University and Microsoft write in a new study that our critical thinking skills are suffering due to overreliance on AI.

The researchers asked 319 so-called knowledge workers – the ones who handle data – to self-report details of how they use generative AI tools at work.

People were then asked to report what kind of tasks they were asked to do, how they used AI to complete them, how confident they were in the AI’s ability to do the task, and how confident they were in their own ability to complete the task without any machine assistance.

The researchers found the pattern quickly. Apparently, people are beginning to rely on AI instead of thinking critically about what’s being produced by such tools.

Workers who participated in the study (PDF), even said they were missing errors made by AI, “raising concerns about long-term reliance and diminished independent problem-solving.”

Trust in AI is the give-away. Users who had less confidence in the AI’s output used more critical thinking and had more confidence in their own ability to both solve the problem and fix the insufficient quality of the AI’s output.

In other words, we’re now outsourcing not just the work itself but our critical engagement with it. We assume the machine will handle this or that problem, and we’re more worried about better AI response integration rather than actual problem-solving.

“Our work suggests that GenAI tools need to be designed to support knowledge workers’ critical thinking by addressing their awareness, motivation, and ability barriers,” said the authors of the study.

To be fair, the history of technology allegedly hurting the quality of human thought is long. Socrates objected to writing, teachers of arithmetic despise calculators, and we’re still grumbling about the internet scattering our minds.

“A key irony of automation is that by mechanizing routine tasks and leaving exception-handling to the human user, you deprive the user of the routine opportunities to practice their judgment and strengthen their cognitive musculature, leaving them atrophied and unprepared when the exceptions do arise,” Lisanne Bainbridge argued in his famed 1983 research paper Ironies of Automation.