New study confirms what we’ve long suspected about AI shortcuts in learning

Relying on artificial intelligence (AI) summaries to learn about new topics can result in a shallower understanding of the subject compared to a traditional Google search.

A new study published in the peer-reviewed journal PNAS Nexus adds to the evidence that while large language models (LLMs) like ChatGPT offer speed and convenience, they may diminish the ability to learn.

The study reviewing seven online and laboratory experiments, which included over 10,000 participants in total, found that those learning about a new topic from LLM summaries reported shallower knowledge compared to those using Google search.

Moreover, content created by LLM users was sparser and more generic, resulting in recipients finding it less informative and actionable.

In one experiment, participants were asked to give a friend advice on how to plant vegetables. They searched for information on the topic either on ChatGPT or Google before forming their advice.

Participants who used ChatGPT spent less time on the search task than those using Google search, suggesting that learning from LLM summaries requires less effort than learning from standard web search results.

ChatGPT users reported less deep knowledge and a lower sense of personal ownership over the knowledge they gained from their search. Additionally, they thought their search yielded less comprehensive information about the topic.

Researchers say learning through web search requires engaging in trial-and-error navigation among result links, as well as interpreting and synthesizing the different pieces of information.

This may result in better learning outcomes because encountering friction in learning leads to devoting more cognitive resources to overcoming it.

AI is slowly replacing Google search

The findings come as AI is already revolutionizing the way we look for things online.

Only 8% of Google searches involving an AI summary result in clicking on a traditional search result link, compared to 15% of those without an AI summary, according to a Pew Research Center survey.

Most strikingly, only 1% of AI summary encounters result in visiting pages cited by the summary.

A recent paper from the Massachusetts Institute of Technology asked participants to write essays using ChatGPT, Google’s search engine, or just their brains. Their brain activity was measured using an electroencephalogram.

ChatGPT users were found to have the lowest brain engagement and underperformed at neural, linguistic, and behavioral levels, further raising concerns about the AI’s impact on cognitive abilities.

While the new study suggests that Google is superior to LLMs when it comes to learning, it’s noteworthy that the arrival of the search engine in the late 1990s was met with similar concerns about the technology’s impact on our ability to focus and think.

Unlock exclusive Cybernews content on YouTube