Who writes better – students or ChatGPT?

It’s crunch time – millions of American students are enduring their spring examinations. Teachers are vigilant – ChatGPT is surely going to be used. But a new study says humans are still much better at writing essays than generative artificial intelligence (AI).

ChatGPT has indeed created considerable anxiety among teachers concerned that students might turn to large language models (LLMs) to write their assignments.

Many of these models are able to create grammatically accurate and coherent texts, thus potentially enabling cheating and undermining literacy and critical thinking skills. Even academics were caught using ChatGPT in their research papers in late 2023.

Now, a new study by the University of East Anglia (UK) researchers has found that essays – of the argumentative kind – are still far from matching the quality of work produced by real students.

The study titled “Does ChatGPT Write Like a Student? Markers of Engagement in Argumentative Essays,” and published in the journal Written Communication, compared the essays of 145 real students with those generated by ChatGPT.

“The findings reveal that student essays are significantly richer in the quantity and variety of engagement features, producing a more interactive and persuasive discourse,” the study concludes.

According to the researchers, the ChatGPT-generated essays exhibited fewer engagement markers, particularly questions and personal asides, indicating its limitations in building interactional arguments. In short, these works lacked one critical aspect: personality.

"We found that the essays written by real students featured a rich array of engagement strategies, making them more interactive and compelling. These included rhetorical questions, personal digressions, and direct appeals to the reader—techniques that enhance clarity, communication, and argument strength. On the other hand, while ChatGPT’s essays demonstrated linguistic fluency, they were more impersonal.

"We found that the essays written by real students featured a rich array of engagement strategies, making them more interactive and compelling,” said Professor Ken Hyland, one of the authors of the study.

“These included rhetorical questions, personal digressions, and direct appeals to the reader – techniques that enhance clarity, communication, and argument strength. On the other hand, while ChatGPT’s essays demonstrated linguistic fluency, they were more impersonal.”

The hope is that these findings will assist educators in identifying instances of cheating in schools, colleges, and universities worldwide by distinguishing AI-generated essays from those written by students.

The caveat, though, is that we don’t yet have reliable tools to detect AI-generated texts. Numerous websites, such as Grammarly or GPTZero, now offer checks for AI plagiarism, but not all of them correctly identify AI-generated text every time.