The Pope’s AI encyclical is triggering AI detectors
After his rousing speech on how “AI needs to be disarmed,” there are claims that the text in Pope Leo’s encyclical, Magnifica Humanitas, could have been partially written by AI.
AI encyclical. Image by Cybernews.
After his rousing speech on how “AI needs to be disarmed,” there are claims that the text in Pope Leo’s encyclical, Magnifica Humanitas, could have been partially written by AI.
- Pope Leo’s Magnifica Humanitas is being questioned after claims that parts of the encyclical may have been AI-assisted.
- AI detectors flagged sections of the text (e.g. Pangram and GPTZero), suggesting some pages looked partly synthetic, but results vary widely.
- Key “evidence” like heavy em dash use is weak, since it’s a common stylistic feature in human religious and academic writing too.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
"The word is strong, I know, but deliberately chosen because this moment needs words capable of attracting attention,” said the Pope in his address on May 25th, in an effort to emphasize human thought and conscious choice.
But after the speech, researcher Linch Zhang said in a post on LessWrong that 62% of the first chapter was flagged as AI-written when Magnifica Humanitas was put through the detection tool Pangram.
Zhang went into detail, in particular regarding the use of the em dash. As the em dash is often cited as a telltale sign of machine composition, Zhang explained that it was used a lot more than in previous papal texts.
Other important findings from Zhang's research are that it was most likely Claude AI that was used, and that the majority of the text's pages returned 0% AI use – it was only fragments that used synthetic means.
When Cybernews put the entire text through GPTZero, an AI detection tool, the results differed slightly from Zhang’s, but did show a similarity: patches of the Pope's text could have been aided by AI. Here, we found that pages 6 and 7 of the text were three-quarters synthetic, and page 15 was completely AI-dependent.
This may come as a surprise, especially when the pope was speaking to a global audience about the perils of AI and compared it to the slave trade in how it is enslaving humans.
And while AI detection tools are known to have mixed results, the topic of AI-assisted writing has come into the spotlight a lot lately. For example, Nobel Prize-winning writer Olga Tokarczuk recently called it “beloved” and confirmed that she uses it in the creative process.
