Stop outsourcing your mind to AI, expert says
Still asking AI for health advice, relationship struggles, or investment opinions? You shouldn’t.
-
Lacking real-world senses, AI confidently generates plausible but fake information because it only matches text patterns.
-
Humans easily stop critical thinking and blindly trust AI simply because it always answers with authority.
-
Human over-reliance combined with models training on their own contaminated data threatens to degrade AI quality over time
Upon hearing that researchers created a fake medical condition, spoon-fed the info to AI, and had it spout out hogwash to the end user, I felt taken aback and on edge.
The disease in question was called Bixonimania, which supposedly is contracted from excessive exposure to blue light, resulting in sore eyes and dark circles.
And the researchers' intention was to demonstrate how prone AI is to insist that something is real, regardless of the training materials it is fed. Large language models, such as ChatGPT, Gemini, and others, always have an answer and don't always care whether it's true.
Scientists invented a fake disease. AI told people it was real
by u/StephieWatts in ChatGPT
Dr. Marko Sarstedt, LMU Munich professor and marketing researcher, quotes philosopher Harry Frankfurt to analogize this kind of AI duplicity: "The bullshitter is somebody who always has an opinion and doesn't care about the truth."
He explains that these AI bullshitters more often than not sound plausible but are definitely not accurate, and that the further you drill down from general questions to individual psychology, the more wonky they become.
Curious what others think about this story? Contribute your thoughts to the debate below.
In his talk on stage at Login tech forum in Vilnius, Lithuania, , Sarstedt poured out about how AI completely lacks sensory coherence – for example, in describing a potato chip. AI bullshits because it has no senses – no smell, no touch, no gut. It's just pattern-matching text.
We are complex creatures that use all five senses to navigate the world. But because of the physical limitations of the digital world, we're often left guessing. So when facing something subjective where you should trust your gut, you shouldn't let the AI system decide for you.
Cognitive surrender
Many of us go through phases where we rely on every little scrap of advice that a chatbot offers, filtering out what other humans are telling us. Sarstedt introduced the concept of "cognitive surrender" – the moment we stop reasoning and blindly accept what AI gives us.
Firstly, you formulate a prompt, thinking about it. Then system two kicks in. You just blindly accept what's there.
For comparison's sake, there might be a human you idolize, be it a friend, colleague, or character, like in Fight Club or The Wizard of Oz, who gives a promising answer, only to be led down the proverbial path where something darker is lurking.
Sarstedt notes, "We know from theology that if you meet somebody who always has an answer, you perceive this person as intelligent."
So the combination of human laziness, plus model performance, is "somewhat dreadful," opines Sarstedt, describing it as “AI autophagy”, when models eat their own contaminated output, which can degrade performance over time.
We don't need a chatbot to describe a smell for us. We can do it ourselves. Just like we don't need it for advice on a fake disease.
By doing a deep-dive into actual reality – it might be time to peel back the AI a bit.
