Let it slop! The AI Con book claims the revolution will devour its children

The AI Con book, written by Emily Bender and Alex Hanna, was published a while ago, in May. We thought it was worth reviewing still, if only because their insights now sound even sharper. The AI revolution isn’t funny anymore.

To AI’s most enthusiastic proponents, the technology is like ultra-premium fish oil that will make our lives healthier, better, and smarter. To its critics, it’s more like fentanyl, a digital opioid bound to kill off our remaining brain cells slowly but surely,

As ever, between the fanatics promising to blow our minds, and the skeptics saying machines will soon rule and doom us, lies a fat body of inbetweeners.

To them, the chatbots are just a bit of fun. They play around, they sext, they cheat in school. It’s not serious, nothing’s going to happen, isn’t it?

As Emily Bender, linguist and professor at the University of Washington, where she directs its Computational Linguistics Laboratory, and Alex Hanna, sociologist and director of research at the Distributed AI Research Institute, explain, though, the AI house of cards is actually dangerous.

Fancy autocomplete and media synthesis

What if you trust the chatbot (and not a human doctor) to provide you with the best health advice and then suffer the terrible consequences? What happens to actual human art when the models can and will just steal the artists’ styles and ideas?

“A few major well-placed players are poised to accumulate significant wealth by extracting value from other people’s creative work, personal data, or labor, and replacing quality services with artificial facsimiles,” the authors argue.

It’s hard to disagree. As a sincere believer that AI (it’s a marketing term, I know, but anyway) is already turning us as thick as the heroes of the Idiocracy kind, I could feel my heart sing whilst reading this belter of a book.

Bender and Hanna are really punchy with words and scornful descriptions of a phenomenon, the symbol of which is OpenAI’s CEO, Sam Altman, and his erratically articulated hogwash about how he needs ever more money from me, you, and the government.

To them, ChatGPT, Gemini, and other AI chatbots are essentially “fancy autocomplete,” trained on – mostly illegal – scrapings from websites. Meta trained its models on LibGen and called it fair use. The cheek!

The large language models (LLMs) should actually be called “synthetic text-extruding machines” or “synthetic media machines.”

Better yet, “giant plagiarism machines.”

Those research papers claiming spectacular advances in AI development? Alas, much of the “science” around these tools has been conducted by the companies themselves.

“Like an industrial plastic process,” the authors explain, text databases are “forced through complicated machinery to produce a product that looks like communicative language, but without any intent or thinking mind behind it.”

No, these machines are not in fact humanlike. They and their drivers – mostly ultra-greedy con men – actually devalue what it means to be human and reduce our condition to one of computability, quantification, and rationality.

They behave and talk as if we’re data points rather than “relational and fully dimensional beings,” as Bender and Hanna put it.

And, for goodness’ sake, no, the LLMs don’t “hallucinate,” they make errors. First, using the term “hallucination” is making light of what can be symptoms of serious mental illness.

Second, “hallucination” refers to the experience of perceiving things that aren’t there. But the LLMs don’t have and can’t have these perceptions because, again, they’re not human. This is unhelpful anthropomorphization: we really shouldn’t assign thought processes to these systems.

And those research papers claiming spectacular advances in AI development? Alas, much of the “science” around these tools has been conducted by the companies themselves. Conflicts of interest, anyone?

The truth is ugly and terrible

The AI doomers – you know, those behind all those public letters warning humanity of a possibility of its extinction – are no better than the Oppenheimers like Altman and the most fervent apostles of artificial general intelligence (no one has been able to define what exactly this means, true story).

It’s still fashionable in the industry to constantly mention p(doom). It’s a term in the AI safety community that represents the estimated chance of catastrophic outcomes, such as human extinction or subjugation. But it’s all a sham, the authors of the book say.

The tactic seems to be to take a series of tropes from science fiction (they’re all nerds) about how some Terminators would wage wars for their right to exist, and then actually make everyone talk about it.

“The language of p(doom) is a ruse to keep us focused on imaginary scenarios, filled with awe at modern robber barons’ allegedly potentially world-ending technology, and too distracted to see the daily harms being done in its name,” they write.

The tactic seems to be to take a series of tropes from science fiction (they’re all nerds) about how some Terminators would wage wars for their right to exist, and then actually make everyone talk about it.

“But this belies what these technologies are doing to the rest of us: threatening stable careers and replacing them with gig work, slashing personnel in government, cheapening our social services, and degrading creativity,” write Bender and Hanna.

There’s a reason corporate executives, especially in the US, keep talking about how AI will “increase productivity.” This is consultant-speak for replacing labor with technology, and the peddlers of Silicon Valley have convinced them it’s doable.

“AI isn’t sentient, it’s not going to make your job easier, and AI doctors aren’t going to cure what ails you. But these claims can make your work worse and reduce your quality of life, unless we fight back against the increasing encroachment of these products into every area of public and private life,” say Bender and Hanna.

“Hype doesn’t occur by accident, but rather because it fulfills a function: scaring workers and promising to save decision-makers and business leaders lots of money.”

In the vast majority of cases, AI is going to replace you. But it’s going to make your job a lot shittier.

You won’t be freed up “to do interesting work” (this line, often used by AI peddlers, is especially infuriating). You will be cleaning after the machines that supposedly save you time. These tasks could even be traumatic to carry out.

Beneath the bubbly surface, terrible factual stories are coming through. In early 2023, Time reported that OpenAI had subcontracted Kenyan workers making less than two dollars a day to filter out gore, hate speech, child sexual abuse material, and pornographic images from ChatGPT and OpenAI’s image generation tool, DALL-E.

Those workers were lured in by the prospect of breaking into the lucrative field of computing, but ended up with PTSD and ravaged personal relationships due to mental health issues.

Generally speaking, what we’re dealing with here is complete hypocrisy. We probably wouldn’t even have the current wave of “AI” if thousands of on-demand cheap laborers weren’t available to be called upon anytime to perform a set of tasks whenever some AI researchers or corporate engineers demanded it.

Profit without blame

Now then, let’s calm down a little. Bender and Hanna themselves write that AI can be useful – even as fancy autocomplete or basic, if unnecessary, shiny automation.

“Automation can be useful, but we should be choosy about what kinds of automation we accept, and which will make us work for machines rather than with them,” they say.

It’s true. There are sensible use cases for AI, such as image processing that helps radiologists.

Additionally, machine learning systems can manage load on the electricity grid more effectively, cut the time required to inspect nuclear facilities, and help reduce emissions in shipping, steelmaking, mining, and trucking industries.

Last year, scientists at Google DeepMind, an AI research laboratory, and an American biochemist were awarded the 2024 Nobel prize in chemistry for “enormous” breakthroughs in predicting and designing the structure of proteins. Changes in drugmaking could be profound.

However, all of this needs to be welcomed rationally, even coolly, after evaluating how each system helps us, both individually and collectively. And all that nonsense about AGI should just stop, please.

A computer can never be held accountable, therefore a computer must never make a management decision.

A famous old rule from IBM.

As even venture capitalists are beginning to see the limits of AI hype, Bender and Hanna do well to insist (with a helpful questionnaire) that each larger case of AI use should be properly scrutinized for its utility, biases, and propensity to displace human jobs.

A famous old rule from IBM: “A computer can never be held accountable, therefore a computer must never make a management decision.”

That’s great, but that’s also why AI and the LLMs are perhaps so tempting to business and political elites. The utopia is handing decision-making capacity to computers and enjoying power and profit without responsibility or blame. We shouldn’t allow this to happen.

So let’s resist, the authors invite us. Yes, pushing back against the hype is a lonely experience, plus, all the FOMO makes it even harder to resist.

“But resisting hype can also be empowering, grounding, and even joyful. It is empowering to reaffirm the value of our skills and expertise. It is grounding to lean into the value of human-to-human connection, of being human together. And it can be flat-out fun to find the silliest excesses of the hype machine and deflate it, with ridicule as praxis,” the book concludes.

Unlock more exclusive Cybernews content on YouTube.