AI Achilles heel: why we shouldn’t bow to our computer “overlords” just yet

We’re living in the Stone Age when it comes to the Artificial Intelligence (AI) systems, and the book “Not with a Bug, But with a Sticker” provides plenty of examples to support that.

This is a review of a book that raises crucial questions surrounding AI and is seasoned with unbelievable yet real-life examples of attacks on AI to keep you engaged and entertained.

“Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them”, written by Ram Shankar Siva Kumar and Hyrum Anderson is a comprehensive documentation on the discourse surrounding AI. It discusses the following:

Who’s who in AI research

Why can’t AI be fair and secure at the same time

What is AI’s Achilles heel

The most curious and creative attacks that fooled AI systems

Why AI systems insists on pandas being gibbons, and vultures – orangutans

Why machine learning systems are called a Potemkin village

The overtrust in AI

I already introduced one of the authors, Kumar, in my previous article. During the flagship hacker conference in Las Vegas, Black Hat 2023, he took to the stage to discuss how we should test and regulate AI systems for them to be fairer and/or more secure.

As it turns out, the AI system, at least at the moment, can’t be secure and fair and the same, presenting us with an immense dilemma of how to embrace the technology without doing more harm than good.

Overtrust in AI

If you want a job done well, do it yourself. That’s somewhat true when it comes to AI. While different systems can offer an immense amount of help and increase productivity, they can’t be left to deal with the task unsupervised.

Oh, but what a temptation we have to just get the load off our shoulders. Alarmingly, we tend to put more trust into a machine just because, well, it’s a machine, no matter how faulty.

The book recalls in detail an experiment by University of Hertfordshire researchers. They invited some participants to a home, and used a white plastic humanoid robot on wheels to give them directions. The robot was intentionally incompetent. However, people blindly followed its instructions, like pouring orange juice into a plant, unlocking a computer with a given password and disclosing the host’s private information, and so on.

Another experiment by the researchers from Georgia Tech is even more shocking. A robot led participants into a conference room to fill out a survey. In a staged scenario, suddenly, smoke filled the hallway and emergency sirens started blaring, urging particpants to evacuate.

Just around the corner, an “emergency guide robot” was placed by researchers to see whether they would trust a machine in an emergency. Not only did people wait for the robot to guide them to safety, they followed it after it made one navigation mistake after another, and even those people who were told the robot was broken continued following the machine instead of trying to evacuate themselves.

These two experiments alone can serve as a wake-up call and motivate people to better understand AI systems. After all, they are built by people, meaning they can, in fact, possess the same characteristics as we do, including strong biases.

The machine is not racist, people are

Maybe overtrust in machines derives from a not very well-grounded belief that numbers don’t lie, and math is queen. However, all complex hardware and software applications are built by people, therefore, bias is unavoidable.

As our contributor Susan Morrow once quite aptly asked, “Is AI biased or just holding a mirror up to ourselves?” You have to factor in many circumstances, including what data a certain AI model has been trained on to the way that algorithms are constructed.

And let’s not forget that people are heavily involved. There’s a profession out there in the AI world called a data labeler. In essence, for a machine learning system to identify a picture of a dog, well, as a picture of a dog, it needs to know how a dog looks. It needs to know how many different species of dogs look so it could later say with confidence that a picture features a dog. So someone needs to label those pictures.

The authors of the book argue that these labelers are paid by the task, and the salary is “embarrassingly low.” So, to maximize the rewards, they don’t exactly do their best job. MIT researchers discovered the following incorrectly assigned labels on ImageNet, a large visual database designed for use in visual object recognition software research:

A person wearing jeans labeled as “bathtub”

Actress Sigourney Weaver labeled as hermaphrodite

An overweight person tagged as a loser

You see, there’s no reason to trust AI systems since they’re flawed. If you teach them to label overweight people as losers, what good can come out of this? Raising the discussion now is extremely important since we don’t really want a faulty system to be responsible for, let’s say, diagnostics.

Adversarial machine learning

I’ve already touched upon adversarial machine learning just a notch in my short explainer article on why a machine learning system misrecognized a panda and thought it was a gibbon.

It’s important because companies are rushing to implement AI to optimize their work processes, and, well, earn a bigger buck as a result. They’re neither incentivized nor motivated enough to make those systems secure and fair. What could be the repercussions?