Apple Intelligence is “a boomer dad” that reportedly prioritizes phishing emails

Users on the social media platforms Reddit and Panic are complaining that Apple’s new artificial intelligence (AI) platform, Apple Intelligence, prioritizes phishing emails.

One user on Reddit posted to the IOSBeta community on Reddit that Apple Intelligence added a phishing email to the user’s priority list.

The post is captioned, “Apple marking a phishing email as ‘priority’ for me to open.” The email was supposedly sent from Xfinity.com, an American telecommunications company that provides internet, mobile plans, and cable TV services.

Some of the Reddit comments under the post demonstrate users’ feelings surrounding Apple Intelligence in beta.

One user said, “Apple Intelligence is a boomer dad,” while another user exclaimed, “A prince from Nigeria needs your help with a frozen bank account,” imitating the scammers and making light of the situation.

The email preview shows an urgent message saying that the user’s account will be suspended due to billing issues unless the user updates their payment information.

Similarly, on the decentralized social media network Panic.com, one user who is the co-founder of the social media site, Cabel Sasser, claimed to be experiencing the same problem.

“Apple Intelligence in 15.1 just flagged a phishing email as “Priority” and moved it to the top of my Inbox. This seems… bad.”

The address affiliated with the email sent to Sasser is [email protected]. According to Shopify, one of the largest e-commerce platforms on the internet, this .co.in domain is the second most popular domain in India.

When using Email Hippo, an online email verification tool, the website explained that there are at least three emails like this one. The rating for this email was marked “bad” as the address is undeliverable.

Although it’s hard to verify the legitimacy of these emails, in part due to advancements in AI, both users claim that they are phishing emails.

Edward Tian, CEO of GPTZero, a website that allows you to check for AI-written content in texts, gave credence to this point. “These days, AI technology has strengthened phishing attempts because it has made email body and subject texts less riddled with human errors, which makes looking at data like sender information more important than ever.”

These malicious messages are usually unsuspecting, urgent, and always problematic.

Why why Apple AI?

Many users of both Reddit and Panic speculate that the use of urgent terms and phrases, similar to those used in phishing emails, is the reason why Apple’s AI has flagged these messages as a priority.

Experts also agree that the content of the email is what’s pushing it into the priority list.

“The main reason why Apple Intelligence is having an issue with marking phishing emails as a priority, rather than marking them as spam, is because of the way their algorithm is designed,” Tian said.

Tian explained that the algorithm is designed to analyze data from the email’s subject line and body.

“Based on the data available, they appear to only be looking at the text and how it’s structured, what sense of urgency and tone it is demonstrating, and overall grammar,” said Joshua Bartolomie, vice president of Global Threat Services at Cofense.

“From what has been seen, they are not considering any of the common or known methods for phishing techniques, such as spoofed senders or domain impersonation, or related validations and checks,” Bartolomie concludes.

On the Apple Intelligence web page, the company even says that “priority messages in Mail elevate time-sensitive messages to the top of your inbox – like an invitation that has a deadline today or a check-in reminder for your flight this afternoon.”

Both emails display a level of urgency, and Sasser’s email was time-sensitive, considering that it urges the recipient to renew their subscription before August 4th, 2024.

Sasser confirmed that if you “write an appropriately urgent-sounding spam message, surely the AI will give it credence and credibility by putting it in its own special little important section.”

Furthermore, GPTZero’s CEO said that “AI technology isn’t analyzing data like the sender’s information, which is an essential aspect of identifying phishing attacks.”

Experts have also said that companies need to train the AI model (which is still in beta) to spot phishing attempts and respond appropriately to them.

“If email security is intended, Apple’s AI might not have enough examples of phishing emails in its training data,” said Eyal Benishti, CEO of IRONSCALES.

“By enhancing the company’s AI with more phishing examples, continuous learning, and user feedback, they can improve detection and prioritization.”

Although these may seem like isolated incidents, scammers aren’t biased, phishing emails will come to everyone eventually. Even some of the users on Reddit complained of similar issues.

Reddit reaction, Panic response

Other Reddit users have experienced the same thing, even saying that they have experienced the same phishing emails from the supposed “Xfinity” address.

Another user said, “I get those Xfinity bill suspension emails all the time now. The last couple of months, they started.”

But it’s important to note, like one user said on Panic, “Apple Intelligence is recognizing those words as an important account notification. Apple Wisdom is knowing not to click on it.”

Whatever you do, don’t click that link

When encountering phishing emails, users should “still always practice discernment and good cybersecurity measures whenever viewing any kind of messages sent to you,” Tian added.

If you receive a phishing email, make sure you “do not reply to the sender, and do not follow any instructions or requests contained in the email,” Benishti said.

When you come across these emails, make sure you report the issue to an official service provider. “Report it as phishing in the email client. If the message is trying to spoof a company, go to that company’s website to report the issue,” said Benishti.

Cabel Sasser, the one who started this saga, implores users who have experienced the same issues to report the matter. “(If you’re at Apple, by using the wild new 'Rate Your Experiences’ feedback system, I apparently filed a bug on this: FB14656882.)”

However, one Redditor made a very good point. They said that these bugs are essentially supposed to be there. The user elaborated by saying:

“You guys do know the point of a beta is to report things like this so it can be fixed before it goes full public release, right? It’s not just early access, so mistakes are gonna happen a lot, not just bugs, but as a beta user, you’re helping the AI learn what to prioritize and what not to and how to spot things such as spam just as a competent human would. This is just for those who take betas as early access releases rather than what it actually is: public/developer testing.”

Will Apple be able to fix this issue?

Sure, with a few tweaks, feeding the AI model more data on phishing attacks, and training it to mark these types of emails correctly, we shouldn’t see this issue surface again.