When Amazon's new AI tool answers shoppers' queries, who benefits?

Amazon.com has begun rolling out a new artificial intelligence assistant that is meant to address shoppers’ product questions, but the feature raises as many questions as it answers.

Rufus, as the software is known, will help users, according to Amazon, by guiding them to the toaster ovens or dinosaur toys that best fit their needs.

Yet Amazon has a history of steering customers towards products that most benefit Amazon, either because they are more profitable or are backed by advertising dollars, according to the Federal Trade Commission's pending 2023 antitrust lawsuit against Amazon.

The FTC accused Amazon of “biasing Amazon’s search results to preference Amazon’s own products over ones that Amazon knows are of better quality.” Further, the FTC alleges the Seattle firm operates a “pay-to-play” system, giving top billing for the products on which marketers were willing to spend the most. The company has denied the claims and said it will contest the suit in court.

Amazon's general counsel said in 2023 that the FTC “is wrong on the facts and the law,” and that its practices “have helped to spur competition and innovation across the retail industry.”

Like for other generative AI services, the algorithm behind Rufus is a closely held secret and Amazon declined to discuss how it operates. It offered some clues, including a press release, that said it is trained using Amazon’s product catalog, reviews, information from the web and question and answer sections.

But Michael Pachter, a Wedbush Securities analyst, said Amazon would be turning money away if it didn’t ostensibly open Rufus’ results up to advertisers.

“You’ll most likely get sponsored results,” he said. “Advertising drives retail and Amazon is no different – why do you think they are generating tens of billions of dollars in advertising a year?”

The company on Thursday reported a 27% jump in ad sales in last year’s fourth quarter to $14.7 billion.

At least initially, Rufus doesn’t appear to be influenced by advertising. Gil Luria, a D.A. Davidson senior software analyst, said that was likely necessary to build trust with consumers. “If they lose your faith in how they answer, you're not going to ask them anymore," Luria said.

Advertising and consumer choices

Yet the company has time and again allowed advertising to creep into its recommendations.

Consumers may have once believed that an “Amazon’s Choice” designation indicated a review or testing process, but Amazon once granted that badge to marketers who were willing to pay for it, according to a 2019 Digiday story citing an Amazon internal deck from 2017.

Amazon has denied that the company has ever allowed sponsored results in “Amazon's Choice” and said the label relies primarily on criteria such as reviews, shipping speed and price, according to a company spokesperson.

Europe’s top antitrust enforcer Margrethe Vestager said Amazon’s ability to limit access to the “Amazon’s Choice” badge was one reason for her objection to the company’s planned takeover of Roomba-maker iRobot. Amazon walked away from the deal last month.

And Amazon previously allowed marketers to place products in customers’ online baby registries without their knowledge, a Wall Street Journal investigation found. It has since ended the practice.

Amazon.com search results included nine sponsored listings on average, a study by Profitero from 2020 to 2021 found, twice the amount as Walmart and four times that of Target.

Jim Tierney, Chief Investment Officer of Concentrated US Growth Equities at AllianceBernstein, said he saw real promise in Rufus “if you actually have some kind of an AI conversational shopper.”

“That just means more engagement and more conversion, which means more revenues and more units (sold),” Tierney said.

But Amazon offered few clues in rolling out Rufus to select customers. In a call with investors, CEO Andy Jassy said Rufus “lets customers discover items in a very different way than they have been able to on e-commerce websites.”

Amazon in a blog post showed how the software might respond to a query about a particular pickleball paddle’s worthiness for beginners.

“This paddle is a great option for beginners,” the software replied, according to a screenshot. “The large sweet spot helps beginner players get the ball over the net consistently.”

In another example it displayed Tyrannosaurus Rex toys for a query about best toys for a “dinosaur obsessed 5-year old.”

The company said Rufus is trained using Amazon’s product catalog, reviews, information from the web, and question and answer sections.

However Rufus evolves, Luria said Amazon has an edge over Google and Microsoft's AI applications because it has a rich trove of purchasing and personalized data that’s quite an advantage at the checkout counter.