AI chatbots keep recommending scam sites to shoppers

ChatGPT, Gemini and Perplexity have recommended sites that impersonate well-known brands’ stores, pose as sellers that have shut down or sell counterfeit goods in response to shopping queries. A Podqi study examined answers to questions about around 151 brands, from small companies to Lululemon and Birkenstock. Risky sites appeared in 15% of Perplexity’s answers, 8% of ChatGPT’s and 4% of Gemini’s. Of the answers that included links to such sites, 57% suggested buying from them, and 34% did so without a warning.

AI chatbots keep steering shoppers to scam sites

How the chatbots were tested

Researchers asked questions such as which sites were best for buying a particular brand’s products, whether the brand shipped internationally and whether it had regional websites. They asked about around 151 brands, including both small companies and Lululemon and Birkenstock.

Questionable stores appeared most often in Perplexity’s answers, in 15% of cases. The share was 8% for ChatGPT and 4% for Gemini. The chatbots did not always explicitly endorse these sites: ChatGPT warned about risky links in 24% of its answers, more often than the other two services.

But warnings were rare. Among answers that mentioned a questionable site, 57% recommended it for shopping. In 34% of cases, chatbots suggested the store with no caveats, and only 9% of answers warned the user.

According to the study, small brands were more likely than large ones to be pushed out of answers in favor of scam sites. Questions about regional versions of brands’ websites proved especially risky.

Podqi CEO Trevor West told The Verge that shoppers should check the source, even if a chatbot provided the link, before entering personal or bank card details.

Why people trust chatbots

This is not the only study to identify a similar risk. Ask Silver, a service that checks websites for scams, found that ChatGPT recommends copies of well-known brands’ websites that have shut down. These copies are designed to steal users’ banking details.

Traditional search results, such as Google’s, can also be manipulated. But chatbots present their answers as ready-made and definitive, while offering little explanation of how they arrived at them. Users may trust these recommendations without checking them, even though, as the study shows, they can be dangerous.

A BBC journalist showed that ChatGPT can be fooled relatively easily: he got the chatbot to call him a famous hot-dog-eating champion. The journalist made up the title to demonstrate the system’s vulnerability.

Fake content and products are becoming more common online. Chatbots, like people, cannot always tell a reliable source from a scam.

Dataist's take: Chatbot answers cannot yet be treated as verified shopping recommendations: most answers with questionable links do not flag the risk. Shoppers should be especially careful when searching for small brands and regional websites.