A Stanford University study has found that AI chatbots are significantly less reliable when answering questions in Hindi than in English — a timely warning as Indian shoppers lean on artificial intelligence this festive season. Google Trends data shows 51 per cent of Diwali shoppers credited AI with helping them decide what to buy, consulting chatbots 7.5 times on average per purchase journey, and an InMobi report found 80 per cent of Indian shoppers ready to use AI shopping assistants.
Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) tested six commercial chatbots on current news questions across six language regions. Hindi ranked last, averaging 79.3 per cent accuracy — nearly 10 percentage points below the next-weakest region. Every chatbot tested performed worst in Hindi, with error volume roughly double any other region’s.
Crucially, the problem is retrieval, not comprehension. The chatbots understood Hindi but struggled to find the right evidence. Two failures explained over 70 per cent of wrong answers: retrieval failure (38.8 per cent), where sufficiently relevant information was missing, and source divergence (32.7 per cent), where material on a similar topic surfaced with conflicting facts. Once the correct source was found, chatbots almost always extracted the right answer.
The citations reveal where the gap begins. English Wikipedia was the most-cited domain for Hindi queries, ahead of every Hindi news outlet, and nine of the ten most-cited domains were primarily English.
Kartik Sharma, CEO and founder of RankinLLM.ai, told NDTV the finding points to a structural flaw. “The models read Hindi perfectly well. They then went looking for the answer in English,” he said. Since the pattern held across all six chatbots, he added, “this is not one company’s blind spot.”
The study tested news questions, not purchases, but Sharma said the same behaviour could shape commercial answers: a Hindi-speaking user asking which insurance plan suits their parents could get an answer drawn from different sources than an English-speaking user would.
For brands, the risk is visibility: a company may appear prominently in an English AI answer yet be nearly absent from the Hindi one, because the two draw on different evidence. “A brand does not have one AI reputation,” Sharma said. “It does not even have one per language.” His advice: stop treating this as a translation task. “Publishing Hindi pages on your own website will not fix it if no credible Hindi source anywhere else corroborates what you claim,” he said. Brands should build credible Hindi coverage beyond their own sites and check whether AI tools actually retrieve it.
The findings carry caveats: accuracy fell sharply when questions contained subtle errors or misleading assumptions, and a free-response test showed a 16-17 per cent fall in absolute accuracy. All queries also ran on US-based servers, which may have steered chatbots toward English sources. Not every Hindi answer is wrong, but strong overall scores can hide large gaps between languages.
With festive spending at its peak, shoppers should verify important claims — prices, product specifications, financial products and admissions details — against credible sources before acting, especially when asking in Hindi.
Original Source: https://www.ndtv.com/artificial-intelligence/festive-shopping-diwali-ai-hindi-queries-inaccurate-responses-stanford-study-12204267#publisher=newsstand
Category:
Tags:
Publish Date: 2026-10-19 07:57:00
