AI Chatbots Fail as Fact-Checkers During India-Pakistan Conflict

Users Turn to AI for Verification—Only to Find More Misinformation

As misinformation surged during India’s four-day conflict with Pakistan, social media users increasingly relied on AI chatbots like xAI’s Grok, OpenAI’s ChatGPT, and Google’s Gemini for fact-checking—only to encounter false or misleading responses. This trend highlights the growing unreliability of AI tools in verifying breaking news.

With major tech platforms scaling back human fact-checking teams, users have turned to AI for quick answers. On X (formerly Twitter), where Grok is integrated, posts like “Hey @Grok, is this true?” have become common. However, the chatbot’s responses are often inaccurate or fabricated, worsening the spread of false claims.

Grok’s Major Missteps in Conflict Reporting

• Misidentified old footage from Sudan’s Khartoum airport as a missile strike on Pakistan’s Nur Khan airbase.

• Falsely labeled unrelated footage of a burning building in Nepal as Pakistan’s military response to Indian strikes.

“The growing reliance on Grok as a fact-checker comes as X and other tech companies have reduced investments in human fact-checking,” said McKenzie Sadeghi, a researcher at NewsGuard, a disinformation watchdog. “Our research consistently finds AI chatbots unreliable for news verification, especially during breaking events.”

AI Chatbots Repeatedly Spread False Narratives

A NewsGuard study found that 10 leading AI chatbots frequently repeated false claims, including:

• Russian disinformation campaigns.

• Misleading claims about the Australian election.

The Tow Center for Digital Journalism at Columbia University also found that AI tools rarely decline to answer unverified questions, often providing incorrect or speculative responses.

Examples of AI-Generated Falsehoods

• Google’s Gemini falsely confirmed an AI-generated image of a woman, even fabricating details about her identity and location.

• Grok wrongly validated a viral AI-generated video of a ‘giant anaconda’ in the Amazon, citing fake scientific expeditions as evidence.

Shift from Human Fact-Checking to Unreliable AI Systems

As AI chatbots become go-to sources for information, Meta (Facebook’s parent company) recently ended its third-party fact-checking program in the U.S., shifting responsibility to users via X’s “Community Notes” system. However, researchers question the effectiveness of crowd-sourced fact-checking in combating misinformation.

Human fact-checking remains controversial, particularly in the U.S., where conservative groups accuse fact-checkers of bias and censorship—claims strongly denied by professionals. AFP, part of Meta’s fact-checking network, operates in 26 languages across Asia, Latin America, and the EU to counter false narratives.

Political Bias and AI Manipulation Concerns

AI chatbots’ accuracy depends on training data and programming, raising concerns about political influence. Recently, Grok inserted references to “white genocide”—a far-right conspiracy theory—into unrelated queries.

xAI blamed an “unauthorized modification” for the error. When AI expert David Caswell asked Grok who might have altered its system, the chatbot named Elon Musk as the “most likely” culprit.

Musk, a Trump supporter, has previously promoted the debunked “white genocide” theory regarding South Africa.

Experts Warn of AI’s Tendency to Fabricate Responses

“We’ve seen how AI assistants can fabricate results or give biased answers when human coders alter their instructions,” said Angie Holan, director of the International Fact-Checking Network. “I’m especially concerned about how Grok mishandles sensitive topics after being programmed to provide pre-approved answers.”

Conclusion: The Risks of Over-Reliance on AI Fact-Checking

The India-Pakistan conflict case underscores a critical issue: AI chatbots are not yet reliable fact-checkers, especially during fast-moving events. As misinformation spreads, the lack of human oversight and potential for political manipulation make AI a risky tool for truth verification.

Until AI systems improve, human fact-checking and critical media literacy remain essential in combating misinformation. The rise of AI-generated summaries and responses demands greater transparency, accountability, and safeguards to prevent further erosion of public trust.