Google raised this question itself when we mapped how People Also Ask differs from AI Overviews: on 27 September 2026 it appeared as a child of “Is AI Mode more accurate than AI Overview?” The honest answer has two halves. In the largest public test, AI Overviews were right about nine times in ten. But on any single answer, you can’t tell which kind you’re looking at without checking the source.
What Google says
Google makes two kinds of statement. The first is about quality. After the U.S. launch, Liz Reid, Google’s head of Search, wrote on 30 May 2024 that AI Overviews “generally don’t ‘hallucinate’ or make things up in the ways that other LLM products might,” and that Google’s tests put their accuracy “on par with” featured snippets (Google). In January 2026 Google told the Guardian that AI Overviews only show up on queries where it has high confidence in the quality of the response (The Guardian).
The second is a warning, printed on its own help page: “AI responses may include mistakes.” Further down, the same page is blunter: “AI Overviews can and will make mistakes” (Google Search Help).
What independent testing found
The biggest public test so far came from Oumi, an AI company, working at the request of The New York Times. It ran 4,326 searches built from SimpleQA, an OpenAI benchmark of hard factual questions, and graded the AI Overviews that appeared (The Decoder).
- Correct answers: 85% with Gemini 2 (tested in October 2025) and about 91% with Gemini 3 (February 2026).
- Supported by sources: with Gemini 3, only 67% of individual claims were backed by the pages the Overview cited. Only 39% of Overviews were both correct and fully supported (Oumi).
- Direction of travel: the newer model was more accurate, but Oumi found its rate of unsupported claims went up too.
Google pushed back. Spokesperson Ned Adriance said the study had “serious holes,” arguing that SimpleQA contains errors and doesn’t reflect what people really search. Oumi replied that Gemini models score about the same, within about 3%, on SimpleQA and on Google’s own fact-checked version, Verified-SimpleQA. Both points deserve weight. A benchmark of hard questions says little about your everyday hit rate, in either direction. But “right answer, unsupported claims” is a real pattern, and it’s the one you meet when you click through to check.
Is AI Overview ever wrong?
Yes, and the cases are well documented. Google reached this exact question in three hops from “people also ask vs ai overview” in our depth pass, by way of “Can AI Overview talk to me?” and “Does AI Overview ever lie?” Here is the public record, with dates.
- May 2024: rocks and glue. In the launch weeks, an AI Overview answered “How many rocks should I eat?” from satire republished on a geological software provider’s site, and others repeated forum advice “like using glue to get cheese to stick to pizza.” Google’s 30 May 2024 post addressed both and described “more than a dozen technical improvements” (Google).
- April 2025: invented idioms. Type a made-up saying plus “meaning” and AI Overviews would often explain it with confidence. Mashable collected examples such as “You can’t lick a badger twice” on 23 April 2025 (Mashable).
- May 2025: the wrong year. On 29 May 2025, Wired asked “is it 2025” three times and got three different AI Overviews, all starting “No, it is not 2025.” Google told Wired it was “actively working on an update to address this type of issue” (Wired).
- January 2026: health. A Guardian investigation found AI Overviews giving normal ranges for liver blood tests with little context, which experts called “dangerous” and “alarming.” Google removed AI Overviews for two of those searches, but the Guardian found slight rewordings still triggered them. For other flagged examples it left up, Google said its clinicians found that “in many instances, the information was not inaccurate” (The Guardian).
Why the mistakes happen
Google’s 2024 explanation still covers most of the list. When AI Overviews get it wrong, Google wrote, “it’s usually for other reasons: misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.” The rocks question was what Google called a “data void,” a topic with little serious content, so satire filled the gap. Made-up idioms are the same problem on purpose: there is nothing real to summarize.
Two more patterns show up in the record:
- Missing context. The liver test answers weren’t invented numbers. The Guardian’s complaint was numbers with no accounting for nationality, sex, ethnicity or age.
- Answers vary. The same prompt can produce different Overviews, as Wired found. Ahrefs has reported that 45% of AI Overview citations change between generations, so even the sources shift (Ahrefs).
How to check an AI Overview
- Click the cited link and find the sentence. If the page doesn’t say it, treat the claim as unverified. Oumi’s numbers suggest this will happen often.
- Ask again in other words. Google’s own advice is to “ask multiple versions of your question.” If the answer changes, that tells you something.
- Check anything important “in more than one place,” as Google’s help page puts it. For health, money or legal questions, go to the primary source.
- Use the Web filter when you want links without AI. Google says it “displays only text-based links without features like AI Overviews.”
- Send feedback when an answer is wrong. Google asks for it on the same help page.
The same habits apply inside People Also Ask, where most answers now carry the AI Overview label too. And if you’re wondering whether Google’s other AI search does better, see is AI Mode more accurate than AI Overview?