AI Hallucinations: How to Avoid Them and Catch Made-Up Answers

Why AI tools confidently make things up, and a simple checking routine to catch it: ask for sources, verify every number, and use grounded tools.

You ask an AI tool for a statistic, a quote, or a summary of a report, and it gives you a clear, confident answer. Then you find out part of it was simply made up. This is called a hallucination, and it happens with every major AI assistant. In this article you'll learn, in plain English, why AI makes things up, which kinds of answers are most likely to be wrong, and a short checking routine you can use every time the facts matter.

What an AI hallucination actually is

OpenAI describes a hallucination as when "a model confidently generates an answer that isn't true." The key word is confidently. The wrong answer doesn't look different from a right one. It has the same polished tone, the same tidy formatting, and often the same level of detail.

That's what makes hallucinations risky in ordinary work. A made-up figure in a client email, a quote that was never said in a board report, or a research paper that doesn't exist in a reference list all read just as smoothly as the real thing. You can't spot them by how they sound. You can only spot them by checking.

Why AI makes things up

In a September 2025 paper, OpenAI argues that models hallucinate because "standard training and evaluation procedures reward guessing over acknowledging uncertainty." In plain terms, there are two causes.

How the model learns

During training, a model learns to predict the next word by reading huge amounts of fluent text. Nothing in that text is labeled "true" or "false." Common facts show up so often that the model gets them right. Rare, arbitrary facts, like a specific person's birthday or an exact figure from a niche report, don't follow a pattern it can learn, so it fills the gap with something that sounds plausible.

How the model is tested

Many tests only score whether an answer is right. Under that scoring, guessing beats saying "I don't know," just like a student who guesses on every multiple-choice question gets a better score than one who leaves blanks. Models tuned to do well on those tests learn to guess.

The paper shows the trade-off with real numbers. On one test, OpenAI's gpt-5-thinking-mini declined to answer 52% of questions and was wrong on 26%. The older o4-mini declined just 1%, was right 24% of the time, and was wrong 75% of the time. Slightly higher accuracy came with far more confident errors.

OpenAI's position is that hallucinations aren't inevitable, because a model can decline when it's unsure. It also says plainly that ChatGPT, including GPT-5, still hallucinates, just less often. One practical result: newer models may hedge or say "I don't know" more often. That's intended, and it's better than a confident guess.

How often it happens, and why there's no single number

You'll see headlines claiming AI is wrong some fixed percentage of the time. Be skeptical. The rate depends heavily on the task, the model, and how the test was scored.

For example, the Vectara hallucination leaderboard measures how often models add unsupported content when summarizing a document they were given. According to one tracker, as of the October 2026 update the best model scored 1.8%, while several widely used models were above 20% on the same test. These rankings are refreshed often, so any specific ranking goes out of date quickly.

OpenAI's GPT-5 system card also reports big improvements: gpt-5-thinking makes "over 5 times fewer factual errors than o3," with browsing on or off. Fewer is good. It is not zero, and the errors that remain are just as confident as before.

What it costs when nobody checks

The most common real-world failures aren't subtle. They're fake citations, invented quotes, and sources that don't exist.

  • Deloitte Australia partially refunded an A$440,000 report for an Australian government department after it was found to include a fabricated quote from a federal court judgment and references to research papers that don't exist. The refund was about A$97,000, and the revised report added a disclosure that generative AI was used.
  • Courts keep seeing filings with AI-invented cases. Researcher Damien Charlotin maintains a public database of these decisions, and secondary reports put the count well over a thousand worldwide in 2026.
  • Judges are sanctioning lawyers. The Alabama Supreme Court, for example, sanctioned an attorney over AI-generated fake citations. As Scientific American reports, lawyers know about the risk, have been warned, and keep doing it anyway.

That last point matters for everyone, not just lawyers. Knowing that AI can hallucinate doesn't protect you. A routine you follow every time does.

A five-step checking routine

Use this whenever an AI answer will go into something other people rely on: a report, a client email, a presentation, a policy, or a decision.

  1. Give the model permission to say "I don't know." Ask it to separate what a source actually says from what it's inferring.
  2. Ask for sources, then open every one. Confirm the source exists and that it says what the AI claims it says.
  3. Check every number, date, name, and quote against the original. Not against a summary, a blog post about it, or another AI's answer.
  4. Use grounded modes for factual work. Turn on web search, or give the AI the documents you want it to work from.
  5. Be most suspicious of specific details. Exact figures, page numbers, case names, and quotations are where made-up content hides.

Step one is a prompt habit. Here's a version you can paste in front of any factual question. For more on structuring prompts, see How to Write a Good AI Prompt.

Ask for an answer that admits uncertainty
Answer the question below. Only state facts you are confident about. If you don't know or aren't sure, say "I don't know" instead of guessing.

For each factual claim, add a tag:
[Source] if it comes from a document I gave you or a source you can link to, with the link or section name.
[Inferred] if it's your own reasoning or estimate.

Question: [paste your question here]

For steps two and three, it helps to turn the AI's answer into a checklist before you start verifying. This prompt does the sorting for you, but leaves the actual checking to you.

Turn a draft into a fact-check list
Below is a draft written with AI help. List every specific claim I need to verify: numbers, percentages, dates, names of people or organizations, quotations, and cited sources.

Put them in a table with three columns: the exact claim, where it appears in the draft, and what kind of original source I should check it against (for example, the company's annual report, the court decision, the original study).

Do not tell me whether the claims are true. Just list them.

Draft:
[paste draft here]

On step three, one lawyer sanctioned in 2026 over 16 fabricated quotations said she had worked from publicly available versions of the cases, according to one report. The lesson: check quotes against the primary source itself. If a figure comes from a spreadsheet, check the spreadsheet cell; our guide to AI for Excel covers checking AI-written formulas too.

Use grounded tools for factual work

"Grounded" means the AI answers from specific material, like live web results or documents you provide, instead of only from memory. OpenAI's system card shows that turning on browsing lowers error rates a lot. It doesn't remove them, so grounding works alongside the checking routine, not instead of it.

There are two common ways to ground an answer:

  • Web search on. Many assistants can search the web and show links next to their answers. This is useful for current facts, but you still need to open the links, because a real link doesn't guarantee the page says what the summary claims.
  • Your own documents. Upload or paste the report, contract, or meeting transcript and tell the AI to answer only from it. This is the safest setup for summaries and questions about a specific document.

Features, names, and limits for these modes change often and differ by tool and plan, so check what yours offers before relying on it. If you're still choosing a tool, Which AI Assistant Should I Use? compares the main options. And before uploading anything sensitive, read Is It Safe to Put Company Data in ChatGPT?

Answer only from a document you provide
Use only the document I've attached (or pasted below) to answer my question. Do not use outside knowledge.

For each point in your answer, quote the exact sentence from the document that supports it and name the section or page it came from.

If the document doesn't answer the question, reply "Not in the document" rather than filling the gap.

Question: [your question]

Document: [paste or attach]

Even with this prompt, spot-check a few of the quotes against the document. If a quote isn't there word for word, treat the whole answer with more caution.

Make checking a team habit

Hallucinations are easiest to catch when checking is expected, not optional. A few simple norms help:

  • Anything with numbers, quotes, or citations gets a second look against the original before it leaves your desk.
  • Drafts note which parts came from AI, so reviewers know where to look harder. Deloitte added an AI disclosure only after the problems surfaced.
  • When the AI says "I don't know," treat that as useful information, not a failure. Go find the answer from a primary source.
  • Keep a shared list of the prompts above so everyone uses the same routine.

If you want help building these habits across a group, see our team training options.

Questions people ask

Can you completely stop AI hallucinations?

No. Newer models hallucinate less often, and grounding answers in web search or your own documents lowers error rates further, but no current tool is error-free. The reliable fix is a checking routine: open every source and verify every number, name, date, and quote.

Why does ChatGPT make up sources and citations?

AI models learn to produce text that sounds right, and a citation has a predictable shape, so the model can generate a plausible-looking one that doesn't exist. OpenAI's research also says training and testing have rewarded guessing over admitting uncertainty. Always click or search for each source yourself.

Does turning on web search stop AI from making things up?

It helps a lot but doesn't stop it. OpenAI's GPT-5 system card shows lower error rates with browsing, yet errors still occur, and a summary can misstate what a real linked page says. Open the links and confirm the details.

Which AI hallucinates the least?

It depends on the task, and rankings change often. Benchmarks like the Vectara leaderboard are updated several times a year, and results vary widely between tests. Instead of chasing the top model, use grounded modes and check the specific facts that matter.

Sources

  1. Why language models hallucinate (OpenAI)
  2. Why Language Models Hallucinate (research paper)
  3. GPT-5 System Card
  4. AI hallucination rate benchmarks (Klevere)
  5. Deloitte refunds part of report fee after AI errors (CFO Dive)
  6. Why lawyers keep citing fake cases invented by AI (Scientific American)
  7. AI hallucination legal cases (GC AI)
  8. AI hallucination sanctions and the duty to verify

Written with AI assistance and checked against the sources above. AI tools change quickly, so confirm features and prices on each vendor's site.

What did you think?

Join the conversation

What's the most surprising thing an AI tool has made up for you, and how did you catch it?

Want help putting this to work?

Promptly Taught offers hands-on AI training for people and teams, online or in person. Bring a real problem and leave with something that works.

Keep learning