An AI hallucination is a confident, fluent answer from a language model that is false or not supported by its sources, such as an invented citation, statistic or software package. This guide explains why hallucinations happen, how to recognize them, and the techniques that reduce them in chat, documents and code.
- Language models predict likely text; they do not look facts up, so a plausible-sounding wrong answer is always possible.
- Hallucinations cluster around specifics: names, numbers, dates, quotes, citations, URLs, legal cases and package names.
- Grounding the model in your documents, allowing it to say "I don't know" and asking for quotes reduce hallucinations a lot.
- For code, tests and type checks catch invented functions; check every new dependency exists before installing it.
- No technique removes hallucinations completely, so verify anything you will publish, sign, ship or act on.
What are AI hallucinations?
AI hallucinations are outputs from a language model that sound confident and fluent but are false, made up, or not supported by the material the model was given. The model is not lying in a human sense. It produces the most plausible-looking text, and plausible is not the same as true.
Typical examples:
- A research summary that cites a paper that does not exist.
- A biography with the wrong birth year or a job the person never held.
- A legal brief citing court cases with realistic names and docket numbers that were never decided.
- Code that calls a function or imports a package that does not exist.
- A document summary that includes a point the document never makes.
Some people prefer "confabulation" because the model fills gaps with invented detail, but "hallucination" is the term most people search for and most vendors use.
Why do AI models hallucinate?
AI models hallucinate because they are trained to predict likely next words, not to retrieve verified facts, and because training and testing have rewarded confident guesses over admitting uncertainty. Both causes are built into how current models work.
Prediction, not lookup
A large language model learns statistical patterns from huge amounts of text. When you ask a question, it generates an answer token by token, each one chosen because it fits what came before. For common facts, the most likely continuation is also the correct one. For rare facts, the model has seen too little to separate true from merely plausible, so it produces something that has the right shape. If you want the mechanics in plain language, see what is a large language model.
Incentives that reward guessing
A 2025 paper by researchers at OpenAI and Georgia Tech, Why Language Models Hallucinate, argues that hallucinations persist partly because popular evaluations score answers as right or wrong and give no credit for "I don't know." Under that scoring, guessing always beats abstaining, so models are, in the authors' words, optimized to be good test-takers. The authors propose changing how mainstream benchmarks score uncertain answers.
Situations that make it worse
- Obscure or very specific facts: small companies, minor historical figures, exact figures.
- Recent events after the model's training cutoff, when it has no search tool.
- Long documents, where details from different parts get mixed up.
- Leading questions that assume something false ("Why did Einstein win two Nobel Prizes?").
- Pressure to produce something: "give me 10 sources" when only 3 good ones exist.
How can you spot an AI hallucination?
Spot hallucinations by checking the specific, verifiable details in an answer, because that is where they cluster. Fluency and confidence tell you nothing; a hallucinated answer reads exactly like a correct one.
| Red flag | Why it is risky | How to check |
|---|---|---|
| Citations, DOIs, URLs | Often invented whole or mismatched | Open each link; search the exact title |
| Exact numbers and statistics | Easy to generate, hard to verify | Find the original report |
| Direct quotes | Frequently paraphrased or fabricated | Search the exact phrase in the source |
| Legal cases and regulations | Realistic names, fake rulings | Check an official court or law database |
| Software packages and API methods | Plausible names that do not exist | Check the registry and official docs |
| Very specific claims about people | Mixes up similar people | Check a primary source about that person |
| Answers that match your assumption | Models tend to go along with the question | Ask the opposite question in a new chat |
Two quick tests help:
- Ask again in a fresh conversation. If the details change between runs, at least one version is wrong. Anthropic's guidance calls this best-of-N verification.
- Ask for the source of one claim. If the model cannot point to where a specific fact comes from, treat the fact as unverified.
Real examples of AI hallucinations causing damage
Hallucinations cause real harm when nobody checks the output before acting on it. Two documented cases show the pattern.
Invented legal cases. In Mata v. Avianca (S.D.N.Y. 2023), lawyers filed a brief that cited court decisions generated by ChatGPT, complete with fake quotes and internal citations. The court sanctioned the lawyers and their firm with a $5,000 penalty in June 2023. The opinion notes there is nothing inherently improper about using a reliable AI tool for assistance; the failure was that the lawyers did not check the citations and then stood by them.
Invented software packages. A study accepted at USENIX Security 2025, We Have a Package for You!, generated 576,000 code samples with 16 models and found that at least 5.2% of recommended packages from commercial models, and 21.7% from open-source models, did not exist. It counted 205,474 unique made-up package names. Attackers can register those names and publish malicious code, a supply-chain risk sometimes called slopsquatting.
How do you reduce AI hallucinations when prompting?
Reduce hallucinations by giving the model permission to be unsure, giving it the source material, and asking it to tie each claim to that material. Anthropic's guide to reducing hallucinations lists these as the basic techniques, and they work with any major model.
1. Allow "I don't know"
Explicitly tell the model what to say when it lacks information. This one line removes much of the pressure to guess.
Answer using only the report below. If the report does not contain the information,say "The report does not say" instead of guessing.2. Provide the source and restrict to it
Paste the document, attach the file, or turn on web search, and tell the model to rely on that material instead of its general knowledge. This is the idea behind retrieval augmented generation; see what is RAG for how applications do this at scale.
3. Extract quotes first, then answer
For long documents, ask for word-for-word quotes before the analysis. Anthropic recommends this for documents over about 20,000 tokens.
1. Extract the exact sentences from the contract that mention termination or notice periods. Number them. If there are none, write "No relevant quotes found."2. Using only those numbered quotes, explain how either party can end the contract. Reference quote numbers for every point.4. Make the model verify its own claims
After a draft, ask the model to find a supporting quote for each claim and delete any it cannot support.
Review your answer above. For each factual claim, quote the sentence from the documentsthat supports it. If you cannot find one, remove the claim and mark the spot with [].5. Ask narrower questions
"List every revenue figure in this report with its page number" hallucinates less than "summarize the financials." Specific tasks give the model less room to fill gaps. The prompt engineering guide covers structuring prompts this way.
When a model offers a list of sources, ask for fewer. "The three most relevant sources you can verify" produces better results than "10 sources."
How do tools and grounding reduce hallucinations?
Tools reduce hallucinations by letting the model check facts instead of recalling them. Web search, file search and code execution turn "what do I remember?" into "what does the source say?"
| Approach | What it fixes | Remaining risk |
|---|---|---|
| Web search or deep research modes | Outdated or obscure facts | Low-quality sources, misread pages |
| Uploading your documents | Facts specific to your business | Mixing details across documents |
| Citations features | Unverifiable claims | Citation points to a passage that does not fully support the claim |
| Code execution | Arithmetic and data analysis errors | Wrong method applied correctly |
| Retrieval (RAG) in apps | Answers outside your knowledge base | Bad retrieval returns the wrong passages |
Some APIs return citations as structured data. Claude's Citations feature, for example, returns the exact passages from your documents that support each part of the answer, so an application can show sources and you can check them.
For research tasks, the guides on using an AI research assistant and on how to chat with a PDF cover source checking in more depth.
How do you handle hallucinations in AI-generated code?
Handle hallucinations in code by making the computer check the code: run it, type-check it, test it, and verify every new dependency before installing it. Code has an advantage over prose here, because many hallucinations fail loudly.
- Run tests after every change. Invented functions and wrong signatures break tests or the build immediately.
- Use type checkers and linters. They catch calls to methods that do not exist.
- Check new packages before installing. Look the package up on the official registry, check its age, downloads, maintainers and repository. A package created recently with no history is a warning sign.
- Pin dependencies and use a lockfile, so a typo-squatted or hallucinated package cannot slip in later.
- Ask for documentation links for any unfamiliar API, and open them.
- Review the diff, especially configuration and security code, where wrong code may still run.
Coding agents that can run tests close most of this loop on their own: they write code, run the suite, read the failures and fix them. That works best when tests exist and the agent is told it may not edit tests to make them pass. The post is AI-generated code safe has a full review checklist. If you run agents unattended, for example on an always-on box such as Orkio, tests are what let you trust the pull request you review in the morning.
How to protect yourself from AI hallucinations today
Use this checklist whenever AI output feeds a decision, a publication or production code:
- Decide the stakes. Casual brainstorming needs little checking; anything published, signed or shipped needs full verification.
- Give the model the sources, or turn on search, instead of relying on its memory.
- Tell it to say "I don't know" when the sources do not answer the question.
- For long documents, ask for quotes first and analysis second.
- Open every citation, link and quote before reusing it.
- Check numbers against the original report, not the summary.
- For code, run tests and type checks, and verify each new package on its registry.
- When details matter, ask the same question in a fresh chat and compare.
Hallucinations come from how language models work, so no setting removes them. Let the model do the drafting and searching, and keep verification with a person or a test.
Frequently asked questions
Why does ChatGPT make up sources?
When asked for references, a model generates text that looks like a citation: plausible authors, a fitting title, a realistic journal. Without a search or document tool it has no way to check that the paper exists. Use a mode that searches the web or your files, and open every source before you rely on it.
Are newer AI models less likely to hallucinate?
Newer models generally hallucinate less on common benchmarks and are more willing to say they are unsure, especially when they can search or read documents. They still make errors, particularly on obscure facts, recent events and exact numbers. Treat improvements as fewer mistakes, not none.
Can you completely stop AI hallucinations?
No. Grounding, citations, verification steps and tools reduce hallucinations substantially, but a model that generates text can always generate something false. Anthropic's own guidance says these techniques reduce hallucinations without eliminating them, so critical information still needs human validation.
What is the difference between a hallucination and a mistake?
The terms overlap. Hallucination usually means content that is invented or unsupported, such as a fake quote or a nonexistent function, rather than a reasoning slip like a wrong calculation. Both are caught the same way: by checking against a source or running a test.
Is AI hallucination dangerous?
It can be when people act on the output without checking. Lawyers have been sanctioned for filing briefs with invented cases, and developers can install malicious packages whose names a model made up. The risk depends on the stakes of the task and whether anyone verifies the result.
