AI Hallucination definition
AI hallucination is when a generative AI model, such as a large language model, produces output that sounds confident and plausible but is false, fabricated or unsupported by its sources. Examples include invented citations, wrong figures and made-up product features. Hallucinations happen because models generate likely text rather than checking facts against a source of truth.
Why do AI models hallucinate?
A language model is trained to produce likely continuations of text, not true ones. Most of the time likely and true overlap, because the model has seen accurate information many times. When it lacks knowledge, it still produces something fluent, because a confident-sounding answer is the most probable shape of a response. Training and benchmarks that reward a guess over "I don't know" reinforce the habit.
Hallucinations cluster in predictable places: obscure facts, exact numbers and dates, citations and URLs, details about people or small companies, events after the training cutoff, and leading questions that assume something false. In retrieval systems they also appear when search returns nothing useful and the model fills the gap from general knowledge.
Examples of AI hallucinations
Hallucinations have already had legal consequences. In 2023 a US federal judge sanctioned lawyers in Mata v. Avianca for filing a brief citing court cases that ChatGPT had invented. In 2024 a Canadian tribunal held Air Canada responsible when its website chatbot gave a customer incorrect information about bereavement fares. In both cases the organization, not the model, carried the responsibility.
- Factual errors: wrong dates, figures or attributions.
- Fabricated sources: citations, case law or URLs that do not exist.
- Unfaithful summaries: details added that the source document never stated.
- Code hallucinations: calls to functions or packages that do not exist.
- Visual hallucinations: misread text or objects described that are not in an image.
How to reduce AI hallucinations
No single technique is enough on its own. Retrieval fails when search misses, instructions are sometimes ignored and verification models make mistakes too. Layering several controls, and matching their strictness to the cost of an error, gives far better results than relying on one clever prompt.
- Ground answers with retrieval-augmented generation and require citations.
- Instruct the model to answer only from provided sources and to say when information is missing.
- Use tools for facts and math: database lookups, search, calculators.
- Constrain outputs with structured formats and allowed values.
- Add a verification step that checks claims against the sources.
- Keep human review for legal, medical, financial or customer-binding outputs.
- Lower the sampling temperature for factual tasks to reduce creative guessing.
How to measure hallucinations
You cannot manage what you do not measure. Build an evaluation set of real questions with known answers, including questions the system should decline. Score faithfulness, meaning whether every claim is supported by the retrieved context, using a mix of human review and LLM-based graders, and track the rate over time. In production, sample conversations for review and give users an easy way to flag wrong answers.
Hallucination cannot be driven to zero with current technology, so the practical goal is a rate that suits the use case, plus a design that limits the damage when one occurs. Nexzem sets these targets with clients before launch and builds the evaluation set that proves whether a release meets them.