Hallucinations

When a model says something that sounds confident and well-formed but is plain wrong.

Why it happens

Two structural reasons — properties of how LLMs work, not bugs to be fixed:

1. Knowledge is compressed, not stored. A 70-billion-parameter model can't memorise the entire web word-for-word. Specific details (names, dates, numbers, citations) are the lossiest part of that compression — patterns survive, exact strings don't.
2. No built-in lookup. With nothing in the prompt and only fuzzy weights to draw from, the model fills the gap with what sounds like an answer. The fluency is a feature; the consequence is fabrication when there's no real signal.

Most likely on: specific verifiable facts (names, dates, citations, URLs), recent events past the training cutoff, niche/long-tail topics, high temperature.

Where it bites in real life

How to reduce them (in order of impact)

1. RAG — put the answer in the prompt. (Medium-track lesson covers this.)
2. Lower temperature — less randomness = fewer inventions.
3. Allow "I don't know" — "If the answer isn't in the context, say 'I don't know.'" Models do follow this.
4. Verify with tools — web search, database, calculator. Don't trust facts the model can't look up.
5. Always verify citations — court cases, paper titles, URLs. Don't ship anything you haven't checked.

Under the hood (optional)

A "groundedness" prompt pattern: strict instruction to use only the supplied context, and say "I don't know" if it's missing. ~12 lines of Python. Skip if you don't code — the concept (force reading, not recalling) is what matters.

GROUNDED = """You are answering questions ONLY using the supplied context.
If the answer isn't in the context, reply exactly: "I don't know."
Do not use outside knowledge.
 
Context:
{context}
 
Question: {question}"""
 
resp = client.messages.create(
    model="claude-sonnet-4-6",
    temperature=0,
    messages=[{"role": "user", "content": GROUNDED.format(context=ctx, question=q)}],
)

Try it yourself (no coding, ~5 minutes)

Witness a hallucination live:

1. Pick a very specific, niche topic — your neighbourhood's history, a cult 1980s book, an obscure library.
2. Ask: "Cite three academic papers about [topic], with author, title, year, journal."
3. Verify each citation in Google Scholar.

Some real, some confabulated — invented authors or real authors with made-up papers. The model isn't lying on purpose; it's filling a gap with what sounds like a citation. Same mechanism that cost the Avianca lawyer his case.

Further reading

• Why language models hallucinate (OpenAI, 2025) — research note.
• TruthfulQA benchmark — open dataset measuring true vs. plausible-sounding answers.
• Retrieval-Augmented Generation (Lewis et al. 2020) — original RAG paper.
• Anthropic — Reduce hallucinations — concrete techniques.