Eval basics

If you can't measure your AI system, you can't improve it. Vibes alone won't catch the regression that affects 0.4% of users — but those users will notice.

What evals actually do

An eval is a repeatable test that scores model output against expected behaviour. Same inputs, same scoring rules, every time you change the prompt or the model. That repeatability is the whole point: it turns "feels good to me" into a number you can track over weeks.

Without evals, you can't tell whether a tweak helped 100% of users or hurt the 0.4% you didn't manually test. Those 0.4% will notice. With evals, you catch the regression before the email comes in.

That's the whole concept. Below: how to actually grade output, what to measure, and how to build a golden set without poisoning it.

When you'd reach for it

Anytime an LLM is part of a real product:

• Before shipping a prompt change — does the new version still pass everything the old one did?
• When swapping models — Sonnet 4.5 → 4.7, GPT → Claude, frontier → cheap.
• Tracking faithfulness in RAG — is the bot still answering from the docs, not from training?
• Investigating a user-reported failure — add it to the golden set so it never silently regresses again.

You wouldn't bother with formal evals on a one-off internal script that gets used twice. The work pays off when prompts touch users repeatedly.

How it's actually built

A maturity ladder most teams climb:

1. Vibes-testing. A few prompts, eyeball the output, ship.
2. Spot checks. A small set of canned inputs re-run after each prompt change.
3. Golden set. Curated dataset of inputs + expected outputs (or grading criteria), versioned in git.
4. Automated evals in CI. Every prompt or model change runs the suite; results gate the merge.
5. Online evals. Sample real production traffic, score it, watch trends.
6. Active learning. Production failures feed back into the golden set; the suite grows over time.

You don't have to skip levels. But every step past 1 pays for itself the first time something breaks.

Three ways to grade output

• Deterministic checks — regex, JSON schema validation, unit tests on generated code. Cheapest. Use whenever you can.
• LLM-as-judge — a different LLM scores against a rubric. Good for qualitative metrics like "faithful to the context".
• Human evaluation — gold standard. Slow, expensive. Used to calibrate the other two.

What to measure

Where it bites in real life

Under the hood (optional)

A bare-bones eval is just unit tests for an LLM: a list of questions paired with what the answer must contain (or must NOT contain). Below in pytest. Skip if you don't code — the idea (a versioned list of expected behaviours, run on every prompt change) is the lesson.

import pytest
 
GOLDEN = [
    {"q": "How long do refunds take?",
     "must_contain": ["5 business days"],
     "must_not_contain": ["1 day", "instantly"]},
    {"q": "What's your support phone number?",
     "must_contain": ["I don't know"],   # ← must refuse, no number in our docs
     "must_not_contain": []},
]
 
@pytest.mark.parametrize("case", GOLDEN)
def test_qa_bot(case):
    answer = bot.answer(case["q"])
    for s in case["must_contain"]:
        assert s.lower() in answer.lower(), f"missing {s!r}"
    for s in case["must_not_contain"]:
        assert s.lower() not in answer.lower(), f"unexpected {s!r}"

Further reading

• Hamel Husain — Your AI Product Needs Evals — the most practical "how to actually build evals" article online.
• OpenAI Evals — open framework — runnable eval harness.
• Anthropic — Evaluating prompts — Claude's first-party eval tooling.
• Patronus AI — Lynx faithfulness model — open hallucination/faithfulness scoring.