What is an LLM, really?

A very, very good autocomplete that has read most of the internet.

What's actually happening

The technical term is autoregressive generation: each step produces a probability distribution over ~100,000 possible tokens (more on tokens next lesson), a sampler picks one, the result becomes part of the input, and the loop continues.

That's worth seeing concretely. At each step the model isn't producing one word — it's producing a score for every word in its vocabulary. The sampler picks one. Then everything starts again with the new prefix.

Input:  "The capital of France is"
                               ↓
        ┌─────────────────────────────────┐
        │  Paris        74%   ← picked    │
        │  located       8%               │
        │  the           5%               │
        │  in            3%               │
        │  a             2%               │
        │  Europe        1%               │
        │  …            …%   (~100k more) │
        └─────────────────────────────────┘

New input: "The capital of France is Paris"
                                       ↓
        ┌─────────────────────────────────┐
        │  ,            42%   ← picked    │
        │  .            18%               │
        │  and          12%               │
        │  which         7%               │
        │  …            …%                │
        └─────────────────────────────────┘

(Numbers above are illustrative — but the shape is real. Every step is a top-k of competing candidates, not a single deterministic answer.)

Two consequences worth keeping in mind:

• The full sequence so far is the only input the model conditions on. No memory beyond the visible context.
• There's no separate "plan" then "execute" stage — the same forward pass runs at every step. Anything that looks like planning emerged from training, not from a different mechanism.

Where you see it

Under the hood (optional)

If you're curious, the whole loop is ~12 lines of Python. Skip if you don't code — you already have the picture.

def generate(prompt, max_new_tokens=50):
    tokens = tokenize(prompt)                  # turn text into chunks the model understands
    for _ in range(max_new_tokens):
        logits = model(tokens)                 # one score per possible next chunk
        probs = softmax(logits[-1])            # convert scores to probabilities
        next_token = sample(probs)             # pick one
        tokens.append(next_token)              # glue it onto the end
        if next_token == END_OF_TEXT:
            break
    return detokenize(tokens)

Try it yourself (no coding, ~5 minutes)

There are two flavours of this exercise. The first is what's easy to try; the second is what actually shows the mechanism. Do both.

A. The "polite-version" experiment

1. Open claude.ai or chatgpt.com.
2. Send exactly The capital of France is — no question mark, just the fragment.
3. Watch the reply.

You'll get something like "The capital of France is Paris, located in northern France and known for…" — a full helpful sentence, not a one-word continuation. That's not the model "ignoring your fragment" — it's still next-token prediction underneath, but post-training has taught it to dress every reply in helpful-assistant scaffolding (intro sentence, full context, polite ending). Strip the scaffolding mentally and you can still see autocompletion: the first useful word it produced was the next-token guess.

B. The "see the actual probabilities" experiment

This is the one that shows what the diagram above describes:

1. Open platform.openai.com/playground (free signup).
2. In the right-hand panel, find the "Show probabilities" option (or the logprobs toggle, depending on UI).
3. Type the same fragment, generate a few tokens.
4. Hover over each generated token — the playground shows you the actual top candidate words and their probabilities at that step. Exactly the diagram from earlier, with real numbers.

That's the closest you'll get to seeing the mechanism without reading code. Worth ten minutes — once you've seen it once, the rest of this curriculum stops feeling abstract.

Further reading

• Andrej Karpathy — Intro to LLMs (1h talk) — best one-hour overview.
• 3Blue1Brown — But what is a GPT? — beautifully visual.
• Jay Alammar — The Illustrated Transformer — once you want what's under the loop.
• Karpathy — Let's build GPT from scratch — code an LLM end-to-end in 2 hours.