Open-source vs frontier models

Frontier models lead on quality. Open-weight models lead on cost, control, and customisation. Most production systems use both.

What this actually means in practice

This isn't a "pick one" decision — most production AI systems use both. Frontier APIs (Claude, GPT, Gemini) are the default starting point because you ship faster, get the best quality on hard tasks, and skip GPU ops entirely. Open-weight models (Llama, Qwen, Mistral, DeepSeek) earn their keep when one or more of these dominates: data privacy, fine-grained customisation, on-device deployment, or sheer cost at high volume.

The pragmatic path most teams converge on: prototype on frontier, identify the high-volume narrow tasks where a smaller fine-tuned open model does ~95% as well at 5–10% of the cost, then route. The router itself becomes a piece of infrastructure you tune.

That's the gist. Below: the comparison table, real cost examples, and the self-hosting reality check.

How the choice plays out

The honest comparison:

A note on terminology: open-weight ≠ open-source. Most open-weight licenses restrict commercial use or redistribution. Truly open (weights + training data + recipe) is rare — OLMo, Pythia, OpenCoder come closest. Read the license.

Where it bites in real life

The pragmatic path

1. Prototype on frontier APIs. Ship fast. Instrument cost and quality.
2. Identify high-volume narrow sub-tasks where open-weight + fine-tune wins.
3. Migrate those; keep frontier for the rest.
4. Build a router that dispatches each request to the right tier.

Under the hood (optional)

A model-routing pattern: a tiny classifier picks one of three categories, then dispatches to the right specialised model. ~10 lines. Skip if you don't code — the idea (small router + tiered models) is what saves the bill.

ROUTER_PROMPT = """Classify this user message into one of:
- 'simple': straightforward question, factual lookup, billing, account.
- 'creative': open-ended writing, brainstorming, complex reasoning.
- 'code': writing, debugging, or explaining code.
 
Respond with one word."""
 
def smart_chat(user_msg):
    label = call_small_model(ROUTER_PROMPT, user_msg).strip().lower()
    if label == "creative":
        return call_frontier(user_msg)
    if label == "code":
        return call_code_model(user_msg)
    return call_finetuned_small(user_msg)

Further reading

• Hugging Face Open LLM Leaderboard — independent benchmarks.
• LMSYS Chatbot Arena — Elo-style human-preference rankings comparing open + closed.
• Artificial Analysis — independent cost/latency/quality across providers.
• vLLM — most widely used open-source LLM serving stack.
• llama.cpp — run quantised open models on CPU/Mac/edge.