The first version took an afternoon. You wired in an LLM, wrote a prompt, and had something genuinely useful working by dinner.

Six months later, there have been forty prompt edits, three model swaps, and at least one regression you can't trace. Is the current version actually better than the old one? You compare a few outputs side by side, but that only gets you so far.

At some point, you make a judgment call and ship.

The problem: the uncertainty moved

Traditional software keeps most of its uncertainty at the edges: user input, networks, and flaky dependencies. Tests mock those away and make assertions against a deterministic core. Given the same input, the code should produce the same output.

LLM applications work differently. Much of the surrounding code is deterministic, but the model at the center is not. It decides what to do, which tool to call, and how to phrase the answer. The same request may take a different path on the next run, and a small prompt change can affect a wide range of outputs. There is no single expected result for a green check to prove.

The statisticians got here first: all models are wrong, but some are useful. LLMs are no exception. You cannot guarantee that every output will be correct, but you can measure how often the system behaves as intended. Accuracy tells you how often it gets the task right. Precision tells you how confident you can be in that estimate. Improving the system raises the first; running more samples improves the second. Instead of one pass/fail assertion, you work with a score, a confidence interval, and a baseline for comparison.

Large AI labs have teams dedicated to evaluation. Most product teams do not. Whether you are building a support tool, a contract summarizer, or a product centered on a model, every meaningful change raises the same questions:

• Can we ship this?
• Which model should we use?
• Will we notice when it gets worse?

Answering them properly takes real infrastructure. You need a harness, isolated sandboxes, reliable graders, secure handling for API keys, and limits on what a runaway task can spend. It is enough work to become its own roadmap item, which means it is often postponed while the product continues to change.

The solution: eval-driven development

Eval-driven development applies the discipline of test-driven development to probabilistic software. You still define what correct behavior looks like, but you measure it across a dataset instead of checking one exact output. A demo shows that the system can work. An eval tells you how reliably it works.

The useful part is the loop. Change a prompt, model, or retrieval step, then run the eval against a pinned baseline. If the result clears the gate, it becomes the new baseline. If it does not, the change never reaches users. When this runs on every pull request, evaluation becomes part of development instead of a report produced every few months. Regressions are tied to specific changes, and improvements come with evidence.

A single score can still mislead you. If you optimize one number aggressively enough, the system may find ways to improve it without improving the product. A support bot judged only on ticket resolution might learn to close tickets by deflecting customers. The resolution rate goes up even as churn gets worse.

This is Goodhart's law: once a measure becomes a target, it becomes less useful as a measure. Choosing a cleverer metric does not solve the problem. The practical answer is to evaluate the system from several angles, so an apparent improvement on one axis can be checked against the others.

Start with accuracy, cost, and latency. Retries and longer reasoning may improve accuracy, but their effect appears in cost and response time. Cutting work short may improve latency while hurting accuracy. A cheaper model lowers the bill, but the quality score shows whether the tradeoff was worth it. Looking at these measurements together makes the compromises visible and gives the team a clearer idea of what to improve next.

This is already how test suites work. No single test defines correctness; confidence comes from the suite as a whole. Benchwright makes it practical to apply the same discipline to AI behavior on every change.

The product: an AI benchmarking factory

Benchwright is a factory for evals. You describe what you want to measure, and it handles the work required to produce a useful score: finding the benchmark, creating sandboxes, running the tasks, and grading the outputs.

For a standard benchmark, a run takes one call. The catalog includes hundreds of options, such as MMLU for broad knowledge, GPQA Diamond for expert reasoning, AIME for competition math, LiveCodeBench for coding, WebArena for tasks in a live environment, and MMMU for images and documents. You can launch any of them from the CLI or with a POST request:

benchwright run "GPQA Diamond" --model qwen/qwen3-32b@openrouter

curl https://benchwright.ai/v1/runs \
  -H "Authorization: Bearer $BW_KEY" \
  -d '{
    "request": "GPQA Diamond",
    "model": {
      "model": "my-model",
      "base_url": "https://your-endpoint.example.com/v1/chat/completions",
      "api_key": "$MODEL_KEY"
    }
  }'

The AI workforce

Operators handle the work inside the factory. These AI agents take a plain-English goal, find an appropriate benchmark, run it against your model, and investigate failures along the way.

Public benchmarks tell you something about a model in general. Your product, however, succeeds or fails on a much narrower set of scenarios. A minimum viable evaluation covers that gap. Give the operator the cases that matter most, and it turns them into a small dataset with graders. The first version will not cover everything, but it can run on every change and grow alongside the product. That is more useful than a comprehensive evaluation that only runs twice a year.

One eval run gives you a number. Repeating it on every change gives you a record of what is safe to ship, which model performs best, and whether the product is drifting over time.

Benchwright is live, and the free tier is enough to try the full workflow. Choose a benchmark that resembles your workload, point it at the model behind your product, and run your first evaluation.

Prompts will keep changing, and so will the models behind them. What matters is knowing whether each change made the product better. Benchwright gives teams a practical way to answer that before they ship.