Benchwright Blog

Notes on agent evaluation, reproducible pipelines, and the plumbing that makes benchmarks trustworthy.

RSS
Tagged · evals
The Fitness You Can't Write Down
Every eval is a proxy for a target you can't fully specify. Optimize the proxy hard enough and it peels away from the target. That's Goodhart's law. You don't escape it with a better number. You triangulate it with several, anchored to something real.
7 min