Benchmark

A benchmark is a standardized evaluation suite used to compare model capabilities across the research community. Examples include MMLU for knowledge breadth, HumanEval for code generation, and TruthfulQA for factuality. Public benchmarks enable reproducible comparisons but also suffer from contamination when training data overlaps with benchmark test sets. Production teams typically supplement public benchmarks with internal task-specific evals that better reflect real user needs.