Evaluation Drift

Evaluation drift occurs when a fixed benchmark gradually stops reflecting real production behavior, either because user queries have shifted or because the system has been tuned to overfit the static test set. It produces reassuring scores that mask real regressions. Countering drift means periodically refreshing the evaluation set from live traffic so it continues to represent the actual usage distribution.