Model Drift

Model drift in LLMOps refers to a degradation in the quality or behavioral consistency of a production model over time. It can originate from upstream causes — such as a provider silently updating a model's weights — or from data distribution shifts where the queries the system receives diverge from those on which prompts and few-shot examples were tuned. Detecting model drift requires continuous evaluation on a held-out reference dataset and alerting when quality metrics fall below a defined threshold.