Reward Model

Also known as: RM, preference model

A reward model is a language model trained to predict a scalar quality score for a given prompt-response pair, representing how much a human would prefer that response. It is trained on human-labeled preference data and then used as a proxy evaluator during RL fine-tuning to provide a reward signal without requiring constant human feedback. Reward model quality is a bottleneck in the RLHF pipeline: a poorly calibrated reward model leads to reward hacking, where the policy exploits scoring loopholes rather than genuinely improving.