Top-K Sampling

Also known as: top-k

Top-K sampling restricts next-token selection to the K tokens with the highest logit scores, zeroing out all other candidates before applying softmax and sampling. It prevents the model from selecting very low-probability tokens that could derail coherent generation. A small K value produces conservative, focused output while a larger K introduces more variability. Top-K is often combined with top-P sampling and temperature for fine-grained decoding control.