Checkpoint Merging

Also known as: model merging, weight merging

Checkpoint merging combines the weights of two or more independently fine-tuned model checkpoints into a single model, often by taking a linear interpolation of their parameters. Techniques like SLERP and TIES merging can blend capabilities from different fine-tuning runs — for example, merging a code-focused checkpoint with an instruction-following checkpoint — without rerunning expensive training. The resulting merged model can exhibit complementary strengths, though careful validation is needed to confirm merged capabilities are retained.