PEFT

Also known as: Parameter-Efficient Fine-Tuning

Parameter-efficient fine-tuning is a family of methods that adapt a pretrained model by training only a small number of additional or selected parameters rather than the entire weight set. PEFT approaches include LoRA, prefix tuning, prompt tuning, and adapter insertion. By reducing the number of trainable parameters by orders of magnitude, PEFT lowers hardware requirements, reduces catastrophic forgetting risk, and makes it practical to maintain many task-specific model variants without storing full copies.