Fine-Tuning

Fine-Tuning & Alignment

Understand LoRA, RLHF, DPO, and when fine-tuning is the right approach versus prompting.

1. What does LoRA (Low-Rank Adaptation) do?

2. What is the first stage of RLHF (Reinforcement Learning from Human Feedback)?

3. How does DPO (Direct Preference Optimization) differ from RLHF?

4. When is fine-tuning clearly preferable to prompt engineering alone?

5. What is 'catastrophic forgetting' in the context of fine-tuning?

6. What role does the KL divergence penalty play in RLHF training?

7. What is QLoRA?