subtopic

Fine-tune with TRL's SFTTrainer

Load a small instruct model (e.g. Qwen2.5-1.5B-Instruct or Llama-3.2-1B-Instruct) in 4-bit, attach LoRA, and train with SFTTrainer: 2–3 epochs, learning rate about 2e-4, gradient accumulation to fit memory. Watch training and validation loss; save the adapter (a few MB).

Builds on: 3.5 Fine-tune the Transformer, 5.6 Fine-tune the model