Who it's for
Platform
Resources
subtopic
Learn the last training stage in practice: instead of 'the right answer', you show the model pairs — a preferred reply and a rejected reply (e.g. polite and on-policy vs rude, unsafe or promising refunds). DPO trains the model to prefer the good one. Build 300–500 pairs from Project 1: approved replies as 'chosen', and rejected or deliberately bad replies as 'rejected'. Train with TRL's DPOTrainer on top of your fine-tuned model and compare reply-quality scores and rule violations before and after.