subtopic

Pretraining, instruction tuning and human feedback

Learn the three training stages: (1) pretraining on trillions of tokens of internet text → a model that continues text; (2) instruction tuning (SFT) on question–answer examples → a model that follows instructions; (3) human feedback (RLHF / DPO) → answers that are helpful, honest and safe. You will do a small version of stage 2 in Project 3.

Builds on: 2.4 Using pretrained models