subtopic

Pretraining, fine-tuning and zero-shot

Learn the 3 ways to use a Transformer: zero-shot (use it as is, e.g. a pipeline), fine-tuning (train it a little more on your labelled data — Projects 1 and 3), and embeddings (use its vectors for search — Project 2). Pretraining (learning from billions of words by predicting missing or next words) is done by big labs; we reuse it — the same transfer-learning idea as in computer vision.

Builds on: 2.5 Improve with transfer learning