Module 2 · Transformers Explained

OBJECTIVE: understand, in simple steps, how a Transformer works and how to use pretrained Transformer models with Hugging Face. WHAT WE COVER: • Attention: the idea of looking at all words at once • Self-attention (Query, Key, Value), multi-head attention, positional encoding • The Transformer block: encoder and decoder • The three model families: BERT (understand text), GPT (generate text), T5/BART (transform text) • Tokenisation into sub-words, padding and attention masks • Pretraining, fine-tuning and zero-shot use • The Hugging Face toolkit: pipeline, tokenizer, model, datasets, Hub AFTER THIS MODULE YOU CAN: • Explain attention and the Transformer in plain words with a diagram • Pick the right model family for a task (BERT vs GPT vs T5) • Run pretrained models for 5 tasks in a few lines and read what the tokenizer does MINI EXERCISE: run 5 Hugging Face pipelines (sentiment, NER, zero-shot classification, summarisation, text generation) and inspect the tokens.