subtopic

Why sequence models were not enough

Recall from the Deep Learning module that RNN/LSTM read text one word at a time. Learn their limits for language: slow (no parallel processing), and they struggle to connect words far apart in a long text. This is the problem Transformers solve (Module 2).

Builds on: 4.3 RNN concepts