Why Transformers Won

RNNs read one step at a time and forgot too much. Transformers process everything at once and reach any distance in one hop. The three reasons the field switched, in full.

For a decade, gated RNNs were how machines processed language. Then in 2017 the Transformer arrived, and within a few years the RNN was gone from serious NLP. This was not fashion. Three concrete advantages compounded until the older architecture had no answer, and understanding them is the payoff of this entire layer.

Reason 1: parallelism

This is the decisive one, and it is about hardware, not linguistics.

An RNN computes its hidden state at step 100 from the state at step 99, which came from step 98, all the way back. The computation is inherently sequential — you cannot start a step until the previous one finishes. On a GPU, which is a machine built to do thousands of operations simultaneously, this leaves almost all the hardware idle. Training on a long document means waiting through it one word at a time.

The Transformer’s self-attention computes the representation of every position at the same time. There is no step-by-step dependency during the attention operation; it is a few big matrix multiplications, and matrix multiplication is exactly what GPUs do fastest. Every word attends to every other word in one parallel pass.

The practical consequence was enormous. Freed from sequential processing, models could train on far more data with the same wall-clock time and hardware budget. That efficiency is what made training on internet-scale corpora feasible, which is the precondition for everything that followed. The RNN could not scale this way no matter how it was tuned; sequentiality was baked into its definition.

Reason 2: direct access at any distance

An RNN carries information across a sequence by relaying it through the hidden state, one step at a time. To connect word 1 to word 200, the signal has to survive 199 hops, and even LSTM gating only stretches how far it can travel before degrading. Long-range dependencies were always the RNN’s weak spot.

In self-attention, any position reaches any other position in a single operation. Word 200 attending to word 1 costs exactly the same as attending to word 199. There is no relay, no decay over distance. The path length between any two tokens is one.

This is why Transformers handle long-range structure — a pronoun and its referent paragraphs apart, a function definition and its distant call site — that RNNs routinely lost. The self-attention walkthrough shows the mechanism in detail; the headline is that distance stopped mattering.

Reason 3: attention was already the useful part

The Bahdanau attention work had shown that when a seq2seq model translated well, attention was doing the heavy lifting — the decoder looking back and focusing on the right input words. The RNN underneath was increasingly just plumbing.

“Attention Is All You Need” took the idea to its conclusion: keep the attention, remove the RNN entirely. Replace sequential recurrence with self-attention stacked in layers, and add a few supporting pieces to make it work as a standalone architecture.

Two of those pieces matter here because they patch what dropping the RNN removed:

• Positional encoding. Attention alone is order-blind — it sees a set of words, not a sequence, because it has no built-in notion of position. So position information is added directly to the token representations. Positional encoding covers how.
• Multiple heads. One attention pass captures one kind of relationship. Running several in parallel lets the model track several at once — syntax in one head, coreference in another. See multi-head attention.

The result is the Transformer: no recurrence, fully parallel, direct access everywhere.

What the RNN did better, honestly

The switch was not free, and pretending otherwise misleads.

Self-attention costs n² comparisons for n tokens — every token scores against every other. An RNN is linear in sequence length. For very long sequences the Transformer’s quadratic cost becomes the dominant expense, which is the whole subject of why attention is expensive. RNNs also process streaming input naturally, one token at a time with constant memory, which suits some real-time settings.

These trade-offs are exactly why RNN-flavored ideas never fully died and why state space models revisit sequential processing with modern tricks. But for the central task of training large models on large data, the Transformer’s parallelism was decisive, and the field followed.

What to remember

• Parallelism was the deciding factor: self-attention processes all positions at once, RNNs cannot, and GPUs reward the difference enormously.
• Direct access: any token reaches any other in one hop, versus the RNN’s lossy step-by-step relay.
• Attention was the useful part of seq2seq already; the Transformer kept it and dropped the recurrence.
• Dropping recurrence required positional encoding and multi-head attention to compensate.
• The cost is quadratic attention, so RNN-style linear processing still has niches — but for scale, Transformers won cleanly.

Next: The Transformer — the full architecture that replaced the RNN and now underlies every large language model.