INDUCTION

A two-layer transformer, trained in this tab, on a task it can only solve one way. Watch the circuit that solves it assemble itself.

Built on gama3d · scena3d · anima3d. No weights are downloaded and no assets are loaded — the model is initialised, trained and drawn from code in front of you.

What am I looking at

A transformer has three independent axes and a diagram on paper can only show two. So this one is laid out in all three.

• Across — position in the sequence. The letters are the tokens.
• Away from you — depth. The input row, then layer 0, then layer 1, then what the model predicts. The bars running the whole length are the residual stream: every block reads from it and adds back into it.
• The arcs — attention, drawn from each query to the keys it reads. Brightness is weight. Each head is tilted to its own angle so you can read them at the same time instead of on top of each other.
• Green letters at the far end are correct predictions, red are wrong. The dim ones are positions where being wrong is the right answer — nothing could know them yet.

Controls

• Drag to orbit, wheel to zoom.
• Space pause or resume training · N a new sequence
• S turn the √d divisor off, and watch the arcs collapse
• M switch the MLPs on, and watch the circuit dissolve

Why this task

Each line is a short random pattern of letters, repeated until the line is full. The pattern's length changes every time.

• Until a letter has appeared once, nothing can predict what follows it. Those positions are pinned at the loss of pure guessing, forever, and the model never beats them. That is the grey curve.
• After it has appeared, the answer is exact — whatever followed it last time. But finding it means matching on content, and that takes two layers: one head to record “the letter before me was X”, and a second to look for a place whose recorded letter matches the one I am holding now. Elhage et al. (2021) show one layer cannot express it.
• The length has to vary. With a fixed period the answer always sits the same distance back, and a single head can find it by counting positions with no content matching at all — which is exactly what the first version of this did, scoring 0.85 on the induction metric for a head doing nothing of the kind.

Vaswani et al. 2017 for the architecture and the √d_k derivation; Elhage et al. 2021 for the residual stream and the two-layer argument; Olsson et al. 2022 for induction heads.