A looped transformer runs the same transformer block again and again, feeding its output back in as its next input, instead of stacking many different layers.
- Shelf
- AI and Computing
- Kind
- Built from a recipe
- Sound
- Yes
- Licenses
- Own work (made in the page)
How to play
Tap it: the tiles loop through one block, sharper each lap, until 3 + 4 = 7.
Its controls
| Control | Kind | Range | Start |
|---|---|---|---|
| Loop until it's sure | Tap | ||
| Think | Button | ||
| View | List | 2 choices | Poster (2D) |
About it
A looped transformer runs the same transformer block again and again, feeding its output back in as its next input, instead of stacking many different layers. Each pass through the loop can refine the answer a little more, a bit like checking your work, so a small model can spend longer thinking about a harder problem.
Tap it: five tiles ride three laps around a track through one block. Each pass sharpens every tile one step, from noise to a blocky mosaic, to nearly right, to exact, until “3 + 4 = 7” settles with the 7 in gold. Pick the poster or a 3D model in the Splash tab.
Details
- Splats
- Opens at 200,000 at Mid Detail, 56,377 at Low, and up to 660,000 at Max
Is it right?
Repeating one block is a valid idea in recurrent transformer designs. Here three laps reveal stored pictures of 3 + 4 = 7; no transformer block or arithmetic inference runs.
Simplified
Transformer computation can repeat across depth.1
The single block and returning track illustrate a recurrent-depth architecture.
Correct
The splash ends with the correct arithmetic statement 3 + 4 = 7.2
The five stored marks are 3, +, 4, =, 7; the last tile images contain only finished bitmap values.
Simplified
The three laps illustrate refinement rather than learned reasoning.1
Each lap chooses another stored bitmap level, with no attention, recurrent tensor update, or learned stopping rule.
- 1Universal Transformers, Dehghani and coauthors, 2018
- 2Multilayer perceptrons, Dive into Deep Learning authors
- Simplified
- The noise, coarse, nearly finished, and exact tile images are prepared stages. The lap count is fixed at three and is not chosen by difficulty.
Sources and credits
Made in Splashery. Its code is MIT.
