1 · The shape
A ring says “clock”. Its arrangement says which clock.
A small network learned to add on a 31-hour clock. What it built is nine rings, each with the thirty-one numbers around it, each counting its way around at a different rate. The network found this on its own; it was never shown a clock.
The rings can be found without being told where to look: groups of neurons that read the same directions give them away (notebook 3). And they can be put to other use: a lens of 64 to 80 trained numbers turns the frozen rings of addition into subtraction, and taking the lens off restores addition (notebook 5).
Interactive figure
The Clock Inside the Network
- A ring says “clock”.
- Its arrangement says which clock.
- A lens is a turn or a mirror.
- How it forms.
Sources in the project: findings/nb01_ring_detection.json, findings/nb03_general_matcher.json, findings/nb05_sharing_lenses.json, figures/data/fig01_rings.json
2 · The payoff
What reuse buys
A lens is a small trained map placed in front of a frozen circuit so that it serves a new task. The frozen circuit already holds the clock; the lens only has to turn it. The figure walks through what that buys, in steps and in examples.
Sources in the project: figures/data/fig03_curves.json, findings/conventions.json
3 · The loop
What the reader sees, what deciding costs, what an adapter weighs
Before it can reuse a circuit, the network has to decide which one. It trains a short probe on the new task and reads the probe's shape: deciding by shape from a 500-step probe of the output end is accurate, cheap and robust to unseen formulas (notebooks 7, 8 and 11).
Run as one loop over a stream of 15 tasks, a single network makes 12 of 13 right calls in every seed and ends at 59–64% of the size of separate networks, with 24–36% of the steps (notebook 8).
Interactive figure
The reader, the stream, the costs, the adapters
- The Reader’s View
- Fifteen Tasks, One Network
- What Deciding Costs
- What an Adapter Weighs
Sources in the project: findings/nb07_learning_to_read.json, findings/nb08_whole_loop.json, findings/nb09_adapters.json, findings/nb11_deciding.json, figures/data/fig02-05_data.json
The record
Where the evidence stands
Every notebook fixes its rules before its fresh seeds run; a rule passes if it holds in at least 4 of 5 seeds. This table is rebuilt from the project's findings each time the page is built.
| No. | Notebook | What it found | Result |
|---|---|---|---|
| 1 | ring detection weekend test | rings exist and are clean with the answer key (10/10, no false alarms); blind whole-layer views fail | passed, with the answer key |
| 2 | grafting oracle test | a perfectly placed 128-weight bridge helps B learn (less data, 1.4-1.9x faster); copying the whole table helps more | passed |
| 3 | general matcher test | neuron groups find the rings without the answer key (1.11-1.15 vs controls 0.11-0.29); end to end B beats no help 5/5 | passed |
| 4 | fewer neurons test | a bridge teaches, it does not share: B copies the rings; size savings not consistent at 30,000 steps | mixed |
| 5 | sharing test | a lens of 64-80 trained numbers turns add's frozen rings into subtraction 5/5; lens off restores add; mul unreachable | sharing passed; fewer neurons did not hold |
| 6 | one network many tasks | one network decides by ring matching, ends at 64-72% of separate networks, 42-79% of the steps, learns reused tasks from 96 pairs | passed (7/7) |
| 7 | learning to read circuits | read the output end (95-96% vs 73-77%); a learned reader works on unseen families from 500-step probes; the features do the work | passed (8/9; the miss is a setup check) |
| 8 | whole loop one network | the whole loop in one network over 15 tasks: 12/13 right calls every seed, 59-64% of the size, 24-36% of the steps, output lens reaches the square-at-the-end decoys | passed (10/10) |
| 9 | adapters which lens | the ring lens is the smallest adapter that does the turn tasks and needs real rings; table tune retires the full lens; freezing matters at 96 pairs | passed (4/4 rules, 10/10 seeds) |
| 10 | one shared network | a one-table shared network cannot learn the mixed stream at any setting; per-task tables fix it; the fair baseline ran as notebook 10b | inconclusive; cause diagnosed |
| 10b | nb10b fair shared network | against a shared network with a table per task, the loop learns a new task in 200-250 steps vs 400-550 (5/5) and from 96 pairs scores 71-87% vs 38-51% (bar met 3/5); the shared network never learns four chain-like tasks | mixed (steps 5/5; data 3/5, short of the 25-point bar) |
| 11 | deciding by shape | the reader decides as well as trying at a fixed 500-step cost, wastes a fifth of guessing, handles unseen formulas; |a-b| and floor((a+b)/2) are its blind spot | passed (5/5) |
| 13 | nb13 chains | stored links compose: all 171 readable chains answer composite tasks such as (a - b)^2 at 94% or better with no training; the right chain is the top scorer among 55 on 80 check pairs every time; chains reach 45 of 45 cube composites where learning reaches 20 | passed (4/4 rules, 5/5 seeds) |
| 14 | nb14 addresses | pieces can be found by what they do: a piece learned again lands at the same address (237 of 237), a chain's address follows from its pieces' (1,397 of 1,399), and a search one piece at a time finds the chain with one run instead of 336 (1,526 of 1,528) or says exactly what is missing (1,300 of 1,300; 420 of 420); but a stored output link does not move to the other circuit (1 of 130) and two rarely stack (117 of 836) | passed (5/5 rules, 5/5 seeds) |
| 15 | nb15 common code | why stored links did not move or stack, and the remedy: a link on raw scores reads its circuit's own pattern (shuffle the 30 losing scores: 1 of 127 links survives); a link that reads the softmax of the scores learns every step (130 of 130, raw 127), works behind the other circuit and another network (130 of 130 each, raw 1 and 3 of 127), and stacks (1,690 of 1,690 pairs; every sequence of six, raw 0%) | passed (5/5 rules, 5/5 seeds) |
| 16 | nb16 library around a transformer | the library carries over when the frozen circuit is a one-layer transformer: output links 130 of 130, input links found by counting 110 of 110, chains 1,430 of 1,430, pairs 1,690 of 1,690, the search 1,560 of 1,560, and pieces move between a plain block and a transformer both ways; the one piece that reaches inside, a table tune learned through the transformer, does not (29 of 110, against 110 of 110 through a plain block) | passed (5/5 rules, 5/5 seeds) |
| 17 | nb17 inside the transformer | the inside of a one-layer transformer is built like a block's: the ring detector finds rings in all 10 trained circuits and none untrained, they are the predicted waves, and a lens that only turns or mirrors those waves (24-32 trained numbers) reuses the frozen circuit: with 96 pairs it averages 95% where the same lens on random planes gets 8% and a free table tune 38%; it never fits a task outside its reach; all 30 links within reach reach 90% | passed (rules 1-5 in 5/5, 4/5, 5/5, 5/5, 5/5 seeds; reach 30 of 30 links) |
passedmixedinconclusive or open
What stands
- Shape carries reuse: the ring lens works only on real rings (nb 5, 9).
- Reuse goes through lenses on frozen circuits; freezing is what makes 96-pair learning possible (nb 5, 9).
- Deciding by shape from a 500-step probe of the output end is accurate, cheap and robust to unseen formulas (nb 7, 8, 11).
- One network runs the loop with the three savings: space, steps, data (nb 6, 8).
- Against a fair shared network, a frozen circuit with a table tune still learns a new task in fewer steps (5/5 seeds) and is ahead with 96 pairs in every seed, by 23-42 points, short of the 25-point bar in two (nb 10b).
- Links are separable pieces: an input link and an output link learned on different tasks compose around a frozen circuit into tasks neither was trained on, and the right chain is found by trying the library on 80 pairs, without names (nb 13).
- Pieces can be found by what they do: an address computed from behaviour puts a piece learned again, and its twin of another kind, at the same place; the address of a chain follows from its pieces' addresses; and a search that looks for one piece at a time finds the chain with one run instead of 336, or says exactly what the library is missing (nb 14).
- Pieces move and stack when they talk through a common code: a link that reads the softmax of a circuit's scores, instead of the raw scores, works behind any circuit and any network and stacks six deep without training; a link on raw scores reads its own circuit's pattern and does neither (nb 15).
- The library does not care what kind of network the circuit is: with a one-layer transformer as the circuit, common-code output links, input links found by counting from the circuit's stored answers, chains, stacks and the search all work, and pieces move between a plain block and a transformer; every such piece stays outside the circuit and names symbols (nb 16).
- Shape carries reuse inside a transformer too: in one-layer transformers trained on the clock tasks the ring detector finds rings from the weights alone (none before training), they are the waves arithmetic predicts, and a lens bound to them reuses the frozen circuit from a fifth of the data, where random planes do nothing and a free table tune needs all the data (nb 17).