Topological reuse · Figures 2–5
Four figures from the trained networks and the results files: the output end of a circuit as a field of waves, the fifteen tasks arriving at one network, the price of each way of deciding, and every adapter placed by its size and what it learns from 96 examples.
This is the table the readers in notebooks 7, 8 and 11 look at: how strongly each hidden neuron votes for each answer. In the add circuit every neuron’s votes make a wave over the answers in counting order; in the mul circuit the same picture appears only when the answers are read in the order 1, 3, 9, 27, 19, … Reorder the answers and one circuit’s waves turn to noise while the other’s appear.
Scrub the training back to step 500. The network still gets 0% of unseen pairs right, yet the top wave of each neuron already carries 50% of its spectrum along the right order against 21% along the wrong one (22% is the noise floor at step 0). That is what a 500-step probe gives the reader: the arrangement is legible at the output end long before the circuit can answer. By step 2,000 the share is 85%; the finished circuit sits at 95%.
Notebook 11’s stream, as the network with the reader in the loop lived it. Each task arrives with 480 examples; a 500-step probe is read at the output end; the reader names a circuit or none; the ladder tries a ring lens, then table tune, on the named circuit; otherwise the probe keeps training and joins the library as a new block of 42,975 numbers.
The upper line is what fifteen separate networks would store. The lower line is the network: it climbs by a block when something new arrives (add, mul, the unlearnable cubesum, diffsq, maxshift, minshift, mix, sqsum) and barely moves when a lens will do (shift, mulscale, subshift on their rings; scale, cubeadd, mulshift and sqadd through table tune). It ends at 56% of the separate networks, with every learnable task at 90% or better. The one visible miss is the last arrival: sqsum could have reused diffsq’s circuit, and the reader did not know that circuit.
Left: the steps spent deciding for one new task, against how many circuits the network already has. Every dot is one task in one seed. Reading the output end costs one 500-step probe whatever the library holds; trying every circuit grows with the library, and with seven circuits it costs about seven times more. The ring gauge at 2,000 steps is a flat line too, four times higher.
Right: what each decider got wrong on the thirteen new tasks, and the lens steps its wrong calls wasted, five seeds each. The reader’s one wrong call per seed is always the same missed reuse (sqsum); it never sent a task to a circuit that could not serve it. Trying every circuit makes no mistakes and pays for it in steps. The ring gauge read at 500 steps misses most relabelings because the rings have not formed yet. Guessing wastes five times the lens steps the reader does.
Notebook 9: ten adapters on the right frozen circuit, each tuned on its own check pairs, ten fresh seeds. Left, the turn tasks (shift, subshift, mulscale), where the new task is a turn or a mirror of every ring. The ring lens, 490–840 stored numbers of which 56–96 are trained, learns them from 96 pairs; the same lens on random planes of the same size learns nothing; the smallest general adapter that does both turn tasks is three times larger.
Right, the folding tasks (scale, cubeadd, sqadd, mulshift), where the relabeling is not a turn. Here the ring lens fails as it should, and retuning the circuit’s own number tables (table tune, 3,968 numbers) beats the full 64 × 64 lens with twice the numbers. Retraining everything, 42,975 numbers, collapses with 96 pairs though it learns fine with 480: freezing the circuit is what makes learning from little data possible.