Cut it open. Add neurons. Don’t stop it.

The setup

Each cell is one association the network has to hold at the same time: a key in, a value back. The network starts undersized so the budget is visible. Teaching continues the whole time. Nothing is pre-loaded.

Past that budget, adding one more makes the others interfere. Recall of older memories decays. That is the capacity law, live. Grow adds neurons to the running network. The original weights stay in place; new cells weave in; the set re-consolidates.

• Green: that memory is held (recall error below the line on the chart).
• Amber: still in the set, slipping.
• Red: lost to interference. Grow is the move, not reset.

The tick on each cell is the target; the white dot is what the network answers now.

What to watch

1. Add memories one at a time. The first few go green and stay green.
2. Keep adding until cells that were green slip amber, then red. The network is still learning — it just does not have the room.
3. Grow +12. Neuron count steps up on a live network. Watch the red cells go green again. The graft is the event; the recovery is the network using the new room.
4. max|w| stays finite through the add, the saturation, and the graft. Boundedness is not paused for surgery.

Why this is not a normal thing to be able to do

Network size is a design-time decision. Run short of capacity in the field and the fix is to train a bigger model and ship a replacement while the deployed unit waits or comes out of service.

The bound is what makes the surgery survivable. Grafting neurons into a live network injects new pathways into a state that is mid-flight. In an ordinary network nothing constrains where that goes: the perturbation is free to compound as training continues, which is divergence. Here the state and the weights are confined by construction, so a graft can cost recall for a moment — it does — but it cannot destabilize. That is why the old weights can be carried across untouched, bit for bit in the isolated graft, and why the same mechanism can add new input channels to a unit that never stops running.

Scope

Capacity is finite and measured. Extra neurons raise the budget because learning is additive below it. This is not a claim that size is free, or that a graft is instant.

Growth is also not monotonic: each graft perturbs what is already held, so repeatedly growing the moment a cell flickers lands worse than growing once and letting the set re-consolidate. More neurons is not automatically better. The claim is narrower and stranger — that the budget can be raised at all on a running, bounded network, without throwing away what it already holds.