Shipped policy dies. The box keeps learning.

The setup

A cart on a rail, a pole balanced on it, pushes left and right to keep it up. The standard hard-mode control problem: unstable, unforgiving, fails visibly.

A policy was trained until it could hold the pole, then shipped with frozen weights, exactly how a model reaches a product today. Then the world changes underneath it.

Reading the stage

• Left pole (purple): the live network. Same weights at the start, but it keeps learning on the box.
• Right pole (gray): the shipped policy, frozen at deploy, running the same physics. It appears once you change the world.
• The pole turns red as it tilts toward the fall angle, and gray while it resets after a drop.

Live champion and Frozen shipped on the right are survival rates, the share of a run each one stays up. ES gens counts search generations while it is re-learning on the box.

What to watch

1. Nominal. The shipped policy balances comfortably. Nothing to see, and that is the point: this is the state everyone validates before shipping.
2. Change the world. Add a crosswind, or double the pole length. The physics the policy was trained on no longer exists.
3. The frozen policy falls. Watch survival collapse on the gray side. It is not broken; it is correct for a world that is gone. Today the fix is a lab retrain and a firmware push.
4. After three falls the live net starts searching on the device, in the new physics, with no lab and no data upload. Watch ES gens climb and live survival recover past 80% while gray stays down.
5. Nominal puts the original physics back; Reset champion reloads the shipped weights so you can run the full sequence again from a clean deploy.

Scope

Re-learning here is episodic search: the pole falls, the episode resets, the search tries again, and the best candidate is kept. That is a real cost and you can watch it happen; it is not instant, in-place magic. The claim is that the search runs on the device, in the new world, unattended, and that it cannot diverge the entire time it searches.