Sleep, and the 81% that survived

Catastrophic forgetting is the oldest embarrassment in neural networks. A model learns something, learns a second thing on top of it, and the second lesson quietly overwrites the first. The standard workaround is to never stop showing the model everything at once, which is a way of avoiding the problem rather than solving it. Brains do not get that luxury. They learn in sequence, all day, and they keep the important parts. The trick is that they stop and sleep.

Mind-1B is built to sleep on purpose. Its memory is not a growing cache of the past; it is written into fast weights during waking, gated by surprise, so the model captures the moments its predictions broke rather than the whole predictable stream. That fast store is deliberately small and a little messy by the end of a day. Sleep is where it gets cleaned.

What a night actually does

The offline phase is three moves, each one lifted from a real thing the sleeping brain is understood to do.

1. Replay. The most surprising episodes of the day are played back through the network, the same way hippocampal replay reactivates the day's salient traces during slow-wave sleep. Surprise is the priority: what broke a prediction is what gets rehearsed.
2. Homeostatic downscale. The overall synaptic gain is turned down, which lifts the signal of the rehearsed, important connections above the noise of everything that was written and did not matter. This is synaptic homeostasis: sleep as a global renormalization.
3. Prune. The weakest connections are driven to zero. Because the core is ternary, a pruned synapse is not a special sparse case to bookkeep; it is just the native 0 state. Pruning literally recovers capacity for free.

Then the model wakes up with the day's lesson burned into its ternary base and the noise gone. We ran that loop for eight wake and sleep cycles on a memory task and measured how much of the earliest material was still recallable at the end.

34% to 81% is not a tuning gain, it is a different regime. Without sleep, the model was on the familiar slide: each new cycle ate into the last, and by the eighth the early memories were mostly gone. With sleep, the early memories were still there. The consolidation loop was the entire difference between the two runs.

Wake and sleep have to be different states

One subtlety made this work, and it is worth naming because it is easy to get wrong. Wake and sleep cannot be the same dynamics with a flag flipped. Consolidation needs the network to be in a genuinely different mode: quieter, more internally driven, replaying rather than responding. In the prototype the two states settle into anticorrelated attractors, with a measured correlation of -0.74 between the wake and sleep activity patterns. They are close to opposites, which is exactly what you want. A model that consolidates in the same state it perceives in tends to overwrite the thing it is trying to protect.

The novel contribution is the memory lifecycle: capture what surprised you awake, and decide what to keep while asleep.

The honest edge of this result

This is gate M2 on the roadmap, and it passed, in numpy, at toy scale. That is precisely as far as it goes. Proving that a replay-and-prune loop beats no loop on a small memory task is not the same as proving it holds inside a 1B model trained on 300B tokens, and the whitepaper is blunt about that gap: the full brain-like assembly has not yet been shown to match a dense model at this scale. What we have shown is that the mechanism is real and that it earns its place on the ladder.

The next rung asks more of it. Train a real model on task A, then B, then C, and require sleep to retain the old tasks materially better than no sleep, on the two-B300 box rather than in numpy. If it holds there, the most speculative piece of the whole design, a language model that sleeps to remember, stops being a bet and becomes a component. If it does not, that failure gets a note too, with what it taught.

Either way, the model is going to sleep tonight.