Research ledger
The claim, the command, and the limit.
This page separates the model we intend to test from the evidence already collected. The architecture section is a specification, not a result. Each evidence card names the exact script, default setup, result, and the thing it does not establish.
Revision 6 // design under test
00The current model, stage by stage
Mind-1B is not six speculative subsystems switched on at once. It is one recurrent spine with a promotion ladder. The first trainable reference is deliberately plain: a bilateral GDN2 core in bf16, fast and slow state/head banks, and a deterministic four-slot workspace. Each later row changes one controlled variable.
| Stage | Component entering | Required comparison | Status |
|---|---|---|---|
| M3a | 18-block bilateral GDN2, fast/slow clocks, deterministic four-slot workspace, bf16 | Single-stream GDN2, 3:1 GDN2/attention hybrid, and Transformer++ at matched parameters and active compute | next run |
| M3b | Progressive ternary QAT with shadow weights | The winning M3a checkpoint before quantization | blocked by M3a |
| M3c | Causal workspace audit | Scrambled, delayed, bypassed, and capacity-matched workspace controls | blocked by M3a |
| M4a | Precision Workspace: belief mean and diagonal uncertainty per slot; predictive compute budget | Deterministic workspace at equal parameters and active compute | specified, untrained |
| M5a | Causal event-boundary head plus bounded entorhinal/DG/CA3/CA1-style episodic path | Fixed chunks, turn-only boundaries, and flat retrieval at the same byte budget | specified, untrained |
| M5b | Scheduled replay with frozen base weights; plastic sleep and REM-like integration only as later ablations | No replay, matched online replay, and replay-only controls | specified, untrained |
| M6 | Scale only the components that passed | Published checkpoint, matched control suite, quality, latency, memory, calibration, and failure table | blocked by M3–M5 |
What “bilateral” means
It does not mean two independent models or a left-brain/right-brain personality split. A shared GDN2 spine maintains two timescales: a fast precision stream for local ordered updates and a slower context stream for integration. Both can propose a write; neither owns the workspace. A matched single-stream model decides whether the extra structure earns its parameters.
What the Precision Workspace adds
M3a slots are ordinary deterministic vectors. M4a may give each slot a belief mean and diagonal variance, then update it with a bounded Kalman-style gain. Reliable evidence can revise an uncertain slot; noisy evidence should be discounted. The module ships only if proper scoring, calibration, state tracking, and language loss improve—or writes/compute fall at matched loss. It is inspired by Bayesian filtering in Kalman Linear Attention; it does not replace GDN2 as the sequence mixer.
What event memory and sleep add
M5a closes an episode only after a causal boundary detector sees the event's final token. A bounded record carries a temporal key, compressed summary, terminal belief and uncertainty, source span, and confidence. The design lesson comes from event-segmented hierarchical memory. M5b then tests scheduled replay into a protected candidate memory before permitting base-weight plasticity. Wake serving never silently rewrites the deployed base model.
Limit. Revision 6 is an executable research specification, not a trained checkpoint. GDN2 itself has published evidence; the bilateral clocks, workspace, event path, and sleep lifecycle do not yet have language-model results in this repository.
Read the full Revision 6 paper / Gated DeltaNet-2 paper / GDN2 reference code / workspace interpretability study
M0 // character-model toy
01Ternary QAT
Run: python3 mind-1b/proto/toy_ternary_lm.py --steps 4000
Limit. This trains on the text of its own source file. It establishes that this quantization recipe can converge in a tiny deterministic setting; it does not establish language-model parity at useful scale.
M3 scout // operation-count check
02Surprise gate
4.14e-6, not mathematically zero.Run: python3 mind-1b/proto/surprise_gating.py
| Redundancy | Threshold | MAC ratio | Relative RMS error |
|---|---|---|---|
| 0.85 | 0.00 | 16.7% | 0.000007 |
| 0.85 | 0.10 | 15.5% | 0.014681 |
| 0.85 | 0.40 | 12.2% | 0.115351 |
| 0.95 | 0.00 | 7.7% | 0.000004 |
| 0.95 | 0.10 | 7.1% | 0.016392 |
| 0.95 | 0.40 | 5.2% | 0.121315 |
Limit. This counts operations for an exact delta-coded linear matvec on a generated stream. It does not measure language-model quality, kernel latency, or training savings.
retained scout // synthetic routing rule
03Phase-routing curiosity
Run: python3 mind-1b/proto/phase_binding.py
| Input kind | Active regions | Relative error |
|---|---|---|
| Simple, one domain | 1.65 | 0.257 |
| Composite, two domains | 2.37 | 0.466 |
| Dense all-region baseline | 8.00 (4.2× compute) | 0.914 |
Limit. The script seeds each region's prototype and expert matrix from the synthetic task's ground-truth centers and rules. It tests a routing rule, not learned specialization or a fair trained-router comparison. Phase routing is not in M3a and earns no flagship claim; if revisited, it must beat a learned router at equal active compute.
M2 // synthetic continual-memory check
04Sleep cleanup
Run: python3 mind-1b/proto/sleep_cleanup.py
| Cycle | With cleanup | Naive control |
|---|---|---|
| 1 | 89.2% | 49.8% |
| 2 | 88.5% | 44.1% |
| 4 | 85.2% | 39.1% |
| 6 | 82.3% | 35.6% |
| 8 | 81.4% | 34.4% |
Limit. Neither arm is a language model. The control is a naive raw-magnitude retention strategy, not an otherwise-identical model "without sleep." This result earns a larger experiment; it does not solve catastrophic forgetting.
Configuration calculator
05Memory budget
Run: python3 mind-1b/calc/params.py and python3 mind-1b/calc/memory_budget.py
| Measure | Value | Scope |
|---|---|---|
| Total parameters | 1.035B | Tied embeddings, 18 GDN2 blocks, workspace, router estimate, and MTP transforms |
| Non-embedding parameters | 969.02M | Arithmetic planning count |
| Training estimate | 14.83 GB/GPU | QAT planning envelope, batch 8, sequence 2,048, checkpointing; excludes runtime overhead |
| Inference minimum | 0.37 GB | Packed core plus fp16 embeddings; excludes unmeasured GDN2 state and future episodic memory |
Limit. These are arithmetic planning estimates, not measured device memory or throughput. The calculator currently sets recurrent-state and hippocampal-memory bytes to zero and warns about both, so 0.37 GB is a floor—not a deployment budget.
Parameter calculator / Memory calculator / Design specification
What earns the next claim
06The gates still open
The next material claim needs a small bf16 GDN2 model that reproduces recurrence correctly, followed by the 100M M3a control ladder: single-stream GDN2, bilateral GDN2, a 3:1 GDN2/attention hybrid, and Transformer++. The first H200 reservation is a ten-hour iteration window for environment validation and smoke runs, not a promise to finish pretraining. Held-out language quality, peak memory, tokens per second, stability, matched active compute, and failed ablations all ship together.
Phase routing, sparse MoE, MatFormer widths, multi-token prediction, custom kernels, and additional brain analogies remain outside the reference model until a simpler stage exposes a measured need. That is how the roadmap stays a living status board instead of a list of features.