Research into how models actually learn — architecture, training, memory — and an attempt to build something that learns like a child. Draw an architecture out of tensor primitives, compile it straight to the GPU, and watch the neurons light up.
Most people use models. This is about understanding them — from the tensor up. If you can draw an architecture, compile it, and race it against another, you build real intuition for why one design wins and another doesn't.
A tiny transformer’s neurons placed on a sunflower (Fibonacci) sphere, lighting up as it reads a sentence. Turning the vectors inside a model into a picture you can actually watch.
A deterministic, human-owned loop that keeps a model learning: it trains, tests itself, rolls back mistakes, prunes memory, and sleeps to consolidate — while the loop itself can never be rewritten by the model. Status: the train-and-test loop runs today; rollback, memory pruning and sleep-consolidation are designed and not all running yet.
Attention drawn by hand from three matmuls and a softmax, raced against the same network without it. On matched compute, attention won — val loss 2.40 vs 2.50. A real result from a hand-drawn network.
Papers pulled from arXiv, rebuilt as runnable architectures and raced against a matched transformer baseline at two scales. Published every week, negative results included.
Every comparison holds these fixed: parameter count matched within a stated tolerance, tokens seen rather than steps, same tokenizer and same train/val split, three seeds with the spread reported. A run that cannot meet these is reported as failed rather than compared. Wall-clock and FLOPs-per-token are logged alongside each result — matched parameters is not matched FLOPs, and the FLOPs column shows which. These are fixed-budget results, not converged ones.
The first table publishes this week.
In progress. The pieces below are being assembled into one system. Some are running, some are designed and not yet built.
The pieces are being assembled into one living system — a “brain” and its organs. The rule that keeps it safe: three kinds of memory (the shared model weights, private per-user memory, and short-term working memory), and only the human-owned Control Plane ever writes to the shared weights.