Marilyn

status

active · ongoing

when

2026-07 → now

came from

shellm, Research corpus, SnapDragon-Coder

led to

LiLM / Lucky Labs, Bench lane, Paper ingestion & arXiv weekly

A language model trained from random weights on my own hardware, to study why a model that has memorised something still fails to use it.

Why the name

She is named after Marilyn vos Savant, the columnist who answered readers’ hardest questions for decades and whose most famous answer, on the Monty Hall problem, was right when thousands of letters, many from people with doctorates, insisted she was wrong. The point of the name is not the record-book IQ. It is that she showed her reasoning, held her answer under pressure, and was vindicated by the arithmetic. In the lab’s own notes the rule for an open question is written as “run the experiment (the Marilyn-vos-Savant way)”.¹

Why it started

shellm proved a from-scratch model could be trained here, and that its base and corpus were far too small to be useful. The next question was stranger. A small model was re-schooled on study material, demonstrably absorbed it, and then could not route it to answer a question: it deflected. The literature calls this the Knowing–Using Gap (the paper is on Reading that shaped it). That gap is the spine of the project.

What it is

Marilyn begins as random weights. Nothing is distilled from a larger model; when she knows something, she learned it here, and I can point at when. Around her:

• a corpus of about 4.3 billion tokens, assembled and gated (research corpus);
• a nursery: a serve-and-learn service with tools, a checkpoint ring, and a promotion gate that decides whether a new checkpoint is better and safe;
• pre-registered proofs for vocabulary size, weight tying, epochs and model size, each settled by a factorial sweep with seed-pair noise floors before a decision was recorded;
• Temporal workflows for every multi-hour run, so a reboot is an event, not a loss;
• Mantis, a native memory taken from the Metis paper and bolted on as the cheapest possible disproof first. Details on the v5 programme;
• a post-training programme for target behaviour rather than facts: calling a calculator instead of guessing, looking things up instead of confabulating, asking for help when out of depth. Taught in the nursery by a local teacher model over an escalating curriculum of knowledge, reasoning and skills, graded by a sandbox, with reinforcement from verifiable rewards where imitation hit its ceiling. The pictures are on Architecture.

Versions: v2 (fed about a seventh of what sizing rules ask for, which taught the sizing lesson), v3 (506M parameters on 4.8B tokens, 0.8575 bits per byte), v4, and v5, a bigger trunk in three variants that are parked as of September 2026 awaiting LiLM’s answer.²

Who does what

I design the experiments, write the pre-registrations, set the gates and review the results. Agents run the builds, the sweeps and the harvests under those gates, and they are not allowed to promote anything. When a page here says “measured”, the measurement was made by a workflow I can re-run and the verdict was mine.

Where it stands

The post-training programme (calculator tool, look-up tool, a tensor lane, a knowing-gap diagnostic) is built and its test suites are green; the exit gates are not yet run. The question in the charter is still open. The lab repositories are private; the numbers on these pages are quoted from their records.³

Lineage

• Came from: shellm, research corpus, the RLVR traces of SnapDragon-Coder
• Led to: LiLM (knowledge outside the weights), bench lane, paper ingestion

Sub-pages

• Architecture: corpus to checkpoint, the nursery, and one teaching cycle with a local teacher
• The v5 programme: single trunk, MoE+, MoE+Mantis, and why all three are parked
• Reading that shaped it: the three papers, what was taken, what was found
• Wins · Losses · Rabbit holes