Projects

What survived every pivot

Thirteen projects, four architectures deleted, and six things that outlived all of them: every long job is a Temporal workflow, so a reboot is an event and not a loss; every gate ships with the attack that must make it fail; every card is held by a lease; every backup is proved by a restore that runs weekly; every model assignment is a bench result, not a preference; and every number on this site is re-runnable. Those six are the product. The models were how I found them.

The lab runs when I am not at it: the nightly backup, the weekly restore drill, the paper ingestion and every training run are Temporal schedules, so the work that takes hours is not work I sit through. What I spend time on is deciding what to measure and reading what came back.

Every project has an overview, and the ones with enough history have three pages behind it. Wins say what worked and what was measured. Losses say what it cost and what it taught. Rabbit holes say where the detour went: a dead end with a reason, or the next project. The losses are the ones to read.

The lab has had three big turns. Each one started as the answer to the previous one’s failure.

Turn	Question	Project	Answer
1	Can a large open model be retrained into a better Go coder?	SnapDragon-Coder	Yes, to a point. Fine-tuning a quantised base left it handicapped. The quantisation was the villain.
2	Can a small model be trained from scratch to be a shell expert?	shellm	It beat gzip. Base and corpus were far too small to be useful.
3	Why does a model that has memorised something still fail to use it?	Marilyn	Open. This is the current work.

Before the turns, five months of agents. Rabbit Holing tells it on one page; the projects are:

First public repos: a 22-model Go benchmark, local-model tooling, Temporal patterns.
Agent Zone: infrastructure knowledge for humans and agents, live at agent-zone.ai.
Dream Team: an organisation of agent pods over Mattermost and an ActiveMQ bus. Merged 0 of 10 real items; deleted 2026-07-06.
Greenfield DevOps platform: its Temporal successor. 40% yield, zero false merges, paused for training.

One decision cuts across every lane: Local or cloud, how a model call is routed and why, from Bifrost profiles to the LiteLLM gateway, with the bench-lane hooks that keep the assignments honest.

Around the three turns sit the lanes that make them possible:

Fleet & operations: the machines, the workflows, the backups, the drills.
Research corpus: 4.3 billion tokens, assembled and gated.
Paper ingestion & arXiv weekly: the literature pipeline that feeds Marilyn’s reviews.
Bench lane: measuring quantisation deltas and serving stacks honestly.
LiLM / Lucky Labs: can a model learn, self-learn and be patched without retraining?
Team enablement: the work-side thread, TL;DR only.

Names you will meet

Marilyn: the from-scratch language model. Versions v2 through v5.
Mantis: Marilyn’s native memory, a mechanism taken from the Metis paper.
MoE+: a shared trunk plus domain experts that read from it; one of three v5 variants.
The nursery: the serve-and-learn service Marilyn lives in, with tools and a promotion gate.
shellm: the earlier byte-level shell model, written in Rust.
SnapDragon-Coder: a fine-tuned Go coder built from a large open model.
LiLM / Lucky Labs: knowledge outside the weights, and the hobby brand for its product face.
carlo, monte, markov, monaco: the machines, by role: control plane, trainer and server, nursery box, the Mac.
RLVR: reinforcement learning from verifiable rewards, where a sandbox grades the answer.
bits per byte (bpb): the loss metric used throughout, comparable across tokenizers; lower is better.
tokens per parameter: how well fed a model is; v2 had about 1.5, v3 about ten, v5 about 43.
δ (delta), as in δ_seed or δ_gate: a noise floor measured before a comparison is made, by re-running with different seeds or bootstrapping over a probe set; a difference smaller than δ is reported as “cannot tell”, never as a win.
nats: the natural-log unit of a loss or a binding strength; like bits, but base e.
arm-C, arm-D: labels for training lineages in the corpus era, before versions were numbered.

Lineage graph

team-enablement ──► early-experiments ──► agent-zone ──► dream-team ──► greenfield-platform
                            │                  │              │                 │
                            │                  │              ▼                 ▼
                            └──────────────────┴────────► snapdragon-coder ──► shellm ──► marilyn ──► lilm
                                                              │                              ▲  │
                                                              └── fleet-ops ─────────────────┘  └── bench-lane
research-corpus ──► marilyn        paper-ingestion ──► marilyn        agent-zone ──► lilm

Status vocabulary

active is being worked on · paused deliberately shelved, restorable · closed a question answered · archived kept for lineage only.

The lab repositories are private. The public ones are on GitHub.