Reading that shaped it
Ideas here were taken from published work. Each entry says what was taken and what I found when I tried it, so the people who published the idea get the credit and the reader gets the result.
- Metis: Memory Foundation Model, Zeyu Zhang and colleagues, MemTensor, arXiv:2607.26760, 2026-07-29. Taken: the native-memory mechanism (their equations 2, 3 and 5) that became Mantis, including initialising the memory projections from the backbone’s own key/value matrices. Found: untrained, it is signal not capability; the +1 stabiliser is unsafe before training; their 14,000-step mid-training on a 4B model is the price of the rest.
- Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning, Lu Dai and colleagues, arXiv:2607.08393, 2026-07-09. Taken: the Knowing–Using Gap as the project’s spine, and self-patching as a diagnostic to separate capacity from circuit misalignment. Found: it described v2’s classroom failure exactly; the v5 diagnostic lane exists to run their test on a properly-fed model. It is one paper, and the lab treats its account as a hypothesis to test, not a settled literature.
- MemoHarness: Agent Harnesses That Learn from Experience, Yue Huang and colleagues, arXiv:2607.14159, 2026-07-14. Taken: memory that lives in the harness rather than the weights, as the tractable half of the memory question. Found: it is the design behind the nursery’s experience bank; the same idea turned inward is Mantis.
Not on this list, because it did not ship in code: the compute-optimal scaling literature that informs everyone’s tokens-per-parameter intuitions. v5’s sizing came from an in-house study of composition versus depth and width on my own runs, and is reported that way.