Recipe card — claim-stack v1
Exact candidate recipe, stock control definition, and matched-pair meaning.
Primary sources:
results/claim_stack_v1/ ↗
(PREREG.md,
MULTI_SEED_CLAIM.md,
seed summaries) ·
reproduce notes ↗.
What is being claimed
Candidate stack (not Muon alone; not joint stock):
Keller Jordan Muon + B32 systems path + timing-cheap split-QKV, on three seeds, first-crosses
val_disjoint ≤ 3.3821about 8% faster than sealed stock AdamW F1_seal (T=3.866 h).
Do not claim matched first-cross speedup vs joint stock.
Hardware / software pin
| Knob | Value |
|---|---|
| GPU | NVIDIA A100-SXM4-40GB exact (80GB rejected by guard; seed 4242 relaunched) |
| Torch | 2.13.0 (resolved 2.13.0+cu130) |
| Metric T | train_time at first logged disjoint val ≤ 3.3821 |
| Horizon | 4768 updates |
| Tokens / update | 524,288 (B32 × accum16 × seq 1024) |
Shared recipe (both claim-stack arms)
| Knob | Value |
|---|---|
| Batch | microbatch 32 × accumulation 16 |
| Optimizer (Muon groups) | lr 0.02 · momentum 0.95 · WD 0.1 |
| Adam groups (embed/head) | lr 0.0018 · WD 0.1 |
| Newton–Schulz steps | 5 |
| Warmup / warmdown | 476 / 1192 |
| z-loss | 0 |
| EMA | off |
| Data | raw FineWeb shards + official val |
| Prior | frozen prior_full.pt |
| Parallel block | on |
| single_process_fastpath | on |
| val every / save every | 128 / 256 |
| Trainer | train_gpt2_split_qkv_cheap_v1.py ↗ — site kit/ · TRAINER_DELTA.html (PREREG SHA freeze) · CLI batch knobs in train_gpt2_t4.py ↗ |
Attribution: Muon → Keller Jordan. Timing-cheap split-QKV → this campaign’s banked mechanism.
Candidate vs stock control (claim-stack)
| Stock control | Candidate | |
|---|---|---|
| QKV | joint (muon_split_qkv=0) | timing-cheap split (muon_split_qkv=1) |
| match_joint | 1 (joint path) | 0 |
| Everything else | identical | identical |
Stock here is joint-QKV Muon stack, not AdamW F1.
F1_seal / F1_Claim-W2 are separate AdamW denominators for under-baseline ΔT%.
Denominators (AdamW)
| Label | Account | Recipe | First-cross T |
|---|---|---|---|
| F1_seal | Clock-W | stock AdamW · raw FineWeb · zloss 0 · no prior/EMA | 3.866 h @4736 |
| F1_Claim-W2 | Claim-W2 | same-class AdamW re-seal | 3.858 h @4768 |
Public headline ΔT% uses F1_seal. Seed 2029 may also report vs F1_Claim-W2 (+7.5%).
Matched-pair meaning
A matched pair = same seed, same account, same pin, same shared recipe, differing only on split-QKV treatment, both arms run to (or through) the first-cross event.
| Seed | Pair? | Notes |
|---|---|---|
| 1337 | Yes (Claim-W1) | Stock never crossed; cand HIT |
| 2029 | Yes (Claim-W2) | Stock never crossed; cand HIT |
| 4242 | No | Candidate-only robustness seed; no stock arm |
Primary endpoint in PREREG was matched ΔT% vs stock. Observed reality: stock quality-missed → that endpoint is N/A. Claim language uses candidate vs F1_seal instead.
ΔT%_vs F1 = 100 × (1 - (T_candidate) / (T_F1_seal))
Pass bar used: candidate first-cross and ΔT% vs F1 ≥ 6%. Sealed: 3/3 seeds (+8.1%, +7.7%, +9.2%).
Sealed candidate results (quick)
| Seed | Cross step | val @ cross | T | vs F1_seal | Terminal CE |
|---|---|---|---|---|---|
| 1337 | 4736 | 3.3741 | 3.553 h | +8.1% | 3.3710 |
| 2029 | 4736 | 3.3726 | 3.567 h | +7.7% | 3.3695 |
| 4242 | 4736 | 3.3733 | 3.511 h | +9.2% | 3.3702 |
Exact ms / more decimals: see seed summaries and MULTI_SEED_CLAIM.md.