Learn the systems path · code-adjacent

B32 systems

Microbatch 32 × accum 16 · fixed 524,288 tokens/update · throughput, not a new algorithm

B32×accum16 is a batching choice on the claim recipe. Same tokens per optimizer update as the older B16 path. Larger microbatch, fewer accumulation steps. Matched systems_v1 screen: ~6% lower core step-time than B16 with tokens/update fixed. That is component evidence for the recipe, not a sealed ΔT%. Not a new optimizer. Not Ortho/NS. Not split-QKV.

systems path 524,288 tok/update ~6% step-time (banked) claim-stack ingredient

NUMBERS_CANON RECIPE_CARD GLOSSARY · B32×accum16 COMPREHENSIVE_REPORT

Problem 00 · scope systems path

Systems vs algorithm

Ortho/NS lives on Muon. Timing-cheap split lives on architecture. B32×accum16 only changes how tokens are packed into one optimizer update.

Predict · Does B32 train on more tokens per update?

No. Tokens/update stay 524,288. Only the factors change (microbatch ↑, accum ↓). Same product → fair comparison of step wallclock.

Definition

One optimizer update sees a fixed token count. Microbatch size × accumulation steps × sequence length = that count. Call that product the effective batch in tokens (not a new algorithm).

Claim pin Tokens per update

tokens/update = microbatch × accum × seq
32 × 16 × 1024 = 524,288
(B16 path: 16 × 32 × 1024 = same 524,288)

Recipe pin: see RECIPE_CARD and NUMBERS_CANON. Horizon stays 4768 updates.

Problem 01 · fixed budget same tokens

What stays constant?

The fair comparison locks tokens per update. You do not “train on more data per step” by moving to B32. You change how those tokens are packed on the GPU.

Visual · One update = one fixed token block Our commentary

Figure · Token budget is the invariant. Step-time is what the systems path tries to move.

Problem 02 · reshape B16 → B32

Same product, different factors

Older path: microbatch 16 × accum 32. Claim path: microbatch 32 × accum 16. Product identical. Fewer, larger microbatches → different GPU occupancy and step wallclock.

Packing steps Auto / Step / Reset

Lock the product

Effective tokens/update must match, or the “faster step” comparison is confounded.

16 × 32 × 1024 = 32 × 16 × 1024 = 524,288

Measure step wallclock

Same tokens, different packing → different core step-time. That is the systems measurement — not a new loss or Ortho.

Validated screen: ~6% lower core step-time (systems_v1) · component evidence ≠ seal

Beat 1 · Product locked. One optimizer update always sees 524,288 tokens — microbatch × accum × seq length.

microbatch —

accum steps —

fwd/bwd passes —

tokens/update 524,288

Step through product → factors → throughput

Keyboard (focus controls): Space or → step · A auto · R reset · keys 1–3 jump beats

Beats: product → factors → throughput. Caption + readouts sync to each beat (microbatch, accum, passes, tokens/update); step-time is schematic (bank ≠ seal).

Problem 03 · banked gain ~6% step-time

What was measured?

Matched campaign screen systems_v1 compared B32 vs B16 with tokens/update fixed. Result banked into the recipe: ~6% faster core step-time. That is component evidence. It is not the sealed multi-seed first-passage ΔT% on the report.

Visual · Step-time bar (schematic) Our commentary

Figure · Illustrates the validated systems delta. Sources: GLOSSARY, CAMPAIGN_OVERVIEW, and COMPREHENSIVE_REPORT.

Claim boundary: B32×accum16 is a systems path (microbatch × accumulation, tokens/update fixed) with matched campaign step-time evidence. It is not a paper novelty claim or Muon Ortho. Systems component evidence is not the sealed percentage. Do not read ~6% core step-time as the sealed mean ΔT% ≈ +8.3% vs F1_seal (3.866 h). Sealed ΔT% is for Muon B32 + timing-cheap split-QKV on the report / NUMBERS_CANON. Under our sealed F1 baseline. Public % vs F1_seal only. Matched cand-vs-stock ΔT% = N/A. Optimizer attribution: Keller Jordan Muon · ecosystem: Modded-NanoGPT.

Problem 04 · claim stack ingredient, not solo

Where B32 sits

Screen candidates, then validate components, then seal. Plain Muon compose failed or lost under the seal. The B32 systems path and timing-cheap split-QKV met their stated component criteria before the full-stack run. Only then: three-seed candidate seals.

1 · Optimizer

Keller Jordan Muon

Ortho/NS on matrices. Muon explained.

2 · Systems

B32 × accum16

This page. Throughput packing.

3 · Architecture

Timing-cheap split-QKV

Validated component · ≠ Ortho.

Process order: failed finals stayed in the ledger; a fair clock was fixed; plain Muon compose missed; systems B32 and timing-cheap split-QKV were validated as components; defaults stayed NS 5 / z-loss 0; then claim-stack seals ran 3/3 vs F1_seal. Detail: screen → validate → seal.

Systems_v1 also excluded NS3 and z-loss 1e-4 on the same path family. Defaults that stayed: NS 5, z-loss 0.