2026-04-09 — Research Day

DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-04-09

val_bpb · 136 of 1273 · 0 warnings

audit summary

{
  "date": "2026-04-09",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 1273,
  "render_records": 136,
  "test_rows": 23,
  "warnings": 0
}

73 commits on 2026-04-09

81972ab5 05:57:19 parameter-golf-lab long-context causal tap mechanism v1.9 adaptive TTT ablation lane
81972ab5 05:57:19 sota_crawler long-context causal tap mechanism v1.9 adaptive TTT ablation lane
81972ab5 05:57:19 SOTA_FINAL long-context causal tap mechanism v1.9 adaptive TTT ablation lane
857de475 07:17:55 parameter-golf-lab Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT — val_bpb 1.0810 (3-seed mean)
857de475 07:17:55 sota_crawler Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT — val_bpb 1.0810 (3-seed mean)
857de475 07:17:55 SOTA_FINAL Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT — val_bpb 1.0810 (3-seed mean)
3fdaaa56 16:59:39 sota_nueral Record all valid session results + stack scripts + corrected ratio sweep
e90539f3 17:27:50 parameter-golf-lab Crawler layer relationship ablation: 5x4 grid sweep (flat 5-9 × crawler 1-4)
e90539f3 17:27:50 sota_crawler Crawler layer relationship ablation: 5x4 grid sweep (flat 5-9 × crawler 1-4)
e90539f3 17:27:50 SOTA_FINAL Crawler layer relationship ablation: 5x4 grid sweep (flat 5-9 × crawler 1-4)
94491303 18:27:41 sota_nueral Save all raw pod logs from 4xH100 session (2026-04-09)
f136b6aa 19:33:20 parameter-golf-lab Crawler symmetry ablation: test C=loops hypothesis (3×3, 4×4, 6×6, 8×8)
f136b6aa 19:33:20 sota_crawler Crawler symmetry ablation: test C=loops hypothesis (3×3, 4×4, 6×6, 8×8)
f136b6aa 19:33:20 SOTA_FINAL Crawler symmetry ablation: test C=loops hypothesis (3×3, 4×4, 6×6, 8×8)
2c045f76 19:44:07 parameter-golf-lab Drop redundant control from symmetry ablation — grid already has 8F+3C
2c045f76 19:44:07 sota_crawler Drop redundant control from symmetry ablation — grid already has 8F+3C
2c045f76 19:44:07 SOTA_FINAL Drop redundant control from symmetry ablation — grid already has 8F+3C
c4846a39 20:08:28 parameter-golf-lab Symmetry ablation: 4x4 only — 6x6 and 8x8 impractical at 785ms/step
c4846a39 20:08:28 sota_crawler Symmetry ablation: 4x4 only — 6x6 and 8x8 impractical at 785ms/step
c4846a39 20:08:28 SOTA_FINAL Symmetry ablation: 4x4 only — 6x6 and 8x8 impractical at 785ms/step
75e07222 21:01:34 parameter-golf-lab multi-crawler descendant 1: 8F+3C production gate (two-variable diff from BWX 9F)
75e07222 21:01:34 sota_crawler multi-crawler descendant 1: 8F+3C production gate (two-variable diff from BWX 9F)
75e07222 21:01:34 SOTA_FINAL multi-crawler descendant 1: 8F+3C production gate (two-variable diff from BWX 9F)
22c7e07f 21:05:10 parameter-golf-lab TK1 gate: safe (3 loops) vs aggressive (4 loops diff battery) on 4xGPU 2k steps
22c7e07f 21:05:10 sota_crawler TK1 gate: safe (3 loops) vs aggressive (4 loops diff battery) on 4xGPU 2k steps
22c7e07f 21:05:10 SOTA_FINAL TK1 gate: safe (3 loops) vs aggressive (4 loops diff battery) on 4xGPU 2k steps
dd2166e1 21:13:22 parameter-golf-lab TK1 gate: switch to 600s wallclock (matches competition 10-min budget)
dd2166e1 21:13:22 sota_crawler TK1 gate: switch to 600s wallclock (matches competition 10-min budget)
dd2166e1 21:13:22 SOTA_FINAL TK1 gate: switch to 600s wallclock (matches competition 10-min budget)
29ff99e9 21:13:49 parameter-golf-lab TK1 gate: 300s wallclock for faster screening on 4xGPU
29ff99e9 21:13:49 sota_crawler TK1 gate: 300s wallclock for faster screening on 4xGPU
29ff99e9 21:13:49 SOTA_FINAL TK1 gate: 300s wallclock for faster screening on 4xGPU
81b6bd7a 21:14:54 parameter-golf-lab Update README leaderboard for April records
81b6bd7a 21:14:54 sota_crawler Update README leaderboard for April records
81b6bd7a 21:14:54 SOTA_FINAL Update README leaderboard for April records
eadf2aae 22:25:58 parameter-golf-lab TK1: add Brotli compression packaging compression packaging compression (default, quality=11) — zstd artifact was 17.95MB
eadf2aae 22:25:58 sota_crawler TK1: add Brotli compression packaging compression packaging compression (default, quality=11) — zstd artifact was 17.95MB
eadf2aae 22:25:58 SOTA_FINAL TK1: add Brotli compression packaging compression packaging compression (default, quality=11) — zstd artifact was 17.95MB
868f0785 22:36:24 parameter-golf-lab Update SCIENCE.md and TK1 ablation with production gate results
868f0785 22:36:24 sota_crawler Update SCIENCE.md and TK1 ablation with production gate results
868f0785 22:36:24 SOTA_FINAL Update SCIENCE.md and TK1 ablation with production gate results
45b27848 23:16:18 parameter-golf-lab long-context causal tap mechanism v1.9.1 stabilize adaptive TTT and drift-lite arms
45b27848 23:16:18 sota_crawler long-context causal tap mechanism v1.9.1 stabilize adaptive TTT and drift-lite arms
45b27848 23:16:18 SOTA_FINAL long-context causal tap mechanism v1.9.1 stabilize adaptive TTT and drift-lite arms
59109e30 23:37:05 parameter-golf-lab feat(records): SP8192 + Muon 0.97 + Legal TTT + Causal N-gram Tilt — val_bpb 1.07983
85b332cd 02:02:09 sota_nueral Add 12L GQA mixed-bit tower (safe) and 12L GQA mixed-bit tower (aggressive) legs
848244cc 02:18:07 parameter-golf-lab BW_9F2C: 9F+2C production gate — single-variable NUM_CRAWLER_LAYERS=2
848244cc 02:18:07 sota_crawler BW_9F2C: 9F+2C production gate — single-variable NUM_CRAWLER_LAYERS=2
848244cc 02:18:07 SOTA_FINAL BW_9F2C: 9F+2C production gate — single-variable NUM_CRAWLER_LAYERS=2
885cf584 02:19:16 parameter-golf-lab BW_9F2C: add Brotli compression packaging compression packaging compression (default, quality=11)
885cf584 02:19:16 sota_crawler BW_9F2C: add Brotli compression packaging compression packaging compression (default, quality=11)
885cf584 02:19:16 SOTA_FINAL BW_9F2C: add Brotli compression packaging compression packaging compression (default, quality=11)
b7c794c4 02:31:53 parameter-golf-lab Add 12L GQA mixed-bit tower (safe) and 12L GQA mixed-bit tower (aggressive) for 8xH100 run
b7c794c4 02:31:53 sota_crawler Add 12L GQA mixed-bit tower (safe) and 12L GQA mixed-bit tower (aggressive) for 8xH100 run
b7c794c4 02:31:53 SOTA_FINAL Add 12L GQA mixed-bit tower (safe) and 12L GQA mixed-bit tower (aggressive) for 8xH100 run
8966f96b 02:48:22 parameter-golf-lab BW_9F2C: downgrade FA3 preflight from fatal to warning
8966f96b 02:48:22 sota_crawler BW_9F2C: downgrade FA3 preflight from fatal to warning
8966f96b 02:48:22 SOTA_FINAL BW_9F2C: downgrade FA3 preflight from fatal to warning
1ede5637 02:52:02 parameter-golf-lab BW_9F2C: FA3 preflight warns instead of exits — SDPA fallback is in train_gpt.py
1ede5637 02:52:02 sota_crawler BW_9F2C: FA3 preflight warns instead of exits — SDPA fallback is in train_gpt.py
1ede5637 02:52:02 SOTA_FINAL BW_9F2C: FA3 preflight warns instead of exits — SDPA fallback is in train_gpt.py
9912ff54 03:33:13 sota_nueral Add run_12L GQA mixed-bit tower.sh — launch any experiment with 12L GQA mixed-bit tower 12L env vars
8f3dcd86 03:34:23 parameter-golf-lab Add run_12L GQA mixed-bit tower.sh — 12L GQA mixed-bit tower 12L env vars for any experiment
8f3dcd86 03:34:23 sota_crawler Add run_12L GQA mixed-bit tower.sh — 12L GQA mixed-bit tower 12L env vars for any experiment
8f3dcd86 03:34:23 SOTA_FINAL Add run_12L GQA mixed-bit tower.sh — 12L GQA mixed-bit tower 12L env vars for any experiment
2a83653c 03:41:27 parameter-golf-lab Fix 12L GQA mixed-bit tower 12L defaults in GPTQ and Black experiments
2a83653c 03:41:27 sota_crawler Fix 12L GQA mixed-bit tower 12L defaults in GPTQ and Black experiments
2a83653c 03:41:27 SOTA_FINAL Fix 12L GQA mixed-bit tower 12L defaults in GPTQ and Black experiments
471d793e 03:41:40 sota_nueral Fix vault defaults to match 12L GQA mixed-bit tower 12L actual config
da1c5352 04:07:40 sota_nueral Fix vault defaults: COMPRESSOR=Brotli compression packaging compression packaging, COPRIME_MAX_LOADED_SHARDS=4
7ab8d0c7 04:07:47 parameter-golf-lab Fix GPTQ+Black: COMPRESSOR=Brotli compression packaging compression packaging, WARMDOWN=3500, cache=4
7ab8d0c7 04:07:47 sota_crawler Fix GPTQ+Black: COMPRESSOR=Brotli compression packaging compression packaging, WARMDOWN=3500, cache=4
7ab8d0c7 04:07:47 SOTA_FINAL Fix GPTQ+Black: COMPRESSOR=Brotli compression packaging compression packaging, WARMDOWN=3500, cache=4

DATE2026-04-09

DAY023

PRIMARY METRICVAL_BPB

TESTS RUN23 unique (9 valid / 0 TTT leakage)

ARCHITECTURE FAMILY12L GQA MIXED-BIT TOWER (×14) · CRAWLER-SAFE TRAIN/EVAL SURFACE (×9)

CLASSIFICATIONEXPERIMENTAL 39% (9)/NEURAL 61% (14)

ARCHITECTURE RECIPE

12L GQA MIXED-BIT TOWERx14

12-layer GQA tower: 8/4 GQA, RoPE-16, Bigram 2048, XSA on the last 11 layers, and mixed-bit packaging.

BODYnum_layers=12 · num_heads=8 · num_kv_heads=4 · rope_dims=16 · xsa_last_n=11 · BigramHash vocab=2048

QUANT / EVALattention int5 · mlp int6 · aux int6 · embed int8 · other int8 · Brotli artifact · 16 MB artifact cap

RUN SHAPE8xH100 · train_seq_len=2048 · eval_seq_len=2048 · 600s wallclock

CRAWLER-SAFE TRAIN/EVAL SURFACEx9

Crawler-safe training/eval surface around a widened shared-crawler stack with final int6 and int8 checks.

BODYvocab_size=1024 · model_dim=512 · num_heads=8 · num_kv_heads=4 · shared crawler block · looped crawler passes

FEATURESBigramHash · XSA · RoPE · crawler loop rope scales · instruction bottleneck

QUANT / EVALint6 roundtrip · int6 sliding-window eval · int8 zlib roundtrip · 8xH100 · 600s wallclock

VAL_BPB SPREAD	n	min	median	max	download
CRAWLER-SAFE TRAIN/EVAL SURFACE	9	1.3177	1.3277	1.3310	download
ALL VALID	9	1.3177	1.3277	1.3310	—

VAL_BPB SPREAD

min

median

max

download

CRAWLER-SAFE TRAIN/EVAL SURFACE

1.3177

1.3277

1.3310

download

ALL VALID

1.3177

1.3277

1.3310

—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 9

#	TIME	HW	ABLATION	VAL_BPB	STATUS	Δ %	OPEN
13	02:44	4×H100	A00_screen_s444_20260409_024421.log	1.3300	OK	↑24.6%	training_log

14	02:44	4×H100	A06_screen_s444_20260409_024421.log	1.3310	OK	↑24.7%	training_log

15	02:44	4×H100	A08_screen_s444_20260409_024421.log	1.3292	OK	↑24.5%	training_log

16	02:44	4×H100	A05_screen_s444_20260409_024421.log	1.3260	OK	↑24.2%	training_log

17	02:44	4×H100	A07_screen_s444_20260409_024421.log	1.3177	OK	↑23.5%	training_log

18	02:44	4×H100	A04_screen_s444_20260409_024421.log	1.3253	OK	↑24.2%	training_log

19	02:44	4×H100	A03_screen_s444_20260409_024421.log	1.3260	OK	↑24.3%	training_log

20	02:44	4×H100	A01_screen_s444_20260409_024421.log	1.3288	OK	↑24.5%	training_log

21	02:44	4×H100	A02_screen_s444_20260409_024421.log	1.3277	OK	↑24.4%	training_log

TIME

ABLATION

VAL_BPB

STATUS

Δ %

OPEN