2026-04-15 — Research Day

DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-04-15

val_bpb · 670 of 2186 · 0 warnings

audit summary

{
  "date": "2026-04-15",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 2186,
  "render_records": 670,
  "test_rows": 38,
  "warnings": 0
}

41 commits on 2026-04-15

1522aeda 06:39:16 sota_crawler rhythm recurrence: add 6F+3C+battery gate (run_6_7) and MoE crawler variant
1522aeda 06:39:16 SOTA_FINAL rhythm recurrence: add 6F+3C+battery gate (run_6_7) and MoE crawler variant
f3240c75 15:01:21 sota_crawler Add pod helpers for 12L eval fix
f3240c75 15:01:21 SOTA_FINAL Add pod helpers for 12L eval fix
44c87aa9 15:26:51 sota_crawler Guard looping-only eval fix in 12L leg
44c87aa9 15:26:51 SOTA_FINAL Guard looping-only eval fix in 12L leg
d2fda302 15:45:58 sota_crawler Add canonical rhythm recurrence run launcher
d2fda302 15:45:58 SOTA_FINAL Add canonical rhythm recurrence run launcher
1c038e1c 15:52:18 sota_crawler Rename rhythm recurrence launcher to 6f3c path
1c038e1c 15:52:18 SOTA_FINAL Rename rhythm recurrence launcher to 6f3c path
fe3fe450 16:15:16 sota_crawler Remove ramp from 6f3c launcher
fe3fe450 16:15:16 SOTA_FINAL Remove ramp from 6f3c launcher
b345452a 16:35:50 sota_crawler Add flat 8F 8k launcher and save 6f3c run note
b345452a 16:35:50 SOTA_FINAL Add flat 8F 8k launcher and save 6f3c run note
212a9d49 16:57:21 sota_crawler Save flat 8F 8k run record
212a9d49 16:57:21 SOTA_FINAL Save flat 8F 8k run record
08ffcddf 17:21:33 sota_crawler Add locked 12L GQA mixed-bit tower build and compress legs
08ffcddf 17:21:33 SOTA_FINAL Add locked 12L GQA mixed-bit tower build and compress legs
f7da96fc 17:27:23 sota_crawler Add litecrawler MoE path
f7da96fc 17:27:23 SOTA_FINAL Add litecrawler MoE path
a9573b27 17:30:17 parameter-golf-lab Non-record: Mamba-3 Hybrid SSM + SP8192 + Legal TTT — 1.1473 bpb
4ea40a6d 17:30:56 sota_crawler Add plain litecrawler launcher
4ea40a6d 17:30:56 SOTA_FINAL Add plain litecrawler launcher
a9dae360 17:32:29 sota_crawler Record 12L GQA mixed-bit tower compression loop-state fix
a9dae360 17:32:29 SOTA_FINAL Record 12L GQA mixed-bit tower compression loop-state fix
94e7bdd0 17:46:05 sota_crawler Fix litecrawler slot OOM
94e7bdd0 17:46:05 SOTA_FINAL Fix litecrawler slot OOM
e5ac4b82 18:11:40 sota_crawler Save litecrawler 8F run record
e5ac4b82 18:11:40 SOTA_FINAL Save litecrawler 8F run record
be7b7114 19:24:26 sota_crawler Add local-trunk litecrawler path
be7b7114 19:24:26 SOTA_FINAL Add local-trunk litecrawler path
ce26f830 20:07:51 sota_crawler Add CPU cold memory litecrawler path
ce26f830 20:07:51 SOTA_FINAL Add CPU cold memory litecrawler path
c58a07b1 20:08:59 sota_crawler Make CPU cold litecrawler prefetch non-blocking
c58a07b1 20:08:59 SOTA_FINAL Make CPU cold litecrawler prefetch non-blocking
bf4cb02f 21:13:32 sota_crawler Add parallel-residual QK-gain litecrawler experiment
bf4cb02f 21:13:32 SOTA_FINAL Add parallel-residual QK-gain litecrawler experiment
8c54a796 21:55:41 sota_crawler Add 12-layer flat 8k launcher
8c54a796 21:55:41 SOTA_FINAL Add 12-layer flat 8k launcher
54f6ccd8 03:43:10 parameter-golf-lab Reframe DG Attention as informative negative result with real H100 data
151dcb54 03:46:00 parameter-golf-lab Drop negative result subtitle from paper title

DATE2026-04-15

DAY029

PRIMARY METRICVAL_BPB

TESTS RUN38 unique (36 valid / 0 TTT leakage)

ARCHITECTURE FAMILY12L GQA MIXED-BIT TOWER (×29) · LEADERBOARD TRANSFORMER VARIANTS (×4) · OTHER (×4) · CRAWLER-SAFE TRAIN/EVAL SURFACE (×1)

CLASSIFICATIONEXPERIMENTAL 3% (1)/NEURAL 97% (37)

ARCHITECTURE RECIPE

12L GQA MIXED-BIT TOWERx29

12-layer GQA tower: 8/4 GQA, RoPE-16, Bigram 2048, XSA on the last 11 layers, and mixed-bit packaging.

BODYnum_layers=12 · num_heads=8 · num_kv_heads=4 · rope_dims=16 · xsa_last_n=11 · BigramHash vocab=2048

QUANT / EVALattention int5 · mlp int6 · aux int6 · embed int8 · other int8 · Brotli artifact · 16 MB artifact cap

RUN SHAPE8xH100 · train_seq_len=2048 · eval_seq_len=2048 · 600s wallclock

CRAWLER-SAFE TRAIN/EVAL SURFACEx1

Crawler-safe training/eval surface around a widened shared-crawler stack with final int6 and int8 checks.

BODYvocab_size=1024 · model_dim=512 · num_heads=8 · num_kv_heads=4 · shared crawler block · looped crawler passes

FEATURESBigramHash · XSA · RoPE · crawler loop rope scales · instruction bottleneck

QUANT / EVALint6 roundtrip · int6 sliding-window eval · int8 zlib roundtrip · 8xH100 · 600s wallclock

VAL_BPB SPREAD	n	min	median	max	download
12L GQA MIXED-BIT TOWER	28	1.1041	1.1055	1.5224	download
LEADERBOARD TRANSFORMER VARIANTS	3	1.1123	1.1123	1.1188	—
OTHER	4	1.1242	1.1323	1.1403	—
CRAWLER-SAFE TRAIN/EVAL SURFACE	1	2.8923	2.8923	2.8923	download
ALL VALID	36	1.1041	1.1063	2.8923	—

VAL_BPB SPREAD

min

median

max

download

12L GQA MIXED-BIT TOWER

1.1041

1.1055

1.5224

download

LEADERBOARD TRANSFORMER VARIANTS

1.1123

1.1188

—

OTHER

1.1242

1.1323

1.1403

—

CRAWLER-SAFE TRAIN/EVAL SURFACE

2.8923

download

ALL VALID

1.1041

1.1063

2.8923

—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 1

#	TIME	HW	ABLATION	VAL_BPB	STATUS	Δ %	OPEN
1	06:39	1×GB10	train_seed4.log	2.8923	OK	↑171.0%	training_log

TIME

ABLATION

VAL_BPB

STATUS

Δ %

OPEN

06:39

1×GB10

train_seed4.log

2.8923

↑171.0%

training_log

NEURAL TEST RUNS — 37

#	TIME	HW	ABLATION	VAL_BPB	STATUS	Δ %	OPEN
2	10:27	8×H100	relaunch_seed444_20260415_102725.log	1.1242	OK	↑5.3%	training_log

3	15:12	8×H100	full_seed444_20260415_151238.log	1.1403	OK	↑6.8%	training_log

4	15:12	8×H100	launcher_seed444_retry_20260415_151238.log	1.1403	OK	↑6.8%	training_log

5	15:27	8×H100	full_seed444_20260415_152725.log	1.1242	OK	↑5.3%	training_log

6	15:49	8×H100	launcher_seed444_20260415_154949.log	1.5224	OK	↑42.7%	training_log

7	15:49	8×H100	full_seed444_20260415_154950.log	1.5224	OK	↑42.7%	training_log

8	17:03	8×H100	launcher_seed444_20260415_170333.log	1.1063	OK	↑3.7%	training_log

9	17:03	8×H100	full_seed444_20260415_170333.log	1.1063	OK	↑3.7%	training_log

10	17:17	8×H100	launcher_seed444_20260415_171733.log	1.4901	OK	↑39.6%	training_log

11	17:17	8×H100	full_seed444_20260415_171733.log	1.4901	OK	↑39.6%	training_log

12	17:26	8×H100	launcher_seed444_20260415_172614.log	1.1051	OK	↑3.5%	training_log

13	17:26	8×H100	full_seed444_20260415_172614.log	1.1051	OK	↑3.5%	training_log

14	17:34	8×H100	launcher_seed444_20260415_173420.log	1.1199	OK	↑4.9%	training_log

15	17:34	8×H100	full_seed444_20260415_173420.log	1.1199	OK	↑4.9%	training_log

16	18:11	8×H100	full_seed444_20260415_181112.log (×3)	1.1055	OK	↑3.6%	training_log 1 training_log 2 training_log 3

17	18:47	8×H100	full_seed444_20260415_184736.log (×2)	3.4840	CRASH	—	training_log 1 training_log 2

18	18:57	8×H100	full_seed444_20260415_185730.log	1.1055	OK	↑3.6%	training_log

19	19:09	8×H100	full_seed444_20260415_190910.log	1.1063	OK	↑3.7%	training_log

20	19:21	8×H100	full_seed444_20260415_192130.log	1.1061	OK	↑3.6%	training_log

21	19:36	8×H100	full_seed444_20260415_193620.log	1.1049	OK	↑3.5%	training_log

22	19:37	8×H100	full_seed444_20260415_193740.log	1.1049	OK	↑3.5%	training_log

23	19:38	8×H100	full_seed444_20260415_193821.log	1.1046	OK	↑3.5%	training_log

24	19:42	8×H100	full_seed444_20260415_194201.log	1.1196	OK	↑4.9%	training_log

25	19:45	8×H100	full_seed444_20260415_194521.log	1.1505	OK	↑7.8%	training_log

26	19:48	8×H100	full_seed444_20260415_194838.log	1.1046	OK	↑3.5%	training_log

27	20:09	1×GB10	full_seed444_20260415_200921.log	1.1049	OK	↑3.5%	training_log

28	20:23	8×H100	full_seed444_20260415_202318.log	1.1055	OK	↑3.6%	training_log

29	20:26	8×H100	full_seed444_20260415_202622.log	1.1046	OK	↑3.5%	training_log

30	20:29	8×H100	full_seed444_20260415_202929.log	1.1055	OK	↑3.6%	training_log

31	20:42	8×H100	full_seed444_20260415_204217.log	1.1068	OK	↑3.7%	training_log

32	20:59	8×H100	full_seed444_20260415_205927.log	1.1049	OK	↑3.5%	training_log

33	21:03	8×H100	full_seed444_20260415_210338.log	1.1049	OK	↑3.5%	training_log

34	21:08	8×H100	full_seed444_20260415_210825.log	1.1041	OK	↑3.5%	training_log

35	22:22	8×H100 (lambda)	launcher_pr1541_20260415_222252.log	1.1123	OK	↑4.2%	training_log

36	22:22	8×H100	full_seed444_20260415_222252.log	1.1123	OK	↑4.2%	training_log

37	22:36	8×H100	full_seed444_20260415_223609.log	1.1188	OK	↑4.8%	training_log

38	22:47	8×H100	full_seed444_20260415_224757.log	3.4831	CRASH	—	training_log

TIME

ABLATION

VAL_BPB

STATUS

Δ %

OPEN