DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-03-25

val_bpb · 170 of 292 · 6 warnings
findings mentioning this date
| 2026-03-25 | 6 | 22 |
audit summary
{
  "date": "2026-03-25",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 292,
  "render_records": 170,
  "test_rows": 22,
  "warnings": 6
}
141 commits on 2026-03-25
  1. 98556888 05:14:11 parameter-golf-lab Notable Non-Record Submission: 1.1239 BPB - 106.2M Binary Asymmetric U-Net + NeoMuon + 4xrelu²MLP + Smear + Fact Tied Emb + Poly5 Softcap + YaRN2048 + 8192BP…
  2. 98556888 05:14:11 sota_crawler Notable Non-Record Submission: 1.1239 BPB - 106.2M Binary Asymmetric U-Net + NeoMuon + 4xrelu²MLP + Smear + Fact Tied Emb + Poly5 Softcap + YaRN2048 + 8192BP…
  3. 98556888 05:14:11 SOTA_FINAL Notable Non-Record Submission: 1.1239 BPB - 106.2M Binary Asymmetric U-Net + NeoMuon + 4xrelu²MLP + Smear + Fact Tied Emb + Poly5 Softcap + YaRN2048 + 8192BP…
  4. 499d0022 05:17:42 parameter-golf-lab Update README.md
  5. 499d0022 05:17:42 sota_crawler Update README.md
  6. 499d0022 05:17:42 SOTA_FINAL Update README.md
  7. 69bc84ee 05:22:39 parameter-golf-lab Record Submission: 1.1570 BPB - 73.7M Ternary U-Net (10L 768d 8192BPE relu² 4xMLP FP8) (#640)
  8. 69bc84ee 05:22:39 sota_crawler Record Submission: 1.1570 BPB - 73.7M Ternary U-Net (10L 768d 8192BPE relu² 4xMLP FP8) (#640)
  9. 69bc84ee 05:22:39 SOTA_FINAL Record Submission: 1.1570 BPB - 73.7M Ternary U-Net (10L 768d 8192BPE relu² 4xMLP FP8) (#640)
  10. 226d817e 05:23:19 parameter-golf-lab Update README.md
  11. 226d817e 05:23:19 sota_crawler Update README.md
  12. 226d817e 05:23:19 SOTA_FINAL Update README.md
  13. d36bdec2 12:38:52 parameter-golf-lab Podracing: add submission.json, training logs, README
  14. 49e72194 14:19:12 parameter-golf-lab Alpha sweep: eval-only script, no retraining needed
  15. 49e72194 14:19:12 sota_crawler Alpha sweep: eval-only script, no retraining needed
  16. 5fdd78e6 16:01:24 parameter-golf-lab Alpha sweep eval script + SDPA/GQA fallback for non-FA3 GPUs
  17. 5fdd78e6 16:01:24 sota_crawler Alpha sweep eval script + SDPA/GQA fallback for non-FA3 GPUs
  18. 871fb73f 16:29:19 parameter-golf-lab Fix: move GPTQ calibration inside training phase
  19. 871fb73f 16:29:19 sota_crawler Fix: move GPTQ calibration inside training phase
  20. 727e67f7 16:39:12 parameter-golf-lab Multi-order backoff (2-7) + entropy-adaptive alpha + GPTQ fix
  21. 727e67f7 16:39:12 sota_crawler Multi-order backoff (2-7) + entropy-adaptive alpha + GPTQ fix
  22. e372620e 16:48:36 parameter-golf-lab Backoff 7-gram run script + results prep
  23. e372620e 16:48:36 sota_crawler Backoff 7-gram run script + results prep
  24. 0bf0f5bb 17:07:07 parameter-golf-lab Cubric n-gram accumulator: online alpha adaptation from n-gram reliability
  25. 0bf0f5bb 17:07:07 sota_crawler Cubric n-gram accumulator: online alpha adaptation from n-gram reliability
  26. dc57d782 17:12:45 parameter-golf-lab Update README.md
  27. dc57d782 17:12:45 sota_crawler Update README.md
  28. dc57d782 17:12:45 SOTA_FINAL Update README.md
  29. 499a606c 17:13:15 parameter-golf-lab Update README.md
  30. 499a606c 17:13:15 sota_crawler Update README.md
  31. 499a606c 17:13:15 SOTA_FINAL Update README.md
  32. bd48a1eb 17:19:49 parameter-golf-lab Cubric n-gram: full A/B sweep with all eval-time variants + RWKV-style recurrence body prep
  33. bd48a1eb 17:19:49 sota_crawler Cubric n-gram: full A/B sweep with all eval-time variants + RWKV-style recurrence body prep
  34. 48283e2f 17:23:39 parameter-golf-lab RWKV-style recurrence body: neural alpha head — learned n-gram interpolation weight
  35. 48283e2f 17:23:39 sota_crawler RWKV-style recurrence body: neural alpha head — learned n-gram interpolation weight
  36. 9f44bc69 17:34:01 parameter-golf-lab Update README.md
  37. 9f44bc69 17:34:01 sota_crawler Update README.md
  38. 9f44bc69 17:34:01 SOTA_FINAL Update README.md
  39. 0e5b1984 17:35:57 parameter-golf-lab Update README.md
  40. 0e5b1984 17:35:57 sota_crawler Update README.md
  41. 0e5b1984 17:35:57 SOTA_FINAL Update README.md
  42. ed062dfc 17:46:52 parameter-golf-lab Podracing II: Electric Bugaloo — 0.9625 mean (3-seed, all sub-0.964)
  43. ed062dfc 17:46:52 sota_crawler Podracing II: Electric Bugaloo — 0.9625 mean (3-seed, all sub-0.964)
  44. 2c3c10c9 17:52:37 parameter-golf-lab Cubric cadence accumulator: N/N/N/C pattern for n-gram table optimization
  45. 2c3c10c9 17:52:37 sota_crawler Cubric cadence accumulator: N/N/N/C pattern for n-gram table optimization
  46. 150213cd 17:52:41 parameter-golf-lab Podracing II: Electric Bugaloo — 0.9620 BPB (best seed), mean 0.9823
  47. 150213cd 17:52:41 sota_crawler Podracing II: Electric Bugaloo — 0.9620 BPB (best seed), mean 0.9823
  48. cfa5ffe2 17:54:21 parameter-golf-lab Cubric cadence C arm: single run script for cadence=10 balanced test
  49. cfa5ffe2 17:54:21 sota_crawler Cubric cadence C arm: single run script for cadence=10 balanced test
  50. 2973429c 17:57:10 parameter-golf-lab Cubric cadence B arm: single run script for cadence=4 aggressive test
  51. 2973429c 17:57:10 sota_crawler Cubric cadence B arm: single run script for cadence=4 aggressive test
  52. 68b645f8 18:24:31 parameter-golf-lab Cubric cadence C-step: clean surgical patch on exact SOTA (podracing-ii)
  53. 68b645f8 18:24:31 sota_crawler Cubric cadence C-step: clean surgical patch on exact SOTA (podracing-ii)
  54. f26350fb 18:43:16 parameter-golf-lab N-gram parameter sweep: 10 arms, one variable each, 1-GPU Vast
  55. f26350fb 18:43:16 sota_crawler N-gram parameter sweep: 10 arms, one variable each, 1-GPU Vast
  56. 71993870 18:53:12 parameter-golf-lab Add proper run script for cubric test — no more paste issues
  57. 71993870 18:53:12 sota_crawler Add proper run script for cubric test — no more paste issues
  58. 9b6ff565 18:56:51 parameter-golf-lab Fix: patch cubric C-step directly onto real SOTA train_gpt.py
  59. 9b6ff565 18:56:51 sota_crawler Fix: patch cubric C-step directly onto real SOTA train_gpt.py
  60. 4f05f10b 18:57:04 parameter-golf-lab cubric racing variant garage: SOTA train_gpt.py + run.sh × 3 safety copies
  61. 52138d43 18:57:04 parameter-golf-lab cubric racing variant garage: SOTA train_gpt.py + run.sh × 3 safety copies
  62. 4f05f10b 18:57:04 sota_crawler cubric racing variant garage: SOTA train_gpt.py + run.sh × 3 safety copies
  63. 52138d43 18:57:04 sota_crawler cubric racing variant garage: SOTA train_gpt.py + run.sh × 3 safety copies
  64. 6efb7d40 19:08:12 parameter-golf-lab Fix: revert SOTA, cubric on separate copy only
  65. 6efb7d40 19:08:12 sota_crawler Fix: revert SOTA, cubric on separate copy only
  66. db416830 19:14:59 parameter-golf-lab Remove unused TTT eval code (was disabled via TTT_EVAL_ENABLED=0)
  67. db416830 19:14:59 sota_crawler Remove unused TTT eval code (was disabled via TTT_EVAL_ENABLED=0)
  68. 8a591508 19:15:02 parameter-golf-lab Cubric garage: 3 test copies + hypotheses + run scripts
  69. 8a591508 19:15:02 sota_crawler Cubric garage: 3 test copies + hypotheses + run scripts
  70. 37478bac 20:11:47 parameter-golf-lab Add n-gram parameter sweep + cubric lite eval variant
  71. 37478bac 20:11:47 sota_crawler Add n-gram parameter sweep + cubric lite eval variant
  72. 0f912707 20:25:51 parameter-golf-lab Add cubric racing variant safe experiment lane
  73. 0f912707 20:25:51 sota_crawler Add cubric racing variant safe experiment lane
  74. d416488f 20:48:57 parameter-golf-lab Fix corrupted SOTA: restore real PR train_gpt.py (147bbccc)
  75. d416488f 20:48:57 sota_crawler Fix corrupted SOTA: restore real PR train_gpt.py (147bbccc)
  76. 653250b4 20:55:04 parameter-golf-lab Add Vast.ai 8xH100 n-gram sweep script
  77. 653250b4 20:55:04 sota_crawler Add Vast.ai 8xH100 n-gram sweep script
  78. 9dea66a5 21:16:45 parameter-golf-lab cubric racing variant lanes: green (cubric lite), purple (clean experimental), frozen backups
  79. 9dea66a5 21:16:45 sota_crawler cubric racing variant lanes: green (cubric lite), purple (clean experimental), frozen backups
  80. 1aaffbfe 21:20:53 parameter-golf-lab Add cubric racing variant green2 lane (cubric lite copy for iteration)
  81. 1aaffbfe 21:20:53 sota_crawler Add cubric racing variant green2 lane (cubric lite copy for iteration)
  82. 7ef1e153 21:41:11 parameter-golf-lab Racing profiles: green + purple with aggressive n-gram params
  83. 7ef1e153 21:41:11 sota_crawler Racing profiles: green + purple with aggressive n-gram params
  84. c9de92cc 21:55:37 parameter-golf-lab cubric racing variant: add cubric-lite racing profile
  85. c9de92cc 21:55:37 sota_crawler cubric racing variant: add cubric-lite racing profile
  86. 1403acf8 21:59:33 parameter-golf-lab cubric racing variant: add dedicated racing profile run scripts
  87. 1403acf8 21:59:33 sota_crawler cubric racing variant: add dedicated racing profile run scripts
  88. f7f6b217 22:23:46 parameter-golf-lab Podracing III submission prep: cubric lite 0.9362 BPB (seed 2045)
  89. f7f6b217 22:23:46 sota_crawler Podracing III submission prep: cubric lite 0.9362 BPB (seed 2045)
  90. ae124fec 22:54:47 parameter-golf-lab Green2: maxed cubric — floor=0.05, cap=4.0, adapt=1.05/0.95, alpha_max=0.80
  91. ae124fec 22:54:47 sota_crawler Green2: maxed cubric — floor=0.05, cap=4.0, adapt=1.05/0.95, alpha_max=0.80
  92. 50390d60 23:07:53 parameter-golf-lab Non-record: Depth Recurrence in Parameter-Constrained Transformers — What Works, What Doesn't, and Why (#363)
  93. 50390d60 23:07:53 sota_crawler Non-record: Depth Recurrence in Parameter-Constrained Transformers — What Works, What Doesn't, and Why (#363)
  94. 50390d60 23:07:53 SOTA_FINAL Non-record: Depth Recurrence in Parameter-Constrained Transformers — What Works, What Doesn't, and Why (#363)
  95. 556904ad 23:21:08 parameter-golf-lab Podracing III: Cubric Lite — 0.9362 BPB (3-seed mean)
  96. 556904ad 23:21:08 sota_crawler Podracing III: Cubric Lite — 0.9362 BPB (3-seed mean)
  97. 67b952d7 23:21:58 parameter-golf-lab Podracing III: Cubric Lite — 0.9362 mean BPB (3-seed)
  98. 69371cb6 23:39:08 parameter-golf-lab Fix: add cubric_cadence to Hyperparameters class in green + green2
  99. 69371cb6 23:39:08 sota_crawler Fix: add cubric_cadence to Hyperparameters class in green + green2
  100. 804c5603 23:41:11 parameter-golf-lab Fix green2: add COMPILE_FULLGRAPH=0 to match working runs
  101. 804c5603 23:41:11 sota_crawler Fix green2: add COMPILE_FULLGRAPH=0 to match working runs
  102. 3fbdc2a3 00:23:27 parameter-golf-lab cubric n-gram mixer: add pod-ready PR779 test launch script
  103. 3fbdc2a3 00:23:27 sota_crawler cubric n-gram mixer: add pod-ready PR779 test launch script
  104. f458d524 01:08:20 parameter-golf-lab cubric n-gram mixer: shared n-gram tables + cubric — cubric racing variant engine with PR#779 insight
  105. f458d524 01:08:20 sota_crawler cubric n-gram mixer: shared n-gram tables + cubric — cubric racing variant engine with PR#779 insight
  106. e751f41b 01:10:30 parameter-golf-lab Move cubric n-gram mixer to concepts/cubric n-gram mixer (beside cubric racing variant, not inside)
  107. e751f41b 01:10:30 sota_crawler Move cubric n-gram mixer to concepts/cubric n-gram mixer (beside cubric racing variant, not inside)
  108. 697e4cb5 01:25:59 parameter-golf-lab Add cubric n-gram mixer-Red_1 alpha-head variant from cubric racing variant
  109. 697e4cb5 01:25:59 sota_crawler Add cubric n-gram mixer-Red_1 alpha-head variant from cubric racing variant
  110. 193f8eb0 01:49:37 parameter-golf-lab cubric n-gram mixer eval speedup: bincount + batch_seqs 128
  111. 193f8eb0 01:49:37 sota_crawler cubric n-gram mixer eval speedup: bincount + batch_seqs 128
  112. 1655d015 02:08:39 parameter-golf-lab Speed up cubric n-gram mixer Red 1 eval: bincount + batch_seqs 128
  113. 1655d015 02:08:39 sota_crawler Speed up cubric n-gram mixer Red 1 eval: bincount + batch_seqs 128
  114. 86134cd1 02:10:08 parameter-golf-lab cubric n-gram mixer submission: 0.5641 BPB 2-seed mean (new world record)
  115. 86134cd1 02:10:08 sota_crawler cubric n-gram mixer submission: 0.5641 BPB 2-seed mean (new world record)
  116. 77a9267a 02:29:50 parameter-golf-lab cubric n-gram mixer: 0.5644 BPB 3-seed mean — new world record
  117. 77a9267a 02:29:50 sota_crawler cubric n-gram mixer: 0.5644 BPB 3-seed mean — new world record
  118. 699cbdca 02:43:25 parameter-golf-lab cubric n-gram mixer v2: per-order entropy gating + cubric (targeting #798)
  119. 699cbdca 02:43:25 sota_crawler cubric n-gram mixer v2: per-order entropy gating + cubric (targeting #798)
  120. 2976c49e 02:54:16 parameter-golf-lab Add eval-only script for fast 2x2 ablation grid
  121. 2976c49e 02:54:16 sota_crawler Add eval-only script for fast 2x2 ablation grid
  122. 475a62da 02:56:42 parameter-golf-lab Add 2x2 ablation grid script for cubric × per-order centers
  123. 475a62da 02:56:42 sota_crawler Add 2x2 ablation grid script for cubric × per-order centers
  124. 4adcdc6a 03:13:20 parameter-golf-lab cubric n-gram mixer Brown: shared tables + per-order entropy gating, no cubric
  125. 4adcdc6a 03:13:20 sota_crawler cubric n-gram mixer Brown: shared tables + per-order entropy gating, no cubric
  126. 390af98b 03:29:14 parameter-golf-lab cubric n-gram mixer Fast: safe speed boosts, no quality loss
  127. 390af98b 03:29:14 sota_crawler cubric n-gram mixer Fast: safe speed boosts, no quality loss
  128. 383ab3a9 03:34:33 parameter-golf-lab cubric n-gram mixer Brown II: brown per-order gating + safe speed boosts
  129. 383ab3a9 03:34:33 sota_crawler cubric n-gram mixer Brown II: brown per-order gating + safe speed boosts
  130. 13c4bcdc 03:47:09 parameter-golf-lab cubric n-gram mixer Yellow: 2D cubric (order × entropy_bin) pattern recognizer
  131. 13c4bcdc 03:47:09 sota_crawler cubric n-gram mixer Yellow: 2D cubric (order × entropy_bin) pattern recognizer
  132. 8c709f13 03:54:10 parameter-golf-lab cubric n-gram mixer Yellow II: 3D cubric monster + orders 2-9
  133. 8c709f13 03:54:10 sota_crawler cubric n-gram mixer Yellow II: 3D cubric monster + orders 2-9
  134. 68436559 03:57:20 parameter-golf-lab cubric n-gram mixer Yellow II: 3D cubric + complementary training — THE MONSTER
  135. 68436559 03:57:20 sota_crawler cubric n-gram mixer Yellow II: 3D cubric + complementary training — THE MONSTER
  136. f3eb3d41 04:04:34 parameter-golf-lab cubric n-gram mixer Yellow III: warm-start cubric at proven converged values
  137. f3eb3d41 04:04:34 sota_crawler cubric n-gram mixer Yellow III: warm-start cubric at proven converged values
  138. 290a2913 04:18:11 parameter-golf-lab cubric n-gram mixer Yellow IV: UNCHARTED — ceiling 2.5 + 16M buckets + orders 2-10
  139. 290a2913 04:18:11 sota_crawler cubric n-gram mixer Yellow IV: UNCHARTED — ceiling 2.5 + 16M buckets + orders 2-10
  140. 659b8b2e 04:22:15 parameter-golf-lab cubric n-gram mixer Yellow II submission: 0.4896 BPB — 3D cubric + complementary training
  141. 659b8b2e 04:22:15 sota_crawler cubric n-gram mixer Yellow II submission: 0.4896 BPB — 3D cubric + complementary training
DATE2026-03-25
DAY008
PRIMARY METRICVAL_BPB
TESTS RUN22 unique (15 valid / 6 TTT leakage)
ARCHITECTURE FAMILYLEADERBOARD TRANSFORMER VARIANTS (×14) · INSTRUCTED CRAWLER RECURRENCE FAMILY (×3) · OTHER (×3) · CUBRIC RACING VARIANT (×2)
CLASSIFICATIONEXPERIMENTAL 82% (18)/NEURAL 18% (4)
ARCHITECTURE RECIPE
INSTRUCTED CRAWLER RECURRENCE FAMILYx3

Crawler-oriented recurrence experiments that tested instruction-conditioned loop behavior and DeltaNet-adjacent crawler variants.

BODYshared crawler recurrence path · content-derived loop instructions · crawler loop rope scaling · DeltaNet crawler recurrence variants
TEST SURFACEGB10 micro probes · H100/runpod follow-up logs · commit-level architecture entries · crawler branch retained as the strongest follow-on direction

SECTION 1.0 — TRAJECTORY

VAL_BPB for 2026-03-25 panel 1 of 2
csv data
VAL_BPB for 2026-03-25 panel 2 of 2
csv data
VAL_BPB SPREADnminmedianmaxdownload
LEADERBOARD TRANSFORMER VARIANTS131.11901.11961.1217—
CUBRIC RACING VARIANT21.12001.12011.1202—
ALL VALID151.11901.11971.1217—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 18

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
100:318×H100 (runpod)f1_car02_t1_ve_spread_s1337_20260325_003137.log1.1195OK↑4.9%
training_log
200:508×H100 (runpod)f1_car02_t2_rope24_s1337_20260325_005049.log1.1192OK↑4.9%
training_log
301:098×H100 (runpod)f1_car02_t3_xsa3_speed_s1337_20260325_010936.log1.1203OK↑5.0%
training_log
401:298×H100 (runpod)f1_car02_iso_var_t2_rope24_ngram5_s1337_20260325_012927.log1.1194OK↑4.9%
training_log
501:538×H100 (runpod)f1_car02_iso_var_t2_rope24_ngram5_s1337_20260325_015359.log1.1197OK↑4.9%
training_log
702:298×H100 (runpod)f1_car02_iso_var_t2_rope24_ngram5_s1337_20260325_022915.log1.1192OK↑4.9%
training_log
802:29—cubric n-gram mixer: 0.5644 BPB 3-seed mean — new world record (×2)0.5644INVALID—
details
902:508×H100 (runpod)f1_car02_iso_var_t2_rope24_ngram5_s69_20260325_025034.log4.1050CRASH—
training_log
1002:568×H100f1_car02_iso_var_t2_rope24_ngram5_s1337_20260325_025620.log (×2)1.1190OK↑4.9%
training_log 1
training_log 2
1103:138×H100f1_car02_iso_var_t2_rope24_ngram5_s42_20260325_031357.log (×2)1.1217OK↑5.1%
training_log 1
training_log 2
1203:318×H100f1_car02_iso_var_t2_rope24_ngram5_s2045_20260325_033133.log1.1200OK↑4.9%
training_log
1416:458×H100f1_car02_iso_var_t2_rope24_ngram5_s1337_20260325_164500.log (×2)1.1195OK↑4.9%
training_log 1
training_log 2
1517:048×H100f1_car02_iso_backoff_7gram_adaptive_s42_20260325_170429.log (×2)1.1210OK↑5.0%
training_log 1
training_log 2
1617:228×H100f1_car02_iso_backoff_7gram_adaptive_s2045_20260325_172241.log1.1196OK↑4.9%
training_log
1717:428×H100f1_car02_iso_backoff_7gram_adaptive_s7_20260325_174208.log (×2)1.1202OK↑5.0%
training_log 1
training_log 2
1817:52—Podracing II: Electric Bugaloo — 0.9620 BPB (best seed), mean 0.9823 (×2)0.9620INVALID—
details
2022:418×H100 (runpod)cubric racing variant.log1.1200OK↑4.9%
training_log
2122:598×H100 (runpod)cubric racing variant.log1.1202OK↑5.0%
training_log

NEURAL TEST RUNS — 4

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
602:10—cubric n-gram mixer submission: 0.5641 BPB 2-seed mean (new world record) (×2)0.5641INVALID—
details
1304:22—cubric n-gram mixer Yellow II submission: 0.4896 BPB — 3D cubric + complement…0.4896INVALID—
details
1922:23—Podracing III submission prep: cubric lite 0.9362 BPB (seed 2045) (×2)0.9362INVALID—
details
2223:21—Podracing III: Cubric Lite — 0.9362 BPB (3-seed mean) (×2)0.9362INVALID—
details