DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-04-11

val_bpb · 174 of 572 · 2 warnings
findings mentioning this date
| 2026-04-11 | 2 | 3 |
audit summary
{
  "date": "2026-04-11",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 572,
  "render_records": 174,
  "test_rows": 3,
  "warnings": 2
}
89 commits on 2026-04-11
  1. 3393a21d 05:04:11 sota_nueral Reorganize corpus around tables
  2. cb47a4dc 05:13:41 sota_nueral Update CORPUS_CRAWLER.md
  3. 8414a329 05:15:03 sota_nueral Update CORPUS_CRAWLER.md
  4. 12503d50 05:16:28 sota_nueral Update CORPUS_CRAWLER.md
  5. e3f221b3 05:17:14 sota_nueral Update CORPUS_CRAWLER.md
  6. 8de7738b 05:48:26 sota_nueral Update CORPUS_CRAWLER.md
  7. 27ed93d1 05:48:48 sota_nueral Update CORPUS_CRAWLER.md
  8. 9d0df67b 05:53:56 sota_nueral Update CORPUS_CRAWLER.md
  9. b3db2ca9 05:54:41 sota_nueral Update CORPUS_CRAWLER.md
  10. 681602f5 05:56:34 sota_nueral Update CORPUS_CRAWLER.md
  11. 7ee6af8d 05:57:26 sota_nueral Update CORPUS_CRAWLER.md
  12. c20419d1 05:58:10 sota_nueral Update CORPUS_CRAWLER.md
  13. 80b79600 05:59:15 sota_nueral Update CORPUS_CRAWLER.md
  14. 514ddec1 06:03:07 parameter-golf-lab Add multi-crawler descendant submission — 1.0742 BPB (4h), 1.1354 BPB (10min), 14.2MB
  15. 514ddec1 06:03:07 sota_crawler Add multi-crawler descendant submission — 1.0742 BPB (4h), 1.1354 BPB (10min), 14.2MB
  16. 514ddec1 06:03:07 SOTA_FINAL Add multi-crawler descendant submission — 1.0742 BPB (4h), 1.1354 BPB (10min), 14.2MB
  17. fbbc6156 06:06:00 sota_nueral Update CORPUS_CRAWLER.md
  18. ebbb3b42 06:10:15 sota_nueral Update CORPUS_CRAWLER.md
  19. 05013959 06:10:30 sota_nueral Fix typo in 'dunning kruger' to 'dunning krugers'
  20. 2aa12464 06:12:37 sota_nueral Add loop-instructed DeltaNet crawler body history to crawler corpus
  21. 78e70eb1 06:12:37 sota_nueral Add loop-instructed DeltaNet crawler body history to crawler corpus
  22. 41144bb6 06:14:48 sota_nueral Update CORPUS_CRAWLER.md
  23. 4d8e4cce 06:19:26 sota_nueral Update CORPUS_CRAWLER.md
  24. f29f2376 06:23:53 sota_nueral Tighten corpus grammar and phrasing
  25. 43e36e1b 06:31:04 parameter-golf-lab Update README.md
  26. 253ad2f0 15:37:51 sota_nueral Vault 12L GQA mixed-bit tower III base + update all orientation docs
  27. 7f0415c7 16:40:54 sota_nueral Cache sweep: COPRIME_MAX_LOADED_SHARDS 4/8/12/24 on 12L GQA mixed-bit tower III
  28. bc6f37e1 16:49:28 parameter-golf-lab Cache sweep: COPRIME_MAX_LOADED_SHARDS 4/8/12/24 on 12L GQA mixed-bit tower III
  29. bc6f37e1 16:49:28 sota_crawler Cache sweep: COPRIME_MAX_LOADED_SHARDS 4/8/12/24 on 12L GQA mixed-bit tower III
  30. bc6f37e1 16:49:28 SOTA_FINAL Cache sweep: COPRIME_MAX_LOADED_SHARDS 4/8/12/24 on 12L GQA mixed-bit tower III
  31. fc949e81 17:14:38 parameter-golf-lab Fix REPO_ROOT path in cache sweep run scripts
  32. fc949e81 17:14:38 sota_crawler Fix REPO_ROOT path in cache sweep run scripts
  33. fc949e81 17:14:38 SOTA_FINAL Fix REPO_ROOT path in cache sweep run scripts
  34. 161d6442 17:48:35 parameter-golf-lab update w/ improvements+logs
  35. 28d1bee5 17:48:40 parameter-golf-lab Add cache=80 arm (full dataset in RAM) to cache sweep
  36. 28d1bee5 17:48:40 sota_crawler Add cache=80 arm (full dataset in RAM) to cache sweep
  37. 28d1bee5 17:48:40 SOTA_FINAL Add cache=80 arm (full dataset in RAM) to cache sweep
  38. 91fde872 20:25:25 sota_nueral Cache sweep results: cache=80 chosen, monotonic speedup, zero quality loss
  39. 3ac78d0e 20:50:32 parameter-golf-lab Add variance ratio measurement script for paper experiments
  40. d695fde9 00:05:55 parameter-golf-lab Fix variance ratio script: clamp tokens to vocab size
  41. 0c9b0fcb 01:10:16 parameter-golf-lab Update README.md
  42. 938e4f8a 01:41:15 sota_nueral Gate legs: iii_lean (no loops) + iii_loop (GPTQ enabled) on 12L GQA mixed-bit tower III base
  43. 61398d37 01:44:49 parameter-golf-lab Gate legs: iii_lean (no loops) + iii_loop (GPTQ enabled) on 12L GQA mixed-bit tower III base
  44. 61398d37 01:44:49 sota_crawler Gate legs: iii_lean (no loops) + iii_loop (GPTQ enabled) on 12L GQA mixed-bit tower III base
  45. 61398d37 01:44:49 SOTA_FINAL Gate legs: iii_lean (no loops) + iii_loop (GPTQ enabled) on 12L GQA mixed-bit tower III base
  46. 5baa7730 01:54:13 parameter-golf-lab Fix run.sh: add PYTHONPATH for flash-attention/hopper (FA3 kernels)
  47. 5baa7730 01:54:13 sota_crawler Fix run.sh: add PYTHONPATH for flash-attention/hopper (FA3 kernels)
  48. 5baa7730 01:54:13 SOTA_FINAL Fix run.sh: add PYTHONPATH for flash-attention/hopper (FA3 kernels)
  49. 4f7530c7 02:16:09 sota_nueral Fix GPTQ plumbing for bank architecture: F.linear interception + per-slice quantization
  50. 01e6d1a5 02:16:25 parameter-golf-lab Fix GPTQ plumbing for bank architecture: F.linear interception + per-slice quantization
  51. 01e6d1a5 02:16:25 sota_crawler Fix GPTQ plumbing for bank architecture: F.linear interception + per-slice quantization
  52. 01e6d1a5 02:16:25 SOTA_FINAL Fix GPTQ plumbing for bank architecture: F.linear interception + per-slice quantization
  53. 01b360e1 02:42:48 sota_nueral Fix GPTQ calibration: remove autocast so data_ptr matching works (v3)
  54. 3cba33bf 02:43:55 parameter-golf-lab Fix GPTQ calibration: remove autocast so data_ptr matching works (v3)
  55. 3cba33bf 02:43:55 sota_crawler Fix GPTQ calibration: remove autocast so data_ptr matching works (v3)
  56. 3cba33bf 02:43:55 SOTA_FINAL Fix GPTQ calibration: remove autocast so data_ptr matching works (v3)
  57. 58d81a50 02:52:35 sota_nueral 12L GQA mixed-bit tower: progressive depth ablation — 12 core + late layers with per-layer quant
  58. bf760ad6 02:52:56 parameter-golf-lab 12L GQA mixed-bit tower: progressive depth ablation — 12 core + late layers with per-layer quant
  59. bf760ad6 02:52:56 sota_crawler 12L GQA mixed-bit tower: progressive depth ablation — 12 core + late layers with per-layer quant
  60. bf760ad6 02:52:56 SOTA_FINAL 12L GQA mixed-bit tower: progressive depth ablation — 12 core + late layers with per-layer quant
  61. 9340bf35 03:09:26 sota_nueral Fix GPTQ calibration v4: temp cast banks to bf16 so .to(x.dtype) is no-op
  62. 66fac399 03:09:33 parameter-golf-lab Fix GPTQ calibration v4: temp cast banks to bf16 so .to(x.dtype) is no-op
  63. 66fac399 03:09:33 sota_crawler Fix GPTQ calibration v4: temp cast banks to bf16 so .to(x.dtype) is no-op
  64. 66fac399 03:09:33 SOTA_FINAL Fix GPTQ calibration v4: temp cast banks to bf16 so .to(x.dtype) is no-op
  65. e8d7e327 03:11:42 parameter-golf-lab Fix 12L GQA mixed-bit tower GPTQ: apply v4 bf16 temp-cast fix
  66. e8d7e327 03:11:42 sota_crawler Fix 12L GQA mixed-bit tower GPTQ: apply v4 bf16 temp-cast fix
  67. e8d7e327 03:11:42 SOTA_FINAL Fix 12L GQA mixed-bit tower GPTQ: apply v4 bf16 temp-cast fix
  68. 8b2dfa9b 03:33:41 parameter-golf-lab 12L GQA mixed-bit tower v2: clean rebuild with late_warmup for torch.compile
  69. 8b2dfa9b 03:33:41 sota_crawler 12L GQA mixed-bit tower v2: clean rebuild with late_warmup for torch.compile
  70. 8b2dfa9b 03:33:41 SOTA_FINAL 12L GQA mixed-bit tower v2: clean rebuild with late_warmup for torch.compile
  71. 4343b583 03:37:36 parameter-golf-lab Flip 12L GQA mixed-bit tower arms to SKIP_GPTQ=1 — no time wasted on broken GPTQ
  72. 4343b583 03:37:36 sota_crawler Flip 12L GQA mixed-bit tower arms to SKIP_GPTQ=1 — no time wasted on broken GPTQ
  73. 4343b583 03:37:36 SOTA_FINAL Flip 12L GQA mixed-bit tower arms to SKIP_GPTQ=1 — no time wasted on broken GPTQ
  74. aa42b093 04:03:04 sota_nueral Record iii_loop v4: GPTQ 72/72 works but quant gap is structural
  75. fb596075 04:08:43 sota_nueral Save session state: iii_lean results, 12L GQA mixed-bit tower v2 (shelved)
  76. 9c1e52ac 04:11:34 parameter-golf-lab 12L GQA mixed-bit tower: 14 layers, no loops, SKIP_GPTQ=1, env-only change
  77. 9c1e52ac 04:11:34 sota_crawler 12L GQA mixed-bit tower: 14 layers, no loops, SKIP_GPTQ=1, env-only change
  78. 9c1e52ac 04:11:34 SOTA_FINAL 12L GQA mixed-bit tower: 14 layers, no loops, SKIP_GPTQ=1, env-only change
  79. 589f8592 04:20:30 parameter-golf-lab 12L GQA mixed-bit tower ablation arms: 15L depth, int6-attn, int8-all
  80. 589f8592 04:20:30 sota_crawler 12L GQA mixed-bit tower ablation arms: 15L depth, int6-attn, int8-all
  81. 589f8592 04:20:30 SOTA_FINAL 12L GQA mixed-bit tower ablation arms: 15L depth, int6-attn, int8-all
  82. 9e90b1dc 04:33:53 sota_nueral 12L GQA mixed-bit tower 14L result: 1.11411 BPB, does not beat leader
  83. 5fe097b8 04:46:37 parameter-golf-lab 12L GQA mixed-bit tower: 14L + TTT eval from winning PR #1493
  84. 5fe097b8 04:46:37 sota_crawler 12L GQA mixed-bit tower: 14L + TTT eval from winning PR #1493
  85. 5fe097b8 04:46:37 SOTA_FINAL 12L GQA mixed-bit tower: 14L + TTT eval from winning PR #1493
  86. 7260c90b 04:56:36 sota_nueral Record 15L: best no-loop score 1.11047 BPB, depth wins at 15 layers
  87. a910fbe6 04:57:46 parameter-golf-lab 12L GQA mixed-bit tower: 16L arm (max depth under 16MB) + bump Special to 15L
  88. a910fbe6 04:57:46 sota_crawler 12L GQA mixed-bit tower: 16L arm (max depth under 16MB) + bump Special to 15L
  89. a910fbe6 04:57:46 SOTA_FINAL 12L GQA mixed-bit tower: 16L arm (max depth under 16MB) + bump Special to 15L
DATE2026-04-11
DAY025
PRIMARY METRICVAL_BPB
TESTS RUN3 unique (3 valid / 0 TTT leakage)
ARCHITECTURE FAMILY12L GQA MIXED-BIT TOWER (×1) · WIDENED CRAWLER BOTTLENECK (×1) · OTHER (×1)
CLASSIFICATIONEXPERIMENTAL 33% (1)/NEURAL 67% (2)
ARCHITECTURE RECIPE
12L GQA MIXED-BIT TOWERx1

12-layer GQA tower: 8/4 GQA, RoPE-16, Bigram 2048, XSA on the last 11 layers, and mixed-bit packaging.

BODYnum_layers=12 · num_heads=8 · num_kv_heads=4 · rope_dims=16 · xsa_last_n=11 · BigramHash vocab=2048
QUANT / EVALattention int5 · mlp int6 · aux int6 · embed int8 · other int8 · Brotli artifact · 16 MB artifact cap
RUN SHAPE8xH100 · train_seq_len=2048 · eval_seq_len=2048 · 600s wallclock
WIDENED CRAWLER BOTTLENECKx1

Widened crawler bottleneck body: 5 flat blocks, 1 shared crawler block, 5 flat blocks.

BODYMODEL_DIM=512 · NUM_FLAT_LAYERS=5 · NUM_CRAWLER_LAYERS=1 · CRAWLER_LOOPS=3 · USE_CRAWLER=1 · num_heads=8 · num_kv_heads=4
CRAWLERCRAWLER_MLP_MULT=6.0 · INST_DIM=32 · CRAWLER_QUANT_INT8=1 · CRAWLER_LOOP_ROPE_SCALES=9,1,1 · CRAWLER_TAP_DIM=32 · CRAWLER_TAP_LAYERS=all
PACKAGINGBIGRAM_VOCAB_SIZE=2048 · XSA_LAST_N=11 · ROPE_DIMS=16 · SKIP_EMA=1 · SKIP_GPTQ=1 · MTP_NUM_HEADS=0

SECTION 1.0 — TRAJECTORY

VAL_BPB for 2026-04-11
csv data
VAL_BPB SPREADnminmedianmaxdownload
12L GQA MIXED-BIT TOWER11.10621.10621.1062download
WIDENED CRAWLER BOTTLENECK11.07421.07421.0742download
OTHER11.11051.11051.1105—
ALL VALID31.07421.10621.1105—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 1

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
306:03—Add multi-crawler descendant submission — 1.0742 BPB (4h), 1.1354 BPB (10min)…1.0742OK↑0.7%
details

NEURAL TEST RUNS — 2

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
104:368×H100screen_seed444_20260411_043644.log1.1062OK↑3.7%
training_log
204:56—Record 15L: best no-loop score 1.11047 BPB, depth wins at 15 layers1.1105OK↑4.1%
details