DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-04-07

val_bpb · 210 of 1626 · 4 warnings
findings mentioning this date
| 2026-04-07 | 4 | 10 |
audit summary
{
  "date": "2026-04-07",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 1626,
  "render_records": 210,
  "test_rows": 10,
  "warnings": 4
}
98 commits on 2026-04-07
  1. 61da6045 05:11:03 parameter-golf-lab crawler variant: gate leg for noisy QAT on 9F base
  2. 61da6045 05:11:03 sota_crawler crawler variant: gate leg for noisy QAT on 9F base
  3. 61da6045 05:11:03 SOTA_FINAL crawler variant: gate leg for noisy QAT on 9F base
  4. a0eafe0c 05:17:04 parameter-golf-lab Record Brotli compression packaging compression packaging+mixed-int compression packaging results + add GPTQ eval sweep (4 arms)
  5. a0eafe0c 05:17:04 sota_crawler Record Brotli compression packaging compression packaging+mixed-int compression packaging results + add GPTQ eval sweep (4 arms)
  6. a0eafe0c 05:17:04 SOTA_FINAL Record Brotli compression packaging compression packaging+mixed-int compression packaging results + add GPTQ eval sweep (4 arms)
  7. 5fe00435 05:29:03 parameter-golf-lab Add MODEL_DIM=528 experiment: Brotli compression packaging compression packaging+mixed-int compression packaging+wider model
  8. 5fe00435 05:29:03 sota_crawler Add MODEL_DIM=528 experiment: Brotli compression packaging compression packaging+mixed-int compression packaging+wider model
  9. 5fe00435 05:29:03 SOTA_FINAL Add MODEL_DIM=528 experiment: Brotli compression packaging compression packaging+mixed-int compression packaging+wider model
  10. d586f9eb 05:33:51 parameter-golf-lab Fix: NUM_LAYERS=12 instead of MODEL_DIM=528 (flash_attn_3 head_dim%8)
  11. d586f9eb 05:33:51 sota_crawler Fix: NUM_LAYERS=12 instead of MODEL_DIM=528 (flash_attn_3 head_dim%8)
  12. d586f9eb 05:33:51 SOTA_FINAL Fix: NUM_LAYERS=12 instead of MODEL_DIM=528 (flash_attn_3 head_dim%8)
  13. f2e925ca 05:50:02 parameter-golf-lab Add 12-layer eval sweep: 5 arms testing quant quality vs size
  14. f2e925ca 05:50:02 sota_crawler Add 12-layer eval sweep: 5 arms testing quant quality vs size
  15. f2e925ca 05:50:02 SOTA_FINAL Add 12-layer eval sweep: 5 arms testing quant quality vs size
  16. 06b6f779 05:55:25 parameter-golf-lab Fix INIT_MODEL_PATH: handle skip_weights shape mismatch gracefully
  17. 06b6f779 05:55:25 sota_crawler Fix INIT_MODEL_PATH: handle skip_weights shape mismatch gracefully
  18. 06b6f779 05:55:25 SOTA_FINAL Fix INIT_MODEL_PATH: handle skip_weights shape mismatch gracefully
  19. 37830a60 06:43:29 parameter-golf-lab Add bank-aware GPTQ experiment: fix calibration for 3D bank tensors
  20. 37830a60 06:43:29 sota_crawler Add bank-aware GPTQ experiment: fix calibration for 3D bank tensors
  21. 37830a60 06:43:29 SOTA_FINAL Add bank-aware GPTQ experiment: fix calibration for 3D bank tensors
  22. 9ce181e7 13:49:38 parameter-golf-lab Non-record: ByteJEPA — True Byte-Level JEPA (val_bpb 1.3496)
  23. ee7454a5 21:23:23 parameter-golf-lab test_lab: track 4k bpb in v1.3 sweep summary and leaderboard
  24. ee7454a5 21:23:23 sota_crawler test_lab: track 4k bpb in v1.3 sweep summary and leaderboard
  25. ee7454a5 21:23:23 SOTA_FINAL test_lab: track 4k bpb in v1.3 sweep summary and leaderboard
  26. 65ebe465 21:40:56 parameter-golf-lab test_lab: log 2026-04-07 v1.3 sweep report and fix arm25 head divisibility
  27. 65ebe465 21:40:56 sota_crawler test_lab: log 2026-04-07 v1.3 sweep report and fix arm25 head divisibility
  28. 65ebe465 21:40:56 SOTA_FINAL test_lab: log 2026-04-07 v1.3 sweep report and fix arm25 head divisibility
  29. 506b40ed 21:44:02 parameter-golf-lab crawler variant: 5-arm loop depth + battery gate for 4h showcase
  30. 506b40ed 21:44:02 sota_crawler crawler variant: 5-arm loop depth + battery gate for 4h showcase
  31. 506b40ed 21:44:02 SOTA_FINAL crawler variant: 5-arm loop depth + battery gate for 4h showcase
  32. b1a554a3 22:01:03 parameter-golf-lab test_lab: add long-context causal tap mechanism v1.4 power sweep and remove v1.3 ngram legal gate
  33. b1a554a3 22:01:03 sota_crawler test_lab: add long-context causal tap mechanism v1.4 power sweep and remove v1.3 ngram legal gate
  34. b1a554a3 22:01:03 SOTA_FINAL test_lab: add long-context causal tap mechanism v1.4 power sweep and remove v1.3 ngram legal gate
  35. 3c1ba7ff 22:06:02 parameter-golf-lab crawler variant: add Brotli compression packaging compression packaging variant of train_gpt.py for production run
  36. 3c1ba7ff 22:06:02 sota_crawler crawler variant: add Brotli compression packaging compression packaging variant of train_gpt.py for production run
  37. 3c1ba7ff 22:06:02 SOTA_FINAL crawler variant: add Brotli compression packaging compression packaging variant of train_gpt.py for production run
  38. 4e8194b3 22:40:48 parameter-golf-lab Add Crawler_Katta gate and log crawler variant gate results
  39. df05e2dc 22:40:48 parameter-golf-lab Add Crawler_Katta gate and log crawler variant gate results
  40. 4e8194b3 22:40:48 sota_crawler Add Crawler_Katta gate and log crawler variant gate results
  41. df05e2dc 22:40:48 sota_crawler Add Crawler_Katta gate and log crawler variant gate results
  42. 4e8194b3 22:40:48 SOTA_FINAL Add Crawler_Katta gate and log crawler variant gate results
  43. 5682e7ed 22:58:06 parameter-golf-lab Fix GPTQ bank calibration: run in float32 to preserve data_ptrs
  44. 5682e7ed 22:58:06 sota_crawler Fix GPTQ bank calibration: run in float32 to preserve data_ptrs
  45. 5682e7ed 22:58:06 SOTA_FINAL Fix GPTQ bank calibration: run in float32 to preserve data_ptrs
  46. 19d8d3f6 23:21:46 parameter-golf-lab Add 12L GQA mixed-bit tower 12L submission candidate
  47. 19d8d3f6 23:21:46 sota_crawler Add 12L GQA mixed-bit tower 12L submission candidate
  48. 19d8d3f6 23:21:46 SOTA_FINAL Add 12L GQA mixed-bit tower 12L submission candidate
  49. 33abaef9 23:33:59 parameter-golf-lab Record 12L GQA mixed-bit tower 12L seed=444: 1.10568 BPB, 15.63MB
  50. 33abaef9 23:33:59 sota_crawler Record 12L GQA mixed-bit tower 12L seed=444: 1.10568 BPB, 15.63MB
  51. 33abaef9 23:33:59 SOTA_FINAL Record 12L GQA mixed-bit tower 12L seed=444: 1.10568 BPB, 15.63MB
  52. 08b6784c 00:03:32 parameter-golf-lab Record 12L GQA mixed-bit tower 12L seed=300: 1.10582 BPB, 15.62MB
  53. 08b6784c 00:03:32 sota_crawler Record 12L GQA mixed-bit tower 12L seed=300: 1.10582 BPB, 15.62MB
  54. 08b6784c 00:03:32 SOTA_FINAL Record 12L GQA mixed-bit tower 12L seed=300: 1.10582 BPB, 15.62MB
  55. 97874fc3 00:38:14 parameter-golf-lab Add 12L GQA mixed-bit tower 12L submission — 1.1060 BPB, 15.63MB
  56. 97874fc3 00:38:14 sota_crawler Add 12L GQA mixed-bit tower 12L submission — 1.1060 BPB, 15.63MB
  57. 64be5dfa 02:31:58 parameter-golf-lab Add Crawler_Katta 1xGPU medium sweep harness
  58. 5184a986 02:31:58 parameter-golf-lab Add Crawler_Katta 1xGPU medium sweep harness
  59. 64be5dfa 02:31:58 sota_crawler Add Crawler_Katta 1xGPU medium sweep harness
  60. 5184a986 02:31:58 sota_crawler Add Crawler_Katta 1xGPU medium sweep harness
  61. 64be5dfa 02:31:58 SOTA_FINAL Add Crawler_Katta 1xGPU medium sweep harness
  62. ffb17f39 02:48:00 parameter-golf-lab Add long-context causal tap mechanism v1.5 context mechanism, submission runners, and workspace updates
  63. ffb17f39 02:48:00 sota_crawler Add long-context causal tap mechanism v1.5 context mechanism, submission runners, and workspace updates
  64. ffb17f39 02:48:00 SOTA_FINAL Add long-context causal tap mechanism v1.5 context mechanism, submission runners, and workspace updates
  65. dd17ead4 02:55:52 parameter-golf-lab Fix long-context causal tap mechanism FA3 head-size guard and make v1.5 arm geometry valid
  66. dd17ead4 02:55:52 sota_crawler Fix long-context causal tap mechanism FA3 head-size guard and make v1.5 arm geometry valid
  67. dd17ead4 02:55:52 SOTA_FINAL Fix long-context causal tap mechanism FA3 head-size guard and make v1.5 arm geometry valid
  68. 40772868 03:09:05 parameter-golf-lab Add long-context causal tap mechanism fast L8 submission arm and launcher
  69. 40772868 03:09:05 sota_crawler Add long-context causal tap mechanism fast L8 submission arm and launcher
  70. 40772868 03:09:05 SOTA_FINAL Add long-context causal tap mechanism fast L8 submission arm and launcher
  71. 0cd6e34c 03:09:41 parameter-golf-lab Guard Katta 1xGPU sweep against tiny fallback datasets
  72. d974a620 03:09:41 parameter-golf-lab Guard Katta 1xGPU sweep against tiny fallback datasets
  73. 0cd6e34c 03:09:41 sota_crawler Guard Katta 1xGPU sweep against tiny fallback datasets
  74. d974a620 03:09:41 sota_crawler Guard Katta 1xGPU sweep against tiny fallback datasets
  75. 0cd6e34c 03:09:41 SOTA_FINAL Guard Katta 1xGPU sweep against tiny fallback datasets
  76. 474054ab 03:12:22 parameter-golf-lab Add long-context causal tap mechanism v1.5 heavy-arm run log (s445, 2026-04-08)
  77. 474054ab 03:12:22 sota_crawler Add long-context causal tap mechanism v1.5 heavy-arm run log (s445, 2026-04-08)
  78. 474054ab 03:12:22 SOTA_FINAL Add long-context causal tap mechanism v1.5 heavy-arm run log (s445, 2026-04-08)
  79. c2646d37 03:16:07 parameter-golf-lab Use token-count preflight for Katta 1GPU dataset guard
  80. bc82a003 03:16:07 parameter-golf-lab Use token-count preflight for Katta 1GPU dataset guard
  81. c2646d37 03:16:07 sota_crawler Use token-count preflight for Katta 1GPU dataset guard
  82. bc82a003 03:16:07 sota_crawler Use token-count preflight for Katta 1GPU dataset guard
  83. c2646d37 03:16:07 SOTA_FINAL Use token-count preflight for Katta 1GPU dataset guard
  84. 85558392 03:24:51 parameter-golf-lab Add long-context causal tap mechanism v1.6 fast-L8 run log (s445, 2026-04-08)
  85. 85558392 03:24:51 sota_crawler Add long-context causal tap mechanism v1.6 fast-L8 run log (s445, 2026-04-08)
  86. 85558392 03:24:51 SOTA_FINAL Add long-context causal tap mechanism v1.6 fast-L8 run log (s445, 2026-04-08)
  87. 17a1c309 04:02:08 parameter-golf-lab Add Katta sweep resume + RK compile fallback + fail logging
  88. a71de35a 04:02:08 parameter-golf-lab Add Katta sweep resume + RK compile fallback + fail logging
  89. 17a1c309 04:02:08 sota_crawler Add Katta sweep resume + RK compile fallback + fail logging
  90. a71de35a 04:02:08 sota_crawler Add Katta sweep resume + RK compile fallback + fail logging
  91. 17a1c309 04:02:08 SOTA_FINAL Add Katta sweep resume + RK compile fallback + fail logging
  92. cf658862 04:33:08 parameter-golf-lab Promote crawler variant A3 loop5 battery as quality-priority candidate
  93. cf658862 04:33:08 sota_crawler Promote crawler variant A3 loop5 battery as quality-priority candidate
  94. cf658862 04:33:08 SOTA_FINAL Promote crawler variant A3 loop5 battery as quality-priority candidate
  95. f2dc4e5c 04:46:16 sota_nueral Add 12L GQA mixed-bit tower.2 economical ablation screening harness
  96. 4177602b 04:51:44 parameter-golf-lab Add 12L GQA mixed-bit tower.2 economical ablation screening harness
  97. 4177602b 04:51:44 sota_crawler Add 12L GQA mixed-bit tower.2 economical ablation screening harness
  98. 4177602b 04:51:44 SOTA_FINAL Add 12L GQA mixed-bit tower.2 economical ablation screening harness
DATE2026-04-07
DAY021
PRIMARY METRICVAL_BPB
TESTS RUN10 unique (9 valid / 0 TTT leakage)
ARCHITECTURE FAMILY12L GQA MIXED-BIT TOWER (×7) · 11L GQA U-NET BODY (×1) · COMPRESSION PACKAGING EXPERIMENT (×1) · TRACE AND FLOW-CAPTURE WRAPPER (×1)
CLASSIFICATIONEXPERIMENTAL 70% (7)/NEURAL 30% (3)
ARCHITECTURE RECIPE
12L GQA MIXED-BIT TOWERx7

12-layer GQA tower: 8/4 GQA, RoPE-16, Bigram 2048, XSA on the last 11 layers, and mixed-bit packaging.

BODYnum_layers=12 · num_heads=8 · num_kv_heads=4 · rope_dims=16 · xsa_last_n=11 · BigramHash vocab=2048
QUANT / EVALattention int5 · mlp int6 · aux int6 · embed int8 · other int8 · Brotli artifact · 16 MB artifact cap
RUN SHAPE8xH100 · train_seq_len=2048 · eval_seq_len=2048 · 600s wallclock
11L GQA U-NET BODYx1

11-layer dim-512 GQA U-Net transformer base with XSA, BigramHash, value embedding, and SmearGate.

BODYvocab_size=1024 · num_layers=11 · model_dim=512 · num_heads=8 · num_kv_heads=4 · head_dim=64 · mlp_mult=3.0 · tie_embeddings=1
STRUCTUREencoder_layers=5 · decoder_layers=6 · skip_weights=5 · GQA attention · RMSNorm · relu_sq MLP · Muon matrix optimizer path
FEATURESBigramHash vocab=2048 dim=128 · XSA last_n=11 · rope_dims=16 · VE dim=128 layers=9,10 · SmearGate · logit_softcap=30

SECTION 1.0 — TRAJECTORY

VAL_BPB for 2026-04-07 panel 1 of 2
csv data
VAL_BPB for 2026-04-07 panel 2 of 2
csv data
VAL_BPB SPREADnminmedianmaxdownload
12L GQA MIXED-BIT TOWER71.10571.10581.1064download
11L GQA U-NET BODY11.10991.10991.1099download
COMPRESSION PACKAGING EXPERIMENT11.11021.11021.1102—
ALL VALID91.10571.10601.1102—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 7

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
100:03—Record 12L GQA mixed-bit tower 12L seed=300: 1.10582 BPB, 15.62MB (×3)1.1058OK↑3.6%
details
405:17—Record Brotli compression packaging compression packaging+mixed-int compressi…1.1102OK↑4.0%
details
513:468×H100train_seed300.log (×2)1.1058OK↑3.6%
training_log 1
training_log 2
613:468×H100train_seed42.log (×2)1.1064OK↑3.7%
training_log 1
training_log 2
713:468×H100train_seed444.log (×2)1.1057OK↑3.6%
training_log 1
training_log 2
913:461×GB10trace and flow-capture wrapper GQA mixed-bit tower.log4.2120CRASH—
training_log
1023:33—Record 12L GQA mixed-bit tower 12L seed=444: 1.10568 BPB, 15.63MB (×3)1.1057OK↑3.6%
details

NEURAL TEST RUNS — 3

#TIMEHWABLATIONVAL_BPBSTATUSΔ %OPEN
200:38—Add 12L GQA mixed-bit tower 12L submission — 1.1060 BPB, 15.63MB (×2)1.1060OK↑3.6%
details
304:03—11L GQA U-Net body1.1099OK↑4.0%
details
813:46—12L GQA mixed-bit tower 12L (×3)1.1060OK↑3.6%
details