2026-03-23 — Research Day

DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-03-23

val_bpb · 568 of 658 · 12 warnings

findings mentioning this date

| 2026-03-23 | 12 | 22 |

audit summary

{
  "date": "2026-03-23",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 658,
  "render_records": 568,
  "test_rows": 22,
  "warnings": 12
}

256 commits on 2026-03-23

899a1db8 05:06:31 parameter-golf-lab Remove post-quant burst — illegal (uses training data during eval)
711eb3a6 05:06:31 parameter-golf-lab Remove post-quant burst — illegal (uses training data during eval)
fcfbd34f 05:06:31 parameter-golf-lab Remove post-quant burst — illegal (uses training data during eval)
217eaad5 05:06:31 parameter-golf-lab Remove post-quant burst — illegal (uses training data during eval)
899a1db8 05:06:31 sota_crawler Remove post-quant burst — illegal (uses training data during eval)
fcfbd34f 05:06:31 sota_crawler Remove post-quant burst — illegal (uses training data during eval)
217eaad5 05:06:31 sota_crawler Remove post-quant burst — illegal (uses training data during eval)
3c049dde 05:14:56 parameter-golf-lab Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
0a22c2b7 05:14:56 parameter-golf-lab Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
286b2adc 05:14:56 parameter-golf-lab Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
f94c09a3 05:14:56 parameter-golf-lab Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
3c049dde 05:14:56 sota_crawler Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
286b2adc 05:14:56 sota_crawler Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
f94c09a3 05:14:56 sota_crawler Add selective int8 for sensitive layers (GPTQ-int8 upgrade path)
fc900eb2 05:18:39 parameter-golf-lab recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
769b01b5 05:18:39 parameter-golf-lab recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
c96450e2 05:18:39 parameter-golf-lab recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
8b22b10f 05:18:39 parameter-golf-lab recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
fc900eb2 05:18:39 sota_crawler recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
c96450e2 05:18:39 sota_crawler recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
8b22b10f 05:18:39 sota_crawler recursive weight-sharing transformer V3: v7 full stack + fractal cadence test cadence test loops (GPTQ + TTT + PQB)
6d4772ce 05:54:45 parameter-golf-lab Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
b5986e4d 05:54:45 parameter-golf-lab Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
ff4a79ef 05:54:45 parameter-golf-lab Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
a0e0841e 05:54:45 parameter-golf-lab Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
6d4772ce 05:54:45 sota_crawler Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
ff4a79ef 05:54:45 sota_crawler Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
a0e0841e 05:54:45 sota_crawler Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD
e605d1c5 06:02:05 parameter-golf-lab Record: GPTQ + Early QAT + TTT EMA — 3-seed mean val_bpb 1.1215
464f8591 06:02:05 parameter-golf-lab Record: GPTQ + Early QAT + TTT EMA — 3-seed mean val_bpb 1.1215
464f8591 06:02:05 sota_crawler Record: GPTQ + Early QAT + TTT EMA — 3-seed mean val_bpb 1.1215
cd7871ef 06:04:46 parameter-golf-lab recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
8fe38f0a 06:04:46 parameter-golf-lab recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
8a195607 06:04:46 parameter-golf-lab recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
cd7871ef 06:04:46 sota_crawler recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
8fe38f0a 06:04:46 sota_crawler recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
8a195607 06:04:46 sota_crawler recursive weight-sharing transformer V3: SwiGLU + AdamW TTT + 8K bigram (PR#462 techniques)
1f6d92a7 06:21:26 parameter-golf-lab Fix DDP unused params: add find_unused_parameters=True
98f3e2f5 06:21:26 parameter-golf-lab Fix DDP unused params: add find_unused_parameters=True
a5724c0d 06:21:26 parameter-golf-lab Fix DDP unused params: add find_unused_parameters=True
1f6d92a7 06:21:26 sota_crawler Fix DDP unused params: add find_unused_parameters=True
98f3e2f5 06:21:26 sota_crawler Fix DDP unused params: add find_unused_parameters=True
a5724c0d 06:21:26 sota_crawler Fix DDP unused params: add find_unused_parameters=True
c4763c7d 06:26:42 parameter-golf-lab SwiGLU + GPTQ: PR #462 architecture with our quant improvements
c97ad546 06:26:42 parameter-golf-lab SwiGLU + GPTQ: PR #462 architecture with our quant improvements
04eb2e54 06:26:42 parameter-golf-lab SwiGLU + GPTQ: PR #462 architecture with our quant improvements
c4763c7d 06:26:42 sota_crawler SwiGLU + GPTQ: PR #462 architecture with our quant improvements
c97ad546 06:26:42 sota_crawler SwiGLU + GPTQ: PR #462 architecture with our quant improvements
04eb2e54 06:26:42 sota_crawler SwiGLU + GPTQ: PR #462 architecture with our quant improvements
dbd79b65 06:43:47 parameter-golf-lab Add OptRot: Hadamard rotation before GPTQ quantization
03e5f541 06:43:47 parameter-golf-lab Add OptRot: Hadamard rotation before GPTQ quantization
b639e15b 06:43:47 parameter-golf-lab Add OptRot: Hadamard rotation before GPTQ quantization
dbd79b65 06:43:47 sota_crawler Add OptRot: Hadamard rotation before GPTQ quantization
03e5f541 06:43:47 sota_crawler Add OptRot: Hadamard rotation before GPTQ quantization
b639e15b 06:43:47 sota_crawler Add OptRot: Hadamard rotation before GPTQ quantization
675f3749 06:49:47 parameter-golf-lab recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
eb09f7ec 06:49:47 parameter-golf-lab recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
5aec5ae3 06:49:47 parameter-golf-lab recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
675f3749 06:49:47 sota_crawler recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
eb09f7ec 06:49:47 sota_crawler recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
5aec5ae3 06:49:47 sota_crawler recursive weight-sharing transformer V3: Star-ReLU SwiGLU + VRL + MLP 4.0 (fits 16MB)
34335573 07:12:44 parameter-golf-lab Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
7020bf6d 07:12:44 parameter-golf-lab Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
ce487d52 07:12:44 parameter-golf-lab Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
34335573 07:12:44 sota_crawler Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
7020bf6d 07:12:44 sota_crawler Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
ce487d52 07:12:44 sota_crawler Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted)
60137e91 13:39:04 parameter-golf-lab recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
addbea95 13:39:04 parameter-golf-lab recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
06952977 13:39:04 parameter-golf-lab recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
60137e91 13:39:04 sota_crawler recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
addbea95 13:39:04 sota_crawler recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
06952977 13:39:04 sota_crawler recursive weight-sharing transformer V4: GEPA-matched config for 16MB compression
e90f9c16 13:54:29 parameter-golf-lab H100-ready: revert to DG v2.1, auto-detect GQA, clean up
01e7c574 13:57:54 parameter-golf-lab recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
5c912b51 13:57:54 parameter-golf-lab recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
5b72e979 13:57:54 parameter-golf-lab recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
01e7c574 13:57:54 sota_crawler recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
5c912b51 13:57:54 sota_crawler recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
5b72e979 13:57:54 sota_crawler recursive weight-sharing transformer V4 A/B: 5x2 and 4x2 variants for H100 calibration
2c594466 14:06:12 parameter-golf-lab Add short TTT experiment: no-EMA, 50 chunks, SGD
324843ba 14:06:12 parameter-golf-lab Add short TTT experiment: no-EMA, 50 chunks, SGD
eec0981a 14:06:12 parameter-golf-lab Add short TTT experiment: no-EMA, 50 chunks, SGD
2c594466 14:06:12 sota_crawler Add short TTT experiment: no-EMA, 50 chunks, SGD
324843ba 14:06:12 sota_crawler Add short TTT experiment: no-EMA, 50 chunks, SGD
eec0981a 14:06:12 sota_crawler Add short TTT experiment: no-EMA, 50 chunks, SGD
d97ad7db 14:23:24 parameter-golf-lab DG: use Flash Attention for designator matching
b6374c09 14:27:02 parameter-golf-lab Add XSA-all(11) + short TTT run script
d36c1595 14:27:02 parameter-golf-lab Add XSA-all(11) + short TTT run script
27833024 14:27:02 parameter-golf-lab Add XSA-all(11) + short TTT run script
b6374c09 14:27:02 sota_crawler Add XSA-all(11) + short TTT run script
d36c1595 14:27:02 sota_crawler Add XSA-all(11) + short TTT run script
27833024 14:27:02 sota_crawler Add XSA-all(11) + short TTT run script
03a4ef6f 14:37:25 parameter-golf-lab Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
4185795d 14:37:25 parameter-golf-lab Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
eac2e723 14:37:25 parameter-golf-lab Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
03a4ef6f 14:37:25 sota_crawler Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
4185795d 14:37:25 sota_crawler Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
eac2e723 14:37:25 sota_crawler Record: v7 GPTQ + Short TTT — val_bpb 1.1207 (seed 1337)
5a5bbad3 14:39:07 parameter-golf-lab recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
874237b6 14:39:07 parameter-golf-lab recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
9fd191d5 14:39:07 parameter-golf-lab recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
5a5bbad3 14:39:07 sota_crawler recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
874237b6 14:39:07 sota_crawler recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
9fd191d5 14:39:07 sota_crawler recursive weight-sharing transformer compression shim on SwiGLU SOTA (1.0763 model)
57314a2a 14:50:31 parameter-golf-lab Add v7 smooth run: proper warmdown + XSA-all
47b7700f 14:50:31 parameter-golf-lab Add v7 smooth run: proper warmdown + XSA-all
57314a2a 14:50:31 sota_crawler Add v7 smooth run: proper warmdown + XSA-all
47b7700f 14:50:31 sota_crawler Add v7 smooth run: proper warmdown + XSA-all
a79acfdf 14:51:37 parameter-golf-lab Reduce bigram buckets 2048→1792 for XSA-11 size fit
da0f71aa 14:51:37 parameter-golf-lab Reduce bigram buckets 2048→1792 for XSA-11 size fit
a79acfdf 14:51:37 sota_crawler Reduce bigram buckets 2048→1792 for XSA-11 size fit
da0f71aa 14:51:37 sota_crawler Reduce bigram buckets 2048→1792 for XSA-11 size fit
510b1bc6 14:59:50 parameter-golf-lab DG v3: hardcoded depth schedule replaces learned β
6046116c 15:09:02 parameter-golf-lab recursive weight-sharing transformer stacked: LeakyReLU(0.5)² + VRL + decoder_lr_mult=2.0
7efd4e32 15:09:02 parameter-golf-lab recursive weight-sharing transformer stacked: LeakyReLU(0.5)² + VRL + decoder_lr_mult=2.0
6046116c 15:09:02 sota_crawler recursive weight-sharing transformer stacked: LeakyReLU(0.5)² + VRL + decoder_lr_mult=2.0
7efd4e32 15:09:02 sota_crawler recursive weight-sharing transformer stacked: LeakyReLU(0.5)² + VRL + decoder_lr_mult=2.0
f96d349e 15:16:28 parameter-golf-lab Add research batch script + GPTQ env var tuning
f523db4a 15:16:28 parameter-golf-lab Add research batch script + GPTQ env var tuning
f96d349e 15:16:28 sota_crawler Add research batch script + GPTQ env var tuning
f523db4a 15:16:28 sota_crawler Add research batch script + GPTQ env var tuning
8ff3e0ef 15:19:40 parameter-golf-lab Record: LeakyReLU² + Legal TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean)
8ff3e0ef 15:19:40 sota_crawler Record: LeakyReLU² + Legal TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean)
8ff3e0ef 15:19:40 SOTA_FINAL Record: LeakyReLU² + Legal TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean)
9fec4314 15:21:45 parameter-golf-lab Add DG Attention non-record submission (val_bpb=1.1898)
b3c24d97 15:27:47 parameter-golf-lab Clean PR: remove dev artifacts and track_10min_16mb files
9dd29bea 15:29:03 parameter-golf-lab Sanitize: remove all illegal TTT from recursive weight-sharing transformer scripts
ddb28362 15:29:03 parameter-golf-lab Sanitize: remove all illegal TTT from recursive weight-sharing transformer scripts
9dd29bea 15:29:03 sota_crawler Sanitize: remove all illegal TTT from recursive weight-sharing transformer scripts
ddb28362 15:29:03 sota_crawler Sanitize: remove all illegal TTT from recursive weight-sharing transformer scripts
736d1bd9 15:41:01 parameter-golf-lab Add recursive weight-sharing transformer SwiGLU sweep: 9 configs across 3 batches
6f8be1f3 15:41:01 parameter-golf-lab Add recursive weight-sharing transformer SwiGLU sweep: 9 configs across 3 batches
736d1bd9 15:41:01 sota_crawler Add recursive weight-sharing transformer SwiGLU sweep: 9 configs across 3 batches
6f8be1f3 15:41:01 sota_crawler Add recursive weight-sharing transformer SwiGLU sweep: 9 configs across 3 batches
214fa9e8 15:51:52 parameter-golf-lab EMERGENCY: Disable all illegal TTT across entire project
496705c9 15:51:52 parameter-golf-lab EMERGENCY: Disable all illegal TTT across entire project
214fa9e8 15:51:52 sota_crawler EMERGENCY: Disable all illegal TTT across entire project
496705c9 15:51:52 sota_crawler EMERGENCY: Disable all illegal TTT across entire project
1a01dc77 15:56:27 parameter-golf-lab CRITICAL: Remove illegal TTT from train_gpt_swiglu.py
560db561 15:56:27 parameter-golf-lab CRITICAL: Remove illegal TTT from train_gpt_swiglu.py
1a01dc77 15:56:27 sota_crawler CRITICAL: Remove illegal TTT from train_gpt_swiglu.py
560db561 15:56:27 sota_crawler CRITICAL: Remove illegal TTT from train_gpt_swiglu.py
605f965a 16:03:38 parameter-golf-lab DG v4: prev-token baseline, single projection, depth-skip
bc4f3420 16:24:27 parameter-golf-lab Add SwiGLU recursive weight-sharing transformer compression sweep (6 configs)
ebf06e18 16:24:27 parameter-golf-lab Add SwiGLU recursive weight-sharing transformer compression sweep (6 configs)
bc4f3420 16:24:27 sota_crawler Add SwiGLU recursive weight-sharing transformer compression sweep (6 configs)
ebf06e18 16:24:27 sota_crawler Add SwiGLU recursive weight-sharing transformer compression sweep (6 configs)
c2da2f2f 16:32:53 parameter-golf-lab SwiGLU F1: VRL + LeakyReLU(0.5)² + legal TTT + int8 attn.proj + seq2048
4a629336 16:32:53 parameter-golf-lab SwiGLU F1: VRL + LeakyReLU(0.5)² + legal TTT + int8 attn.proj + seq2048
c2da2f2f 16:32:53 sota_crawler SwiGLU F1: VRL + LeakyReLU(0.5)² + legal TTT + int8 attn.proj + seq2048
4a629336 16:32:53 sota_crawler SwiGLU F1: VRL + LeakyReLU(0.5)² + legal TTT + int8 attn.proj + seq2048
4e5de0b7 16:39:38 parameter-golf-lab Fix DDP: find_unused_parameters=True for VRL block 0
77a8a188 16:39:38 parameter-golf-lab Fix DDP: find_unused_parameters=True for VRL block 0
4e5de0b7 16:39:38 sota_crawler Fix DDP: find_unused_parameters=True for VRL block 0
77a8a188 16:39:38 sota_crawler Fix DDP: find_unused_parameters=True for VRL block 0
2aae6b0a 16:45:42 parameter-golf-lab Add GPTQ re-quantization sweep (no training, ~2 min)
d66040ee 16:45:42 parameter-golf-lab Add GPTQ re-quantization sweep (no training, ~2 min)
2aae6b0a 16:45:42 sota_crawler Add GPTQ re-quantization sweep (no training, ~2 min)
d66040ee 16:45:42 sota_crawler Add GPTQ re-quantization sweep (no training, ~2 min)
ffc909b0 16:51:09 parameter-golf-lab Save Gold Standard (GS): v7 GPTQ 1.1206 BPB — 3 copies
3e624068 16:51:09 parameter-golf-lab Save Gold Standard (GS): v7 GPTQ 1.1206 BPB — 3 copies
ffc909b0 16:51:09 sota_crawler Save Gold Standard (GS): v7 GPTQ 1.1206 BPB — 3 copies
3e624068 16:51:09 sota_crawler Save Gold Standard (GS): v7 GPTQ 1.1206 BPB — 3 copies
9fefc754 16:52:47 parameter-golf-lab Restore original train_gpt_swiglu.py, new F1 variant as separate file
4b9ed8b2 16:52:47 parameter-golf-lab Restore original train_gpt_swiglu.py, new F1 variant as separate file
9fefc754 16:52:47 sota_crawler Restore original train_gpt_swiglu.py, new F1 variant as separate file
4b9ed8b2 16:52:47 sota_crawler Restore original train_gpt_swiglu.py, new F1 variant as separate file
0340c66f 17:00:01 parameter-golf-lab Log SwiGLU F1 run: 1.1208 BPB (20.6MB, needs compression)
16db94f0 17:00:01 parameter-golf-lab Log SwiGLU F1 run: 1.1208 BPB (20.6MB, needs compression)
0340c66f 17:00:01 sota_crawler Log SwiGLU F1 run: 1.1208 BPB (20.6MB, needs compression)
16db94f0 17:00:01 sota_crawler Log SwiGLU F1 run: 1.1208 BPB (20.6MB, needs compression)
139d573e 17:13:03 parameter-golf-lab Fix pre-TTT BPB, TTT gains, and steps to match logs exactly
139d573e 17:13:03 sota_crawler Fix pre-TTT BPB, TTT gains, and steps to match logs exactly
139d573e 17:13:03 SOTA_FINAL Fix pre-TTT BPB, TTT gains, and steps to match logs exactly
b08d72a9 17:22:25 parameter-golf-lab Fix author attributions: PR #493 @parinzee, PR #461 @Christopher-Lee-McClendon
b08d72a9 17:22:25 sota_crawler Fix author attributions: PR #493 @parinzee, PR #461 @Christopher-Lee-McClendon
b08d72a9 17:22:25 SOTA_FINAL Fix author attributions: PR #493 @parinzee, PR #461 @Christopher-Lee-McClendon
b82c50dd 17:36:14 parameter-golf-lab Update README leaderboard with merged records
b82c50dd 17:36:14 sota_crawler Update README leaderboard with merged records
b82c50dd 17:36:14 SOTA_FINAL Update README leaderboard with merged records
d74c0b58 17:38:56 parameter-golf-lab Use GitHub usernames in new leaderboard rows
d74c0b58 17:38:56 sota_crawler Use GitHub usernames in new leaderboard rows
d74c0b58 17:38:56 SOTA_FINAL Use GitHub usernames in new leaderboard rows
8a77849b 17:41:35 parameter-golf-lab Describe leaderboard entries by base-run diff
8a77849b 17:41:35 sota_crawler Describe leaderboard entries by base-run diff
8a77849b 17:41:35 SOTA_FINAL Describe leaderboard entries by base-run diff
c7311ad2 18:00:18 parameter-golf-lab DG v4.1: eliminate second matmul via linearity trick
89914646 20:33:11 parameter-golf-lab Add TTT calibration sweep: 11-config grid to close 1.11 BPB gap
0efae917 20:33:11 parameter-golf-lab Add TTT calibration sweep: 11-config grid to close 1.11 BPB gap
89914646 20:33:11 sota_crawler Add TTT calibration sweep: 11-config grid to close 1.11 BPB gap
0efae917 20:33:11 sota_crawler Add TTT calibration sweep: 11-config grid to close 1.11 BPB gap
a89a1426 21:23:53 parameter-golf-lab Add GPTQ re-quant sweep for GS v7 — no retraining needed
d6dad03c 21:23:53 parameter-golf-lab Add GPTQ re-quant sweep for GS v7 — no retraining needed
a89a1426 21:23:53 sota_crawler Add GPTQ re-quant sweep for GS v7 — no retraining needed
d6dad03c 21:23:53 sota_crawler Add GPTQ re-quant sweep for GS v7 — no retraining needed
513fd618 21:43:51 parameter-golf-lab Update recursive weight-sharing transformer PR with full experiment history and research directions
6abf4436 21:43:51 parameter-golf-lab Update recursive weight-sharing transformer PR with full experiment history and research directions
513fd618 21:43:51 sota_crawler Update recursive weight-sharing transformer PR with full experiment history and research directions
6abf4436 21:43:51 sota_crawler Update recursive weight-sharing transformer PR with full experiment history and research directions
d0c4271d 22:00:21 parameter-golf-lab Rewrite recursive weight-sharing transformer PR: honest framing, clean architecture/training separation
d0c4271d 22:00:21 sota_crawler Rewrite recursive weight-sharing transformer PR: honest framing, clean architecture/training separation
da89c9ee 22:13:07 parameter-golf-lab Submission: XSA-11 + GPTQ block64/percdamp002
8fbd41db 22:13:07 parameter-golf-lab Submission: XSA-11 + GPTQ block64/percdamp002
d6748136 22:13:07 parameter-golf-lab Submission: XSA-11 + GPTQ block64/percdamp002
da89c9ee 22:13:07 sota_crawler Submission: XSA-11 + GPTQ block64/percdamp002
8fbd41db 22:13:07 sota_crawler Submission: XSA-11 + GPTQ block64/percdamp002
0100f8e9 23:15:01 parameter-golf-lab XSA-11 + GPTQ block64/percdamp002 — 3-seed mean 1.1208 BPB
0100f8e9 23:15:01 sota_crawler XSA-11 + GPTQ block64/percdamp002 — 3-seed mean 1.1208 BPB
303192e9 23:16:37 parameter-golf-lab XSA-11 + GPTQ block64/percdamp002 — 3-seed mean val_bpb 1.1208
303192e9 23:16:37 sota_crawler XSA-11 + GPTQ block64/percdamp002 — 3-seed mean val_bpb 1.1208
54f14bd9 00:37:50 parameter-golf-lab Add int5 sizing sweep — test how many params fit at int5 vs int6
44ae7a85 00:40:58 parameter-golf-lab Add int5 sizing sweep — test how many params fit at int5
44ae7a85 00:40:58 sota_crawler Add int5 sizing sweep — test how many params fit at int5
7bbb09cd 00:50:04 parameter-golf-lab Int5 test: 33.6M params (MHA 8/8, MLP 3.5x) with int5 GPTQ
7bbb09cd 00:50:04 sota_crawler Int5 test: 33.6M params (MHA 8/8, MLP 3.5x) with int5 GPTQ
bd415d5c 01:14:49 parameter-golf-lab Add 576+ single-file runner and aggressive edge test launchers
bd415d5c 01:14:49 sota_crawler Add 576+ single-file runner and aggressive edge test launchers
30a61762 02:47:29 parameter-golf-lab Add TTT sweep infra + Vast.ai pipeline + int5mix run record
30a61762 02:47:29 sota_crawler Add TTT sweep infra + Vast.ai pipeline + int5mix run record
f1b68023 02:48:19 parameter-golf-lab Add RunPod setup script for TTT calibration sweep
f1b68023 02:48:19 sota_crawler Add RunPod setup script for TTT calibration sweep
8f94780c 03:10:53 parameter-golf-lab Add Micro Crawler architecture — asymmetric fractal cadence test cadence test with flat+crawler split
8f94780c 03:10:53 sota_crawler Add Micro Crawler architecture — asymmetric fractal cadence test cadence test with flat+crawler split
b3b408bb 03:13:15 parameter-golf-lab Update micro crawler to balanced 4f+2cx2 config at dim=640
b3b408bb 03:13:15 sota_crawler Update micro crawler to balanced 4f+2cx2 config at dim=640
65f42e86 03:41:23 parameter-golf-lab Add crawler cadence + per-loop GPTQ to H100 micro crawler
65f42e86 03:41:23 sota_crawler Add crawler cadence + per-loop GPTQ to H100 micro crawler
c9e2aff8 03:53:47 parameter-golf-lab Cadence N/N/N/N/C + activation-aware per-loop GPTQ blending
c9e2aff8 03:53:47 sota_crawler Cadence N/N/N/N/C + activation-aware per-loop GPTQ blending
ee034506 03:59:51 parameter-golf-lab Pre-flight: backup scripts + pod setup for micro crawler 8xH100
ee034506 03:59:51 sota_crawler Pre-flight: backup scripts + pod setup for micro crawler 8xH100
5a1e8b76 04:10:39 parameter-golf-lab Recursive cadence: ramp N count as LR warms down
5a1e8b76 04:10:39 sota_crawler Recursive cadence: ramp N count as LR warms down
81ed4752 04:21:36 parameter-golf-lab recursive weight-sharing transformer v2: trigram hash + lr_mul compile-spike fix
81ed4752 04:21:36 sota_crawler recursive weight-sharing transformer v2: trigram hash + lr_mul compile-spike fix
de856119 04:24:00 parameter-golf-lab Fix lr_mul compile spike + shrink dim 640→624 for artifact budget
de856119 04:24:00 sota_crawler Fix lr_mul compile spike + shrink dim 640→624 for artifact budget
84f32ff5 04:25:12 parameter-golf-lab Freeze run1 scripts — 1.1377 sliding window (broken LR, dim=640)
84f32ff5 04:25:12 sota_crawler Freeze run1 scripts — 1.1377 sliding window (broken LR, dim=640)
a35820ee 04:25:47 parameter-golf-lab Fix dim 624→630 (must be divisible by num_heads=10)
a35820ee 04:25:47 sota_crawler Fix dim 624→630 (must be divisible by num_heads=10)
3cfebf8d 04:26:51 parameter-golf-lab Fix dim 630→620 (head_dim must be even for RoPE)
3cfebf8d 04:26:51 sota_crawler Fix dim 630→620 (head_dim must be even for RoPE)
357b9aff 04:29:14 parameter-golf-lab Fix dim 620→560 (FA3 needs head_dim multiple of 8: 560/10=56)
357b9aff 04:29:14 sota_crawler Fix dim 620→560 (FA3 needs head_dim multiple of 8: 560/10=56)
61dce75b 04:31:22 parameter-golf-lab Add quantfix training script: learned scales, earlier QAT, selective int8
61dce75b 04:31:22 sota_crawler Add quantfix training script: learned scales, earlier QAT, selective int8
94cd3eba 04:39:29 parameter-golf-lab Run2: GPTQ Hessian quant + dim=640 + trigram_vocab=2048
94cd3eba 04:39:29 sota_crawler Run2: GPTQ Hessian quant + dim=640 + trigram_vocab=2048
9d50ab72 04:42:29 parameter-golf-lab Launcher: dim=640, trigram_vocab=2048 (fit budget, test lr_mul fix only)
9d50ab72 04:42:29 sota_crawler Launcher: dim=640, trigram_vocab=2048 (fit budget, test lr_mul fix only)
1a6ab1b3 04:43:10 parameter-golf-lab Keep trigram_vocab=8192 to match run1 — isolate lr_mul fix only
1a6ab1b3 04:43:10 sota_crawler Keep trigram_vocab=8192 to match run1 — isolate lr_mul fix only
bcccc19d 04:53:13 parameter-golf-lab Run3: Self-referential crawler with deliberation gate
bcccc19d 04:53:13 sota_crawler Run3: Self-referential crawler with deliberation gate

DATE2026-03-23

DAY006

PRIMARY METRICVAL_BPB

TESTS RUN22 unique (22 valid / 0 TTT leakage)

ARCHITECTURE FAMILYLEADERBOARD TRANSFORMER VARIANTS (×11) · CROSS-SEQUENCE ATTENTION VARIANT (×3) · STATE-SPACE TTT VARIANT (×2) · POST-TRAINING QUANTIZATION CALIBRATION (×2) · RECURSIVE WEIGHT-SHARING TRANSFORMER (×1) · SLIDING-WINDOW EVALUATION VARIANT (×1) · +2 MORE (×2)

CLASSIFICATIONEXPERIMENTAL 82% (18)/NEURAL 18% (4)

SECTION 1.0 — TRAJECTORY

VAL_BPB SPREAD	n	min	median	max	download
LEADERBOARD TRANSFORMER VARIANTS	11	1.0672	1.1208	1.4138	—
CROSS-SEQUENCE ATTENTION VARIANT	3	1.1208	1.1208	2.0644	—
STATE-SPACE TTT VARIANT	2	1.1201	1.1262	1.1322	—
POST-TRAINING QUANTIZATION CALIBRATION	2	1.1206	1.1210	1.1215	download
RECURSIVE WEIGHT-SHARING TRANSFORMER	1	1.1478	1.1478	1.1478	download
SLIDING-WINDOW EVALUATION VARIANT	1	1.1377	1.1377	1.1377	—
COMPACT TRANSFORMER RECIPE	1	1.1207	1.1207	1.1207	—
CRAWLER ABLATION CONTROL SET	1	1.3766	1.3766	1.3766	—
ALL VALID	22	1.0672	1.1209	2.0644	—

VAL_BPB SPREAD

min

median

max

download

LEADERBOARD TRANSFORMER VARIANTS

1.0672

1.1208

1.4138

—

CROSS-SEQUENCE ATTENTION VARIANT

1.1208

2.0644

—

STATE-SPACE TTT VARIANT

1.1201

1.1262

1.1322

—

POST-TRAINING QUANTIZATION CALIBRATION

1.1206

1.1210

1.1215

download

RECURSIVE WEIGHT-SHARING TRANSFORMER

1.1478

download

SLIDING-WINDOW EVALUATION VARIANT

1.1377

—

COMPACT TRANSFORMER RECIPE

1.1207

—

CRAWLER ABLATION CONTROL SET

1.3766

—

ALL VALID

1.0672

1.1209

2.0644

—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 18

#	TIME	HW	ABLATION	VAL_BPB	STATUS	Δ %	OPEN
2	04:21	—	recursive weight-sharing transformer v2: trigram hash + lr_mul compile-spike …	1.1478	OK	↑2.4%	details

3	04:25	—	Freeze run1 scripts — 1.1377 sliding window (broken LR, dim=640) (×2)	1.1377	OK	↑1.5%	details

4	05:54	—	Add AdamW TTT option — PR #462 shows 5x better TTT gain vs SGD (×7)	1.0672	OK	↓4.8%	details

6	06:26	—	SwiGLU + GPTQ: PR #462 architecture with our quant improvements (×6)	1.0672	OK	↓4.8%	details

7	07:12	—	Log session results: 1.0763 SwiGLU (size problem), 1.1215 v7 (submitted) (×6)	1.0763	OK	↓4.0%	details

8	14:35	8×H100	run_s1337.log (×3)	1.1201	OK	↓0.1%	training_log 1 training_log 2 training_log 3

10	16:24	1×GB10	A_bigram1536_xsa11.log	2.0644	OK	↑84.2%	training_log

11	16:51	—	Save Gold Standard (GS): v7 GPTQ 1.1206 BPB — 3 copies (×4)	1.1206	OK	↓0.0%	details

13	17:03	1×GB10	S1_loops3_baseline.log	1.3766	OK	↑22.8%	training_log

14	17:39	1×GB10	S2_loops4.log	1.4058	OK	↑25.4%	training_log

15	18:17	1×GB10	S3_loops5.log	1.4138	OK	↑26.1%	training_log

16	18:53	1×GB10	S4_share3_loops4.log	1.4076	OK	↑25.6%	training_log

17	19:06	1×GB10	S5_loops4_bigram4096.log	1.4123	OK	↑26.0%	training_log

18	22:17	8×H100 (runpod)	submit_s1337_20260323_221742.log	1.1203	OK	↓0.1%	training_log

19	22:38	8×H100 (runpod)	submit_s42_20260323_223824.log	1.1211	OK	↑0.0%	training_log

20	22:56	8×H100 (runpod)	submit_s7_20260323_225635.log	1.1205	OK	↓0.0%	training_log

21	23:15	—	XSA-11 + GPTQ block64/percdamp002 — 3-seed mean 1.1208 BPB (×2)	1.1208	OK	↓0.0%	details

22	23:16	—	XSA-11 + GPTQ block64/percdamp002 — 3-seed mean val_bpb 1.1208 (×2)	1.1208	OK	↓0.0%	details

TIME

ABLATION

VAL_BPB

STATUS

Δ %

OPEN