2026-03-22 — Research Day

DAILY RESEARCH REPORTBACK TO MASTER SHEET2026-03-22

val_bpb · 697 of 718 · 11 warnings

findings mentioning this date

| 2026-03-22 | 11 | 15 |

audit summary

{
  "date": "2026-03-22",
  "fatal": 0,
  "headline_metric": "val_bpb",
  "raw_records": 718,
  "render_records": 697,
  "test_rows": 15,
  "warnings": 11
}

479 commits on 2026-03-22

0b29043b 05:00:34 parameter-golf-lab Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
00b8d97f 05:00:34 parameter-golf-lab Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
b34c3de3 05:00:34 parameter-golf-lab Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
a9e5d065 05:00:34 parameter-golf-lab Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
0b29043b 05:00:34 sota_crawler Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
b34c3de3 05:00:34 sota_crawler Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
a9e5d065 05:00:34 sota_crawler Add PR374-depth: 12L/4KV/2.625xMLP (same params, +1 layer) + EMA + QAT + warmdown
b485ef27 05:38:29 parameter-golf-lab Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
0b92cd95 05:38:29 parameter-golf-lab Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
7fba7559 05:38:29 parameter-golf-lab Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
9049b860 05:38:29 parameter-golf-lab Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
b485ef27 05:38:29 sota_crawler Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
7fba7559 05:38:29 sota_crawler Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
9049b860 05:38:29 sota_crawler Add minified train_gpt (30KB vs 74KB) with EMA+SWA edge
641a4817 05:42:25 parameter-golf-lab Fix log0 keyword arg bug in minified script
4ff70c45 05:42:25 parameter-golf-lab Fix log0 keyword arg bug in minified script
48711f65 05:42:25 parameter-golf-lab Fix log0 keyword arg bug in minified script
602a6d2f 05:42:25 parameter-golf-lab Fix log0 keyword arg bug in minified script
641a4817 05:42:25 sota_crawler Fix log0 keyword arg bug in minified script
48711f65 05:42:25 sota_crawler Fix log0 keyword arg bug in minified script
602a6d2f 05:42:25 sota_crawler Fix log0 keyword arg bug in minified script
71d05d19 05:43:27 parameter-golf-lab Fix Block kwarg: layer_idx -> li in minified script
6aa7a9f0 05:43:27 parameter-golf-lab Fix Block kwarg: layer_idx -> li in minified script
c328493c 05:43:27 parameter-golf-lab Fix Block kwarg: layer_idx -> li in minified script
6990d604 05:43:27 parameter-golf-lab Fix Block kwarg: layer_idx -> li in minified script
71d05d19 05:43:27 sota_crawler Fix Block kwarg: layer_idx -> li in minified script
c328493c 05:43:27 sota_crawler Fix Block kwarg: layer_idx -> li in minified script
6990d604 05:43:27 sota_crawler Fix Block kwarg: layer_idx -> li in minified script
a5bc2d49 05:44:47 parameter-golf-lab Fix a.te -> a.tie_embeddings in minified script
8ac59a55 05:44:47 parameter-golf-lab Fix a.te -> a.tie_embeddings in minified script
e190db52 05:44:47 parameter-golf-lab Fix a.te -> a.tie_embeddings in minified script
833efbf7 05:44:47 parameter-golf-lab Fix a.te -> a.tie_embeddings in minified script
a5bc2d49 05:44:47 sota_crawler Fix a.te -> a.tie_embeddings in minified script
e190db52 05:44:47 sota_crawler Fix a.te -> a.tie_embeddings in minified script
833efbf7 05:44:47 sota_crawler Fix a.te -> a.tie_embeddings in minified script
4bfec05a 05:46:58 parameter-golf-lab Add pr374_submit: trimmed winner for submission
a51766bd 05:46:58 parameter-golf-lab Add pr374_submit: trimmed winner for submission
59f6d3de 05:46:58 parameter-golf-lab Add pr374_submit: trimmed winner for submission
5fff32a0 05:46:58 parameter-golf-lab Add pr374_submit: trimmed winner for submission
4bfec05a 05:46:58 sota_crawler Add pr374_submit: trimmed winner for submission
59f6d3de 05:46:58 sota_crawler Add pr374_submit: trimmed winner for submission
5fff32a0 05:46:58 sota_crawler Add pr374_submit: trimmed winner for submission
6f542202 05:50:15 parameter-golf-lab Add PolyAttention, DiffAttention, DGAttention variants
2fb39c20 05:50:26 parameter-golf-lab Remove duplicate stride=64 eval from submit script
91023651 05:50:26 parameter-golf-lab Remove duplicate stride=64 eval from submit script
038b2e7c 05:50:26 parameter-golf-lab Remove duplicate stride=64 eval from submit script
8aa618ad 05:50:26 parameter-golf-lab Remove duplicate stride=64 eval from submit script
2fb39c20 05:50:26 sota_crawler Remove duplicate stride=64 eval from submit script
038b2e7c 05:50:26 sota_crawler Remove duplicate stride=64 eval from submit script
8aa618ad 05:50:26 sota_crawler Remove duplicate stride=64 eval from submit script
cc4752e6 05:52:18 parameter-golf-lab Revert "Remove duplicate stride=64 eval from submit script"
f48b98f7 05:52:18 parameter-golf-lab Revert "Remove duplicate stride=64 eval from submit script"
8595a9a4 05:52:18 parameter-golf-lab Revert "Remove duplicate stride=64 eval from submit script"
2ebc6315 05:52:18 parameter-golf-lab Revert "Remove duplicate stride=64 eval from submit script"
cc4752e6 05:52:18 sota_crawler Revert "Remove duplicate stride=64 eval from submit script"
8595a9a4 05:52:18 sota_crawler Revert "Remove duplicate stride=64 eval from submit script"
2ebc6315 05:52:18 sota_crawler Revert "Remove duplicate stride=64 eval from submit script"
b2f12dd8 05:58:08 parameter-golf-lab Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
9692858b 05:58:08 parameter-golf-lab Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
e8692f5d 05:58:08 parameter-golf-lab Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
194cf3df 05:58:08 parameter-golf-lab Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
b2f12dd8 05:58:08 sota_crawler Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
e8692f5d 05:58:08 sota_crawler Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
194cf3df 05:58:08 sota_crawler Add fractal cadence test cadence test cadence H100 script: 4 unique × 3 loops + ortho positions
bbbd3a77 06:04:05 parameter-golf-lab Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
6eaaf819 06:04:05 parameter-golf-lab Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
f0eab081 06:04:05 parameter-golf-lab Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
a47bfa24 06:04:05 parameter-golf-lab Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
bbbd3a77 06:04:05 sota_crawler Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
f0eab081 06:04:05 sota_crawler Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
a47bfa24 06:04:05 sota_crawler Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 steps) + H100 script
191d368d 06:08:37 parameter-golf-lab Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
f43c7202 06:08:37 parameter-golf-lab Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
7d93af3e 06:08:37 parameter-golf-lab Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
e3770d17 06:08:37 parameter-golf-lab Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
191d368d 06:08:37 sota_crawler Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
7d93af3e 06:08:37 sota_crawler Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
e3770d17 06:08:37 sota_crawler Add pr374_slim: comment-stripped pr374_safe (67KB vs 74KB, -6.4KB)
0b98a304 06:13:40 parameter-golf-lab Add fractal cadence test cadence test cadence auto-research sweep runner
fb551c27 06:13:40 parameter-golf-lab Add fractal cadence test cadence test cadence auto-research sweep runner
594315f7 06:13:40 parameter-golf-lab Add fractal cadence test cadence test cadence auto-research sweep runner
cfc576e0 06:13:40 parameter-golf-lab Add fractal cadence test cadence test cadence auto-research sweep runner
0b98a304 06:13:40 sota_crawler Add fractal cadence test cadence test cadence auto-research sweep runner
594315f7 06:13:40 sota_crawler Add fractal cadence test cadence test cadence auto-research sweep runner
cfc576e0 06:13:40 sota_crawler Add fractal cadence test cadence test cadence auto-research sweep runner
15efb77e 06:20:29 parameter-golf-lab Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
d04199b2 06:20:29 parameter-golf-lab Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
64a606aa 06:20:29 parameter-golf-lab Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
86b1161a 06:20:29 parameter-golf-lab Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
15efb77e 06:20:29 sota_crawler Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
64a606aa 06:20:29 sota_crawler Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
86b1161a 06:20:29 sota_crawler Fix FA3 head_dim limit: 12 heads / 6 kv_heads for dim=768 (head_dim=64)
6c294a28 06:26:45 parameter-golf-lab Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
2f3e804a 06:26:45 parameter-golf-lab Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
545f4d0d 06:26:45 parameter-golf-lab Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
166d3b4e 06:26:45 parameter-golf-lab Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
6c294a28 06:26:45 sota_crawler Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
545f4d0d 06:26:45 sota_crawler Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
166d3b4e 06:26:45 sota_crawler Add pr374_ttt: PR374 + EMA + TTT(20ep,SAM) — rock the house
278715c9 06:28:46 parameter-golf-lab Fix early QAT trigger: skip warmdown/QAT for first 100 steps
469974c9 06:28:46 parameter-golf-lab Fix early QAT trigger: skip warmdown/QAT for first 100 steps
ee16ade9 06:28:46 parameter-golf-lab Fix early QAT trigger: skip warmdown/QAT for first 100 steps
7351d10e 06:28:46 parameter-golf-lab Fix early QAT trigger: skip warmdown/QAT for first 100 steps
278715c9 06:28:46 sota_crawler Fix early QAT trigger: skip warmdown/QAT for first 100 steps
ee16ade9 06:28:46 sota_crawler Fix early QAT trigger: skip warmdown/QAT for first 100 steps
7351d10e 06:28:46 sota_crawler Fix early QAT trigger: skip warmdown/QAT for first 100 steps
14bf4e29 06:37:46 parameter-golf-lab Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
4a4908a3 06:37:46 parameter-golf-lab Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
80958ddc 06:37:46 parameter-golf-lab Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
79f951a3 06:37:46 parameter-golf-lab Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
14bf4e29 06:37:46 sota_crawler Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
80958ddc 06:37:46 sota_crawler Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
79f951a3 06:37:46 sota_crawler Optimize fractal cadence test cadence test: tuned H100 defaults + focused sweep grid
41003f59 06:48:46 parameter-golf-lab Add TTT eval: test-time training on already-graded tokens
33638667 06:48:46 parameter-golf-lab Add TTT eval: test-time training on already-graded tokens
fa6f6d6d 06:48:46 parameter-golf-lab Add TTT eval: test-time training on already-graded tokens
a31c39dc 06:48:46 parameter-golf-lab Add TTT eval: test-time training on already-graded tokens
41003f59 06:48:46 sota_crawler Add TTT eval: test-time training on already-graded tokens
fa6f6d6d 06:48:46 sota_crawler Add TTT eval: test-time training on already-graded tokens
a31c39dc 06:48:46 sota_crawler Add TTT eval: test-time training on already-graded tokens
10fa0414 06:55:25 parameter-golf-lab Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
ed2e0e02 06:55:25 parameter-golf-lab Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
a043ab39 06:55:25 parameter-golf-lab Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
170c6b77 06:55:25 parameter-golf-lab Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
10fa0414 06:55:25 sota_crawler Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
a043ab39 06:55:25 sota_crawler Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
170c6b77 06:55:25 sota_crawler Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM)
fd674e23 07:09:19 parameter-golf-lab Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
4392ca76 07:09:19 parameter-golf-lab Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
307bc81b 07:09:19 parameter-golf-lab Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
30b63ba6 07:09:19 parameter-golf-lab Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
fd674e23 07:09:19 sota_crawler Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
307bc81b 07:09:19 sota_crawler Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
30b63ba6 07:09:19 sota_crawler Save pr374_throne: our #1 non-TTT record holder (1.1243 BPB)
614c3196 07:14:24 parameter-golf-lab Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
5bc584be 07:14:24 parameter-golf-lab Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
c0adc01b 07:14:24 parameter-golf-lab Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
346fd879 07:14:24 parameter-golf-lab Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
614c3196 07:14:24 sota_crawler Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
c0adc01b 07:14:24 sota_crawler Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
346fd879 07:14:24 sota_crawler Add autonomous overnight fractal cadence test cadence test optimizer (Karpathy auto-research)
a2204b02 07:21:10 parameter-golf-lab Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
d0ff238a 07:21:10 parameter-golf-lab Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
c8195b63 07:21:10 parameter-golf-lab Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
525d7550 07:21:10 parameter-golf-lab Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
a2204b02 07:21:10 sota_crawler Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
c8195b63 07:21:10 sota_crawler Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
525d7550 07:21:10 sota_crawler Rewrite autoresearch: Qwen-guided fractal cadence test cadence test optimizer via Ollama
15776db6 07:48:01 parameter-golf-lab Record: 11L EMA + GPTQ-lite + warmdown3500 + QAT@0.15 (val_bpb=1.1233, 3-seed mean)
15776db6 07:48:01 sota_crawler Record: 11L EMA + GPTQ-lite + warmdown3500 + QAT@0.15 (val_bpb=1.1233, 3-seed mean)
15776db6 07:48:01 SOTA_FINAL Record: 11L EMA + GPTQ-lite + warmdown3500 + QAT@0.15 (val_bpb=1.1233, 3-seed mean)
678ff804 16:19:40 parameter-golf-lab Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
9ae4ed29 16:19:40 parameter-golf-lab Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
6f1339f9 16:19:40 parameter-golf-lab Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
eb7f8dfd 16:19:40 parameter-golf-lab Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
678ff804 16:19:40 sota_crawler Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
6f1339f9 16:19:40 sota_crawler Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
eb7f8dfd 16:19:40 sota_crawler Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB)
75b9ace5 16:20:59 parameter-golf-lab Add pod setup script for leapfrog variants (v1/v2/v3)
e03dd345 16:20:59 parameter-golf-lab Add pod setup script for leapfrog variants (v1/v2/v3)
2205ab1e 16:20:59 parameter-golf-lab Add pod setup script for leapfrog variants (v1/v2/v3)
2717b3c5 16:20:59 parameter-golf-lab Add pod setup script for leapfrog variants (v1/v2/v3)
75b9ace5 16:20:59 sota_crawler Add pod setup script for leapfrog variants (v1/v2/v3)
2205ab1e 16:20:59 sota_crawler Add pod setup script for leapfrog variants (v1/v2/v3)
2717b3c5 16:20:59 sota_crawler Add pod setup script for leapfrog variants (v1/v2/v3)
5b9b4dcd 16:22:17 parameter-golf-lab Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
b12f7441 16:22:17 parameter-golf-lab Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
f38c563f 16:22:17 parameter-golf-lab Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
05beadf9 16:22:17 parameter-golf-lab Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
5b9b4dcd 16:22:17 sota_crawler Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
f38c563f 16:22:17 sota_crawler Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
05beadf9 16:22:17 sota_crawler Fix setup_pod.sh: apply FA3 build fixes from RUNPOD_SETUP.md
e4d4ab2d 16:36:50 parameter-golf-lab Update fractal cadence test cadence test H100 with overnight Qwen findings
3dc64e44 16:36:50 parameter-golf-lab Update fractal cadence test cadence test H100 with overnight Qwen findings
3daeff7b 16:36:50 parameter-golf-lab Update fractal cadence test cadence test H100 with overnight Qwen findings
b8914a5e 16:36:50 parameter-golf-lab Update fractal cadence test cadence test H100 with overnight Qwen findings
e4d4ab2d 16:36:50 sota_crawler Update fractal cadence test cadence test H100 with overnight Qwen findings
3daeff7b 16:36:50 sota_crawler Update fractal cadence test cadence test H100 with overnight Qwen findings
b8914a5e 16:36:50 sota_crawler Update fractal cadence test cadence test H100 with overnight Qwen findings
c2493ce7 16:41:26 parameter-golf-lab Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
5a08f56c 16:41:26 parameter-golf-lab Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
4073091d 16:41:26 parameter-golf-lab Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
82d16b44 16:41:26 parameter-golf-lab Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
c2493ce7 16:41:26 sota_crawler Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
4073091d 16:41:26 sota_crawler Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
82d16b44 16:41:26 sota_crawler Fix v1 TTT burst: apply EMA first, then QAT-aware sharpening
33d475eb 17:05:51 parameter-golf-lab Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
cd89c3ce 17:05:51 parameter-golf-lab Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
92a48d22 17:05:51 parameter-golf-lab Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
57cc969b 17:05:51 parameter-golf-lab Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
33d475eb 17:05:51 sota_crawler Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
92a48d22 17:05:51 sota_crawler Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
57cc969b 17:05:51 sota_crawler Add inner-TTT to fractal cadence test cadence test eval: recursive weight improvement per loop
fcd433ff 17:11:12 parameter-golf-lab v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
18f41c0c 17:11:12 parameter-golf-lab v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
12b55384 17:11:12 parameter-golf-lab v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
0ee493d2 17:11:12 parameter-golf-lab v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
fcd433ff 17:11:12 sota_crawler v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
12b55384 17:11:12 sota_crawler v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
0ee493d2 17:11:12 sota_crawler v1 combo: burst+QAT before EMA + 15 GPTQ percentiles
ab6c889f 17:17:22 parameter-golf-lab Record: 11L TTT Burst + GPTQ-15 + EMA + QAT (val_bpb=1.1232, seed 1337)
ab6c889f 17:17:22 sota_crawler Record: 11L TTT Burst + GPTQ-15 + EMA + QAT (val_bpb=1.1232, seed 1337)
481f3501 17:20:07 parameter-golf-lab Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
e239db2f 17:20:07 parameter-golf-lab Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
f45beefc 17:20:07 parameter-golf-lab Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
7236a59d 17:20:07 parameter-golf-lab Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
481f3501 17:20:07 sota_crawler Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
f45beefc 17:20:07 sota_crawler Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
7236a59d 17:20:07 sota_crawler Isolate TTT to fractal cadence test cadence test layers only: blocks + loop_pos + skip_weights
62f8f758 17:27:04 parameter-golf-lab Add TTT drift gate: snap block weights back toward trained originals
b5c90dd2 17:27:04 parameter-golf-lab Add TTT drift gate: snap block weights back toward trained originals
66ebae7d 17:27:04 parameter-golf-lab Add TTT drift gate: snap block weights back toward trained originals
0e6fd680 17:27:04 parameter-golf-lab Add TTT drift gate: snap block weights back toward trained originals
62f8f758 17:27:04 sota_crawler Add TTT drift gate: snap block weights back toward trained originals
66ebae7d 17:27:04 sota_crawler Add TTT drift gate: snap block weights back toward trained originals
0e6fd680 17:27:04 sota_crawler Add TTT drift gate: snap block weights back toward trained originals
131166a9 17:32:37 parameter-golf-lab Update submission with accurate 2-seed results and correct train_gpt.py
131166a9 17:32:37 sota_crawler Update submission with accurate 2-seed results and correct train_gpt.py
d9eb0497 17:32:42 parameter-golf-lab Remove accidentally staged files from submission branch
d9eb0497 17:32:42 sota_crawler Remove accidentally staged files from submission branch
ad9f7bea 17:48:43 parameter-golf-lab Add v2 self-distillation variant (val_bpb=1.1233, seed 1337)
ad9f7bea 17:48:43 sota_crawler Add v2 self-distillation variant (val_bpb=1.1233, seed 1337)
75bb432d 17:52:47 parameter-golf-lab Add v4: burst + distill + train_seq_len=1024 for more steps
3c6f5bc1 17:52:47 parameter-golf-lab Add v4: burst + distill + train_seq_len=1024 for more steps
39c346b1 17:52:47 parameter-golf-lab Add v4: burst + distill + train_seq_len=1024 for more steps
50638c64 17:52:47 parameter-golf-lab Add v4: burst + distill + train_seq_len=1024 for more steps
75bb432d 17:52:47 sota_crawler Add v4: burst + distill + train_seq_len=1024 for more steps
39c346b1 17:52:47 sota_crawler Add v4: burst + distill + train_seq_len=1024 for more steps
50638c64 17:52:47 sota_crawler Add v4: burst + distill + train_seq_len=1024 for more steps
9cad0e01 18:01:23 parameter-golf-lab Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
c75145c6 18:01:23 parameter-golf-lab Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
f2ed46ee 18:01:23 parameter-golf-lab Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
1529671e 18:01:23 parameter-golf-lab Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
9cad0e01 18:01:23 sota_crawler Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
f2ed46ee 18:01:23 sota_crawler Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
1529671e 18:01:23 sota_crawler Add SOTA autoresearch: Qwen-guided edge finding for 1.1233 record
39ad4e9a 18:05:27 parameter-golf-lab Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
5808dddd 18:05:27 parameter-golf-lab Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
2ba8b6fd 18:05:27 parameter-golf-lab Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
1f65df52 18:05:27 parameter-golf-lab Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
39ad4e9a 18:05:27 sota_crawler Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
2ba8b6fd 18:05:27 sota_crawler Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
1f65df52 18:05:27 sota_crawler Fix TTT graph bug + upgrade to 3 blocks x 896d (23M params, ~14MB)
af6b29a5 18:22:48 parameter-golf-lab Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
1a9b6816 18:22:48 parameter-golf-lab Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
c3b0643e 18:22:48 parameter-golf-lab Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
041a8543 18:22:48 parameter-golf-lab Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
af6b29a5 18:22:48 sota_crawler Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
c3b0643e 18:22:48 sota_crawler Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
041a8543 18:22:48 sota_crawler Save leapfrog experiment results: 6 variants tested, v1 wins at 1.12319
fbfa775d 18:36:51 parameter-golf-lab fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
3bb985c6 18:36:51 parameter-golf-lab fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
c3eda014 18:36:51 parameter-golf-lab fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
d14df67c 18:36:51 parameter-golf-lab fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
fbfa775d 18:36:51 sota_crawler fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
c3eda014 18:36:51 sota_crawler fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
d14df67c 18:36:51 sota_crawler fractal cadence test cadence test v2: dim=960 (25.7M, ~15.8MB), Muon LR back to 0.025
cc78f04b 18:39:26 parameter-golf-lab Disable TTT in v2 (backward graph bug), focus on bigger model
41d743f5 18:39:26 parameter-golf-lab Disable TTT in v2 (backward graph bug), focus on bigger model
1a06df24 18:39:26 parameter-golf-lab Disable TTT in v2 (backward graph bug), focus on bigger model
7b5232a5 18:39:26 parameter-golf-lab Disable TTT in v2 (backward graph bug), focus on bigger model
cc78f04b 18:39:26 sota_crawler Disable TTT in v2 (backward graph bug), focus on bigger model
1a06df24 18:39:26 sota_crawler Disable TTT in v2 (backward graph bug), focus on bigger model
7b5232a5 18:39:26 sota_crawler Disable TTT in v2 (backward graph bug), focus on bigger model
7d9b1d9c 18:40:17 parameter-golf-lab Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
6a6802b9 18:40:17 parameter-golf-lab Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
e5834b45 18:40:17 parameter-golf-lab Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
3699d190 18:40:17 parameter-golf-lab Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
7d9b1d9c 18:40:17 sota_crawler Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
e5834b45 18:40:17 sota_crawler Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
3699d190 18:40:17 sota_crawler Add v5: QAT percentile fix + TrigramHash + EMA-SWA blend
9c9a2e1e 18:41:23 parameter-golf-lab Fix TTT graph bug: fresh vc + detached inputs per loop
7e1220db 18:41:23 parameter-golf-lab Fix TTT graph bug: fresh vc + detached inputs per loop
9273a987 18:41:23 parameter-golf-lab Fix TTT graph bug: fresh vc + detached inputs per loop
79abf6cf 18:41:23 parameter-golf-lab Fix TTT graph bug: fresh vc + detached inputs per loop
9c9a2e1e 18:41:23 sota_crawler Fix TTT graph bug: fresh vc + detached inputs per loop
9273a987 18:41:23 sota_crawler Fix TTT graph bug: fresh vc + detached inputs per loop
79abf6cf 18:41:23 sota_crawler Fix TTT graph bug: fresh vc + detached inputs per loop
63c9b9e2 18:55:19 parameter-golf-lab Two v2 scripts: with TTT and without, for A/B comparison
05000048 18:55:19 parameter-golf-lab Two v2 scripts: with TTT and without, for A/B comparison
b73d2298 18:55:19 parameter-golf-lab Two v2 scripts: with TTT and without, for A/B comparison
bf17d79b 18:55:19 parameter-golf-lab Two v2 scripts: with TTT and without, for A/B comparison
63c9b9e2 18:55:19 sota_crawler Two v2 scripts: with TTT and without, for A/B comparison
b73d2298 18:55:19 sota_crawler Two v2 scripts: with TTT and without, for A/B comparison
bf17d79b 18:55:19 sota_crawler Two v2 scripts: with TTT and without, for A/B comparison
b4a02886 19:14:04 parameter-golf-lab fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
0243779c 19:14:04 parameter-golf-lab fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
52c49dc5 19:14:04 parameter-golf-lab fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
61aee008 19:14:04 parameter-golf-lab fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
b4a02886 19:14:04 sota_crawler fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
52c49dc5 19:14:04 sota_crawler fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
61aee008 19:14:04 sota_crawler fractal cadence test cadence test v3: MLP 3.0->3.3 fills 16MB budget (27.4M params, ~15.2MB)
a23d65ae 19:32:33 parameter-golf-lab fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
a7348276 19:32:33 parameter-golf-lab fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
c7dac39c 19:32:33 parameter-golf-lab fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
672d0375 19:32:33 parameter-golf-lab fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
a23d65ae 19:32:33 sota_crawler fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
c7dac39c 19:32:33 sota_crawler fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
672d0375 19:32:33 sota_crawler fractal cadence test cadence test v4: 1.5x batch (1.18M tokens/step) to use spare VRAM
e7f7a502 19:35:14 parameter-golf-lab Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
b42deae8 19:35:14 parameter-golf-lab Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
c28434ed 19:35:14 parameter-golf-lab Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
5f9b9e9b 19:35:14 parameter-golf-lab Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
e7f7a502 19:35:14 sota_crawler Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
c28434ed 19:35:14 sota_crawler Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
5f9b9e9b 19:35:14 sota_crawler Log recursive weight-sharing transformer: fractal cadence test cadence test cadence baseline at 1.2113 BPB
d435755c 19:51:21 parameter-golf-lab Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
659707bc 19:51:21 parameter-golf-lab Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
8a50d685 19:51:21 parameter-golf-lab Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
e7585446 19:51:21 parameter-golf-lab Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
d435755c 19:51:21 sota_crawler Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
8a50d685 19:51:21 sota_crawler Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
e7585446 19:51:21 sota_crawler Add v6: fractal cadence test cadence test 6L×2 loops (12 effective) + 480d/16H/4xMLP
95053441 20:02:40 parameter-golf-lab recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
9ea76e2e 20:02:40 parameter-golf-lab recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
feef78ee 20:02:40 parameter-golf-lab recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
f196e844 20:02:40 parameter-golf-lab recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
95053441 20:02:40 sota_crawler recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
feef78ee 20:02:40 sota_crawler recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
f196e844 20:02:40 sota_crawler recursive weight-sharing transformer v5: TTT warmup-then-freeze to capture 1.19 peak
0a319512 20:03:10 parameter-golf-lab Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
bbbb285e 20:03:10 parameter-golf-lab Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
b60ee7ad 20:03:10 parameter-golf-lab Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
7fba307b 20:03:10 parameter-golf-lab Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
0a319512 20:03:10 sota_crawler Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
b60ee7ad 20:03:10 sota_crawler Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
7fba307b 20:03:10 sota_crawler Log all recursive weight-sharing transformer results: v3 TTT peak 1.1901, v4 batch 1.2186
89890d91 20:04:27 parameter-golf-lab Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
3e0182b7 20:04:27 parameter-golf-lab Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
4d029957 20:04:27 parameter-golf-lab Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
5f4eb5f4 20:04:27 parameter-golf-lab Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
89890d91 20:04:27 sota_crawler Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
4d029957 20:04:27 sota_crawler Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
5f4eb5f4 20:04:27 sota_crawler Fix v6: 512d/16H/8KV (head_dim=32, FA3 requires multiple of 8)
e26df9b8 20:24:20 parameter-golf-lab Log v5/v6 results + add fractal cadence test cadence test sweep script
1017b30b 20:24:20 parameter-golf-lab Log v5/v6 results + add fractal cadence test cadence test sweep script
cb5acc1c 20:24:20 parameter-golf-lab Log v5/v6 results + add fractal cadence test cadence test sweep script
74eebb6a 20:24:20 parameter-golf-lab Log v5/v6 results + add fractal cadence test cadence test sweep script
e26df9b8 20:24:20 sota_crawler Log v5/v6 results + add fractal cadence test cadence test sweep script
cb5acc1c 20:24:20 sota_crawler Log v5/v6 results + add fractal cadence test cadence test sweep script
74eebb6a 20:24:20 sota_crawler Log v5/v6 results + add fractal cadence test cadence test sweep script
df42c95c 20:56:11 parameter-golf-lab Add seed 2024: 1.1239 BPB, 15.50 MB — 3-seed validation complete
df42c95c 20:56:11 sota_crawler Add seed 2024: 1.1239 BPB, 15.50 MB — 3-seed validation complete
cbcd7f71 20:56:19 parameter-golf-lab Remove accidentally staged files
cbcd7f71 20:56:19 sota_crawler Remove accidentally staged files
a474a327 20:58:10 parameter-golf-lab Add recursive weight-sharing transformer deep calibration autoresearch
6b333a81 21:03:24 parameter-golf-lab recursive weight-sharing transformer 8-hour single GPU longrun
67ca84b8 21:03:24 parameter-golf-lab recursive weight-sharing transformer 8-hour single GPU longrun
26bb918f 21:03:24 parameter-golf-lab recursive weight-sharing transformer 8-hour single GPU longrun
bd37700e 21:03:24 parameter-golf-lab recursive weight-sharing transformer 8-hour single GPU longrun
6b333a81 21:03:24 sota_crawler recursive weight-sharing transformer 8-hour single GPU longrun
26bb918f 21:03:24 sota_crawler recursive weight-sharing transformer 8-hour single GPU longrun
bd37700e 21:03:24 sota_crawler recursive weight-sharing transformer 8-hour single GPU longrun
aec30f3e 21:29:00 parameter-golf-lab Fix PolyAttention: use cubic x^3 with learned dynamic scaling per paper
0cba9336 22:19:31 parameter-golf-lab DGAttention v2: asymmetric D, EMA baseline, hybrid payload
91a95e78 22:25:43 parameter-golf-lab DG v2.1: replace sequential EMA loop with parallel cumsum baseline
475acf31 00:54:14 parameter-golf-lab Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
e0527c1e 00:54:14 parameter-golf-lab Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
4394e446 00:54:14 parameter-golf-lab Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
cc2ff3a3 00:54:14 parameter-golf-lab Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
475acf31 00:54:14 sota_crawler Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
4394e446 00:54:14 sota_crawler Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
cc2ff3a3 00:54:14 sota_crawler Complete recursive weight-sharing transformer documentation: all runs, findings, architecture analysis
670ca139 00:55:00 parameter-golf-lab Add v7: legal score-first TTT eval (PR #461 recipe)
b34f4249 00:55:00 parameter-golf-lab Add v7: legal score-first TTT eval (PR #461 recipe)
16336852 00:55:00 parameter-golf-lab Add v7: legal score-first TTT eval (PR #461 recipe)
dec45942 00:55:00 parameter-golf-lab Add v7: legal score-first TTT eval (PR #461 recipe)
670ca139 00:55:00 sota_crawler Add v7: legal score-first TTT eval (PR #461 recipe)
16336852 00:55:00 sota_crawler Add v7: legal score-first TTT eval (PR #461 recipe)
dec45942 00:55:00 sota_crawler Add v7: legal score-first TTT eval (PR #461 recipe)
8027566a 01:56:17 parameter-golf-lab Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
03849f53 01:56:17 parameter-golf-lab Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
d03b8aad 01:56:17 parameter-golf-lab Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
735b22f7 01:56:17 parameter-golf-lab Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
8027566a 01:56:17 sota_crawler Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
d03b8aad 01:56:17 sota_crawler Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
735b22f7 01:56:17 sota_crawler Add v8: mutual learning — flat GPT + fractal cadence test cadence test GPT co-training
0b97cd81 01:59:40 parameter-golf-lab v7: add TTT early stopping — train only first 60 chunks
c4463514 01:59:40 parameter-golf-lab v7: add TTT early stopping — train only first 60 chunks
7e42a13e 01:59:40 parameter-golf-lab v7: add TTT early stopping — train only first 60 chunks
4fbad042 01:59:40 parameter-golf-lab v7: add TTT early stopping — train only first 60 chunks
0b97cd81 01:59:40 sota_crawler v7: add TTT early stopping — train only first 60 chunks
7e42a13e 01:59:40 sota_crawler v7: add TTT early stopping — train only first 60 chunks
4fbad042 01:59:40 sota_crawler v7: add TTT early stopping — train only first 60 chunks
b23d437e 02:10:02 parameter-golf-lab Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
8fd43260 02:10:02 parameter-golf-lab Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
4ead1eed 02:10:02 parameter-golf-lab Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
6d68de38 02:10:02 parameter-golf-lab Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
b23d437e 02:10:02 sota_crawler Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
4ead1eed 02:10:02 sota_crawler Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
6d68de38 02:10:02 sota_crawler Update v6 to sweep winner: 4×3/512d/8H/4KV/4xMLP
b452f849 02:40:55 parameter-golf-lab Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
7c9d8cc8 02:40:55 parameter-golf-lab Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
d3b1e2fd 02:40:55 parameter-golf-lab Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
bd1e51d7 02:40:55 parameter-golf-lab Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
b452f849 02:40:55 sota_crawler Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
d3b1e2fd 02:40:55 sota_crawler Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
bd1e51d7 02:40:55 sota_crawler Log v7 TTT results: early stop at 60 chunks = 1.12312 (best)
9fea2b6d 03:04:30 parameter-golf-lab The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
19acd25d 03:04:30 parameter-golf-lab The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
961dbbd2 03:04:30 parameter-golf-lab The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
afba0768 03:04:30 parameter-golf-lab The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
9fea2b6d 03:04:30 sota_crawler The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
961dbbd2 03:04:30 sota_crawler The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
afba0768 03:04:30 sota_crawler The recursive weight-sharing depth variant: 6L x 2 loops, dim=640, MLP 4x, 28.5M params
cd3d5bc6 03:10:11 parameter-golf-lab Log higher LR (0.030) + MTP results — both worse
504d3d99 03:10:11 parameter-golf-lab Log higher LR (0.030) + MTP results — both worse
c51d394e 03:10:11 parameter-golf-lab Log higher LR (0.030) + MTP results — both worse
4fafae88 03:10:11 parameter-golf-lab Log higher LR (0.030) + MTP results — both worse
cd3d5bc6 03:10:11 sota_crawler Log higher LR (0.030) + MTP results — both worse
c51d394e 03:10:11 sota_crawler Log higher LR (0.030) + MTP results — both worse
4fafae88 03:10:11 sota_crawler Log higher LR (0.030) + MTP results — both worse
c0db6ddd 03:11:00 parameter-golf-lab Fix DG v2.1 dk reshape: use num_kv_heads * d_head_dim for dk projection
924b77ca 03:28:39 parameter-golf-lab The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
c8552f43 03:28:39 parameter-golf-lab The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
0263f445 03:28:39 parameter-golf-lab The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
c40bfce6 03:28:39 parameter-golf-lab The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
924b77ca 03:28:39 sota_crawler The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
0263f445 03:28:39 sota_crawler The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
c40bfce6 03:28:39 sota_crawler The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lead
f5b78fb4 03:30:34 parameter-golf-lab Stack distillation onto recursive weight-sharing depth variant
7f556e65 03:30:34 parameter-golf-lab Stack distillation onto recursive weight-sharing depth variant
d60df798 03:30:34 parameter-golf-lab Stack distillation onto recursive weight-sharing depth variant
57062a5c 03:30:34 parameter-golf-lab Stack distillation onto recursive weight-sharing depth variant
f5b78fb4 03:30:34 sota_crawler Stack distillation onto recursive weight-sharing depth variant
d60df798 03:30:34 sota_crawler Stack distillation onto recursive weight-sharing depth variant
57062a5c 03:30:34 sota_crawler Stack distillation onto recursive weight-sharing depth variant
1ea90b74 03:34:25 parameter-golf-lab Prep recursive weight-sharing depth variant PR submission (draft — needs image)
f1ac766e 03:34:25 parameter-golf-lab Prep recursive weight-sharing depth variant PR submission (draft — needs image)
868accf4 03:34:25 parameter-golf-lab Prep recursive weight-sharing depth variant PR submission (draft — needs image)
eef3124d 03:34:25 parameter-golf-lab Prep recursive weight-sharing depth variant PR submission (draft — needs image)
1ea90b74 03:34:25 sota_crawler Prep recursive weight-sharing depth variant PR submission (draft — needs image)
868accf4 03:34:25 sota_crawler Prep recursive weight-sharing depth variant PR submission (draft — needs image)
eef3124d 03:34:25 sota_crawler Prep recursive weight-sharing depth variant PR submission (draft — needs image)
9e0b23a4 03:36:31 parameter-golf-lab Add train script to recursive weight-sharing transformer submission folder
edea2e13 03:36:31 parameter-golf-lab Add train script to recursive weight-sharing transformer submission folder
132f4494 03:36:31 parameter-golf-lab Add train script to recursive weight-sharing transformer submission folder
2c918975 03:36:31 parameter-golf-lab Add train script to recursive weight-sharing transformer submission folder
9e0b23a4 03:36:31 sota_crawler Add train script to recursive weight-sharing transformer submission folder
132f4494 03:36:31 sota_crawler Add train script to recursive weight-sharing transformer submission folder
2c918975 03:36:31 sota_crawler Add train script to recursive weight-sharing transformer submission folder
fb530e28 03:51:24 parameter-golf-lab TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
fe27bfb8 03:51:24 parameter-golf-lab TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
5229c66a 03:51:24 parameter-golf-lab TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
cb4f8790 03:51:24 parameter-golf-lab TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
fb530e28 03:51:24 sota_crawler TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
5229c66a 03:51:24 sota_crawler TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
cb4f8790 03:51:24 sota_crawler TTT stabilization: EMA scoring + fixed cosine LR + embed freeze
46894842 04:17:43 parameter-golf-lab recursive weight-sharing transformer: Recursive Weight Sharing + MLP 4x (1.1478 BPB)
37aa1fe4 04:17:43 parameter-golf-lab recursive weight-sharing transformer: Recursive Weight Sharing + MLP 4x (1.1478 BPB)
46894842 04:17:43 sota_crawler recursive weight-sharing transformer: Recursive Weight Sharing + MLP 4x (1.1478 BPB)
37aa1fe4 04:17:43 sota_crawler recursive weight-sharing transformer: Recursive Weight Sharing + MLP 4x (1.1478 BPB)
08635604 04:21:57 parameter-golf-lab Add train log to recursive weight-sharing transformer submission
5305a17c 04:21:57 parameter-golf-lab Add train log to recursive weight-sharing transformer submission
08635604 04:21:57 sota_crawler Add train log to recursive weight-sharing transformer submission
5305a17c 04:21:57 sota_crawler Add train log to recursive weight-sharing transformer submission
93205f17 04:23:45 parameter-golf-lab Add GPTQ quantization: Hessian-aware error compensation for int6
fe467672 04:23:45 parameter-golf-lab Add GPTQ quantization: Hessian-aware error compensation for int6
f20e3876 04:23:45 parameter-golf-lab Add GPTQ quantization: Hessian-aware error compensation for int6
ffad368a 04:23:45 parameter-golf-lab Add GPTQ quantization: Hessian-aware error compensation for int6
93205f17 04:23:45 sota_crawler Add GPTQ quantization: Hessian-aware error compensation for int6
f20e3876 04:23:45 sota_crawler Add GPTQ quantization: Hessian-aware error compensation for int6
ffad368a 04:23:45 sota_crawler Add GPTQ quantization: Hessian-aware error compensation for int6
939c140c 04:29:03 parameter-golf-lab Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
5ef90aef 04:29:03 parameter-golf-lab Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
011e7d94 04:29:03 parameter-golf-lab Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
8e0a01f7 04:29:03 parameter-golf-lab Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
939c140c 04:29:03 sota_crawler Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
011e7d94 04:29:03 sota_crawler Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
8e0a01f7 04:29:03 sota_crawler Stack low-hanging fruit: fix GPTQ, earlier QAT, matched clipping
e523ad13 04:32:18 parameter-golf-lab DG v2.2: blended baseline (previous token + global mean)
177e03c8 04:39:30 parameter-golf-lab Add per-layer β/γ diagnostic logging for DG attention
e15e5edf 04:59:35 parameter-golf-lab Add post-quant training burst: repair quant damage before eval
17b34e91 04:59:35 parameter-golf-lab Add post-quant training burst: repair quant damage before eval
988e83b0 04:59:35 parameter-golf-lab Add post-quant training burst: repair quant damage before eval
4f3bde64 04:59:35 parameter-golf-lab Add post-quant training burst: repair quant damage before eval
e15e5edf 04:59:35 sota_crawler Add post-quant training burst: repair quant damage before eval
988e83b0 04:59:35 sota_crawler Add post-quant training burst: repair quant damage before eval
4f3bde64 04:59:35 sota_crawler Add post-quant training burst: repair quant damage before eval

DATE2026-03-22

DAY005

PRIMARY METRICVAL_BPB

TESTS RUN15 unique (15 valid / 0 TTT leakage)

ARCHITECTURE FAMILYLEADERBOARD TRANSFORMER VARIANTS (×4) · TTT8 STRIDE-32 TEST FAMILY (×3) · RECURSIVE WEIGHT-SHARING TRANSFORMER (×2) · TTT-ONLY MODEL TEST (×1) · PROGRESSIVE MLP AND FRACTAL CADENCE TESTS (×1) · STATE-SPACE TTT VARIANT (×1) · +3 MORE (×3)

CLASSIFICATIONEXPERIMENTAL 80% (12)/NEURAL 20% (3)

SECTION 1.0 — TRAJECTORY

VAL_BPB SPREAD	n	min	median	max	download
LEADERBOARD TRANSFORMER VARIANTS	4	1.1232	1.1233	1.1243	—
TTT8 STRIDE-32 TEST FAMILY	3	1.1301	1.1307	1.1313	download
RECURSIVE WEIGHT-SHARING TRANSFORMER	2	1.1478	1.1478	1.1478	download
TTT-ONLY MODEL TEST	1	1.1240	1.1240	1.1240	download
PROGRESSIVE MLP AND FRACTAL CADENCE TESTS	1	1.6575	1.6575	1.6575	—
STATE-SPACE TTT VARIANT	1	1.1223	1.1223	1.1223	—
POST-TRAINING QUANTIZATION CALIBRATION	1	1.1232	1.1232	1.1232	download
CRAWLER ABLATION CONTROL SET	1	1.2113	1.2113	1.2113	—
OTHER	1	1.1232	1.1232	1.1232	—
ALL VALID	15	1.1223	1.1243	1.6575	—

VAL_BPB SPREAD

min

median

max

download

LEADERBOARD TRANSFORMER VARIANTS

1.1232

1.1233

1.1243

—

TTT8 STRIDE-32 TEST FAMILY

1.1301

1.1307

1.1313

download

RECURSIVE WEIGHT-SHARING TRANSFORMER

1.1478

download

TTT-ONLY MODEL TEST

1.1240

download

PROGRESSIVE MLP AND FRACTAL CADENCE TESTS

1.6575

—

STATE-SPACE TTT VARIANT

1.1223

—

POST-TRAINING QUANTIZATION CALIBRATION

1.1232

download

CRAWLER ABLATION CONTROL SET

1.2113

—

OTHER

1.1232

—

ALL VALID

1.1223

1.1243

1.6575

—

SECTION 2.0 — TEST RUNS

EXPERIMENTAL TEST RUNS — 12

#	TIME	HW	ABLATION	VAL_BPB	STATUS	Δ %	OPEN
1	00:48	8×H100	run_s1337.log (×6)	1.1240	OK	↑0.3%	training_log 1 training_log 2 training_log 3 training_log 4 training_log 5 training_log 6

2	01:45	8×H100	run_s42.log (×2)	1.1307	OK	↑0.9%	training_log 1 training_log 2

3	02:05	8×H100	run_s7.log	1.1313	OK	↑0.9%	training_log

4	02:22	8×H100	run_s137.log	1.1301	OK	↑0.8%	training_log

5	03:28	—	The recursive weight-sharing depth variant: 1.1478 BPB — 0.025 from world lea…	1.1478	OK	↑2.4%	details

6	04:17	—	recursive weight-sharing transformer: Recursive Weight Sharing + MLP 4x (1.14…	1.1478	OK	↑2.4%	details

7	06:04	—	Add fractal cadence test cadence test cadence long run (1.6575 BPB @ 3929 ste…	1.6575	OK	↑47.9%	details

8	06:55	—	Log depth + TTT results: 1.1223 BPB (12L+TTT20ep+SAM) (×7)	1.1223	OK	↑0.1%	details

10	16:19	—	Add 3 leapfrog variants based on PR#414 SOTA (1.1233 BPB) (×7)	1.1233	OK	↑0.2%	details

13	17:48	—	Add v2 self-distillation variant (val_bpb=1.1233, seed 1337) (×2)	1.1233	OK	↑0.2%	details

14	19:35	—	Log recursive weight-sharing transformer: fractal cadence test cadence test c…	1.2113	OK	↑8.1%	details

15	20:56	—	Add seed 2024: 1.1239 BPB, 15.50 MB — 3-seed validation complete (×2)	1.1232	OK	↑0.2%	details

TIME

ABLATION

VAL_BPB

STATUS

Δ %

OPEN

00:48

8×H100

run_s1337.log (×6)

1.1240

↑0.3%

training_log 1

training_log 2

training_log 3

training_log 4

training_log 5

training_log 6

01:45

8×H100

run_s42.log (×2)

1.1307

↑0.9%

training_log 1

training_log 2