12L GQA MIXED-BIT TOWERx2
12-layer GQA tower: 8/4 GQA, RoPE-16, Bigram 2048, XSA on the last 11 layers, and mixed-bit packaging.
BODYnum_layers=12 · num_heads=8 · num_kv_heads=4 · rope_dims=16 · xsa_last_n=11 · BigramHash vocab=2048
QUANT / EVALattention int5 · mlp int6 · aux int6 · embed int8 · other int8 · Brotli artifact · 16 MB artifact cap
RUN SHAPE8xH100 · train_seq_len=2048 · eval_seq_len=2048 · 600s wallclock

