CPU cached decode

FP32, 16 threads, batch 1

Tokens / sec

Supra2-100M-Base56.0
Speck1-140M55.1
SmolLM2-135M47.4
GPT-X2.5-135M47.2
BananaMind-2-Pro43.0

RTX 3090 cached decode

BF16, batch 1

Tokens / sec

Supra2-100M-Base298.1
Speck1-140M247.3
SmolLM2-135M157.7
BananaMind-2-Pro140.3
GPT-X2.5-135M125.0

Local batch-one measurements excluding tokenization. CPU uses FP32 with 16 threads; RTX 3090 uses BF16.