MTP doubled my generation, but at 64k context it cuts total latency by only ~3%. Measured the prefill wall on a 3090.

MTP (Multi-Token Prediction) doubled generation speed but provided only ~3% total latency reduction at 64k context length on an RTX 3090 GPU. The limited overall benefit at large context sizes suggests a prefill bottleneck where context processing, rather than token generation, dominates the latency profile. This finding indicates MTP's effectiveness diminishes significantly with longer contexts due to GPU memory bandwidth constraints during the prefill phase.

Source: r/ollama | 2026-06-10