Research
Banger paper from Microsoft and Cornell. If you have looked at thinking tokens and decided you cannot afford the context, read this one. (bo…
Banger paper from Microsoft and Cornell. If you have looked at thinking tokens and decided you cannot afford the context, read this one. (bookmark it) A pause token buys the model extra compute for ea
Banger paper from Microsoft and Cornell. If you have looked at thinking tokens and decided you cannot afford the context, read this one. (bookmark it) A pause token buys the model extra compute for each next-token prediction, and it pays for that compute with a sequence position. Free pause tokens carry the same compute in a parallel prediction stream over a weight-shared backbone, riding an existing position instead of adding one. At inference it adds nothing to context length, leaves the KV cache unchanged, and costs essentially no extra latency, since the additional flops are not the throughput bottleneck. On a 1B model it improves next-token prediction by 2 to 3 centinats. The cost moves to training, where the overhead against an optimized pretraining pipeline comes down to about 1.14x while keeping most of the benefit. Paper: https://arxiv.org/abs/2609.03807 Chat with Paper: https://academy.dair.ai/papers/free-pause-tokens-2609.03807
Related
- Banger paper on harness continual learning. (bookmark it) If you already are allowing your agents to rewrite their own prompts, skills, or m…
- Banger paper from Stanford on efficient test-time scaling. If you run agents that think for a long time, this one is worth your time. (bookm…
- Great paper from Google and colleagues. It proposes an interesting approach to improve agents on long-horizon tasks. Long-running agents slo…
Source: DAIR.AI (X) | 2026-09-05