Research
Prefix Sliding for efficient test-time scaling
arXiv:2608.26070v1 Announce Type: new Abstract: Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models k
arXiv:2608.26070v1 Announce Type: new Abstract: Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window. Our code is at https://github.com/Muennighoff/prefix-sliding
Related
- Adaptive Test-Time Compute Allocation for Block Diffusion Language Models in Complex Reasoning
- Optimal Aggregation of LLM and PRM Signals for Efficient Test-Time Scaling
- When to Ponder: Adaptive Compute Allocation for Code Generation via Test-Time Training
- Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling
Source: arXiv cs.CL | 2026-08-27