A Decomposition Perspective to Long-context Reasoning for LLMs

arXiv:2604.07981v1 Announce Type: new Abstract: Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7% (improving from 46.3% to 54.0%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR.

Related

• DYCP: Dynamic Context Pruning for Long-Form Dialogue with LLMs
• Stacked from One: Multi-Scale Self-Injection for Context Window Extension
• SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMs
• SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
• SubSearch: Intermediate Rewards for Unsupervised Guided Reasoning in Complex Retrieval

Source: arXiv cs.CL | 2026-04-10