ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition

arXiv:2601.03822v2 Announce Type: replace Abstract: Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multiple-Choice Knapsack Problem (OS-MCKP). This perspective highlights a meta-cognitive requirement---anticipating problem difficulty and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. First, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost levels and decide whether a problem is worth attempting before generation. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Experiments across mathematical reasoning, multiple-choice question answering, and code generation show that ROI-Reasoning improves scores and generally reduces reference-order regret under tight budgets, with up to a extbf{417%} relative score improvement over the strongest same-backbone baseline in mathematics.

Related

• Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimization
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
• Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
• Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planning

Source: arXiv cs.AI | 2026-10-15