SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents

arXiv:2610.13157v1 Announce Type: new Abstract: Reinforcement learning (RL) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information (E_{ext}) and fabricate an incorrect answer. We argue that this fabrication arises from an over-reliance on unreliable internal knowledge (E_{int}) as shortcuts due to the inherent opacity of reasoning trajectories. To address this, we propose SOAR (SOurce-Aware Reinforcement learning), a paradigm that transforms monolithic reasoning into source-explicit trajectories. SOAR (i) disentangles E_{ext} and E_{int} into parallel, observable streams via a Source-Aware Workflow, and (ii) incorporates Source-Consistent Reward to guide the policy evolution. By making information-source attribution explicit, SOAR enables precise credit assignment between different knowledge streams. Evaluations show that SOAR outperforms baselines on 3B and 7B backbones by 7.3% and 5.4%, while reducing evidence fabrication by 74.8%. These confirm that SOAR significantly improves performance on knowledge-intensive benchmarks while effectively mitigating evidence fabrication, fostering more faithful and grounded reasoning with enhanced robustness.

Related

• BiCAA: Bidirectional Credit Assignment for Search-Augmented Agent
• COSEARCH: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
• RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents

Source: arXiv cs.CL | 2026-10-12