World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments

arXiv:2607.01470v1 Announce Type: new Abstract: Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: once clinical SMEs encode decision logic into a verifier, that verifier grades unlimited rollouts without per-episode annotation. But applying RL requires a sound feedback channel and sufficient base capability. We audit MedAgentBench v1/v2, find a 41.7% silent-finish ceiling that makes inaction the RL dominant strategy, and construct extbf{MedAgentBench-v3 (MAB-v3)} (508 tasks, 8.9% ceiling). Training Qwen3-8B exposes two structural barriers: a capability ceiling (10/20 task types have 0% base performance, zero gradient) and a format-knowledge barrier (3/20 types require exact clinical codes undiscoverable by exploration). Pure RL reaches 18.2% pass@1 vs. 34.1% for rule-based SFT; the 15.9~pp gap is attributable entirely to these barriers. A decision/format-knowledge/lookup taxonomy predicts RL learnability and prescribes the fix: SFT to inject codes, RL to learn conditionals.

Source: arXiv cs.AI | 2026-07-03