Safety
Inference-Time Agentic Decision Rules Beat Longer Evolving Search for Multi-Image Medical Reasoning
arXiv:2607.27564v1 Announce Type: new Abstract: Multi-image medical VQA is not merely a prompt-length problem; it is a fundamental challenge of agentic decision-making. Medical vision-language agents
arXiv:2607.27564v1 Announce Type: new Abstract: Multi-image medical VQA is not merely a prompt-length problem; it is a fundamental challenge of agentic decision-making. Medical vision-language agents must aggregate evidence across ordered images, remain robust to answer-order perturbations, and avoid overfitting to noisy search-time feedback. We study MedFrameQA through a controlled comparison of five inference-time agentic strategies, optimized using the same high-budget ShinkaEvolve configuration and evaluated on a reproducible internal frozen split (1,331 evolution, 665 holdout, 855 final test). Across five independent repeated runs, the strongest method emerges as the simplest robust aggregator: the extbf{order-vote} policy achieves 57.89 pm 0.65% final-test accuracy, significantly outperforming the fixed baseline (52.73 pm 0.42%) and the more complex, albeit brittle, order-rerank variant (55.79 pm 0.43%). Paired bootstrap analysis confirms these significant gains. Extending the evolutionary search budget from 50 to 100 generations yields no generalization benefit: while holdout performance marginally increases, final-test accuracy drops from 57.89% to 56.02%. Our findings suggest that for multi-image medical reasoning, defining the correct agentic decision rule is substantially more impactful than expanding the optimization search budget.
Related
- MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
- Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Models
- Retrieval-Guided Generation for Safer Histopathology Image Captioning
Source: arXiv cs.CV | 2026-07-31