RxScribe Bench: A Multi-Axis Benchmark for Evaluating Vision-Language Models on Indian Outpatient Prescriptions

arXiv:2609.13280v1 Announce Type: new Abstract: Prescription transcription errors are not interchangeable. A model that fabricates a drug and a model that misreads a legible dose pose very different clinical risks, yet prescription-transcription accuracy is typically reported as a single blended figure that treats the two as equivalent. We introduce RxScribe Bench, a benchmark for evaluating vision-language models on handwritten prescription digitization that decomposes performance into four axes tied to clinical severity, rather than folding everything into a single aggregated score. Given only a prescription image and an output schema, a model produces a structured record, which is then compared field by field against a human-authored ground truth of identical shape, with each field also labeled for visibility and legibility. The four axes isolate distinct failure modes, namely Correctness, Hallucination, Engagement, and Robustness. The Robustness axis withholds its hard-field results rather than reporting one when the supporting sample falls below a minimum-evidence threshold. We evaluate frontier vision-language models on real prescriptions across independent cold runs per image, and find that no single model wins across all four axes.

Related

• JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation
• MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs
• MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models
• EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models

Source: arXiv cs.CV | 2026-09-15