Local Ai
Quality Action Assurance: Multimodal Verification of Examiner Claims in VR OSCEs
arXiv:2607.19063v2 Announce Type: replace Abstract: Objective Structured Clinical Examinations (OSCEs) are the gold standard for assessing clinical competence, yet scoring remains vulnerable to examin
arXiv:2607.19063v2 Announce Type: replace Abstract: Objective Structured Clinical Examinations (OSCEs) are the gold standard for assessing clinical competence, yet scoring remains vulnerable to examiner subjectivity, fatigue, and cognitive bias. Standard examiner validation via inter-rater statistics lacks explanatory power regarding the source of errors, as it neither analyzes examiner reasoning nor verifies examiner claims against actual events. Thus, we introduce Quality Action Assurance (QAA), a multimodal framework that verifies examiner claims in Virtual Reality (VR) pediatric OSCEs by comparing actions claimed by examiners against the true sequence of events, constructed from video, VR logs, and actor data. QAA combines a constrained temporal action alignment model, which performs action localization and actor source attribution, with a large language model that extracts examiner claims and checks them against the record. Across a 5-fold cross-validation, QAA achieves 99.2% pm 0.7% Actor F1 and 93.4% pm 1.9% W@16 for temporal alignment. Overall, QAA detects examiner errors with 69.9% precision and 76.7% recall, improving factual correctness from 39.2% to 79.2%, enabling fairer OSCE assessment.
Related
- OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration
- LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
- GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
Source: arXiv cs.AI | 2026-07-28