Safety
RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
arXiv:2608.00147v1 Announce Type: new Abstract: Vision-language pretraining learns rich medical image representations from radiology reports, but previous model variants commonly operate within a sing
arXiv:2608.00147v1 Announce Type: new Abstract: Vision-language pretraining learns rich medical image representations from radiology reports, but previous model variants commonly operate within a single shared embedding space, so concept-level structure and interpretability must be recovered post hoc, limiting model transparency and, hence, clinical utility. We introduce RadPRISM, which makes a clinician-defined radiology schema a designated stratification axis: an on-premise large language model extracts per-concept text spans from free-text reports, and each clinical concept is aligned in its own dedicated visual subspace, turning concept stratification into direct, top-level alignment supervision. Instantiated on chest radiographs with a 19-concept schema over 203{,}602 examinations from an internal multi-year archive, RadPRISM improved internal dataset zero-shot classification from 0.717 (95% CI, 0.710-0.723) to 0.868 (95% CI, 0.863-0.872) macro AUROC over a matched global-alignment baseline, performed on par with the purpose-built CARZero reference in external zero-shot classification while substantially outperforming it (up to 4.3-fold) in pointing-game visual grounding. In addition, a radiologist reader study demonstrated concept-stratified retrieval ability (0.78 macro retrieval correctness rate within rank 3), surfacing disentangled descriptive findings that report-level retrieval and fixed-label vocabularies cannot express. RadPRISM yields discriminative, spatially faithful, natively concept-stratified representations shaped by and transparently inspectable by clinicians.
Related
- Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance
- CA-GCL: Cross-Anatomy Global-Local Contrastive Learning for Robust 3D Medical Image Understanding
- CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation
- Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
- GTA-Net: Cooperative Game Theory for Vision-Language Alignment in Chest X-Ray Report Generation
Source: arXiv cs.CV | 2026-08-04