MedPlex: Deep Vision-Language Co-Adaptation for Clinically Grounded Medical Segmentation

arXiv:2608.13690v1 Announce Type: cross Abstract: Medical image segmentation is still largely treated as a vision-only problem, although clinical interpretation often relies on textual knowledge of anatomy, location, appearance, and surrounding context. Existing text-guided segmentation methods within the Vision-Language Model (VLM) paradigm often use language only as a late conditioning signal, limiting its influence on visual representation learning. We introduce MedPlex (Medical Plexus of Vision and Language), an end-to-end VLM framework that makes text guidance a continuous, clinically grounded component of segmentation learning. Through Bi-Fusion (Bidirectional Fusion), visual and textual representations evolve jointly across the encoding hierarchy. MedPlex further introduces class-level and region-level concept alignment to organize the shared representation at complementary granularities. Class-level alignment anchors each anatomical target to an aggregated clinical concept profile, while region-level alignment preserves individual concepts, such as shape, location, appearance, and texture, through class-specific visual evidence. In this way, language provides structured supervision throughout the encoder rather than serving only as a late-stage cue. MedPlex achieves state-of-the-art performance across CT and MR benchmarks for multi-organ, cardiac substructure, and tumor segmentation, including settings with real free-text clinical supervision. Code: https://github.com/rafiibnsultan/MedPlex.

Related

• DistMedVL: Distributional Vision-Language Alignment for Uncertainty-Aware Medical Image Segmentation
• Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation
• MedUP: Awakening Unified Understanding and Perception in Medical Vision-Language Models
• Dual-Domain Cross-Modal Decoding for Clinical Text-Guided Medical Image Segmentation

Source: arXiv cs.AI | 2026-08-17