From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation

arXiv:2604.17455v1 Announce Type: new Abstract: Visual prompting has emerged as a powerful method for adapting pre-trained models to new domains without updating model parameters. However, existing prompting methods typically optimize a single prompt per domain and apply it uniformly to all inputs, limiting their ability to generalize under intra and inter-domain variability, which is especially critical in the medical field. To address this, we propose APEX, an Adaptive Prompt EXtraction framework that retrieves input-specific prompts from a learnable prompt memory. The memory stores diverse, domain-discriminative prompt representations and is queried via domain features extracted from the Fourier spectrum. To learn robust and discriminative domain features, we introduce a novel Low-Frequency Feature Contrastive (LFC) learning framework that clusters representations from the same domain while separating those from different domains. Extensive experiments on two medical segmentation tasks demonstrate that APEX significantly improves generalization across both seen and unseen domains. Furthermore, it complements any existing backbones and consistently enhances performance, confirming its effectiveness as a plug-and-play prompting solution in medical fields. The code is available at https://github.com/cetinkayaevren/apex/

Related

• Rethinking Cross-Dose PET Denoising: Mitigating Averaging Effects via Residual Noise Learning
• M-IDoL: Information Decomposition for Modality-Specific and Diverse Representation Learning in Medical Foundation Model
• Noise Injection: Improving Out-of-Distribution Generalization for Limited Size Datasets
• Understanding Task Transfer in Vision-Language Models

Source: arXiv cs.CV | 2026-04-21