Confidence-Calibrating Regularization for Robust Brain MRI Segmentation Under Domain Shift

arXiv:2509.23176v2 Announce Type: replace Abstract: The Segment Anything Model (SAM) exhibits strong zero-shot performance on natural images but suffers from domain shift and overconfidence when applied to medical volumes. We propose extbf{CalSAM}, a lightweight adaptation framework that (i) reduces encoder sensitivity to domain shift via a Feature Fisher Information Penalty (FIP) computed on 3D feature maps and (ii) penalizes overconfident voxel-wise errors through a Confidence Misalignment Penalty (CMP). The combined loss, (L_{CalSAM}) fine-tunes only the mask decoder while keeping SAM's encoders frozen. On cross-center and scanner-shift evaluations, CalSAM substantially improves accuracy and calibration: e.g., on the BraTS scanner split (SiemensoGE) CalSAM shows a +7.4% relative improvement in DSC (80.1% vs. 74.6%), a -26.9% reduction in HD95 (4.6 mm vs. 6.3 mm), and a -39.5% reduction in ECE (5.2% vs. 8.6%). On ATLAS-C (motion corruptions), CalSAM achieves a +5.3% relative improvement in DSC (75.9%) and a -32.6% reduction in ECE (5.8%). Ablations show FIP and CMP contribute complementary gains (p<0.01), and the Fisher penalty incurs a modest sim15% training-time overhead. CalSAM therefore delivers improved domain generalization and better-calibrated uncertainty estimates for brain MRI segmentation, while retaining the computational benefits of freezing SAM's encoder.

Related

• Segment Anything with Robust Uncertainty-Accuracy Correlation
• DSA-CycleGAN: A Domain Shift Aware CycleGAN for Robust Multi-Stain Glomeruli Segmentation
• Improving Prostate Gland Segmentation Using Transformer based Architectures
• Selective, Regularized, and Calibrated: Harnessing Vision Foundation Models for Cross-Domain Few-Shot Semantic Segmentation
• Robustness Evaluation of a Foundation Segmentation Model Under Simulated Domain Shifts in Abdominal CT: Implications for Health Digital Twin Deployment

Source: arXiv cs.CV | 2026-08-14