MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation

arXiv:2609.08283v1 Announce Type: new Abstract: Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in zero-shot transfer but often struggles with dense prediction tasks due to low spatial resolution and the loss of structural information. To address these limitations, we propose MARS-CLIP (Multi-resolution and Attention Refined Segmentation for CLIP), a novel framework for zero-shot semantic segmentation. Our approach introduces two key strategies: (i) a multi-resolution feature extraction module that fuses local fine-grained features with global context to overcome input resolution constraints, and (ii) an attention refinement mechanism that injects spatial and color biases from intermediate layers into the final self-attention block to accurately restore object boundaries. A set of experiments on six public datasets demonstrates that MARS-CLIP significantly outperforms state-of-the-art methods.

Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation
Repurposing CLIP to Localize at Pixel Level
OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation
Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation
MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration

Source: arXiv cs.CV | 2026-09-09

Related