What Does Prompt Learning Change? -A Natural-Language Concept Analysis of Vision-Language Models

arXiv:2608.24142v1 Announce Type: new Abstract: Prompt learning adapts vision-language models such as CLIP by optimizing continuous prompt vectors, but the learned prompts are difficult to interpret in natural language. We present PromptSpLiCE, a post-hoc method that expresses each class-conditioned text embedding as a sparse combination of concepts from a fixed natural-language dictionary. Using the same dictionary before and after prompt learning allows us to compare changes in their concept profiles. We evaluate PromptSpLiCE on CoOp, a representative prompt-learning method, across 11 image-classification datasets. The concept profiles change substantially: on average, only 1.6 of the initial top-10 concepts remain in the top 10 after learning. Across datasets, profile change is positively associated with accuracy gain. We also derive a local gradient expression that provides geometric intuition for why image-aligned concept directions distinct from the current prompt can have greater loss sensitivity.

Related

• Near OOD Detection for Vision-Language Prompt Learning with Contrastive Logit Score
• ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
• Prompt Sensitivity in Vision-Language Grounding: How Small Changes in Wording Affect Object Detection
• Intrinsic Gradient Suppression for Label-Noise Prompt Tuning in Vision-Language Models
• Few-Shot Concept Prompt Learning for Segmentation Foundation Models via Visual Grounding

Source: arXiv cs.CV | 2026-08-26