Safety
DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization
arXiv:2610.18609v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has become a widely adopted approach for aligning vision-language models (VLMs), yet the security of multimodal p
arXiv:2610.18609v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has become a widely adopted approach for aligning vision-language models (VLMs), yet the security of multimodal preference data remains largely unexplored. Existing preference poisoning attacks either corrupt explicit supervision signals or require visible triggers, making them detectable under human inspection. We introduce DPOison-VL, the first clean preference poisoning attack against VLM-DPO, which manipulates model alignment solely through imperceptible perturbations on a small fraction of training images, while leaving all prompts, responses, and preference annotations semantically intact. We formulate poison construction as a bilevel optimization and derive a gradient alignment surrogate that efficiently steers image perturbations toward the attacker-specified preference. We further introduce a preference displacement score for principled poison selection with provable first-order optimality under a fixed poison budget, and derive a sufficient condition for preference inversion. Extensive experiments across diverse attack tasks, different VLM architectures and scales, and baseline comparisons demonstrate that DPOison-VL achieves up to 98% ASR with only 2% poisoned data and exhibits cross-model transferability, while largely preserving clean accuracy and general utility. Our findings reveal a critical vulnerability in VLM-DPO, where imperceptible image perturbations in preference training data can silently redirect alignment.
Related
- Challenging Vision-Language Models with Physically Deployable Multimodal Semantic Lighting Attacks
- Towards Bridging the Gap Between Offline and Iterative Alignment via Preference Distillation
- MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledge
- It Takes Little to Rewrite Perception: Targeted Semantic Substitution in Vision-Language Models at epsilon leq 4/255
Source: arXiv cs.CV | 2026-10-14