EvolvAttack: Backdooring self Evolving VLM Agents

arXiv:2610.15244v1 Announce Type: new Abstract: Self-evolving mechanisms let vision-language model (VLM) agents continually adapt through evolving memory, but they also allow malicious behavior to emerge only after deployment. We introduce a new attack surface, the self-evolving backdoor, whose payload stays dormant in the initial agent and is progressively activated as the agent evolves. Unlike conventional backdoors, which can be exposed by testing triggered inputs on a static model, our attack requires both a visual trigger and a sufficiently evolved memory state; static scanning with clean memory thus yields near-zero attack success and may deem the agent benign. We realize this through memory-aware backdoor training, which couples trigger activation with the agent's evolving memory while preserving benign behavior unless the trigger co-occurs with payload-related memory. Across three VLM backbones, four agent pipelines, and recency-, semantic-, and reward-based memory evolution, the attack preserves normal functionality and stays nearly inactive before evolution (0.5% attack success rate), yet exceeds 95% attack success within 100 interactions, revealing a stealthy threat unique to self-evolving agents.

Related

• AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?
• FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills
• RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents
• EXG: Self-Evolving Agents with Experience Graphs

Source: arXiv cs.LG | 2026-10-14