TRACER: Trace-Regularized Training Against Indirect Prompt Injection

arXiv:2610.19133v1 Announce Type: new Abstract: Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context. We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an answer-anchored, evidence-gated reward with ATRC, which aligns attacked reasoning with clean evidence when an injection is invalid. On GSM8K with Qwen2.5-3B-Instruct, TRACER reduces RAG-framed ASR from 65.5% to 6.1% while preserving 78-82% clean accuracy. Joint RAG+tool training achieves 13.9% average ASR across framings at 3B scale, while a single epoch at 7B reaches 9.4%. These results demonstrate that robust defenses must explicitly train and evaluate across deployment channels.

Related

• Untrusted Content Masking for Web Agents with Security Guarantees
• SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection
• AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

Source: arXiv cs.LG | 2026-10-15