A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG

arXiv:2607.03739v1 Announce Type: cross Abstract: We release a benchmark and failure-mode-aware evaluation framework for grounded QA under coordinated retrieval poisoning. The framework partitions reader outputs into four mutually exclusive categories (gold, hijack, abstention, drift), with instance-level paired clean-to-poison transition matrices and a Forced Exposure protocol isolating reader-side conflict resolution from retrieval variance. We introduce polymorphic sybil poisoning, a coordinated attack class in which S lexically diverse passages jointly support an attacker-chosen target while evading lexical near-duplicate filters that fully detect monomorphic baselines (capturing the residual 14.2% with E5 cosine raises false-positive rate 9imes on legitimate same-topic pairs). A monomorphic-polymorphic ablation under Forced Exposure isolates the diversity dimension and reveals a +18.8pp hijack amplification (95% paired bootstrap CI [+15.4, +22.4], B{=}5{,}000): monomorphic copies register only 4.0% as hijack while polymorphic surface diversity recovers 22.8% -- a 5.7imes amplification of the ASR-visible attack channel. ASR alone treats every non-target output identically; under attack, abstention and drift together hold 47-66% of output mass, unmonitored by ASR+ACC, and two readers at nearly identical ASR (within 0.2pp) differ by 16.5pp on abstention and 17.2pp on drift -- failure profiles invisible to ASR. We release the frozen benchmark (3{,}145 questions, 2{,}982 retained sybil groups; S{=}6 chosen to dominate top-10 retrieval slots, Sref{sec:setup}), the official four-way evaluator, paired-transition utilities, and the Forced Exposure harness across five readers (7B-120B), two retrievers, and two cross-validation datasets (TriviaQA, 2Wiki), under CCBY-SA4.0 (data) and MIT (software); release information in Sref{sec:release}.

Source: arXiv cs.AI | 2026-07-07