When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

arXiv:2609.15106v1 Announce Type: new Abstract: Large language models can hallucinate even when the knowledge required for a correct answer is already available. We study this failure through a latent-key view of inference, where answer selection depends on competition among associations acquired during pretraining. We show that model predictions can be highly sensitive to individual query keywords, that these influential keywords exhibit entity-specific binding, and that their effects are systematically shaped by pretraining frequency. Multiple bindings can also compete and exhibit higher-order interactions within the same query. Based on this mechanism, we introduce a two-stage keyword-perturbation method for hallucination detection. By removing influential keywords and measuring how the model reorganizes its prediction, the method distinguishes errors caused by misleading key associations from correct decisions supported by diagnostic evidence. Across multiple models and benchmarks, perturbation provides a strong and transferable detection signal, reaching .910 AUROC on probe-known ScientistQA. Finally, we extend the same probabilistic framework to four hallucination regimes: knowledge deficit, wrong knowledge, context distraction, and unstable inference. Their operational distributions across benchmarks provide diagnostic context for why different detector families succeed in different settings.

Related

• Enhancing Hallucination Detection through Noise Injection
• Score imes Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation
• Hallucination Detection in Large Language Models Using Diversion Decoding

Source: arXiv cs.CL | 2026-09-15