What LLMs Do in the Shadows: Reading Context Reliance from the Unembedding Dark Space

arXiv:2610.23182v1 Announce Type: new Abstract: Detecting faithfulness hallucinations in large language models (LLMs) is crucial for their safe deployment in high-stakes domains. Recent approaches address this by contrasting contextual against parametric knowledge utilization, yet measuring context reliance typically demands noticeable overhead, such as materializing attention maps, running additional forward passes, or training auxiliary probes. In this work, we show that context reliance can be estimated by projecting hidden states onto the dark space - directions that compose weakly with the unembedding matrix. Beyond its recognized function as an initial-token attention sink, we show that this subspace actively mediates context processing, and that its share of hidden-state norm - termed the Dark Matter Ratio (DMR) - provides a direct score of context utilization. Pairing DMR with a parametric knowledge score yields DarkLens, a lightweight faithfulness detector operating strictly within a single forward pass. Evaluated across three QA and summarization benchmarks on six open-weight LLMs (4B-13B parameters), DarkLens matches or outperforms a wide range of baselines without requiring extra forward passes or probe training.

Related

• Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
• Enhancing Hallucination Detection through Noise Injection
• When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

Source: arXiv cs.CL | 2026-10-19