Model Releases
Towards Reliable Long-Context Reasoning: Detecting Unanswerable Questions via FactGuard
arXiv:2504.05607v2 Announce Type: replace-cross Abstract: Despite the success of Large Language Models (LLMs) in long-context tasks, their reliability in recognizing unanswerable questions remains a c
arXiv:2504.05607v2 Announce Type: replace-cross Abstract: Despite the success of Large Language Models (LLMs) in long-context tasks, their reliability in recognizing unanswerable questions remains a critical challenge. In information-seeking scenarios, models often suffer from overconfidence and hallucinations when evidence is missing or misleading. In this paper, we propose FACTGUARD, a systematic multi-task workflow for generating evidence-grounded answerable and unanswerable questions. We introduce FACTGUARD-BENCH, a benchmark comprising 25,220 examples with context lengths up to 128K, specifically targeting Lack of Evidence and Misleading Evidence scenarios. Evaluations of eleven representative open-source and proprietary LLMs reveal a significant reliability gap: models struggle to reject unanswerable questions as context scales, with the leading model achieving only 71.62% accuracy. By training with FACTGUARD-BENCH, we improve overall accuracy to 81.17% and significantly enhance models' reasoned rejection capabilities. Our work provides a scalable path toward building more faithful and trustworthy long-context LLMs. Our source is publicly available at https://github.com/biaoliu-kiritsugu/FactGuard.
Related
- Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
- AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
- Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
Source: arXiv cs.AI | 2026-10-16