Local Ai
Hallucination Detection and Evaluation of Large Language Model
arXiv:2512.22416v2 Announce Type: replace Abstract: Hallucinations in Large Language Models (LLMs) pose a significant challenge, generating misleading or unverifiable content that undermines trust and
arXiv:2512.22416v2 Announce Type: replace Abstract: Hallucinations in Large Language Models (LLMs) pose a significant challenge, generating misleading or unverifiable content that undermines trust and reliability. Existing evaluation methods, such as KnowHalu, employ multi-stage verification but suffer from high computational costs. To address this, we integrate the Hughes Hallucination Evaluation Model (HHEM), a lightweight classification-based framework that operates independently of LLM-based judgments, significantly improving efficiency while maintaining high detection accuracy. We conduct a comparative analysis of hallucination detection methods across various LLMs, evaluating True Positive Rate (TPR), True Negative Rate (TNR), and Accuracy on question-answering (QA) and summarization tasks. Our results show that HHEM reduces evaluation time from 8 hours to 10 minutes, while HHEM with non-fabrication checking achieves the highest accuracy (82.2%) and TPR (78.9%). However, HHEM struggles with localized hallucinations in summarization tasks. To address this, we introduce segment-based retrieval, improving detection by verifying smaller text components. Additionally, our cumulative distribution function (CDF) analysis indicates that larger models (7B-9B parameters) generally exhibit fewer hallucinations, while intermediate-sized models show higher instability. These findings highlight the need for structured evaluation frameworks that balance computational efficiency with robust factual validation, enhancing the reliability of LLM-generated content.
Related
- An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinations
- When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
- Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms
- The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
- ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation
Source: arXiv cs.CL | 2026-04-10