Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations

arXiv:2604.16217v1 Announce Type: cross Abstract: Large language models are increasingly deployed in settings where reliability matters, yet output-level uncertainty signals such as token probabilities, entropy, and self-consistency can become brittle under calibration--deployment mismatch. Conformal prediction provides finite-sample validity under exchangeability, but its practical usefulness depends on the quality of the nonconformity score. We propose a conformal framework for LLM question answering that uses internal representations rather than output-facing statistics: specifically, we introduce Layer-Wise Information (LI) scores, which measure how conditioning on the input reshapes predictive entropy across model depth, and use them as nonconformity scores within a standard split conformal pipeline. Across closed-ended and open-domain QA benchmarks, with the clearest gains under cross-domain shift, our method achieves a better validity--efficiency trade-off than strong text-level baselines while maintaining competitive in-domain reliability at the same nominal risk level. These results suggest that internal representations can provide more informative conformal scores when surface-level uncertainty is unstable under distribution shift.

Related

• TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
• Learning Uncertainty from Sequential Internal Dispersion in Large Language Models
• Hallucination as output-boundary misclassification: a composite abstention architecture for language models
• Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMs

Source: arXiv cs.AI | 2026-04-20