Applications
The Privacy-Hallucination Tradeoff in Differentially Private Language Models
arXiv:2609.00492v1 Announce Type: new Abstract: Both privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a privacy-hallucination
arXiv:2609.00492v1 Announce Type: new Abstract: Both privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a privacy-hallucination tradeoff in differentially private (DP) language models. First, we empirically show that models pre-trained or fine-tuned with DP tend to produce more hallucinations than non-DP counterparts, with increased severity as the privacy budget grows stricter. Second, we investigate model properties driving this tradeoff, demonstrating that DP mechanisms flatten output distributions, potentially redistributing probability mass toward factually incorrect alternatives. Third, through experiments where we control fact frequency in training data, we characterize how information frequency can reduce hallucination risks in DP models. Overall, our findings underscore the need for more nuanced privacy-preserving interventions that offer rigorous privacy guarantees without compromising factual accuracy.
Related
- Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation
- Identifying the Achilles' Heel: An Iterative Method for Dynamically Uncovering Factual Errors in Large Language Models
- Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies
- Differentially Private De-identification of Dutch Clinical Notes: A Comparative Evaluation
Source: arXiv cs.AI | 2026-09-02