Safety
Weighted Ensemble Sampling for Rare-Event Estimation in Language Models
arXiv:2610.19202v1 Announce Type: new Abstract: Rare but severe failures of large language models can become practically significant at deployment scale. Estimating their probabilities with direct Mon
arXiv:2610.19202v1 Announce Type: new Abstract: Rare but severe failures of large language models can become practically significant at deployment scale. Estimating their probabilities with direct Monte Carlo sampling is highly inefficient, while proposal-based methods typically require auxiliary learning of an additional proposal distribution. We introduce extbf{Weighted Ensemble (WE)}, a training-free framework for rare-event estimation in language models. WE samples exclusively from the original language model and improves upon direct Monte Carlo by periodically reallocating the sampling budget toward prefixes that appear more likely to reach the target rare event. To guide this reallocation without learning a separate proposal distribution, we exploit internal LLM representations as prefix-level signals and empirically validate their effectiveness. We provide a theoretical analysis of the proposed algorithm. Experiments on safety-related tasks spanning both small-scale and modern large-scale LLMs demonstrate that WE improves the efficiency and reliability of rare-event probability estimation.
Related
- Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation
- Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
- DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning
Source: arXiv cs.CL | 2026-10-15