Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators

arXiv:2609.19072v1 Announce Type: new Abstract: Large language models are increasingly used for content moderation, but most evaluations still report aggregate accuracy on individual benchmarks. We introduce Safety-Flag, which places seven widely used safety benchmarks (BeaverTails, XSTest, Ethics, WildGuard, Aegis, ToxiChat, and ToxiGen) into a single balanced flag / do-not-flag protocol. We release item-level decisions and confidence scores for six general-purpose LLMs and four dedicated guards, together with three reference models, evaluated on the same items. Safety-Flag measures three dimensions of moderator reliability: error direction, probability calibration, and confidence-based error ranking for human review. They often disagree. Aggregate accuracy does not reveal error direction: one model flags 85% of benign content, whereas another misses 54% of harmful content. All six general-purpose models are overconfident; fitting one temperature per model reduces calibration error by 2.8--6.0imes without changing predicted labels or confidence ordering. Confidence-based abstention lowers selective risk for every model, although the gains depend on how well confidence ranks errors. Dedicated guards produce fewer false alarms and are better calibrated, but several have higher miss rates outside their documented coverage. We release the benchmark, fixed item lists, evaluation code, per-item model outputs, and leaderboard at: https://github.com/yibo-hu-lab/safety-flag-benchmark.

Related

• Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation
• Bielik Guard: Efficient Polish Language Safety Classifiers for LLM Content Moderation
• ML-Bench&Guard: Policy-Grounded Multilingual Safety Benchmark and Guardrail for Large Language Models
• Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety

Source: arXiv cs.CL | 2026-09-17