Safety
Is Word Error Rate Enough? Rethinking Privacy Evaluation in Speech with Entity-Aware Metrics
arXiv:2610.08831v1 Announce Type: cross Abstract: As the use of smart devices continues to increase, their potential to capture sensitive speech content raises growing privacy concerns. It is therefor
arXiv:2610.08831v1 Announce Type: cross Abstract: As the use of smart devices continues to increase, their potential to capture sensitive speech content raises growing privacy concerns. It is therefore critical to develop techniques that prevent information leakage while preserving the utility of the audio, and evaluation metrics that accurately quantify the level of privacy without overestimating it. In this work, we evaluate the effectiveness of two obfuscation techniques in protecting speech content, with particular emphasis on named entities, by adapting entity-aware privacy metrics from the Natural Language Processing field to the speech privacy domain. Further, we investigate several attack scenarios and show that fine-tuning on entity-rich data improves attack performance for some entity categories but not others. Finally, we provide guidance on metric selection based on whether the obfuscation method preserves temporal alignment.
Related
- ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance
- Neural Speaker Diarization via Multilingual Training: Evaluation on Low-Resource Nepali-Hindi Speech
- How Does Differential Privacy Affect Social Bias in LLMs? A Systematic Evaluation
Source: arXiv cs.CL | 2026-10-08