Model Releases
Sleight of Word Benchmark: Can Language Models Notice If Their Own Output Was Tampered With?
arXiv:2608.29921v1 Announce Type: cross Abstract: The output of a Language Model can be tampered with while the model is writing it. A simple test can thus be constructed by evaluating the model's per
arXiv:2608.29921v1 Announce Type: cross Abstract: The output of a Language Model can be tampered with while the model is writing it. A simple test can thus be constructed by evaluating the model's perception of this external perturbation. In this spirit, a simple benchmark is built in which a single word is consistently substituted with another in the generation process. We call this method Sleight of Word. Two distinct axes are measured: metrics that relate to the model's surprise, as well as an evaluation of the textual reaction for 19 different open-weight language models.
Related
- Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
- Prompt2Fingerprint: Plug-and-Play LLM Fingerprinting via Text-to-Weight Generation
Source: arXiv cs.AI | 2026-09-01