Safety
Limitations on Autoregressive Token Generation via Sequential Sampling
arXiv:2610.14615v1 Announce Type: cross Abstract: We study autoregressive token generation through the notion of sequential samplers: A circuit that samples the next symbol only given the previous one
arXiv:2610.14615v1 Announce Type: cross Abstract: We study autoregressive token generation through the notion of sequential samplers: A circuit that samples the next symbol only given the previous ones and possibly a small amount of state. We show that (1) a sequential sampler with k bits of state can be simulated by a stateless one at a roughly O(2^{k}) size overhead; (2) efficient insertional sampling (Stern, Chan, Kiros, Uszkoreit, ICML 2019) is more powerful than efficient sequential sampling in any fixed order; (3) sequential samplers with small state cannot establish certain forms of covert cryptographic channels. Result (1), together with the pseudoentropy vs. sequential sampleability duality of Vadhan and Zheng (STOC 2012), characterizes efficient sequential sampleability with small state. Result (3) provides a potential safety mechanism against subliminal coordination among malicious LLM agents.
Related
- Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner-Ziv Bounds for KV Cache Compression
- TAAL: Mitigating Early Beam Pruning in Generative Recommendation via Temporal Autoregressive Alignment
- Sampling Decisions: Exact Path-Space Correction, Prior Cancellation and Local-Boltzmann Guidance
- Continuity Laws for Sequential Models
Source: arXiv cs.AI | 2026-10-15