Research
A Quantitative Study of Sustained Focus in Large Language Models via Repetitive Deterministic Prediction Tasks
arXiv:2511.00763v3 Announce Type: replace Abstract: We investigate the performance of large language models (LLMs) on repetitive deterministic prediction tasks and study how the sequence accuracy rate
arXiv:2511.00763v3 Announce Type: replace Abstract: We investigate the performance of large language models (LLMs) on repetitive deterministic prediction tasks and study how the sequence accuracy rate (SAR) scales with output length. Each such task involves the repetition of the same operation N times. Examples of such tasks include letter replacement in letter strings following a given rule, integer addition, and multiplication of string operators in many-body quantum mechanics. If the LLM performs the task by a simple repetition algorithm, the success rate would follow an exponential decay with sequence length. In contrast, our experiments on leading LLMs reveal a crossover that is sharper than exponential: -logSAR grows super-linearly with N, and accuracy collapses around a characteristic length N_, the accuracy cliff that separates reliable from unreliable generation. The hypothesis of independent per-step errors is rejected for every model and task we studied. The crossover is well described by a double-exponential accumulation law, SAR=exp(-eta_0 Nalpha^{N-1}), whose crossover scale N_ does not depend on the functional form chosen to fit it. To interpret this behaviour we introduce a minimal effective model in which step-correctness variables interact through dense random couplings and compete with an external field set by the prompt. Solved by direct enumeration, the model reproduces the super-linear error accumulation and the accuracy cliff qualitatively, and it assigns to each model--task pair two interpretable parameters, an intrinsic error rate and an error-accumulation factor.
Related
- Daily and Weekly Periodicity in Large Language Model Performance and Its Implications for Research
- How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problem
- Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting
Source: arXiv cs.AI | 2026-10-01