Identifying the Periodicity of Information in Natural Language

arXiv:2510.27241v2 Announce Type: replace Abstract: Recent theoretical advancement of information density in natural language has brought the following question on desk: To what degree does natural language exhibit periodicity pattern in its encoded information? We address this question by introducing a new method called AutoPeriod of Surprisal (APS). APS adopts a canonical periodicity detection algorithm and is able to identify any significant periods that exist in the surprisal sequence of a single document. By applying the algorithm to a set of corpora, we have obtained the following interesting results: Firstly, a considerable proportion of human language demonstrates a strong pattern of periodicity in information; Secondly, new periods that are outside the distributions of typical structural units in text (e.g., sentence boundaries, elementary discourse units, etc.) are found and further confirmed via harmonic regression modeling. We conclude that the periodicity of information in language is a joint outcome from both structured factors and other driving factors that take effect at longer distances. The advantages of our periodicity detection method and its potentials in LLM-generation detection are further discussed.

Related

• Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences
• Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
• Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Models
• Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics

Source: arXiv cs.CL | 2026-04-27