Vocabulary Growth Fundamentals: Bernstein Functions and Hausdorff Sequences

arXiv:2608.29449v1 Announce Type: cross Abstract: We survey the theory of vocabulary growth founded in the setting of stochastic processes. In particular, we model the expected number of types through Bernstein functions and Hausdorff sequences. These classes of mathematical objects, defined by alternating signs of their derivatives or differences, can be related to continuous-time Poisson point processes and discrete-time IID processes, respectively. Building on previous accounts of the vocabulary growth, we integrate the broader theories of Bernstein functions and Hausdorff sequences and connect them with recently developed hapax rate models. In particular, we prove that the logistic hapax rate model has a non-negative spectrum and hence it defines a Bernstein function, thereby solving an earlier posed problem. We also analyze the limitations of the Bernstein--Hausdorff theory of the vocabulary growth by considering its generalizations under stationary and Weibull renewal processes.

Related

• Scaling limit of the Random Language Model
• Comment on 'Modeling rapid language learning by distilling Bayesian priors into artificial neural networks'
• Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP

Source: arXiv cs.CL | 2026-09-01