Scaling Laws for Mixture Pretraining Under Data Constraints
Apple derives a repetition-aware mixture scaling law showing scarce target data can be reused ~15–20× when mixed with generic corpora.
Knowledge catalogue
Apple derives a repetition-aware mixture scaling law showing scarce target data can be reused ~15–20× when mixed with generic corpora.