Is More Data Worth the Cost? Dataset Scaling Laws in a Tiny Attention-Only Decoder
arXiv:2604.09389v1 Announce Type: cross Abstract: Training Transformer language models is expensive, as performance typically improves with increasing dataset size and computational budget. Although s