Towards Understanding Self-Pretraining for Sequence Classification
arXiv:2605.21070v1 Announce Type: new Abstract: Amos et al. (2024) showed that the accuracy of Transformer models in sequence classification can be significantly improved by first pretraining with a m