TST works in two phases. In phase 1, which covers the first 20-40% of training, the model reads contiguous bags of k tokens, with input embe…
DGX agentTST works in two phases. In phase 1, which covers the first 20-40% of training, the model reads contiguous bags of k tokens, with input embeddings averaged within each bag, and predicts the next bag o