DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
arXiv:2605.11005v1 Announce Type: new Abstract: Mixture-of-experts (MoE) architectures enable trillion-parameter LLMs with sparsely activated experts. Expert parallelism (EP) is a widely adopted MoE t