FreeBalance: Pre-Routing Online Moe Load Balancing via Residual Workload Prediction
arXiv:2608.14205v1 Announce Type: new Abstract: Load imbalance poses a major bottleneck to the efficiency of expert parallelism in distributed inference of Mixture-of-Experts (MoE) models. The most he