SpecMD: A Comprehensive Study on Speculative Expert Prefetching
Mixture-of-Experts (MoE) models enable sparse expert activation, meaning that only a subset of the model’s parameters is used during each inference. However, to translate this sparsity into practical