Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
arXiv:2502.04416v3 Announce Type: replace-cross Abstract: Scaling large language models (LLMs) improves performance but significantly increases inference costs, with feed-forward networks (FFNs) consu