Agents
Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systems
arXiv:2609.17306v1 Announce Type: cross Abstract: Multi-agent Systems (MAS) combine multiple model outputs to solve complex reasoning tasks. However, despite rapid growth of available open-source mode
arXiv:2609.17306v1 Announce Type: cross Abstract: Multi-agent Systems (MAS) combine multiple model outputs to solve complex reasoning tasks. However, despite rapid growth of available open-source models, there is limited research on how to select optimal model candidates out of this massive pool. We systematically evaluate 8 model selection strategies (including model size, accuracy and answer diversity) across before-generation (routing) and after-generation (majority-voting, LLM-as-a-judge) MAS architectures on challenging scientific benchmarks. Our findings show a significant gap between theoretical oracle potential and actual performance: Expanding candidate pool sizes often degrades performance below that of the top performing base-model. We find that candidate selection within a single model family is the strategy that yields the best relative performance over a standalone model. These results demonstrate that adding arbitrary models to a heterogeneous MAS can introduce system instability, highlighting model selection as a critical design choice for multi-agent systems.
Related
- MAS-on-the-Fly: In-Context Structural Adaptation of LLM-Based Multi-Agent Systems
- Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition
- Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information
- Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systems
Source: arXiv cs.AI | 2026-09-16