Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems

arXiv:2609.13559v1 Announce Type: new Abstract: Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware. At its core, a lightweight k-NN predictor operating in a unified semantic-lexical embedding space estimates query-specific accuracy, delay, and power consumption on each edge tier. These predictions are then combined with real-time grid carbon intensity to route every query to the lowest-emission tier capable of executing it successfully. Evaluated on state-of-the-art function-calling benchmarks and LLM families, our framework matches cloud-level accuracy while reducing operational carbon emissions by 4imes on average.

Related

• AgentFactory: Towards Automated Agentic System Design and Optimization
• GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Calling
• Resource-Aware LLM Reasoning for Mobile Edge General Intelligence
• Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration

Source: arXiv cs.AI | 2026-09-15