Bilevel Optimization of Agent Skills via Monte Carlo Tree Search

arXiv:2604.15709v1 Announce Type: new Abstract: Agent exttt{skills} are structured collections of instructions, tools, and supporting resources that help large language model (LLM) agents perform particular classes of tasks. Empirical evidence shows that the design of exttt{skills} can materially affect agent task performance, yet systematically optimizing exttt{skills} remains challenging. Since a exttt{skill} comprises instructions, tools, and supporting resources in a structured way, optimizing it requires jointly determining both the structure of these components and the content each component contains. This gives rise to a complex decision space with strong interdependence across structure and components. We therefore represent these two coupled decisions as exttt{skill} structure and component content, and formulate exttt{skill} optimization as a bilevel optimization problem. We propose a bilevel optimization framework in which an outer loop employs Monte Carlo Tree Search to determine the exttt{skill} structure, while an inner loop refines the component content within the structure selected by the outer loop. In both loops, we employ LLMs to assist the optimization procedure. We evaluate the proposed framework on an open-source Operations Research Question Answering dataset, and the experimental results suggest that the bilevel optimization framework improves the performance of the agents with the optimized exttt{skill}.

Source: arXiv cs.AI | 2026-04-20

Related