Online Learning with LLM Experts from Limited Feedback

arXiv:2609.05820v1 Announce Type: new Abstract: We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online setting with limited feedback. We formulate it as a bandit problem with K actions that represent experts and d features that encode prompts, over a horizon of T rounds. We propose algorithms that strategically select and observe rewards to minimize regret. In the full-information setting, we achieve a regret of ilde{O}(d T / sqrt{m}), while in the bandit setting we achieve ilde{O}(d T sqrt{K / m}), where m ll T is a budget on feedback. Our experiments show that we efficiently learn high-quality routing strategies across diverse LLMs from limited feedback.

Source: arXiv cs.LG | 2026-09-09