UCS: Estimating Unseen Coverage for Improved In-Context Learning

arXiv:2604.12015v1 Announce Type: cross Abstract: In-context learning (ICL) performance depends critically on which demonstrations are placed in the prompt, yet most existing selectors prioritize heuristic notions of relevance or diversity and provide limited insight into the coverage of a demonstration set. We propose Unseen Coverage Selection (UKS), a training-free, subset-level coverage prior motivated by the principle that a good demonstration set should expose the model to latent cluster unrevealed by the currently selected subset. UCS operationalizes this idea by (1) inducing discrete latent clusters from model-consistent embeddings and (2) estimating the number of unrevealed clusters within a candidate subset via a Smoothed Good--Turing estimator from its empirical frequency spectrum. Unlike previous selection methods, UCS is coverage-based and training-free, and can be seamlessly combined with both query-dependent and query-independent selection baselines via a simple regularized objective. Experiments on multiple intent-classification and reasoning benchmarks with frontier Large Language Models show that augmenting strong baselines with UCS consistently improves ICL accuracy by up to 2-6% under the same selection budget, while also yielding insights into task- and model-level latent cluster distributions. Code is available at https://github.com/Raina-Xin/UCS.

Related

• Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
• LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning
• Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
• p1: Better Prompt Optimization with Fewer Prompts
• Testing the Assumptions of Active Learning for Translation Tasks with Few Samples

Source: arXiv cs.CL | 2026-04-15