Research
Retrieval-Based In-Context Learning: A Domain Adaptation Framework
arXiv:2610.05717v1 Announce Type: cross Abstract: In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based o
arXiv:2610.05717v1 Announce Type: cross Abstract: In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution P of the database may differ from the target distribution Q of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work itep{li2024fine,guo2025retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and language tasks.
Related
- CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion
- Membership Inference Attacks for Retrieval Based In-Context Learning for Document Question Answering
- Multi-layer Cross-Attention is Provably Optimal for Multi-modal In-context Learning
Source: arXiv cs.LG | 2026-10-06