Finding the Right Balance: Relevance and Diversity in LLM Retrieval

arXiv:2610.09412v1 Announce Type: cross Abstract: Retrieval diversification is widely available in retrieval-augmented generation (RAG) frameworks, yet prior studies disagree on whether it improves retrieval and answer quality. We show that its effectiveness varies primarily with candidate-pool redundancy, in a pattern consistent with the number of distinct evidence pieces a query requires. Using controlled near-duplicate injection and production-style overlapping chunking, we find that diversification harms relevance, evidence coverage and answer quality on clean pools, but becomes beneficial on multi-evidence tasks when redundancy causes nearest-neighbor retrieval to select repeated passages. We therefore introduce a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-k selection falls below the query's evidence requirement. Computed from existing embeddings, the rule captures most of the achievable gain, transfers across datasets and encoders and automatically reduces to nearest-neighbor retrieval for single-evidence queries. We also introduce RNG-Score, a geometric reranker with an exact nearest-neighbor fallback whose margin indicates duplicate structure. Overall, we conclude that diversification should be used selectively, based on observable redundancy and evidence requirements.

Related

• Re:CAP - Auditing Retrieval Coverage in Production RAG Pipelines
• The Laws of Context Allocation: Causal Measurement and Closed-Loop Orchestration in Generative Search
• Better Together: Complementary Query Rewriting Under a Strong RAG Baseline
• Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models
• GroupRAG: Cognitively Inspired Group-Aware Retrieval and Reasoning via Knowledge-Driven Problem Structuring

Source: arXiv cs.CL | 2026-10-08