Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering

arXiv:2604.24334v1 Announce Type: new Abstract: Standard Retrieval-Augmented Generation (RAG) chunking methods often create excessive redundancy, increasing storage costs and slowing retrieval. This study explores chunk filtering strategies, such as semantic, topic-based, and named-entity-based methods in order to reduce the indexed corpus while preserving retrieval quality. Experiments are conducted on multiple corpora. Retrieval performance is evaluated using a token-based framework based on precision, recall, and intersection-over-union metrics. Results indicate that entity-based filtering can reduce vector index size by approximately 25% to 36% while maintaining high retrieval quality close to the baseline. These findings suggest that redundancy introduced during chunking can be effectively reduced through lightweight filtering, improving the efficiency of retrieval-oriented components in RAG pipelines.

Related

• Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
• An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation
• Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximation
• BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning

Source: arXiv cs.CL | 2026-04-28