Research
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
arXiv:2604.10167v1 Announce Type: cross Abstract: Multi-vector models dominate Visual Document Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational
arXiv:2604.10167v1 Announce Type: cross Abstract: Multi-vector models dominate Visual Document Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational costs present a major barrier to practical deployment. In this paper, we propose ColChunk, a plug-and-play framework that introduces multimodal late chunking to construct efficient, contextualized multi-vectors. Unlike existing pruning or fixed-token approaches, ColChunk employs hierarchical clustering on patch-level embeddings, fused with a 2D position prior to ensure spatial-semantic coherence. This adaptive grouping allows for a content-aware representation that preserves global context while drastically reducing the vector count. Evaluations across 24 VDR datasets demonstrate ColChunk achieves over a 90% reduction in storage requirements while simultaneously delivering a 9-point average improvement in nDCG@5 across representative single-vector models. ColChunk provides a practical solution for balancing retrieval accuracy and efficiency in visual document systems.
Related
- Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models
- Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains
- Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
- Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models
Source: arXiv cs.CL | 2026-04-14