Bottlenecked semantic compression by representation reconstruction

arXiv:2610.17412v1 Announce Type: new Abstract: Vision Transformers incur high computational costs when processing long visual token sequences. Existing reduction methods typically rank tokens independently and discard those deemed unimportant or redundant. We propose a reconstruction-guided framework for visual token condensation (ReConV), which summarizes the complete visual sequence with a compact Summary Bank (SMB) and retains tokens whose information is not adequately captured by the SMB, as indicated by their reconstruction residuals. At each compression layer, a lightweight position-conditioned decoder reconstructs the current tokens, leaving shared information compressed in the SMB while explicitly preserving poorly reconstructed details. Experiments on image classification and multimodal understanding demonstrate favorable performance-efficiency trade-offs: ReConV surpasses the uncompressed baseline with 48% fewer GMACs on ImageNet and enables reusable image-level representations for efficient multi-turn inference. Code is available at TokenDrop.

Related

• Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
• Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs
• Representation Dynamics Reveal Semantic Saliency and Similarity for Visual Token Pruning in MLLMs

Source: arXiv cs.CV | 2026-10-14