SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

arXiv:2608.01899v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient extit{extbf{Spatio}-vision extbf{L}anguage extbf{M}odels (SpatioLM)}, that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at href{https://github.com/xiaomi-research/spatio-lm}{faGithub~spatio-lm}.

Related

• Reasmory: 3D Reconstruction as Explicit Memory for VLMs Spatial Reasoning
• SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation
• SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
• MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence

Source: arXiv cs.CL | 2026-08-04