SmoGVLM: A Small, Graph-enhanced Vision-Language Model

arXiv:2604.16517v1 Announce Type: cross Abstract: Large vision-language models (VLMs) achieve strong performance on multimodal tasks but often suffer from hallucination and poor grounding in knowledge-intensive reasoning. We propose SmoGVLM, a small, graph-enhanced VLM that integrates structured knowledge with visual and textual modalities, using Graph Neural Networks. We investigate the effects of our method across a range of model sizes, from tiny (1.3B) to large (13B) models. Our results demonstrate that, when trained using our approach, a small model can achieve performance gains upto 16.24%, and surpass its larger counterparts, outperforming larger VLMs and strong fine-tuned baselines. These results highlight the potential of structured knowledge augmentation for efficient, smaller-scale multimodal reasoning systems.

Related

• VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
• HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Models
• Mitigating Multimodal Hallucination via Phase-wise Self-reward
• Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies
• Pay Less Attention to Function Words for Free Robustness of Vision-Language Models

Source: arXiv cs.CL | 2026-04-21