Search: “multimodal”

CorridorVLA: Explicit Spatial Constraints for Generative Action Heads via Sparse Anchors

arXiv:2604.21241v1 Announce Type: cross Abstract: Vision--Language--Action (VLA) models often use intermediate representations to connect multimodal inputs with continuous control, yet spatial guidanc

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

DiagramBank: A Large-scale Dataset of Diagram Design Exemplars with Paper Metadata for Retrieval-Augmented Generation

DGX agent

arXiv:2604.20857v1 Announce Type: cross Abstract: Recent advances in autonomous ``AI scientist'' systems have demonstrated the ability to automatically write scientific manuscripts and codes with exec

agentsarxiv-cs-ai

24 Apr 2026

InVitroVision: a Multi-Modal AI Model for Automated Description of Embryo Development using Natural Language

arXiv:2604.21061v1 Announce Type: new Abstract: The application of artificial intelligence (AI) in IVF has shown promise in improving consistency and standardization of decisions, but often relies on

researcharxiv-cs-ai

TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval

arXiv:2604.21806v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) is an important image retrieval paradigm that enables users to retrieve a target image using a multimodal query that cons

Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue

arXiv:2604.21144v1 Announce Type: cross Abstract: Situated dialogue requires speakers to maintain a reliable representation of shared context rather than reasoning only over isolated utterances. Curre

IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory

arXiv:2604.20136v1 Announce Type: cross Abstract: Correcting errors in long-video understanding is disproportionately costly: existing multimodal pipelines produce opaque, end-to-end outputs that expo

23 Apr 2026

MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror

arXiv:2604.14785v2 Announce Type: replace Abstract: Recent progress in Multimodal Large Language Models (MLLMs) has demonstrated remarkable advances in perception and reasoning, suggesting their poten

23 Apr 2026

Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensing

arXiv:2512.19302v2 Announce Type: replace Abstract: Large Vision--Language Models (LVLMs) hold great promise for advancing optical remote sensing (RS) analysis, yet existing reasoning segmentation fra

GeoLaux: A Benchmark for Evaluating MLLMs' Geometry Performance on Long-Step Problems Requiring Auxiliary Lines

arXiv:2508.06226v2 Announce Type: replace Abstract: Geometry problem solving (GPS) poses significant challenges for Multimodal Large Language Models (MLLMs) in diagram comprehension, knowledge applica

PC2Model: ISPRS benchmark on 3D point cloud to model registration

arXiv:2604.19596v1 Announce Type: new Abstract: Point cloud registration involves aligning one point cloud with another or with a three-dimensional (3D) model, enabling the integration of multimodal d

Toward Clinically Acceptable Chest X-ray Report Generation: A Qualitative Retrospective Pilot Study of CXRMate-2

arXiv:2604.18967v1 Announce Type: new Abstract: Chest X-ray (CXR) radiology report generation (RRG) models have shown rapid progress, yet their clinical utility remains uncertain due to limited evalua

VideoAgent: Personalized Synthesis of Scientific Videos

arXiv:2509.11253v2 Announce Type: replace Abstract: The technical complexity of research papers often limits their reach, necessitating more accessible formats like scientific videos to disseminate ke

Adaptive Forensic Feature Refinement via Intrinsic Importance Perception

arXiv:2604.16879v1 Announce Type: new Abstract: With the rapid development of generative models and multimodal content editing technologies, the key challenge faced by synthetic image detection (SID)

Local Ai

An Edge-Host-Cloud Architecture for Robot-Agnostic, Caregiver-in-the-Loop Personalized Cognitive Exercise: Multi-Site Deployment in Dementia Care

arXiv:2604.16408v1 Announce Type: new Abstract: We present Speaking Memories, a distributed, stakeholder-in-the-loop robotic interaction platform for personalized cognitive exercise support. Rather th

local-aiarxiv-cs-ro

Agents

AutoVQA-G: Self-Improving Agentic Framework for Automated Visual Question Answering and Grounding Annotation

arXiv:2604.17488v1 Announce Type: new Abstract: Manual annotation of high-quality visual question answering with grounding (VQA-G) datasets, which pair visual questions with evidential grounding, is c

agentsarxiv-cs-cv

Bridging Coarse and Fine Recognition: A Hybrid Approach for Open-Ended Multi-Granularity Object Recognition in Interactive Educational Games

arXiv:2604.16785v1 Announce Type: new Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have enabled open-ended object recognition, yet they struggle with fine-grained tasks. In co

Can Explicit Physical Feasibility Benefit VLA Learning? An Empirical Study

arXiv:2604.17896v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models map multimodal inputs directly to robot actions and are typically trained through large-scale imitation learning. Wh

safetyarxiv-cs-lg

Composed Vision-Language Retrieval for Skin Cancer Case Search via Joint Alignment of Global and Local Representations

arXiv:2603.09108v2 Announce Type: replace Abstract: Medical image retrieval aims to identify clinically relevant lesion cases to support diagnostic decision making, education, and quality control. In

Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping

arXiv:2510.09741v3 Announce Type: replace Abstract: Multimodal large language models (MLLMs) often miss small details and spatial relations in cluttered scenes, leading to errors in fine-grained perce

EgoSound: Benchmarking Sound Understanding in Egocentric Videos

arXiv:2602.14122v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have recently achieved remarkable progress in vision-language understanding. Yet, human perception is inher

From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarization

arXiv:2601.16397v2 Announce Type: replace Abstract: Deploying multimodal large language models (MLLMs) for clinical summarization demands not only fluent generation but also transparency about where e

researcharxiv-cs-cl

From Handwriting to Structured Data: Benchmarking AI Digitisation of Handwritten Forms

arXiv:2604.16504v1 Announce Type: new Abstract: Manual digitisation of structured handwritten documents is slow and costly. We benchmark 17 leading frontier multi-modal large language models and open-

GaLa: Hypergraph-Guided Visual Language Models for Procedural Planning

arXiv:2604.17241v1 Announce Type: new Abstract: Implicit spatial relations and deep semantic structures encoded in object attributes are crucial for procedural planning in embodied AI systems. However

researcharxiv-cs-ro

HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval

arXiv:2604.18037v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image through a multimodal quer

How Much Data is Enough? The Zeta Law of Discoverability in Biomedical Data, featuring the enigmatic Riemann zeta function

arXiv:2604.17581v1 Announce Type: new Abstract: How much data is enough to make a scientific discovery? As biomedical datasets scale to millions of samples and AI models grow in capacity, progress inc

researcharxiv-cs-lg

INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval

arXiv:2604.18051v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) is a challenging image retrieval paradigm that enables to retrieve target images based on multimodal queries consisting o

MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems

arXiv:2503.16549v2 Announce Type: replace Abstract: Despite strong results on many tasks, multimodal large language models (MLLMs) still underperform on visual mathematical problem solving, especially

PBSBench: A Multi-Level Vision-Language Framework and Benchmark for Hematopathology Whole Slide Image Interpretation

arXiv:2604.17570v1 Announce Type: new Abstract: Peripheral Blood Smear (PBS) is a critical microscopic examination in hematopathology that yields whole-slide imaging (WSI). Unlike solid tissue patholo

SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamics

arXiv:2604.16358v1 Announce Type: cross Abstract: MLLMs are increasingly deployed in multi-turn settings, where attackers can escalate unsafe intent through the evolving visual-text history and exploi

safetyarxiv-cs-cl

SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning

arXiv:2604.17385v1 Announce Type: new Abstract: Spatial intelligence, which refers to the ability to reason about geometric and physical structure from visual observations, remains a core challenge fo

TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation

arXiv:2504.18269v2 Announce Type: replace Abstract: When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible. However, th

researcharxiv-cs-cl

Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training

arXiv:2603.23885v3 Announce Type: replace Abstract: Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Tradit

VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis

arXiv:2509.16538v3 Announce Type: replace-cross Abstract: We propose VC-Inspector, a lightweight, open-source large multimodal model (LMM) for reference-free evaluation of video captions, with a focus

researcharxiv-cs-cl

VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction

model-releasesarxiv-cs-cl

arXiv:2505.20279v4 Announce Type: replace-cross Abstract: The rapid advancement of Large Multimodal Models (LMMs) for 2D images and videos has motivated extending these models to understand 3D scenes,

Waking Up Blind: Cold-Start Optimization of Supervision-Free Agentic Trajectories for Grounded Visual Perception

arXiv:2604.17475v1 Announce Type: cross Abstract: Small Vision-Language Models (SVLMs) are efficient task controllers but often suffer from visual brittleness and poor tool orchestration. They typical

safetyarxiv-cs-cl

When Text Hijacks Vision: Benchmarking and Mitigating Text Overlay-Induced Hallucination in Vision Language Models

arXiv:2604.17375v1 Announce Type: new Abstract: Recent advances in Vision-Language Models (VLMs) have substantially enhanced their ability across multimodal video understanding benchmarks spanning tem

CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval

arXiv:2604.15663v1 Announce Type: cross Abstract: Code search, framed as information retrieval (IR), underpins modern software engineering and increasingly powers retrieval-augmented generation (RAG),

Tutorials

Cross-modal learning for plankton recognition

arXiv:2603.16427v2 Announce Type: replace Abstract: This paper considers self-supervised cross-modal coordination as a strategy enabling utilization of multiple modalities and large volumes of unlabel

tutorialsarxiv-cs-cv

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models

arXiv:2604.15488v1 Announce Type: cross Abstract: Large language models (LLMs) often exhibit undesirable behaviors, such as safety violations and hallucinations. Although inference-time steering offer

safetyarxiv-cs-ai

Hierarchical Codec Diffusion for Video-to-Speech Generation

arXiv:2604.15923v1 Announce Type: cross Abstract: Video-to-Speech (VTS) generation aims to synthesize speech from a silent video without auditory signals. However, existing VTS methods disregard the h

Agents

Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval

arXiv:2604.15951v1 Announce Type: new Abstract: Generative AI, particularly Large Language Models, increasingly integrates graph-based representations to enhance reasoning, retrieval, and structured d

agentsarxiv-cs-ai

Applications

MRGEN: A Conceptual Framework for LLM-Powered Mixed Reality Authoring Tools for Education

arXiv:2604.15341v1 Announce Type: cross Abstract: Mixed Reality (MR) offers immersive and multimodal opportunities for education but remains difficult for teachers to author without technical expertis

applicationsarxiv-cs-ai

ReactBench: A Benchmark for Topological Reasoning in MLLMs on Chemical Reaction Diagrams

arXiv:2604.15994v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) excel at recognizing individual visual elements and reasoning over simple linear diagrams. However, when faced

SIMMER: Cross-Modal Food Image--Recipe Retrieval via MLLM-Based Embedding

arXiv:2604.15628v1 Announce Type: cross Abstract: Cross-modal retrieval between food images and recipe texts is an important task with applications in nutritional management, dietary logging, and cook

safetyarxiv-cs-cl

VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck

arXiv:2601.05547v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable progress in multimodal tasks, but remain susceptible to hallucinations, where gener

researcharxiv-cs-ai

Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Tools

arXiv:2510.08480v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable potential in bridging visual and textual reasoning, yet their reliance on text

Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions

arXiv:2604.15823v1 Announce Type: new Abstract: Embodied robotic agents often perceive movies through an egocentric screen-view interface rather than native cinematic footage, introducing domain shift

Bird-SR: Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolution

arXiv:2602.07069v2 Announce Type: replace Abstract: Powered by multimodal text-to-image priors, diffusion-based super-resolution excels at synthesizing intricate details; however, models trained on sy