Search: “multimodal”

arXiv:2607.07016v1 Announce Type: cross Abstract: Accurate forecasting of cellular network traffic is essential for network planning, resource allocation, and quality-of-service assurance in modern mo

applicationsarxiv-cs-ai

9 Jul 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

DGX agent

arXiv:2607.06618v1 Announce Type: cross Abstract: Following the CMIVQA, MMI-VQA, and M4IVQA challenges in NLPCC 2023--2025, we introduce the Difficulty-Aware Medical Instructional Video Question Answe

model-releasesarxiv-cs-ai

9 Jul 2026

Safety

GaussFusion: Towards Multimodal 3D Gaussian Pretraining

arXiv:2607.05906v1 Announce Type: new Abstract: 3D Gaussian Splatting provides an explicit representation that jointly models geometry and appearance, serving as a scalable foundation for 3D represent

safetyarxiv-cs-cv

8 Jul 2026

A Multimodal Reasoning Typology for Grounding Chart-Image Coherence in Science Communication

arXiv:2607.05222v1 Announce Type: new Abstract: Charts and images appear together throughout scientific publications, yet most computational work does not characterize their coherence. We argue that a

local-aiarxiv-cs-cv

AppAgent: Multimodal Agents as Smartphone Users

arXiv:2312.13771v3 Announce Type: replace Abstract: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper i

agentsarxiv-cs-cv

BanglaMemeEvidence: A Multimodal Benchmark Dataset for Explanatory Evidence Detection in Bengali Memes

arXiv:2607.03981v1 Announce Type: cross Abstract: Memes have become influential communication tools on social media, combining viral visuals with concise messaging to convey impactful ideas. While sub

Applications

From Geometric Labels to Semantic Understanding of Indoor Building Components Using Multimodal Large Language Models

arXiv:2607.03661v1 Announce Type: new Abstract: Point cloud-based understanding has become an important enabler for facility operation and maintenance involving indoor building components. However, ex

applicationsarxiv-cs-cv

Glance-Say: Multimodal Human-Robot Collaboration and Intent Recognition via Sticky Glance

arXiv:2603.06121v2 Announce Type: replace Abstract: Gaze and speech are promising interaction modalities for individuals with motor impairments, yet robust intent recognition in multi-object environme

researcharxiv-cs-ro

Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion

arXiv:2607.02862v1 Announce Type: new Abstract: Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit signifi

researcharxiv-cs-cl

K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

arXiv:2607.02680v1 Announce Type: cross Abstract: MLLMs have shown strong zero-shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, appl

Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models

arXiv:2607.02909v1 Announce Type: cross Abstract: Taxonomies provide key information about the semantic relationships between concepts and the inherent organization of vision and language. Despite the

researcharxiv-cs-ai

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

arXiv:2607.04733v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain

local-aiarxiv-cs-cl

MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments

arXiv:2512.22867v2 Announce Type: replace Abstract: Socially compliant navigation requires structured reasoning about dynamic pedestrians and physical constraints to ensure safe and interpretable deci

Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

arXiv:2512.14926v2 Announce Type: replace-cross Abstract: Focusing on low-resource languages is an essential step toward democratizing generative AI. In this work, we contribute to reducing the multim

Applications

PedestrianDiffusion: Multimodal Generative Denoising and Dense State Estimation for Inertial Navigation

arXiv:2607.03349v1 Announce Type: cross Abstract: The accuracy of consumer-grade inertial navigation is bottlenecked by the stochastic noise of Micro-Electro-Mechanical Systems (MEMS). Traditional det

applicationsarxiv-cs-ai

Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain

arXiv:2510.17801v2 Announce Type: replace-cross Abstract: Building robots that can perceive, reason, and act in dynamic, unstructured environments remains a central challenge. Recent embodied systems

Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation

arXiv:2607.02593v1 Announce Type: cross Abstract: While knowledge distillation (KD) is widely adopted for training lightweight models by leveraging supervision from larger teacher models, relying sole

researcharxiv-cs-cl

MKGR: Multimodal Knowledge-Graph Representation Learning for Cold-Start Protein-Protein Interaction Prediction

arXiv:2607.01627v1 Announce Type: cross Abstract: Accurate protein-protein interaction (PPI) prediction is central to functional genomics, disease mechanism discovery, and drug development. A difficul

3 Jul 2026

VisionAId: An Offline-First Multimodal Android Assistant for People with Visual Impairment, Featuring Personalized Object Retrieval

arXiv:2607.02371v1 Announce Type: cross Abstract: Over 285 million people worldwide live with a visual impairment, for whom everyday tasks such as avoiding obstacles, locating personal belongings, rec

3 Jul 2026

From Technical Metrics to User Perception: A User Study of a Multimodal Human-Robot Interaction System for Object Detection and Grasping

arXiv:2607.00530v1 Announce Type: cross Abstract: Improvements in the technical performance of human--robot interaction (HRI) systems do not automatically translate into differences that human users c

2 Jul 2026

QuaMoE-DRF: Proactive Beam and Rate Adaptation via Multimodal Dynamic Radio Map Forecasting in ISAC Networks

arXiv:2607.00974v1 Announce Type: cross Abstract: Static radio maps provide location-dependent propagation priors, but they cannot capture short-term blockage caused by moving objects. Direct sensing-

2 Jul 2026

Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap

arXiv:2607.00606v1 Announce Type: new Abstract: Recent work on Thinking with Images makes vision a dynamic part of reasoning, but does so through generation: the model invokes external tools, synthesi

2 Jul 2026

ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection

arXiv:2606.30646v1 Announce Type: cross Abstract: Speech recruits the same executive, attentional, and working memory processes underlying instrumental activities of daily living, or IADLs, providing

researcharxiv-cs-ai

LLM-Powered Interactive Robotic Action Synthesis from Multimodal Speech, Gestures, and Music

arXiv:2606.31158v1 Announce Type: cross Abstract: The quest for intuitive and natural human-robot interaction (HRI) remains a significant challenge in robotics. Traditional methods often rely on rigid

researcharxiv-cs-ai

Tutorials

Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

arXiv:2506.23102v2 Announce Type: replace-cross Abstract: Current CT report generation frameworks predominantly rely on global feature representations, often failing to capture region-specific details

tutorialsarxiv-cs-cv

Robust Autonomous UAV Landing on Maritime Platforms via Multimodal Agentic AI and Active Wave Compensation

arXiv:2606.31613v1 Announce Type: new Abstract: Autonomous aerial inspection of marine infrastructure is frequently compromised by stochastic sea states, introducing risks of high-kinetic impacts, pos

agentsarxiv-cs-cv

Safety

Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning

arXiv:2606.29812v1 Announce Type: new Abstract: Inductive biases steer learning toward generalizable solutions by encoding task structure. In this work, we identify a crucial missing bias in MLLMs: cr

safetyarxiv-cs-cv

Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification

arXiv:2606.29023v1 Announce Type: cross Abstract: Spatio-temporal grounding in long videos requires precise temporal localization and robust object tracking conditioned on natural-language queries. Wh

local-aiarxiv-cs-ai

Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI

arXiv:2606.28991v1 Announce Type: new Abstract: Cardiac magnetic resonance imaging (CMR) routinely records structured acquisition metadata, yet most CMR foundation models rely primarily on image-only

researcharxiv-cs-cv

LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion

arXiv:2606.29900v1 Announce Type: cross Abstract: Personality recognition in asynchronous video interviews (AVIs) has become increasingly important due to their widespread adoption in modern recruitme

Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework

arXiv:2606.29808v1 Announce Type: cross Abstract: Chart data extraction, which reverse-engineers data tables from chart images, is essential for reproducibility, analysis, retrieval, and redesign. Exi

ManimAgent: Self-Evolving Multimodal Agents for Visual Education

arXiv:2606.30296v1 Announce Type: new Abstract: Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode:

agentsarxiv-cs-ai

Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models

arXiv:2606.28369v1 Announce Type: cross Abstract: Semantic search and recommendation of similar documents, such as news and reports about unusual environmental events (e.g., a dead whale washed ashore

local-aiarxiv-cs-ai

Tutorials

NIVA: A Multimodal Foundation Model for Actionable Earth System Intelligence

arXiv:2606.28546v1 Announce Type: new Abstract: Recent advances in AI-driven weather and climate modeling have improved forecast skill while reducing computational cost. However, existing data-driven

tutorialsarxiv-cs-lg

OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data

arXiv:2606.30019v1 Announce Type: new Abstract: Music-driven dance video generation aims to synthesize expressive human motion that is temporally aligned with music while maintaining high visual fidel

researcharxiv-cs-cv

Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

arXiv:2606.28104v1 Announce Type: new Abstract: Vision-based assessment can provide convenient and cost-effective evaluation in Traditional Chinese Medicine (TCM) rehabilitation training, where action

researcharxiv-cs-cv

Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videos

arXiv:2606.27484v1 Announce Type: new Abstract: Autism spectrum disorder (ASD) affects 1 in 31 US children, yet median age at diagnosis exceeds four years. Artificial intelligence pipelines that provi

Multimodal Evaluator Preference Collapse: Cross-Modal Coupling in Self-Evolving Agents

model-releasesarxiv-cs-cl

arXiv:2606.16682v3 Announce Type: replace-cross Abstract: When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge. We show that Evaluator Preferen

ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

arXiv:2603.19466v2 Announce Type: replace Abstract: Effective collaboration begins with knowing when to ask for help. For example, when trying to identify an occluded object, a human would ask someone

ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

arXiv:2606.27974v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fi

agentsarxiv-cs-ai

BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning

arXiv:2602.22284v3 Announce Type: replace Abstract: Recent advancements in deep learning have actively addressed complex challenges within the Computer-Aided Design (CAD) domain.However, most existing

agentsarxiv-cs-lg

26 Jun 2026

Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars

arXiv:2606.26107v1 Announce Type: cross Abstract: Sign language communication systems, that integrate emotional expression remain underexplored, particularly for low-resource languages. This pilot stu

26 Jun 2026

An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

arXiv:2606.25174v1 Announce Type: cross Abstract: Field-scale retrieval of surface soil moisture (SM), leaf area index (LAI), and plant height (PH) is essential for precision agriculture, yet it remai

25 Jun 2026

Safety

SAC^2-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition

arXiv:2606.25542v1 Announce Type: new Abstract: Micro-expression recognition (MER) is challenging due to subtle facial movements, limited data, and the ambiguous relationship between Action Units (AUs

safetyarxiv-cs-cv

25 Jun 2026

3D Masked Autoencoders are Robust Learners of Volumetric and Multimodal Cellular Representations for Microscopy

arXiv:2606.23964v1 Announce Type: cross Abstract: Self-supervised learning in fluorescence microscopy often relies on 2D projections, despite the inherently three-dimensional nature of cells. We prese

local-aiarxiv-cs-cv

E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis

arXiv:2606.23888v1 Announce Type: cross Abstract: While Vision-Language Models (VLMs) show great promise in volumetric medical report generation, they frequently suffer from visual hallucinations and

local-aiarxiv-cs-ai

MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

model-releasesarxiv-cs-cl

arXiv:2606.24155v1 Announce Type: new Abstract: Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a red

SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

arXiv:2606.24350v1 Announce Type: new Abstract: Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods of

researcharxiv-cs-ro