Search: “multimodal”

Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras

arXiv:2609.06928v1 Announce Type: new Abstract: Emotion analysis is a fundamental task in computer vision, but its practical deployment remains constrained by the privacy risks inherent to conventiona

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

M3-Tele: A Unified Multimodal Teleoperational Framework for Compliant Whole-Body Mobile Manipulation

DGX agent

arXiv:2609.07859v1 Announce Type: new Abstract: Executing contact-rich tasks efficiently requires the seamless integration of whole-body coordination and physical compliance regulation. However, exist

safetyarxiv-cs-ro

9 Sept 2026

MM-SVGEdit: A Multimodal-Driven SVG Editing for UI Design

arXiv:2609.06116v1 Announce Type: cross Abstract: In the field of UI design, Scalable Vector Graphics (SVG) is widely used as a design medium. However, traditional SVG editing techniques have high ent

Multiple Myeloma Lesion Segmentation on Whole-Body Diffusion-Weighted Imaging via Efficient Anatomical Anticipation and Multimodal Confirmation

arXiv:2609.06165v1 Announce Type: new Abstract: Whole-body diffusion-weighted imaging (WB-DWI) is widely used for multiple myeloma (MM) assessment, yet automated lesion segmentation remains challengin

Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation

arXiv:2609.06078v1 Announce Type: new Abstract: This report summarizes the 8th Large-scale Video Object Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. The challenge evaluates vide

agentsarxiv-cs-cv

Spatiotemporal Heterogeneity of AI-Driven Traffic Flow Patterns and Land Use Interaction: A GeoAI-Based Analysis of Multimodal Urban Mobility

arXiv:2603.05581v2 Announce Type: replace Abstract: Urban traffic flow is governed by the complex, nonlinear interaction between land use configuration and spatiotemporally heterogeneous mobility dema

Wearable Multimodal Human-Machine Interface for Integrated Hand Intentions Decoding in Dynamic Teleoperation

arXiv:2609.07495v1 Announce Type: new Abstract: Under ubiquitous teleoperation environments with optically challenging conditions, an interface for tele-operated grasping that combines wearability wit

researcharxiv-cs-ro

From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making

arXiv:2609.05149v1 Announce Type: new Abstract: Vision-Language Models are commonly evaluated through their final predictions, but understanding whether these decisions are grounded in visual evidence

researcharxiv-cs-cl

7 Sept 2026

KhatianDoc: A Human-Verified Benchmark Diagnosing Multimodal LLM Failure on Bengali Legal Land Records

arXiv:2609.03597v1 Announce Type: new Abstract: Land ownership in Bangladesh is recorded in Ana-Ganda-Kora-Kranti-Til, a base-16 positional fraction system with dedicated Unicode glyphs, no mainstream

4 Sept 2026

PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment

arXiv:2609.02231v1 Announce Type: new Abstract: Interview assessment requires per-criterion judgments grounded in behavioral evidence, yet surging applicant volumes have made human-only evaluation cos

safetyarxiv-cs-ai

3 Sept 2026

Joint Training Is Not Enough: Conditioned Cross-Granularity Training for Multimodal Document Understanding

arXiv:2609.00756v1 Announce Type: new Abstract: The Mutual Reinforcement Effect (MRE) asks whether a fine, span-level and a coarse, document-level task help each other when one model handles both. We

researcharxiv-cs-cl

2 Sept 2026

Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languages

arXiv:2608.29239v1 Announce Type: new Abstract: Low-resource ASR remains difficult because scarce transcripts provide limited supervised evidence for target-side generation. To address this gap, we pr

researcharxiv-cs-cl

Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence

arXiv:2606.15932v3 Announce Type: replace Abstract: While Large Language Models (LLMs) have substantially advanced text-to-code generation, many real programming tasks specify intent through visual ar

agentsarxiv-cs-cl

Do large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effects

arXiv:2608.28626v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate peer reviews, prompting examination of their capacity for critical evaluation. This stu

Multimodal Large Language Models Predict Urban Safety Perception but Encode Non-Neutral Demographic Priors

arXiv:2503.00610v2 Announce Type: replace-cross Abstract: Understanding how people perceive urban environments is essential for inclusive planning, yet conventional surveys are costly and difficult to

safetyarxiv-cs-ai

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

arXiv:2608.27795v1 Announce Type: new Abstract: Robust perception is essential for the deployment of autonomous underwater robots. However, optical cameras become unreliable under poor illumination an

agentsarxiv-cs-cv

31 Aug 2026

Local Ai

A Safety-Gated Multimodal AI Backend for Mental-Health Support: Hierarchical State Representation, Conservative Risk Fusion, and Controlled Generation in Anian

arXiv:2608.26162v1 Announce Type: new Abstract: Safety-critical mental-health support systems must distinguish when supportive conversation is appropriate from when free-form generation should be bloc

local-aiarxiv-cs-ai

28 Aug 2026

Case2Flow: Bridging Patient Cases and Guideline Flowcharts through Multimodal Retrieval

arXiv:2608.26414v1 Announce Type: new Abstract: Medical guidelines encode rich, evidence-based decision logic, yet the specific decision artifact a clinician needs is hard to locate within a guideline

safetyarxiv-cs-cl

28 Aug 2026

Applications

Drift-Aware Multimodal User Representation Learning via Multi-Scale Temporal Modeling and Sparse Mixture-of-Experts

arXiv:2608.25773v1 Announce Type: new Abstract: Understanding user preferences from noisy and temporally evolving social media behaviors is fundamentally challenging due to interest drift, where user

applicationsarxiv-cs-lg

27 Aug 2026

PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans

arXiv:2608.26091v1 Announce Type: cross Abstract: Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the

27 Aug 2026

StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models

arXiv:2608.26067v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a sin

27 Aug 2026

MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models

arXiv:2608.23646v1 Announce Type: new Abstract: Molecular embedding models can serve as foundational infrastructure for computational chemistry and drug discovery, where reusable vector representation

26 Aug 2026

Applications

AI-based worker guidance in assembly and disassembly operations using multimodal ego/exo-centric data capture and structured task knowledge

arXiv:2608.22617v1 Announce Type: cross Abstract: Assembly and disassembly processes rely on expert knowledge that is difficult to document, reuse, and transfer. This paper presents a data-centric app

applicationsarxiv-cs-ai

DesignAgent3D: Interactive 3D Scene Editing via Designer-like Multimodal Reasoning

arXiv:2608.21438v1 Announce Type: new Abstract: Text guided 3D scene editing provides an intuitive interface for modifying reconstructed environments, but remains difficult because natural language de

safetyarxiv-cs-cv

DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts

arXiv:2608.23363v1 Announce Type: cross Abstract: Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deepf

From Multimodal Observation to Interpretable Suggestions: Counterfactual Time-Expanded Relational Modeling of Surgical Teams

arXiv:2608.23254v1 Announce Type: new Abstract: In surgery, patient safety is threatened not only by technical issues but also by poor teamwork. However, existing surgical AI-based solutions focus mai

UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Models

arXiv:2608.22869v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-focused finetuning to achieve strong performance on long-

safetyarxiv-cs-cv

Recognition-Conditioned Reasoning: A Training-Free Multimodal-LLM Pipeline for Fine-Grained Micro-Action Understanding

arXiv:2608.21022v1 Announce Type: new Abstract: Micro-actions are subtle, short, low-amplitude body movements, such as a fidgeting hand or a slight head tilt, that humans perform with little conscious

24 Aug 2026

Representation Affects Retrieval: A Case Study of Skill Discovery and Routing in a Multimodal Agent Harness

arXiv:2608.20389v1 Announce Type: new Abstract: A production agent harness must discover and rank, from a growing library of skills, the one most appropriate for a user's task. At small scale this sel

24 Aug 2026

DeepSeek debuts multimodal language model competitive with Opus 4.8

model-releasessiliconangle

DeepSeek today debuted a new addition to its flagship V4 series of large language models. On launch, V4 Flash Vision Exp is only available via the Chinese startup’s paid developer platform. The compan

21 Aug 2026

Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis

arXiv:2608.20312v1 Announce Type: new Abstract: The capability to perceive and synthesize human-human interactions is fundamental to developing intelligent digital human systems. However, existing dat

21 Aug 2026

Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference

arXiv:2608.18591v1 Announce Type: new Abstract: Uniformly allocating inference reasoning budgets to LLMs is expensive and prone to over-thinking penalties; especially in document tasks where visual la

DentAgent: Evidence-Centric Multi-Agent Coordination for Multimodal Dental Reasoning

arXiv:2608.18878v1 Announce Type: new Abstract: Oral diseases affect billions of people worldwide, underscoring a pressing need for accurate and reliable dental assessment that integrates heterogeneou

agentsarxiv-cs-ai

Eyes on the Image: Gaze Supervised Multimodal Learning for Chest X-ray Diagnosis and Report Generation

arXiv:2508.13068v2 Announce Type: replace Abstract: Medical vision-language models still struggle to match radiologists' attention and to verbalize findings with explicit spatial grounding. We address

Multimodal Rapport Estimation in Real-World HRI

arXiv:2608.18401v1 Announce Type: cross Abstract: Evaluating interaction quality in real-world HRI is an important challenge. If interaction quality can be estimated reliably, the results can be used

Auditing Exposure to Harmful Content on TikTok using Multimodal Language Models: A Cross-National, Age-Stratified Study

arXiv:2608.17583v1 Announce Type: new Abstract: Online video platforms can expose young users to harmful content, but independent audits remain difficult because video annotation is costly and moderat

19 Aug 2026

Closing the Affective Loop: Multimodal Speaker-Listener Emotion-Dynamics-Aware Empathetic Social Robots

arXiv:2608.16686v1 Announce Type: cross Abstract: Empathetic social robots should respond not only to what users say, but also to how their emotions dynamically evolve during interaction. However, exi

safetyarxiv-cs-cl

Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns

arXiv:2607.11621v2 Announce Type: replace Abstract: Aphasia following stroke commonly produces systematic naming errors with characteristic profiles, but whether general-purpose language models not de

NARRATE: A Multimodal Real-World Australian Driving Dataset for Human-Centred Explanations in Automated Driving

arXiv:2608.14767v1 Announce Type: cross Abstract: Automated vehicles must explain their decisions in ways that passengers can understand, monitor, and trust. Existing language-annotated driving datase

PathFinder: Joint Decompositions of Linked Multimodal Datasets

arXiv:2608.14951v1 Announce Type: new Abstract: Low-rank matrix decompositions can uncover patterns and structure in data and have a number of different applications across many disciplines. Extension

researcharxiv-cs-lg

Sequential Multimodal Evidence Optimization for Product Media Ranking in E-Commerce

arXiv:2608.15662v1 Announce Type: new Abstract: On modern e-commerce stores, customers consume ordered slates of heterogeneous product media, such as images, videos, and 3D renders, before making purc

Tutorials

Generalizable Operating Room Expert with Multimodal Enhancement

arXiv:2508.08199v2 Announce Type: replace Abstract: Precise spatial modeling in the operating room (OR) is essential for intraoperative awareness, hazard avoidance, and surgical decision-making. Altho

tutorialsarxiv-cs-cv

14 Aug 2026

How effective are VLMs in assisting humans in inferring the quality of mental models from Multimodal short answers?

arXiv:2603.00056v2 Announce Type: replace-cross Abstract: STEM Mental models can play a critical role in assessing students' conceptual understanding of a topic. They not only offer insights into what

13 Aug 2026

Local Ai

Embodied Multimodal Grounding for Open-Vocabulary Mobile Manipulation via Semantic 3D Gaussian Splatting

arXiv:2608.10756v1 Announce Type: cross Abstract: Embodied mobile manipulation requires language, visual observations, three-dimensional scene structure, and action feasibility to be aligned before ex

local-aiarxiv-cs-cv

12 Aug 2026

Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

arXiv:2608.10522v1 Announce Type: cross Abstract: While vision-language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent

12 Aug 2026

Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations

arXiv:2608.07895v1 Announce Type: cross Abstract: Robot demonstration datasets used to train vision-language-action policies can contain a subtle but harmful failure mode: trajectories that are behavi

11 Aug 2026

From Visual to Multimodal: Systematic Ablation of Encoders and Fusion Strategies in Animal Identification

arXiv:2603.02270v2 Announce Type: replace Abstract: Automated animal identification is a practical task for reuniting lost pets with their owners, yet current systems often struggle due to limited dat

11 Aug 2026

Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models

arXiv:2604.02048v2 Announce Type: replace Abstract: Developing vision-language models (VLMs) that generalize across diverse tasks requires large-scale training datasets with diverse content. In Englis