Search: “multimodal”

Efficient Hybrid Hard--Soft Signal Temporal Logic MPC for Driving under Multimodal Uncertainty

arXiv:2610.15934v1 Announce Type: new Abstract: Motion planning for automated vehicles in interactive traffic has to simultaneously ensure safety, respect traffic rules, and maintain comfortable progr

safetyarxiv-cs-ro

14 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Multimodal Fusion of RGB and Complementary Modalities for Semantic Segmentation

DGX agent

arXiv:2610.17800v1 Announce Type: new Abstract: Multi-modal semantic segmentation adds an auxiliary sensing stream to RGB images (RGB-X), such as thermal, LiDAR, depth, event, polarization, or light f

model-releasesarxiv-cs-cv

14 Oct 2026

Agents

Multimodal Sensor-Gated Mid-Air Launch and Vision-Based Recovery of a Child UAV on a Hovering Mothership Drone

arXiv:2610.15740v1 Announce Type: new Abstract: Launching a micro unmanned aerial vehicle (UAV) from a hovering carrier and recovering it in flight requires reliable confirmation that the child is sec

agentsarxiv-cs-ro

14 Oct 2026

Parameter-Efficient Multimodal Brain Tumor Segmentation With Stage-Wise Shared Adapters

arXiv:2610.18571v1 Announce Type: new Abstract: Different MRI modalities reveal different parts of a brain tumor. Segmentation models combine information from these modalities to outline tumor regions

14 Oct 2026

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

arXiv:2610.13397v1 Announce Type: cross Abstract: We develop a model-free, noise-aware Bayesian clustering framework that constructs a martingale posterior over clusterings from a predictive Bayesian

12 Oct 2026

DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

arXiv:2610.11993v1 Announce Type: cross Abstract: Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared w

Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers

arXiv:2610.11315v1 Announce Type: cross Abstract: In diffusion transformers, low-rank branches can mitigate 4-bit weight--activation (W4A4) post-training quantization (PTQ) loss by decomposing each we

Applications

EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams

arXiv:2610.12248v1 Announce Type: new Abstract: Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user's activity through first-pe

applicationsarxiv-cs-cl

Agents

OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

arXiv:2610.12419v1 Announce Type: new Abstract: Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and

agentsarxiv-cs-cv

Immiscible Diffusion Policy: Preserving Multimodal Robot Actions through Label-Free Noise Assignment

arXiv:2610.09369v1 Announce Type: cross Abstract: When diffusion policies were first introduced, they were expected to recover multi-modal action distributions. However, we find this expectation does

safetyarxiv-cs-lg

Tutorials

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

arXiv:2610.09355v1 Announce Type: cross Abstract: Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalogra

tutorialsarxiv-cs-cv

PrismSSL: One Interface, Many Modalities; A Single-Interface Library for Multimodal Self-Supervised Learning

model-releasesarxiv-cs-lg

arXiv:2511.17776v2 Announce Type: replace Abstract: We present PrismSSL, a Python library that unifies state-of-the-art self-supervised learning (SSL) methods across audio, vision, graphs, and cross-m

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carr

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning

arXiv:2610.06571v1 Announce Type: new Abstract: Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences,

6 Oct 2026

Conditional Flow Matching for Single-Neuron Electrophysiology: Capturing Multimodal Responses Across Stimuli

arXiv:2610.06520v1 Announce Type: new Abstract: Neurons of the brain exhibit a rich repertoire of electrophysiology dynamics with the same repeated stimulus eliciting very different voltage responses

6 Oct 2026

Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing

arXiv:2610.00825v1 Announce Type: new Abstract: Dubbing quality control requires a reference-free judge that can determine whether a candidate text line matches a speaker's visible articulation in bot

2 Oct 2026

Same Reward, Different Skills: When Multimodal RL Learns to Look

model-releasesarxiv-cs-lg

arXiv:2610.01908v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With im

2 Oct 2026

Local Ai

Spherical Interpolation for Backward-Compatible Multimodal Representations

arXiv:2609.39836v1 Announce Type: new Abstract: Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natural

local-aiarxiv-cs-cv

1 Oct 2026

Local Ai

FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning

arXiv:2609.37582v1 Announce Type: cross Abstract: Federated knowledge must remain usable by recipients with different modalities, private architectures, and tasks. We present FedSocket, which makes re

local-aiarxiv-cs-ai

Tutorials

Multimodal Detection of Higher-Order Behavioral Constructs: Self-Compassion in Structured Reflective Interaction

arXiv:2609.37148v1 Announce Type: cross Abstract: Many of the qualities that matter most in how people learn and grow, how someone regulates their emotions, reflects on a setback, or stays aware of ot

tutorialsarxiv-cs-cl

OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells

arXiv:2609.37773v1 Announce Type: new Abstract: Artificial Intelligence Virtual Cells (AIVCs) are envisioned as scientific agents that simulate cellular responses, explain underlying mechanisms, and s

Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation

arXiv:2609.36995v1 Announce Type: cross Abstract: Few-step streaming audio--video generation requires both causal modeling and step distillation, yet standard training recipes face two context-related

safetyarxiv-cs-ai

VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict

arXiv:2609.37324v1 Announce Type: new Abstract: Conflicting emotional cues can be individually valid: a subdued voice may reflect a blocked goal while a smile satisfies a social obligation. Their inte

AMBIT: Anticipatory Multimodal Body Recruitment for Bimanual Tracking on a Humanoid

arXiv:2609.33484v1 Announce Type: new Abstract: A humanoid with 5-DoF arms cannot track generic bimanual end-effector trajectories with its arms alone; pelvis and waist motion must be recruited, but w

researcharxiv-cs-ro

29 Sept 2026

Refinement Symmetry in Multimodal Transformers

arXiv:2609.32669v1 Announce Type: cross Abstract: Attention weights depend on token counts, which change with the representation of a signal. We study refinement symmetry: splitting a representation w

29 Sept 2026

The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond

arXiv:2609.33280v1 Announce Type: cross Abstract: Medical images are released with the reports that describe them, and protecting the image does not protect the report. This paper measures the text co

29 Sept 2026

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language re

28 Sept 2026

Multimodal Routing and Region Refinement for Language-Guided Medical Image Segmentation

arXiv:2609.28860v1 Announce Type: new Abstract: Textual descriptions can reduce ambiguity in medical image segmentation by specifying the finding and location to be delineated. Existing text-guided me

Multimodal Thinking with Renderable Programs

model-releasesarxiv-cs-cl

arXiv:2609.30130v1 Announce Type: cross Abstract: Current vision-language models (VLMs) excel at visual content understanding and text-based reasoning, yet their structure limits the advancement of in

OncoVision: Integrating Mammography and Clinical Data through Attention-Driven Multimodal AI for Enhanced Breast Cancer Diagnosis

arXiv:2511.19667v2 Announce Type: replace Abstract: OncoVision is a privileged-information training framework that uses mammography images and clinical features during training and performs inference

WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks

arXiv:2507.00938v4 Announce Type: replace-cross Abstract: Foundation models now enable autonomous agents to interact with real-world websites, but existing benchmarks emphasize general-purpose browsin

Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

model-releasesarxiv-cs-cl

arXiv:2609.27353v1 Announce Type: new Abstract: Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reprod

24 Sept 2026

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

24 Sept 2026

AIGC Video Detection based on the fusion of spatial-frequency-optical flow multimodal features

arXiv:2609.26274v1 Announce Type: new Abstract: The rapid evolution of generative AI (e.g., Sora, Hunyuan) makes it essential to develop effective detection strategies that can generalize across ever-

safetyarxiv-cs-cv

Designing Task-Induced Arousal: A Multimodal Stress Induction Method for Interactive Experiments

arXiv:2609.26156v1 Announce Type: cross Abstract: HCI and HRI studies often require short, repeatable arousal manipulations that can run while participants continue interacting with a device or robot.

researcharxiv-cs-ro

Destination Support Restoration for Finite-Set Multimodal Trajectory Prediction

arXiv:2609.25942v1 Announce Type: new Abstract: Robots operating around pedestrians often reason over a finite set of predicted human futures. Repeated online updates can concentrate this limited pred

researcharxiv-cs-ro

ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion

arXiv:2609.26648v1 Announce Type: cross Abstract: Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under chall

Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA

arXiv:2609.22628v1 Announce Type: new Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isol

Zephyron: Integrated Design and Analytical Evaluation of a Solar-Assisted Mobile Manipulator for Multimodal Environmental Reconnaissance and Distributed Visual Inference

arXiv:2609.25709v1 Announce Type: new Abstract: Environmental reconnaissance needs mobile platforms that carry sensors, preserve measurement context, and return interpretable evidence under limited en

safetyarxiv-cs-ro

Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing

arXiv:2609.24791v1 Announce Type: cross Abstract: Challenging behaviors including aggression, self-injury, and property destruction are observed in 68% of autistic youth and pose risks to youth and ca

22 Sept 2026

Offline Multimodal Large Language Models for Decision Support in Air Operations

arXiv:2609.21390v1 Announce Type: new Abstract: Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In

21 Sept 2026

Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders

model-releasesarxiv-cs-cl

arXiv:2609.20845v1 Announce Type: new Abstract: A decoder that turns video or audio into text conventionally consumes the entire input before emitting a word. Offline this is merely more than the task

21 Sept 2026

Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

arXiv:2603.23393v2 Announce Type: replace Abstract: Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when

safetyarxiv-cs-ro

21 Sept 2026

Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment

arXiv:2609.20199v1 Announce Type: new Abstract: Cognitive workload reflects the mental effort required during task performance and is central to the design of adaptive human-machine systems. The use o

18 Sept 2026

Understanding Role Switching in Human-AI Collaboration through Multimodal Behavioral Signals

arXiv:2509.20666v2 Announce Type: replace-cross Abstract: Human-AI collaboration often requires dividing complex tasks into complementary subtasks. As a task unfolds, users may want to shift which sub

18 Sept 2026

DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment

arXiv:2601.18493v2 Announce Type: replace Abstract: Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-cen

16 Sept 2026

Local Ai

On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

arXiv:2609.11961v1 Announce Type: new Abstract: Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-pre

local-aiarxiv-cs-lg

14 Sept 2026

Physics as the label for measuring and correcting materials reasoning in multimodal models

arXiv:2609.12181v1 Announce Type: new Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate ph