Search: “multimodal”

Building multimodal models for spatial reasoning

What is spatial reasoning? AI is moving beyond the digital world. The next generation of AI systems needs to understand and interact with physical environments, from robots to autonomous systems navig

agentslambda-labs

24 Sept 2026

Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2609.27353v1 Announce Type: new Abstract: Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reprod

model-releasesarxiv-cs-cl

24 Sept 2026

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

model-releasesarxiv-cs-cv

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

24 Sept 2026

AIGC Video Detection based on the fusion of spatial-frequency-optical flow multimodal features

arXiv:2609.26274v1 Announce Type: new Abstract: The rapid evolution of generative AI (e.g., Sora, Hunyuan) makes it essential to develop effective detection strategies that can generalize across ever-

safetyarxiv-cs-cv

Designing Task-Induced Arousal: A Multimodal Stress Induction Method for Interactive Experiments

arXiv:2609.26156v1 Announce Type: cross Abstract: HCI and HRI studies often require short, repeatable arousal manipulations that can run while participants continue interacting with a device or robot.

researcharxiv-cs-ro

Destination Support Restoration for Finite-Set Multimodal Trajectory Prediction

arXiv:2609.25942v1 Announce Type: new Abstract: Robots operating around pedestrians often reason over a finite set of predicted human futures. Repeated online updates can concentrate this limited pred

researcharxiv-cs-ro

ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion

arXiv:2609.26648v1 Announce Type: cross Abstract: Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under chall

researcharxiv-cs-cv

Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA

arXiv:2609.22628v1 Announce Type: new Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isol

Zephyron: Integrated Design and Analytical Evaluation of a Solar-Assisted Mobile Manipulator for Multimodal Environmental Reconnaissance and Distributed Visual Inference

arXiv:2609.25709v1 Announce Type: new Abstract: Environmental reconnaissance needs mobile platforms that carry sensors, preserve measurement context, and return interpretable evidence under limited en

Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing

arXiv:2609.24791v1 Announce Type: cross Abstract: Challenging behaviors including aggression, self-injury, and property destruction are observed in 68% of autistic youth and pose risks to youth and ca

22 Sept 2026

Offline Multimodal Large Language Models for Decision Support in Air Operations

arXiv:2609.21390v1 Announce Type: new Abstract: Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In

21 Sept 2026

Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders

model-releasesarxiv-cs-cl

arXiv:2609.20845v1 Announce Type: new Abstract: A decoder that turns video or audio into text conventionally consumes the entire input before emitting a word. Offline this is merely more than the task

21 Sept 2026

Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

arXiv:2603.23393v2 Announce Type: replace Abstract: Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when

21 Sept 2026

Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment

arXiv:2609.20199v1 Announce Type: new Abstract: Cognitive workload reflects the mental effort required during task performance and is central to the design of adaptive human-machine systems. The use o

18 Sept 2026

Understanding Role Switching in Human-AI Collaboration through Multimodal Behavioral Signals

arXiv:2509.20666v2 Announce Type: replace-cross Abstract: Human-AI collaboration often requires dividing complex tasks into complementary subtasks. As a task unfolds, users may want to shift which sub

18 Sept 2026

DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment

model-releasesarxiv-cs-cv

arXiv:2601.18493v2 Announce Type: replace Abstract: Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-cen

16 Sept 2026

Intern-S2-397B (multimodal, reasoning, coding, and scientific agent capabilities)

Model: https://huggingface.co/internlm/Intern-S2-397B Collection: https://huggingface.co/collections/internlm/intern-s2 From Intern Large Models on 𝕏: https://x.com/intern_lm/status/209942518458737097

local-air-localllama

14 Sept 2026

On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

arXiv:2609.11961v1 Announce Type: new Abstract: Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-pre

local-aiarxiv-cs-lg

14 Sept 2026

Physics as the label for measuring and correcting materials reasoning in multimodal models

model-releasesarxiv-cs-cv

arXiv:2609.12181v1 Announce Type: new Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate ph

14 Sept 2026

Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36

model-releasesr-localllama

I find this new model at HF: 'Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding. Architecture Agnes-3.0-Flash

12 Sept 2026

AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

arXiv:2609.09212v1 Announce Type: cross Abstract: This paper presents an end-to-end evaluation framework for image-triggered command injection against computer-use agents (CUAs). The goal is to test w

local-aiarxiv-cs-ai

11 Sept 2026

An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks

arXiv:2609.09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read

11 Sept 2026

A Multimodal Label Forecasting Method for Aperiodic Visuo-Motor Time Series

model-releasesarxiv-cs-ro

arXiv:2609.07930v1 Announce Type: new Abstract: Deep learning models have been increasingly applied to Time Series Forecasting (TSF) in recent years. Transformer-based and MLP-based models have both b

9 Sept 2026

Applications

KODAMA: Multimodal Digital Twin Reconstruction for Urban RF Propagation Modelling

arXiv:2609.07298v1 Announce Type: new Abstract: 3D reconstruction typically strives for geometric fidelity or visual plausibility. Radio frequency digital twins (RFDT) are instead judged by whether co

applicationsarxiv-cs-cv

9 Sept 2026

BioSync: Transformer-Based Cross-Modal Fusion for a Multimodal Physiological Digital Biomarker

arXiv:2609.04504v1 Announce Type: new Abstract: Cardiac, neural, behavioral, and speech measurements from wearable and mobile devices provide partial, noise-sensitive views of physiological state. Bio

Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction

arXiv:2609.05361v1 Announce Type: new Abstract: Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and robots in real-world environments. This paper introduce

local-aiarxiv-cs-ro

Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection

arXiv:2603.05623v2 Announce Type: replace-cross Abstract: Camera-LiDAR fusion is widely used in autonomous driving to enable accurate 3D object detection. However, bird's-eye view (BEV) fusion detecto

Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

arXiv:2609.04801v1 Announce Type: new Abstract: Contextualized visual personalization can retrieve a true record yet apply it to the wrong visual subject. We formalize when a record may condition an a

Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents

arXiv:2609.03438v1 Announce Type: new Abstract: Graphical user interface (GUI) agents are increasingly used to execute natural-language instructions on user interfaces, yet real users may issue infeas

4 Sept 2026

Landmark-Based Discrimination of Injury-Associated Athlete-Sessions from Minute-Resolution Multimodal Football Monitoring Data

arXiv:2609.03790v1 Announce Type: new Abstract: Athlete monitoring data may be recorded minute by minute throughout a match or training session, while injury information may only indicate whether the

4 Sept 2026

Agentic Multimodal Models for Environmental Hyperspectral Unmixing

arXiv:2609.01289v1 Announce Type: new Abstract: Hyperspectral unmixing is a key task in remote sensing that aims to decompose mixed pixels in hyperspectral images into their constituent material signa

agentsarxiv-cs-cv

2 Sept 2026

Differentially Private Paired Table-Image Multimodal Synthesis

arXiv:2609.00708v1 Announce Type: cross Abstract: Differentially private (DP) synthesis has been extensively studied for tabular and image data separately, yet many real-world datasets contain images

safetyarxiv-cs-ai

2 Sept 2026

Cross-Regional Grapevine Cold Hardiness Prediction via Learned Multimodal Latent Representations

arXiv:2608.31097v1 Announce Type: new Abstract: Accurate daily predictions of cold hardiness in woody plants are critical in regions where freezing temperatures can damage dormant buds and reduce seas

MIRAGE-CAD: Construction-Mediated Multimodal Generation of Executable CAD Programs

arXiv:2608.28669v1 Announce Type: new Abstract: Recovering an executable parametric CAD program from an observed object is fundamentally ambiguous, because the same final geometry can result from diff

researcharxiv-cs-cv

Parametric Multimodal User Memory: Storing What Captions Cannot Carry

arXiv:2608.28609v1 Announce Type: cross Abstract: A personalized agent needs a user memory: a persistent model of who its user is. Today it is almost always text -- transcripts and captions retrieved

agentsarxiv-cs-ai

SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks

arXiv:2606.15872v2 Announce Type: replace Abstract: Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of

agentsarxiv-cs-cl

Relaxation-Aware Multimodal Sensing of Soft Gripper Driven by Structure-Perception-Learning

arXiv:2608.26622v1 Announce Type: new Abstract: Achieving stable, sustained grasping with soft robotic hands remains a fundamental challenge. Compliance enables safe and adaptive contact, yet the intr

researcharxiv-cs-ro

28 Aug 2026

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

arXiv:2608.24325v1 Announce Type: new Abstract: Reliable underwater perception requires complementary sensing under variable visibility. Optical cameras capture appearance and semantics but degrade ra

26 Aug 2026

LetzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents

model-releasesarxiv-cs-cl

arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-li

Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules

arXiv:2608.22642v1 Announce Type: cross Abstract: Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and i

Multimodal-Language-Model-Driven Interaction and Companionship for Service Robots in Elderly-Care Facilities

arXiv:2608.21387v1 Announce Type: new Abstract: Service robots are increasingly deployed in elderly-care facilities to alleviate caregiver workload and enhance the quality of daily care. However, most

Toward a Foundation Plug-and-Play Prior for Computed Tomography Reconstruction via a Multimodal Diffusion Model

arXiv:2608.23190v1 Announce Type: new Abstract: Computed tomography (CT) throughput is limited by scan time, which grows with both the number of projections acquired and the detector integration time

researcharxiv-cs-cv

Expressivity In Multimodal Contrastive Learning

arXiv:2608.17203v1 Announce Type: cross Abstract: Contrastive learning has become a cornerstone of modern representation learning, powering CLIP-style models that underpin text-to-image generation, vi

19 Aug 2026

Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal Policies

arXiv:2608.17484v1 Announce Type: new Abstract: Frozen vision-language-action (VLA) policies are increasingly improved at test time by sampling additional policy behaviors or introducing external demo

19 Aug 2026

Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance

arXiv:2608.14651v1 Announce Type: new Abstract: Effective disaster risk communication is a foundational humanitarian challenge, yet current emergency infrastructure fails to meet the needs of individu

MIRROR: Multimodal Intelligent Radiology Reasoning and Observation Reporter

arXiv:2608.16709v1 Announce Type: cross Abstract: A radiologist reading a model's output faces two problems. The model returns a number and no reason, and any system that turns that number into readab

Multi-Agent Closed-Loop Reasoning for Organic Structure Elucidation from Multimodal Spectra

arXiv:2608.14720v1 Announce Type: cross Abstract: Following the molecular discovery and synthesis revolutions, scalable automated structure elucidation from routine spectroscopic data remains an outst

agentsarxiv-cs-ai

PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Models

arXiv:2608.14741v1 Announce Type: cross Abstract: We introduce PolyComp, a procedurally generated and verified benchmark that stresses visual recognition and compositional spatial reasoning. In each p