Search: “multimodal”

Label-Free Deep-Tissue Peripheral Nerve Detection with a Handheld Multimodal OCT Probe and NerveDetNet

arXiv:2608.13807v1 Announce Type: cross Abstract: Peripheral nerves buried beneath intact tissue are difficult to visualize during surgery and remain inaccessible to white light wide-field imaging and

17 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval

DGX agent

arXiv:2608.12570v1 Announce Type: new Abstract: Fashion retrieval often requires satisfying multiple attributes at once, such as category, color, pattern, and demographic. Monolithic embeddings mix th

researcharxiv-cs-cv

14 Aug 2026

Hardware

OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research

arXiv:2604.03781v2 Announce Type: replace Abstract: Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled

hardwarearxiv-cs-ro

Semantic Steering for Controllable Generation: Tuning-Free Concept Erasure in Multimodal Diffusion Transformers

arXiv:2608.12829v1 Announce Type: new Abstract: Multimodal Diffusion Transformers (MM-DiTs) have demonstrated remarkable text-to-image generation performance, surpassing traditional U-Net-based diffus

safetyarxiv-cs-cv

The Role of Natural Language Understanding in Multimodal Video-Based Dengue Diagnosis

arXiv:2608.12677v1 Announce Type: new Abstract: Detecting infection-related behavioral changes in mosquitoes from video data is challenging because mosquitoes are small, move rapidly and irregularly,

We promised open weights for Qwen3.8. Now, time to meet them! 🎉 ⚡ Qwen3.8-27B: - A native multimodal dense model. With just 27B parameters,…

We promised open weights for Qwen3.8. Now, time to meet them! 🎉 ⚡ Qwen3.8-27B: - A native multimodal dense model. With just 27B parameters, it outperforms Qwen3.7-Plus overall and shines in real-world

model-releasesqwen--x

Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting

arXiv:2608.11655v1 Announce Type: cross Abstract: Motion-centric video reasoning is fundamental to interactive applications such as robotic manipulation and autonomous navigation. However, multimodal

13 Aug 2026

Applications

MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation

arXiv:2606.17598v2 Announce Type: replace-cross Abstract: Humans naturally leverage diverse sensing modalities to interact with the physical world, while most Vision-Language-Action (VLA) models for r

applicationsarxiv-cs-cv

13 Aug 2026

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

arXiv:2608.10154v1 Announce Type: cross Abstract: We present results from reconstructing multiple-choice model (MCM) and three-parameter logistic (3PL) model curves using a fine-tuned multimodal large

12 Aug 2026

EMMR: Emotion-Mediated Multimodal Reasoning for Personality Assessment in Asynchronous Video Interviews

arXiv:2608.07512v1 Announce Type: cross Abstract: Asynchronous Video Interviews (AVIs) have become increasingly popular for personality assessment. Recent large language models (LLMs) have shown poten

researcharxiv-cs-ai

Hardware

EsaacSim: A Multimodal Event Camera Add-on for NVIDIA Isaac Sim

arXiv:2608.08522v1 Announce Type: new Abstract: Event-based vision is becoming an increasingly important sensing paradigm for robotics, yet its adoption remains limited by sensor availability and the

hardwarearxiv-cs-ro

FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

arXiv:2608.08736v1 Announce Type: new Abstract: Fitness Action Quality Assessment (AQA) is important for intelligent sports training, yet the capabilities of Multimodal Large Language Models (MLLMs) i

Improving Generalization Robustness of Multimodal RLVR

arXiv:2608.08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasi

Tutorials

Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs

arXiv:2604.05695v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in 2D visual tasks but still struggle to understand physical space in rea

tutorialsarxiv-cs-cv

MMArch: Benchmarking Multimodal Reasoning Grounded in Architectural Evidence

arXiv:2608.09281v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) perform strongly on engineering imagery, yet existing benchmarks mostly test drawing recognition, information e

TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs

arXiv:2601.03100v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) typically rely on a single late-layer feature from a frozen vision encoder, leaving the encoder's ric

researcharxiv-cs-ai

TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases

arXiv:2608.08727v1 Announce Type: cross Abstract: To address this gap, we introduce TomaMMU, a large-scale Tomato leaf disease MultiModal Understanding dataset, alongside TomaBench, a benchmark for ev

VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs

arXiv:2510.16598v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks from the massive number of visual tokens generat

Deep Evidential Regression for Sparse Forest Height Estimation from Multimodal Satellite Imagery

arXiv:2608.06406v1 Announce Type: new Abstract: Accurate estimation of forest height from satellite imagery is essential for applications such as carbon accounting, biodiversity monitoring, and ecosys

10 Aug 2026

Meta released Muse Glimmer 30B: multimodal model for your Claw/Pi setups 🔥 we tested and fine-tuned the model for you, and shipped day-0 su…

model-releasesgeorgi-gerganov--x

Meta released Muse Glimmer 30B: multimodal model for your Claw/Pi setups 🔥 we tested and fine-tuned the model for you, and shipped day-0 support in transformers and llama.cpp, including DFlash for 2-4

10 Aug 2026

M^3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

arXiv:2608.05817v1 Announce Type: new Abstract: Metaphor enables the understanding of abstract concepts through cross-domain mappings while conveying affective attitudes. In multimodal scenarios, visu

7 Aug 2026

MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMs

arXiv:2603.05997v2 Announce Type: replace-cross Abstract: Irregularly sampled time series (ISTS) are widespread in real-world scenarios, exhibiting asynchronous observations on uneven time intervals a

7 Aug 2026

PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs

arXiv:2601.21124v2 Announce Type: replace-cross Abstract: Current multimodal LLMs process audio as a mono stream, ignoring the rich spatial information essential for embodied AI. Existing spatial audi

7 Aug 2026

AI-driven Multimodal Representation Learning for Latent Mediation Structure Discovery of Socioeconomic Disadvantage, Psychosocial Factors, and Cardiometabolic Multimorbidity: Insights from the All of Us Research Program

arXiv:2608.04016v1 Announce Type: cross Abstract: Social disadvantage is associated with multimorbidity, but the pathways linking social conditions to disease burden remain poorly understood. We devel

researcharxiv-cs-ai

Q-CueGraph: Query-Conditioned Visual Evidence Graphs for Multimodal Reasoning

arXiv:2608.04452v1 Announce Type: cross Abstract: High-resolution pixels and crop or zoom tools give multimodal large language models the ability to inspect an image, but they do not provide a reliabl

Local Ai

Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs

arXiv:2608.04759v1 Announce Type: cross Abstract: Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inc

local-aiarxiv-cs-cl

UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction

arXiv:2608.04949v1 Announce Type: cross Abstract: Unified Multimodal Relation Extraction (UMRE) aims to identify intra-modal and cross-modal relations between textual entities and visual objects. Howe

When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning

arXiv:2608.04052v1 Announce Type: cross Abstract: Backdoor attacks in multimodal contrastive learning (MCL) have garnered growing attention in recent years, as many downstream tasks critically depend

When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning

arXiv:2608.04726v1 Announce Type: new Abstract: Multimodal large language models increasingly reason over screenshots and documents where the task itself may be written in pixels. Yet benchmarks usual

Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering

arXiv:2608.02907v1 Announce Type: new Abstract: Multimodal retrievers are essential for knowledge-based visual question answering, where they retrieve external evidence for image-question pairs. Howev

researcharxiv-cs-lg

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding

arXiv:2608.03826v1 Announce Type: new Abstract: Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations,

Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training

arXiv:2608.03660v1 Announce Type: new Abstract: Reinforcement fine-tuning (RFT) is widely believed to inherently resist catastrophic forgetting in continual post-training of multimodal large language

Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent

arXiv:2608.03979v1 Announce Type: cross Abstract: We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense s

Tutorials

Deep Multimodal Fusion Detection through Spatial Mask and Channel Fusion

arXiv:2608.02092v1 Announce Type: new Abstract: Deep multimodal fusion for object detection has demonstrated good performance through mining modal characteristics. However, existing feature-level fusi

tutorialsarxiv-cs-cv

EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records

arXiv:2506.04831v3 Announce Type: replace-cross Abstract: Forecasting how a patient's condition is likely to evolve, including possible deterioration, recovery, treatment needs, and care transitions,

researcharxiv-cs-cl

FAU at ImageCLEF 2026 Task on Multimodal Reasoning Robust Candidate Scoring and Concise Multilingual Visual Answering

arXiv:2608.01664v1 Announce Type: new Abstract: We present our ImageCLEF 2026 Multimodal Reasoning system for the Visual Multiple Choice Question Answering (Visual MCQ) and Visual Open Question Answer

GeoArbiter: Verifiability-Guided Grounding for Remote-Sensing Multimodal LLMs

arXiv:2608.00877v1 Announce Type: new Abstract: Remote-sensing multimodal large language models (MLLMs) often assert facts that imagery cannot establish, such as a facility's identity or function. Coo

safetyarxiv-cs-lg

Mistral releases Shieldstral, a 3B multimodal safety classifier that it says matches models up to 7x its size on text safety, available under Apache 2.0 (Mistral AI Blog)

Mistral AI Blog: Mistral releases Shieldstral, a 3B multimodal safety classifier that it says matches models up to 7x its size on text safety, available under Apache 2.0 — Every product that ships a m

model-releasestechmeme

Models as Tools: An Agentic Coordination Framework for Unified Multimodal Visual Tracking

arXiv:2608.00847v1 Announce Type: new Abstract: Most current visual trackers adopt a matching-based architecture trained exclusively on tracking datasets, whose performance gains depend heavily on the

Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams

arXiv:2608.00012v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster

RHEA: Reliability-Harmonized Reconstruction and Assignment for Robust Multimodal-Attributed Graph Clustering

arXiv:2608.00621v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry heterogeneous attributes such as text and images over a relational structure, have become a funda

researcharxiv-cs-lg

Structured Proxy Features for Multimodal NSCLC Survival Prediction from Pretreatment CT

arXiv:2608.00446v1 Announce Type: new Abstract: Lung cancer results in roughly 1.8 million fatalities annually worldwide, with non-small cell lung cancer (NSCLC) comprising the majority of cases. Desp

Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning

arXiv:2608.01488v1 Announce Type: new Abstract: Unified multimodal object tracking has achieved remarkable robustness by leveraging complementary sensor data (e.g., RGB, Thermal, Depth), yet the heavy

safetyarxiv-cs-cv

Where did the ambiguity go? Examining how multimodal models interpret polysemous words

arXiv:2608.00410v1 Announce Type: cross Abstract: Human language is highly polysemous. Many common words (e.g., 'bank' or 'palm') carry several distinct meanings that shape what humans communicate and

researcharxiv-cs-cl

Faster but Different: Diagnosing and Controlling Content Drift in Accelerated Multimodal Diffusion Language Models

arXiv:2607.29079v1 Announce Type: new Abstract: Training-free acceleration makes diffusion-based multimodal large language models (dMLLMs) more deployable, but it may silently change generated content

safetyarxiv-cs-cl

3 Aug 2026

Agents

CaM-Wolf: Causal-Aware Multimodal Agents for Social Deduction Games

arXiv:2607.26393v1 Announce Type: new Abstract: Social deduction games (SDGs) such as Werewolf have become challenging testbeds for AI agents. These games require complex social skills such as reasoni

agentsarxiv-cs-ai

31 Jul 2026

Scaling medical imaging report generation with multimodal reinforcement learning

arXiv:2601.17151v2 Announce Type: replace-cross Abstract: Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit ma

31 Jul 2026

VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation