Search: “multimodal”

arXiv:2501.19274v2 Announce Type: replace Abstract: The Great Outdoors (GO) dataset is a multi-modal annotated data resource aimed at advancing ground robotics research in unstructured environments. E

agentsarxiv-cs-ro

MMGist: A Comprehensive Multimodal Benchmark for 2027

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

arXiv:2606.22437v1 Announce Type: new Abstract: We conduct a systematic study of 18 widely used vision-language benchmarks and identify three major issues: 1) many items do not rely on visual cues and

model-releasesarxiv-cs-cv

23 Jun 2026

Retrieval-Augmented Multimodal Learning for Enzyme-Substrate Interaction Prediction Under Low-Homology Shift

arXiv:2606.22823v1 Announce Type: new Abstract: Enzyme substrate interaction (ESI) prediction is a fundamental computational task for biocatalyst discovery and reaction screening in large biochemical

researcharxiv-cs-lg

SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

arXiv:2606.22144v1 Announce Type: new Abstract: Gastrointestinal cancers represent a growing health burden in the South Asian region, driven largely by rapid changes in socio-economic conditions & lif

Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation

arXiv:2606.19120v2 Announce Type: replace-cross Abstract: On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned

When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents

model-releasesarxiv-cs-lg

arXiv:2606.22864v1 Announce Type: new Abstract: Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for

When Does a Video-Language Model Stop Watching? Reward Strength Controls the Formation and Reversal of Visual Shortcuts in Multimodal RLVR

arXiv:2606.22043v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is increasingly applied to large vision-language models (LVLMs), yet outcome-only optimization c

researcharxiv-cs-cv

Embed the world: Multimodal AI for searchable aerial imagery at scale

model-releasesaws-ml-blog

In this post, we walk through the problem space, our architecture on Amazon Bedrock and Amazon OpenSearch Serverless, the evaluation methodology we built on OpenStreetMap ground truth, four experiment

22 Jun 2026

Context-Aware Multimodal Claim Verification in Spoken Dialogues

model-releasesarxiv-cs-cl

arXiv:2606.11420v1 Announce Type: new Abstract: Every day, millions absorb claims from podcasts and streams that no fact-checker ever sees. Spoken misinformation is built through conversation, where c

11 Jun 2026

ParseFixer: An Agentic Framework for Document Parsing via Selective Multimodal Correction

arXiv:2606.11977v1 Announce Type: new Abstract: In this report, we present our third-place solution for the DataMFM Challenge Track 1: Document Parsing. This track requires models to recover structure

11 Jun 2026

Agents

Sustainability assessment using multimodal AI agents

arXiv:2507.17012v2 Announce Type: replace Abstract: Reducing the rapidly growing environmental impact of the computing industry requires assessing the emissions of electronics at scale. However, a tra

agentsarxiv-cs-ai

11 Jun 2026

ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations

arXiv:2606.11188v1 Announce Type: new Abstract: This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-

10 Jun 2026

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

arXiv:2606.10819v1 Announce Type: cross Abstract: RS-MLLMs enable natural-language understanding and spatial reasoning over earth observation imagery. However, existing models support only a narrow ra

10 Jun 2026

MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents

model-releasesarxiv-cs-lg

arXiv:2606.10742v1 Announce Type: cross Abstract: External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences. However,

10 Jun 2026

Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

arXiv:2606.09770v1 Announce Type: cross Abstract: Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topog

researcharxiv-cs-lg

9 Jun 2026

MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework

arXiv:2606.07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language

9 Jun 2026

Local Ai

When Vision Misleads, Let Location Speak: A Worldwide Image Geo-Localization Method via Location Attention Mechanism and Large Multimodal Models

arXiv:2606.08918v1 Announce Type: new Abstract: Worldwide image geo-localization aims to determine the capture location of an image on a global scale. Existing methods often mislocalize images by matc

local-aiarxiv-cs-cv

9 Jun 2026

Tutorials

A robust PPG foundation model using multimodal physiological supervision

arXiv:2606.07365v1 Announce Type: cross Abstract: Photoplethysmography (PPG), a non-invasive measure of changes in blood volume, is widely used in both wearable devices and clinical settings. Recent P

tutorialsarxiv-cs-ai

8 Jun 2026

BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding

arXiv:2606.05515v1 Announce Type: new Abstract: Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the na

researcharxiv-cs-cv

5 Jun 2026

LMM-IR: Large-Scale Netlist-Aware Multimodal Framework for Static IR-Drop Prediction

arXiv:2511.12581v2 Announce Type: replace Abstract: Static IR drop analysis is a fundamental and critical task in the field of chip design. Nevertheless, this process can be quite time-consuming, pote

researcharxiv-cs-lg

4 Jun 2026

Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM

arXiv:2606.04719v1 Announce Type: new Abstract: The Transformer's quadratic complexity with input length imposes an unsustainable computational load on large language models (LLMs). In contrast, the S

researcharxiv-cs-cl

4 Jun 2026

Auditing Engagement Incentives in the Kidfluencer Ecosystem: A Multimodal Weak Supervision Approach

model-releasesarxiv-cs-lg

arXiv:2606.03173v1 Announce Type: cross Abstract: The rise of `kidfluencers' on YouTube has raised ethical concerns about child digital labor and exploitation. While emerging legislation attempts to r

Benchmarking Visual State Tracking in Multimodal Video Understanding

arXiv:2606.03920v1 Announce Type: new Abstract: Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacit

Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding

arXiv:2606.03604v1 Announce Type: new Abstract: When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author i

researcharxiv-cs-cl

CP-Agent: Context-Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations

arXiv:2606.03435v1 Announce Type: new Abstract: Cell Painting combines multiplexed fluorescent staining, high-content imaging, and quantitative analysis to generate high-dimensional phenotypic readout

safetyarxiv-cs-ai

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

arXiv:2509.22468v2 Announce Type: replace-cross Abstract: High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce. W

Local Ai

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

arXiv:2606.02956v1 Announce Type: new Abstract: Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We prese

local-aiarxiv-cs-cv

Brain-Atlas-Guided Generative Counterfactual Attention for Explainable Cognitive Decline Diagnosis Using Multimodal Connectomes

arXiv:2606.01237v1 Announce Type: new Abstract: Mild cognitive impairment (MCI) and subjective cognitive decline (SCD) are closely associated with the early Alzheimer's disease continuum, where accura

Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark

arXiv:2606.02246v1 Announce Type: new Abstract: To operate in the physical world, embodied agents must perceive their environment in an 'always-on' fashion, selectively accessing the most informative

MMTalker: Multiresolution 3D Talking Head Synthesis with Multimodal Feature Fusion

arXiv:2604.02941v2 Announce Type: replace Abstract: Speech-driven three-dimensional (3D) facial animation synthesis aims to build a mapping from one-dimensional (1D) speech signals to time-varying 3D

researcharxiv-cs-cv

ObjEmbed: Towards Universal Multimodal Object Embeddings

arXiv:2602.01753v3 Announce Type: replace Abstract: Aligning objects with corresponding textual descriptions is a fundamental challenge and a realistic requirement in vision-language understanding. Wh

Applications

Property Prediction of Stacked Bilayer Materials: A Multimodal Learning Approach

arXiv:2606.01012v1 Announce Type: new Abstract: AI for materials science is a critical topic within AI for science, aiming to accelerate materials discovery and produce accurate property predictions.

applicationsarxiv-cs-ai

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

arXiv:2601.04946v3 Announce Type: replace-cross Abstract: Automatic metrics are widely used to evaluate text-to-image models, often replacing human judgment in benchmarking, model selection, and large

Tutorials

SCL: Towards Domain Generalization via Single-Temporal Multimodal Contrastive Learning for Remote Sensing Change Detection

arXiv:2404.11326v5 Announce Type: replace Abstract: In recent years, change detection and anomaly detection models based on CNN and transformer have achieved remarkable success across various datasets

tutorialsarxiv-cs-cv

Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval

arXiv:2605.31229v1 Announce Type: cross Abstract: While retrieval is a core function of vision-language models, continually updating these models for retrieval tasks remains critically underexplored.

ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment

arXiv:2605.30965v1 Announce Type: cross Abstract: Recent advancements in text-guided audio generation have yielded promising results in diverse domains, including sound effects, speech, and music. How

safetyarxiv-cs-ai

Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search

arXiv:2605.30917v1 Announce Type: cross Abstract: As large-scale visual-document corpora such as arXiv papers and enterprise PDFs continue to grow, visual-document retrieval has gained increasing atte

TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation

model-releasesarxiv-cs-cl

arXiv:2605.30673v1 Announce Type: new Abstract: Classroom videos contain observable teaching practices, but their pedagogical and visual signals are rarely organized in forms suitable for model evalua

COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings

arXiv:2605.29628v1 Announce Type: cross Abstract: Contrastive Language-Audio Pretraining (CLAP) models are widely used for audio understanding and support modality-agnostic condition swapping in many

Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment

arXiv:2605.28822v1 Announce Type: new Abstract: Defect grading of power transmission equipment (DGPTE) is crucial to the stability of electric energy transmission. Although existing machine learning m

researcharxiv-cs-cl

Multimodal LLMs See Sentiment

arXiv:2508.16873v3 Announce Type: replace Abstract: Understanding how visual content conveys sentiment is increasingly important in a digital landscape dominated by imagery. However, sentiment percept

Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models

arXiv:2605.29299v1 Announce Type: cross Abstract: Evaluations of dental vision-language models remain fragmented across datasets, task definitions and metrics, and often ignore their computational cos

ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control

arXiv:2605.29425v1 Announce Type: new Abstract: Reinforcement learning (RL) has shown promise in traffic signal control (TSC). However, its reliance on predefined states limits responsiveness to obser

safetyarxiv-cs-ai

Hardware

Run Step 3.7 Flash on NVIDIA GPUs with Enterprise-Ready Multimodal AI

Step 3.7 Flash is a 198B-parameter Mixture-of-Experts vision-language model designed for enterprise-scale production workloads, featuring native image and video input, a 256k context window, and confi

hardwarenvidia-developer

Agents

Adaptive Multimodal Agents-Based Framework for Automatic Workflow Execution

arXiv:2605.28607v1 Announce Type: new Abstract: Modern information systems require autonomous agents capable of navigating complex workflows, yet current methodologies often struggle with the transiti

agentsarxiv-cs-ai

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

arXiv:2605.28774v1 Announce Type: new Abstract: Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning

safetyarxiv-cs-cl

Local Ai

Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance

arXiv:2602.03491v2 Announce Type: replace-cross Abstract: Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-

local-aiarxiv-cs-cl

FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales

arXiv:2605.28174v1 Announce Type: cross Abstract: Foundation models offer a promising route to transferable remote sensing representations, but many current approaches depend on very large pretraining