Search: “multimodal”

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

arXiv:2606.10819v1 Announce Type: cross Abstract: RS-MLLMs enable natural-language understanding and spatial reasoning over earth observation imagery. However, existing models support only a narrow ra

10 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents

DGX agent

arXiv:2606.10742v1 Announce Type: cross Abstract: External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences. However,

model-releasesarxiv-cs-lg

10 Jun 2026

Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

arXiv:2606.09770v1 Announce Type: cross Abstract: Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topog

9 Jun 2026

MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework

arXiv:2606.07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language

9 Jun 2026

Local Ai

When Vision Misleads, Let Location Speak: A Worldwide Image Geo-Localization Method via Location Attention Mechanism and Large Multimodal Models

arXiv:2606.08918v1 Announce Type: new Abstract: Worldwide image geo-localization aims to determine the capture location of an image on a global scale. Existing methods often mislocalize images by matc

local-aiarxiv-cs-cv

9 Jun 2026

Tutorials

A robust PPG foundation model using multimodal physiological supervision

arXiv:2606.07365v1 Announce Type: cross Abstract: Photoplethysmography (PPG), a non-invasive measure of changes in blood volume, is widely used in both wearable devices and clinical settings. Recent P

tutorialsarxiv-cs-ai

8 Jun 2026

BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding

arXiv:2606.05515v1 Announce Type: new Abstract: Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the na

researcharxiv-cs-cv

5 Jun 2026

LMM-IR: Large-Scale Netlist-Aware Multimodal Framework for Static IR-Drop Prediction

arXiv:2511.12581v2 Announce Type: replace Abstract: Static IR drop analysis is a fundamental and critical task in the field of chip design. Nevertheless, this process can be quite time-consuming, pote

4 Jun 2026

Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM

arXiv:2606.04719v1 Announce Type: new Abstract: The Transformer's quadratic complexity with input length imposes an unsustainable computational load on large language models (LLMs). In contrast, the S

researcharxiv-cs-cl

4 Jun 2026

Auditing Engagement Incentives in the Kidfluencer Ecosystem: A Multimodal Weak Supervision Approach

model-releasesarxiv-cs-lg

arXiv:2606.03173v1 Announce Type: cross Abstract: The rise of `kidfluencers' on YouTube has raised ethical concerns about child digital labor and exploitation. While emerging legislation attempts to r

Benchmarking Visual State Tracking in Multimodal Video Understanding

arXiv:2606.03920v1 Announce Type: new Abstract: Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacit

Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding

arXiv:2606.03604v1 Announce Type: new Abstract: When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author i

researcharxiv-cs-cl

CP-Agent: Context-Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations

arXiv:2606.03435v1 Announce Type: new Abstract: Cell Painting combines multiplexed fluorescent staining, high-content imaging, and quantitative analysis to generate high-dimensional phenotypic readout

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

arXiv:2509.22468v2 Announce Type: replace-cross Abstract: High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce. W

Local Ai

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

arXiv:2606.02956v1 Announce Type: new Abstract: Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We prese

local-aiarxiv-cs-cv

Brain-Atlas-Guided Generative Counterfactual Attention for Explainable Cognitive Decline Diagnosis Using Multimodal Connectomes

arXiv:2606.01237v1 Announce Type: new Abstract: Mild cognitive impairment (MCI) and subjective cognitive decline (SCD) are closely associated with the early Alzheimer's disease continuum, where accura

Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark

arXiv:2606.02246v1 Announce Type: new Abstract: To operate in the physical world, embodied agents must perceive their environment in an 'always-on' fashion, selectively accessing the most informative

MMTalker: Multiresolution 3D Talking Head Synthesis with Multimodal Feature Fusion

arXiv:2604.02941v2 Announce Type: replace Abstract: Speech-driven three-dimensional (3D) facial animation synthesis aims to build a mapping from one-dimensional (1D) speech signals to time-varying 3D

researcharxiv-cs-cv

ObjEmbed: Towards Universal Multimodal Object Embeddings

arXiv:2602.01753v3 Announce Type: replace Abstract: Aligning objects with corresponding textual descriptions is a fundamental challenge and a realistic requirement in vision-language understanding. Wh

safetyarxiv-cs-cv

Applications

Property Prediction of Stacked Bilayer Materials: A Multimodal Learning Approach

arXiv:2606.01012v1 Announce Type: new Abstract: AI for materials science is a critical topic within AI for science, aiming to accelerate materials discovery and produce accurate property predictions.

applicationsarxiv-cs-ai

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

arXiv:2601.04946v3 Announce Type: replace-cross Abstract: Automatic metrics are widely used to evaluate text-to-image models, often replacing human judgment in benchmarking, model selection, and large

Tutorials

SCL: Towards Domain Generalization via Single-Temporal Multimodal Contrastive Learning for Remote Sensing Change Detection

arXiv:2404.11326v5 Announce Type: replace Abstract: In recent years, change detection and anomaly detection models based on CNN and transformer have achieved remarkable success across various datasets

tutorialsarxiv-cs-cv

Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval

arXiv:2605.31229v1 Announce Type: cross Abstract: While retrieval is a core function of vision-language models, continually updating these models for retrieval tasks remains critically underexplored.

ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment

arXiv:2605.30965v1 Announce Type: cross Abstract: Recent advancements in text-guided audio generation have yielded promising results in diverse domains, including sound effects, speech, and music. How

Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search

arXiv:2605.30917v1 Announce Type: cross Abstract: As large-scale visual-document corpora such as arXiv papers and enterprise PDFs continue to grow, visual-document retrieval has gained increasing atte

TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation

model-releasesarxiv-cs-cl

arXiv:2605.30673v1 Announce Type: new Abstract: Classroom videos contain observable teaching practices, but their pedagogical and visual signals are rarely organized in forms suitable for model evalua

COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings

arXiv:2605.29628v1 Announce Type: cross Abstract: Contrastive Language-Audio Pretraining (CLAP) models are widely used for audio understanding and support modality-agnostic condition swapping in many

Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment

arXiv:2605.28822v1 Announce Type: new Abstract: Defect grading of power transmission equipment (DGPTE) is crucial to the stability of electric energy transmission. Although existing machine learning m

researcharxiv-cs-cl

Multimodal LLMs See Sentiment

arXiv:2508.16873v3 Announce Type: replace Abstract: Understanding how visual content conveys sentiment is increasingly important in a digital landscape dominated by imagery. However, sentiment percept

Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models

arXiv:2605.29299v1 Announce Type: cross Abstract: Evaluations of dental vision-language models remain fragmented across datasets, task definitions and metrics, and often ignore their computational cos

ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control

arXiv:2605.29425v1 Announce Type: new Abstract: Reinforcement learning (RL) has shown promise in traffic signal control (TSC). However, its reliance on predefined states limits responsiveness to obser

Agents

Adaptive Multimodal Agents-Based Framework for Automatic Workflow Execution

arXiv:2605.28607v1 Announce Type: new Abstract: Modern information systems require autonomous agents capable of navigating complex workflows, yet current methodologies often struggle with the transiti

agentsarxiv-cs-ai

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

arXiv:2605.28774v1 Announce Type: new Abstract: Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning

safetyarxiv-cs-cl

Local Ai

Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance

arXiv:2602.03491v2 Announce Type: replace-cross Abstract: Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-

local-aiarxiv-cs-cl

FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales

arXiv:2605.28174v1 Announce Type: cross Abstract: Foundation models offer a promising route to transferable remote sensing representations, but many current approaches depend on very large pretraining

Applications

FLUID: From Ephemeral IDs to Multimodal Semantic Codes for Industrial-Scale Livestreaming Recommendation

arXiv:2605.21832v2 Announce Type: replace Abstract: Modern recommender systems rely heavily on ID-based collaborative filtering: each item is represented by a unique ID embedding that accumulates coll

applicationsarxiv-cs-ai

Semantic-Aware Interpretable Multimodal Music Auto-Tagging

arXiv:2505.17233v3 Announce Type: replace Abstract: Music auto-tagging is essential for organizing and discovering music in extensive digital libraries. While foundation models achieve exceptional per

When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?

arXiv:2605.27932v1 Announce Type: cross Abstract: Think-with-image reasoning is emerging as a new inference paradigm for large vision-language models, but its safety implications remain poorly underst

Tutorials

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

arXiv:2605.27310v1 Announce Type: new Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry nee

tutorialsarxiv-cs-cv

27 May 2026

Particle-Lund Multimodality in Jet Taggers

arXiv:2605.26821v1 Announce Type: cross Abstract: The Lund plane offers a physics-motivated, hierarchical representation of QCD radiation within jets, while transformer-based taggers have reached stat

27 May 2026

Agents

QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents

arXiv:2605.27068v1 Announce Type: cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM)

agentsarxiv-cs-ai

27 May 2026

FairJudge: Abstention-Aware Multimodal Judges for Fairness and Alignment Evaluation in Text-to-Image Models

arXiv:2510.22827v3 Announce Type: replace-cross Abstract: Evaluating text-to-image (T2I) systems requires judging not only whether an image matches a prompt, but also whether socially salient attribut

safetyarxiv-cs-lg

26 May 2026

ORACAL: A Robust and Explainable Multimodal Framework for Smart Contract Vulnerability Detection with Causal Graph Enrichment

model-releasesarxiv-cs-lg

arXiv:2603.28128v2 Announce Type: replace Abstract: Although Graph Neural Networks (GNNs) have shown promise for smart contract vulnerability detection, they still face significant limitations. Homoge

26 May 2026

ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models

model-releasesarxiv-cs-cl

arXiv:2605.23694v1 Announce Type: new Abstract: Chart descriptions are essential for accessibility, cross-modal retrieval, and assisting readers in extracting insights from complex visualizations. As

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

arXiv:2510.08945v3 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduc

Agents

FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence

arXiv:2605.22868v1 Announce Type: new Abstract: Autonomous systems and smart-industry deployments increasingly split computation across near-sensor, edge, and cloud resources, where tight energy, late

agentsarxiv-cs-lg

Inconsistency-aware Multimodal Schrodinger Bridge for Deepfake Localization

arXiv:2605.23113v1 Announce Type: new Abstract: Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under singl

researcharxiv-cs-cv

Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems

model-releasesarxiv-cs-lg

arXiv:2605.22891v1 Announce Type: new Abstract: Evaluation in scientific reconstruction is dominated by pointwise metrics - RMSE, MAE, per-event resolution - under the implicit assumption that lower e