Search: “multimodal”

FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales

arXiv:2605.28174v1 Announce Type: cross Abstract: Foundation models offer a promising route to transferable remote sensing representations, but many current approaches depend on very large pretraining

28 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

FLUID: From Ephemeral IDs to Multimodal Semantic Codes for Industrial-Scale Livestreaming Recommendation

DGX agent

arXiv:2605.21832v2 Announce Type: replace Abstract: Modern recommender systems rely heavily on ID-based collaborative filtering: each item is represented by a unique ID embedding that accumulates coll

applicationsarxiv-cs-ai

28 May 2026

Semantic-Aware Interpretable Multimodal Music Auto-Tagging

arXiv:2505.17233v3 Announce Type: replace Abstract: Music auto-tagging is essential for organizing and discovering music in extensive digital libraries. While foundation models achieve exceptional per

28 May 2026

When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?

arXiv:2605.27932v1 Announce Type: cross Abstract: Think-with-image reasoning is emerging as a new inference paradigm for large vision-language models, but its safety implications remain poorly underst

28 May 2026

Tutorials

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

arXiv:2605.27310v1 Announce Type: new Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry nee

tutorialsarxiv-cs-cv

27 May 2026

Particle-Lund Multimodality in Jet Taggers

arXiv:2605.26821v1 Announce Type: cross Abstract: The Lund plane offers a physics-motivated, hierarchical representation of QCD radiation within jets, while transformer-based taggers have reached stat

27 May 2026

QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents

arXiv:2605.27068v1 Announce Type: cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM)

agentsarxiv-cs-ai

27 May 2026

FairJudge: Abstention-Aware Multimodal Judges for Fairness and Alignment Evaluation in Text-to-Image Models

arXiv:2510.22827v3 Announce Type: replace-cross Abstract: Evaluating text-to-image (T2I) systems requires judging not only whether an image matches a prompt, but also whether socially salient attribut

safetyarxiv-cs-lg

26 May 2026

ORACAL: A Robust and Explainable Multimodal Framework for Smart Contract Vulnerability Detection with Causal Graph Enrichment

model-releasesarxiv-cs-lg

arXiv:2603.28128v2 Announce Type: replace Abstract: Although Graph Neural Networks (GNNs) have shown promise for smart contract vulnerability detection, they still face significant limitations. Homoge

26 May 2026

ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models

arXiv:2605.23694v1 Announce Type: new Abstract: Chart descriptions are essential for accessibility, cross-modal retrieval, and assisting readers in extracting insights from complex visualizations. As

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

arXiv:2510.08945v3 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduc

FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence

arXiv:2605.22868v1 Announce Type: new Abstract: Autonomous systems and smart-industry deployments increasingly split computation across near-sensor, edge, and cloud resources, where tight energy, late

agentsarxiv-cs-lg

Inconsistency-aware Multimodal Schrodinger Bridge for Deepfake Localization

arXiv:2605.23113v1 Announce Type: new Abstract: Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under singl

Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems

model-releasesarxiv-cs-lg

arXiv:2605.22891v1 Announce Type: new Abstract: Evaluation in scientific reconstruction is dominated by pointwise metrics - RMSE, MAE, per-event resolution - under the implicit assumption that lower e

Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models

arXiv:2605.22732v1 Announce Type: cross Abstract: We investigate whether acoustic emotion recognition models can serve as proxies for the Pathos dimension in political speech analysis, as operationali

22 May 2026

Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?

arXiv:2605.22170v1 Announce Type: new Abstract: In recent years, several Speech Language Models (SLMs) that represent speech and written text jointly have been presented. The question then emerges abo

researcharxiv-cs-cl

22 May 2026

Towards Selection of Large Multimodal Models as Engines for Burned-in Protected Health Information Detection in Medical Images

model-releasesarxiv-cs-cv

arXiv:2511.02014v2 Announce Type: replace Abstract: The detection of Protected Health Information (PHI) in medical imaging is critical for safeguarding patient privacy and ensuring compliance with reg

22 May 2026

EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

model-releasesarxiv-cs-cv

arXiv:2605.19004v1 Announce Type: new Abstract: Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensin

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction

arXiv:2605.19765v1 Announce Type: new Abstract: Existing affective-computing, social-signal-processing, and meeting corpora capture important parts of human interaction, but they rarely support analys

How does longer temporal context enhance multimodal narrative video processing in the brain?

arXiv:2602.07570v2 Announce Type: replace-cross Abstract: Understanding how humans and artificial intelligence systems process complex narrative videos is a fundamental challenge at the intersection o

StampFormer: A Physics-Guided Material-Geometry-Coupled Multimodal Model for Rapid Prediction of Physical Fields in Sheet Metal Stamping

arXiv:2605.18835v1 Announce Type: new Abstract: Traditional sheet metal forming relies on time-consuming and expensive Finite Element Analysis (FEA) for design validation, a process that significantly

CAM-VFD: Cross-Attention Multimodal Video Forgery Detection

arXiv:2605.17133v1 Announce Type: cross Abstract: The rapid advancement of Deepfake technologies and video manipulation tools poses a critical challenge to multimedia forensics, judicial evidence inte

researcharxiv-cs-ai

F2IND-IT! -- Multimodal Fuzzy Fake Indian News Detection using Images and Text

arXiv:2605.17115v1 Announce Type: new Abstract: Biased manipulation of facts across regional and national media outlets complicates misinformation detection in diverse landscapes like India. This pape

researcharxiv-cs-ai

Gesture First, LLM-Assisted Voice Complement: Exploring Multimodal Robot 'Puppeteer' Teleoperation Via Virtual Counterpart in Augmented Reality

arXiv:2506.13189v3 Announce Type: replace-cross Abstract: Robot teleoperation via augmented reality (AR) offers a promising path toward more intuitive human-robot interaction (HRI). We present a head-

researcharxiv-cs-ro

M^2FedAQI: Multimodal Federated Learning for Air Quality Prediction on Heterogeneous Edge Devices

model-releasesarxiv-cs-lg

arXiv:2605.16375v1 Announce Type: new Abstract: Accurate air quality prediction is essential for public health, environmental monitoring, and industrial safety. However, most existing approaches rely

Multimodal Cultural Heritage Knowledge Graph Extension with Language and Vision Models

arXiv:2605.17669v1 Announce Type: new Abstract: The preservation and interpretation of cultural heritage increasingly rely on digital technologies, among which Knowledge Graphs (KGs) stand out for the

On Improving Multimodal Pedestrian Trajectory Prediction with CVAE: A Study on Benchmark and Robot Data

model-releasesarxiv-cs-ro

arXiv:2605.18262v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction is crucial for autonomous systems operating in complex environments, such as modular buses and delivery robots

Ranking-Aware Calibration for Reliable Multimodal Reinforcement Learning

arXiv:2605.16999v1 Announce Type: new Abstract: Reinforcement learning post-training has substantially improved the reasoning accuracy of vision-language models, yet the resulting policies remain poor

safetyarxiv-cs-lg

MAgSeg: Segmentation of Agricultural Landscapes in High-Resolution Satellite Imagery using Multimodal Large Language Models

arXiv:2605.16179v1 Announce Type: new Abstract: Agricultural landscape segmentation in the Global South is challenging as it is characterized by fragmented plots, high intra-class variance, and a scar

safetyarxiv-cs-cv

18 May 2026

MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

arXiv:2510.23477v2 Announce Type: replace Abstract: Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While multimoda

Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation

arXiv:2508.05008v2 Announce Type: replace Abstract: Vision-Language Models (VLMs), such as CLIP, have demonstrated remarkable zero-shot capabilities in various computer vision tasks. However, their ap

ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

arXiv:2605.14113v1 Announce Type: cross Abstract: While interpretable prototype networks offer compelling case-based reasoning for clinical diagnostics, their raw continuous outputs lack the semantic

agentsarxiv-cs-ai

TeachAnything: A Multimodal Crowdsourcing Platform for Training Embodied AI Agents in Symmetrical Reality

arXiv:2605.14556v1 Announce Type: new Abstract: Symmetrical Reality (SR) is emerging as a future trend for human-agent coexistence, placing higher demands on agents to acquire human-like intelligence.

agentsarxiv-cs-ai

Local Ai

Learning to See What You Need: Gaze Attention for Multimodal Large Language Models

arXiv:2605.13080v1 Announce Type: new Abstract: When humans describe a visual scene, they do not process the entire image uniformly; instead, they selectively fixate on regions relevant to their inten

local-aiarxiv-cs-cv

14 May 2026

Multitask Multimodal Fusion with Tabular Foundation Models for Peak and Durability Prediction of Pertussis Booster Response

arXiv:2605.12852v1 Announce Type: new Abstract: Pertussis booster vaccination produces immune responses that vary widely across individuals in both peak magnitude and long-term durability. These two p

14 May 2026

VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence

arXiv:2605.13798v1 Announce Type: new Abstract: Cross-modal 3D medical image analysis requires voxelwise representations that remain anatomically consistent across imaging contrasts, scanners, and acq

14 May 2026

Applications

DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis

arXiv:2605.08138v1 Announce Type: new Abstract: Synthetic data has emerged as a crucial solution to the data scarcity bottleneck in large language models (LLMs), particularly for specialized domains a

applicationsarxiv-cs-lg

EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding

arXiv:2605.08847v1 Announce Type: new Abstract: In the context of today's high-pressure, aging society, the demand for large-scale emotional models capable of providing empathetic support is more crit

Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning

arXiv:2605.09395v1 Announce Type: new Abstract: In this paper, we propose the first VLnderline{extbf{M}} nderline{extbf{a}}gentic nderline{extbf{r}}easoning framework for few-nderline{extbf{s}}hot mul

EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving

model-releasesarxiv-cs-cv

arXiv:2605.10556v1 Announce Type: new Abstract: As large language models span dense, mixture-of-experts, and state-space architectures and are deployed on heterogeneous accelerators under increasingly

FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

arXiv:2605.08820v1 Announce Type: cross Abstract: Artificial Intelligence (AI)-generated images have become increasingly realistic and readily adaptable to concrete real-world claims, creating new cha

Local Ai

iPay: Integrated Payment Action Recognition via Multimodal Networks and Adaptive Spatial Prior Learning

arXiv:2605.10732v1 Announce Type: cross Abstract: Automated transit payment analysis is vital for scalable fare auditing and passenger analytics, yet practice still relies on limited manual inspection

local-aiarxiv-cs-ai

Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology

arXiv:2605.09152v1 Announce Type: new Abstract: Deciphering animal intent is a fundamental challenge in computational ethology, largely because of semantic aliasing, the phenomenon where identical ext

Neuroscience-Inspired Analyses of Visual Interestingness in Multimodal Transformers

arXiv:2605.08188v1 Announce Type: cross Abstract: Human attention is the gateway to conscious perception, memory and decision-making. However, its role in modern transformer models remains largely une

researcharxiv-cs-ai

Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoning

arXiv:2605.09614v1 Announce Type: new Abstract: Long chain-of-thought (CoT) reasoning improves large vision--language models, but visual information often fades during generation, limiting long-horizo

safetyarxiv-cs-cv

The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence?

arXiv:2605.09352v1 Announce Type: new Abstract: Understanding why independently trained neural networks from different modalities converge toward shared representations, and where this convergence lea

UM-Text: A Unified Multimodal Model for Image Understanding and Visual Text Editing

arXiv:2601.08321v3 Announce Type: replace Abstract: With the rapid advancement of image generation, visual text editing using natural language instructions has received increasing attention. The main

Tutorials

Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment