Search: “multimodal”

ProductWebGen: Benchmarking Multimodal Product Webpage Generation

arXiv:2606.01022v1 Announce Type: cross Abstract: Crafting a product display webpage from a source product image, along with layout and visual content instructions, holds significant practical value f

2 Jun 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Response-Aware Multimodal Learning for Post-Treatment Visual Acuity Forecasting

DGX agent

arXiv:2606.00588v1 Announce Type: new Abstract: Long-term visual acuity (VA) outcomes after anti-VEGF therapy are central to patient counseling, expectation setting, and follow-up planning in diabetic

local-aiarxiv-cs-cv

2 Jun 2026

Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design

arXiv:2606.02507v1 Announce Type: cross Abstract: Inverse materials design is shifting materials discovery from forward prediction to targeted proposal of candidates that satisfy objectives under phys

tutorialsarxiv-cs-lg

2 Jun 2026

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

arXiv:2605.30912v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers.

tutorialsarxiv-cs-cl

Local Ai

Best-Arm Identification-Based Trust Region Selection for Bayesian Optimization on Multimodal Functions

arXiv:2605.31050v1 Announce Type: new Abstract: Gaussian process-based Bayesian optimization (BO) is a popular approach for expensive black-box optimization, but its performance often degrades on comp

local-aiarxiv-cs-lg

UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis

arXiv:2510.15710v3 Announce Type: replace Abstract: Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central

Your Multimodal Speech Model Says I Have a Face for Radio

model-releasesarxiv-cs-cl

arXiv:2605.30472v1 Announce Type: new Abstract: As large neural models have become better at language tasks, researchers are increasingly building multi- and omnimodal models that handle more modaliti

Archon: A Unified Multimodal Model for Holistic Digital Human Generation

arXiv:2605.30311v1 Announce Type: cross Abstract: Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visu

researcharxiv-cs-ai

Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance

arXiv:2411.14279v2 Announce Type: replace-cross Abstract: Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. However, despite showing promising per

safetyarxiv-cs-cl

OmniCD: A Foundational Framework for Remote Sensing Image Change Detection Guided by Multimodal Semantics

arXiv:2605.30168v1 Announce Type: new Abstract: Change detection (CD) in remote sensing is vital for applications such as urban monitoring and disaster assessment, yet traditional methods struggle wit

Local Ai

UniNote: A Unified Embedding Model for Multimodal Representation and Ranking

arXiv:2605.29287v1 Announce Type: cross Abstract: Item-to-Item (I2I) retrieval is a fundamental part of modern content platforms, supporting critical industrial workflows from recommendation engines t

local-aiarxiv-cs-cv

VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis

arXiv:2605.28978v1 Announce Type: new Abstract: Finite Element Analysis (FEA) serves as the cornerstone of modern engineering design. However, its workflow is inherently complex and relies heavily on

agentsarxiv-cs-ai

From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competition

arXiv:2605.27451v1 Announce Type: new Abstract: The 10th Affective & Behavior Analysis in-the-Wild (ABAW) Workshop and Competition, held at CVPR 2026, continues to advance research on modelling, analy

28 May 2026

MeniOmni: A Structured Multimodal Benchmark for Holistic Meniscus Injury Assessment

model-releasesarxiv-cs-cv

arXiv:2605.28161v1 Announce Type: new Abstract: Clinical diagnosis of meniscus injuries requires radiologists to integrate volumetric MRI evidence with patient context (e.g., sex, age, BMI) and to pro

28 May 2026

Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models

arXiv:2605.27243v1 Announce Type: new Abstract: Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent trajectories

agentsarxiv-cs-cv

27 May 2026

Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos

arXiv:2605.26232v1 Announce Type: new Abstract: Pre-trained video large language models excel at visual reasoning. However, they struggle when videos arrive with auxiliary streams, such as audio, dept

27 May 2026

PolyFusionAgent: A Multimodal Foundation Model and Autonomous AI Assistant for Polymer Property Prediction and Inverse Design

arXiv:2605.26543v1 Announce Type: new Abstract: Polymer discovery is central to fields ranging from energy storage to biomedicine, but it is hindered by an astronomically large chemical design space a

agentsarxiv-cs-ai

27 May 2026

Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filters

arXiv:2505.18979v2 Announce Type: replace Abstract: Text-to-image (T2I) models can generate not-safe-for-work (NSFW) content, motivating multi-stage safety pipelines with both text and image filters.

safetyarxiv-cs-lg

26 May 2026

Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation

arXiv:2605.23961v1 Announce Type: cross Abstract: The design of RNA molecules that interact with specific proteins is a critical challenge in experimental and computational biology. Despite recent pro

26 May 2026

RAG4Outcome: A Retrieval-Augmented Multimodal Framework for Prognostic Prediction in Chronic Osteomyelitis

arXiv:2605.22833v1 Announce Type: cross Abstract: Chronic osteomyelitis presents substantial prognostic challenges due to its high recurrence risk and complex postoperative recovery trajectories. Trad

25 May 2026

Case-Aware Medical Image Classification with Multimodal Knowledge Graphs and Reliability-Guided Refinement

arXiv:2605.22547v1 Announce Type: new Abstract: Deep learning has brought significant progress to medical image classification, yet most existing methods still rely on isolated visual evidence and can

22 May 2026

Two-Stage Multimodal Framework for Emotion Mimicry Intensity Prediction

arXiv:2605.21869v1 Announce Type: new Abstract: We present our submission to the Hume-ABAW10 Emotional Mimicry Intensity (EMI) Challenge, which aims to predict six continuous emotion intensity dimensi

22 May 2026

ArPoMeme: An Annotated Arabic Multimodal Dataset for Political Ideology and Polarization

arXiv:2605.20967v1 Announce Type: new Abstract: Memes have become a prominent medium of political communication in the Arab world, reflecting how humor, imagery, and text interact to express ideologic

researcharxiv-cs-cl

21 May 2026

QwenSafe: Multimodal Content Rating Description Identification via Preference-Aligned VLMs

model-releasesarxiv-cs-cv

arXiv:2605.20584v1 Announce Type: new Abstract: Mobile app marketplaces require developers to disclose standardized content rating descriptors (CRDs) to inform users about potentially sensitive or res

21 May 2026

Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing

arXiv:2605.19135v1 Announce Type: new Abstract: Causal representation learning (CRL) seeks to uncover meaningful latent variables and their corresponding causal structure from high-dimensional observa

researcharxiv-cs-lg

20 May 2026

Towards Trust Calibration in Socially Interactive Agents: Investigating Gendered Multimodal Behaviors Generation with LLMs

model-releasesarxiv-cs-cl

arXiv:2605.19798v1 Announce Type: new Abstract: As Socially Interactive Agents (SIAs) become increasingly integrated into daily life, the ability to calibrate user trust to an agent's actual capabilit

20 May 2026

CatalyticMLLM: A Graph-Text Multimodal Large Language Model for Catalytic Materials

arXiv:2605.17254v1 Announce Type: new Abstract: Property prediction and inverse structural design of catalytic materials are typically modeled as two independent tasks: the former predicts target prop

Cross-modal Affinity-aligned Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Game-Based Learning

arXiv:2605.16806v1 Announce Type: cross Abstract: Collaborative game-based learning environments offer rich opportunities for small-group knowledge construction, yet automatically predicting student c

Local Ai

How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A

arXiv:2605.16359v1 Announce Type: cross Abstract: Vision-language models improve perception by feeding increasingly long visual token sequences into language backbones, but the resulting inference cos

local-aiarxiv-cs-ai

MUSE: Multimodal Uncertainty Quantification of State Estimation

arXiv:2605.17421v1 Announce Type: new Abstract: Accurate visual state estimation has been a central topic in robotics with a wide range of applications in robot navigation, autonomous driving, and aut

agentsarxiv-cs-ro

SkyNative: A Native Multimodal Framework for Remote Sensing Visual Evidence Reasoning

model-releasesarxiv-cs-cv

arXiv:2605.17949v1 Announce Type: new Abstract: Remote sensing vision-language models commonly rely on pretrained visual encoders to convert images into semantic features before language-model reasoni

Missing-Data-Induced Phase Transitions in Spectral PLS for Multimodal Learning

arXiv:2601.21294v2 Announce Type: replace Abstract: Partial Least Squares (PLS) learns shared structure from paired data via the top singular vectors of the empirical cross-covariance (PLS-SVD), but m

safetyarxiv-cs-lg

18 May 2026

Multimodal Object Detection Under Sparse Forest-Canopy Occlusion

arXiv:2605.15326v1 Announce Type: new Abstract: Reliable detection of humans beneath forest canopy remains a difficult remote-sensing challenge due to sparse, structured, and viewpoint-dependent occlu

18 May 2026

Second-Order Multi-Level Variance Correction for Modality Competition in Multimodal Models

arXiv:2605.16165v1 Announce Type: cross Abstract: Autoregressive next-token training offers a unified formulation for image generation and text understanding, but it also creates strong modality compe

18 May 2026

Learning Multimodal Embeddings for Traffic Accident Prediction and Causal Estimation

arXiv:2512.02920v3 Announce Type: replace-cross Abstract: We consider analyzing traffic accident patterns using both road network data and satellite images aligned to road graph nodes. Previous work f

15 May 2026

Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

arXiv:2605.15168v1 Announce Type: cross Abstract: Reconstructing precise clinical timelines is essential for modeling patient trajectories and forecasting risk in complex, heterogeneous conditions lik

15 May 2026

Multimodal Graph-based Classification of Esophageal Motility Disorders

arXiv:2605.13623v1 Announce Type: new Abstract: Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry (HRIM) data and variab

tutorialsarxiv-cs-lg

14 May 2026

Multimodal Hidden Markov Models for Persistent Emotional State Tracking

arXiv:2605.12838v1 Announce Type: new Abstract: Tracking an interpretable emotional arc of a conversation via the sentiment of individual utterances processed as a whole is central to both understandi

researcharxiv-cs-ai

14 May 2026

Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts

arXiv:2605.11444v1 Announce Type: new Abstract: All-in-one image restoration seeks to recover clean images from inputs affected by diverse and unknown degradations using a unified framework. Recent me

13 May 2026

Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Making

arXiv:2602.07668v2 Announce Type: replace Abstract: The looking-in-looking-out (LILO) framework has enabled intelligent vehicle applications that understand both the outside scene and the driver state

13 May 2026

Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution

arXiv:2605.08887v1 Announce Type: new Abstract: Self-evolving agents present a promising path toward continual adaptation by distilling task interactions into reusable knowledge artifacts. In practice

agentsarxiv-cs-ai

BEACON: A Multimodal Dataset for Learning Behavioral Fingerprints from Gameplay Data

arXiv:2605.10867v1 Announce Type: cross Abstract: Continuous authentication in high-stakes digital environments requires datasets with fine-grained behavioral signals under realistic cognitive and mot

CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation

arXiv:2605.10873v1 Announce Type: cross Abstract: Recovering editable CAD programs from images or 3D observations is central to AI-assisted design, but progress is difficult to measure because existin

Dual Tuning for Reasoning Efficacy-Driven Data Curation in Multimodal LLM Training

arXiv:2603.04415v2 Announce Type: replace Abstract: Reasoning post-training improves Large Language Models (LLMs) on complex tasks such as mathematics and coding, but its benefits across diverse multi

tutorialsarxiv-cs-cl

HOME-KGQA: A Benchmark Dataset for Multimodal Knowledge Graph Question Answering on Household Daily Activities

arXiv:2605.09348v1 Announce Type: cross Abstract: Large Language Models (LLMs) provide flexible natural language processing capabilities, while knowledge graphs (KGs) offer explicit and structured kno

MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image

model-releasesarxiv-cs-cl

arXiv:2605.10616v1 Announce Type: cross Abstract: Tabular Foundation Models have recently established the state of the art in supervised tabular learning, by leveraging pretraining to learn generaliza

Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner

arXiv:2506.01301v2 Announce Type: replace Abstract: Theory-of-Mind (ToM) enables humans to infer mental states-such as beliefs, desires, and intentions-forming the foundation of social cognition. Howe

researcharxiv-cs-ai

PIDNet: Progressive Implicit Decouple Network for Multimodal Action Quality Assessment

arXiv:2605.08945v1 Announce Type: new Abstract: Action quality assessment (AQA) aims to automatically quantify the execution quality of human actions in videos and is valuable for applications such as