Search: “multimodal”

Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval

arXiv:2604.19135v1 Announce Type: new Abstract: This paper presents the first exploration of text-to-image diffusion models for zero-shot sketch-based 3D shape retrieval (ZS-SBSR). Existing sketch-bas

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning

DGX agent

arXiv:2604.18829v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved impressive performance on visual perception and reasoning tasks with RGB imagery, yet they remain

model-releasesarxiv-cs-cv

22 Apr 2026

How Far Are Video Models from True Multimodal Reasoning?

arXiv:2604.19193v1 Announce Type: new Abstract: Despite remarkable progress toward general-purpose video models, a critical question remains unanswered: how far are these models from achieving true mu

Agents

Introducing Kimi K2.6 from @Kimi_Moonshot, a multimodal agentic model with Agent Swarm scaling to 300 sub-agents and long-horizon coding sta…

Introducing Kimi K2.6 from @Kimi_Moonshot, a multimodal agentic model with Agent Swarm scaling to 300 sub-agents and long-horizon coding stability. AI natives can now use Kimi K2.6 on Together AI and

agentstogether-ai--x

Multimodal Transformer for Sample-Aware Prediction of Metal-Organic Framework Properties

arXiv:2604.19383v1 Announce Type: cross Abstract: Metal-organic frameworks (MOFs) are a major target of machine-learning-based property prediction, yet most models assume that a single framework repre

tutorialsarxiv-cs-ai

SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

arXiv:2604.19638v1 Announce Type: new Abstract: Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address saf

Bias-constrained multimodal intelligence for equitable and reliable clinical AI

arXiv:2604.16884v1 Announce Type: new Abstract: The integration of medical imaging and clinical text has enabled the emergence of generalist artificial intelligence (AI) systems for healthcare. Howeve

Dynamic Emotion and Personality Profiling for Multimodal Deception Detection

arXiv:2604.17037v1 Announce Type: new Abstract: Deception detection is of great significance for ensuring information security and conducting public opinion analysis, with personality factors and emot

safetyarxiv-cs-cl

EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling

arXiv:2604.17087v1 Announce Type: new Abstract: Recent Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language understanding tasks, yet their inference efficie

MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation

model-releasesarxiv-cs-cl

arXiv:2604.16943v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown impressive capabilities, yet they often struggle to effectively capture the fine-grained textual inf

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

arXiv:2604.18360v1 Announce Type: cross Abstract: Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, the

applicationsarxiv-cs-cl

PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction

arXiv:2604.17278v1 Announce Type: new Abstract: Effective pest recognition and management are crucial for sustainable agricultural development. However, collecting pest data in real scenarios is often

applicationsarxiv-cs-cv

Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning

arXiv:2502.02871v2 Announce Type: replace Abstract: Scientific reasoning, the process through which humans apply logic, evidence, and critical thinking to explore and interpret scientific phenomena, i

researcharxiv-cs-cl

Agents

Sensorimotor Self-Recognition in Multimodal Large Language Model-Driven Robots

arXiv:2505.19237v2 Announce Type: replace-cross Abstract: Self-recognition -- the ability to maintain an internal representation of one's own body within the environment -- underpins intelligent, auto

agentsarxiv-cs-ro

Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark

arXiv:2510.13759v3 Announce Type: replace Abstract: Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. E

Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning

model-releasesarxiv-cs-cl

arXiv:2603.23404v2 Announce Type: replace-cross Abstract: Existing Multimodal Large Language Models (MLLMs) struggle with 3D spatial reasoning, as they fail to construct structured abstractions of the

Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs

arXiv:2604.16054v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved impressive progress on vision language benchmarks, yet their capacity for visual cognitive and

20 Apr 2026

EuropeMedQA Study Protocol: A Multilingual, Multimodal Medical Examination Dataset for Language Model Evaluation

model-releasesarxiv-cs-cl

arXiv:2604.14306v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated high proficiency on English-centric medical examinations, their performance often declines when fac

Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding

arXiv:2604.15210v1 Announce Type: cross Abstract: Humor is one of the few cognitive tasks where getting the reasoning right matters as much as getting the answer right. While recent work evaluates hum

safetyarxiv-cs-cl

Power video semantic search with Amazon Nova Multimodal Embeddings

In this post, we show you how to build a video semantic search solution on Amazon Bedrock using Nova Multimodal Embeddings that intelligently understands user intent and retrieves accurate video resul

tutorialsaws-ml-blog

TennisTV: Do Multimodal Large Language Models Understand Tennis Rallies?

arXiv:2509.15602v5 Announce Type: replace Abstract: Multimodal large language models (MLLMs) excel at general video understanding but struggle with fast, high-frequency sports like tennis, where rally

Decoding the Delta: Unifying Remote Sensing Change Detection and Understanding with Multimodal Large Language Models

arXiv:2604.14044v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) excel in general vision-language tasks, their application to remote sensing change understanding is hinde

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining

researchapple-ml-research

This paper was accepted at the Workshop on Navigating and Addressing Data Problems for Foundation Models (NADPFM) at ICLR 2026. Principled domain reweighting can substantially improve sample efficienc

Out of Context: Reliability in Multimodal Anomaly Detection Requires Contextual Inference

model-releasesarxiv-cs-lg

arXiv:2604.13252v1 Announce Type: new Abstract: Anomaly detection aims to identify observations that deviate from expected behavior. Because anomalous events are inherently sparse, most frameworks are

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive 3D worlds from text, im…

applicationsclem-delangue--x

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive *3D worlds* from text, images, and videos. Outputs can be integrated into game engine

AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition

arXiv:2604.12735v1 Announce Type: new Abstract: LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states. In this p

Human-Inspired Context-Selective Multimodal Memory for Social Robots

arXiv:2604.12081v1 Announce Type: new Abstract: Memory is fundamental to social interaction, enabling humans to recall meaningful past experiences and adapt their behavior accordingly based on the con

researcharxiv-cs-ai

Probabilistic Feature Imputation and Uncertainty-Aware Multimodal Federated Aggregation

arXiv:2604.12970v1 Announce Type: cross Abstract: Multimodal federated learning enables privacy-preserving collaborative model training across healthcare institutions. However, a fundamental challenge

RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation

arXiv:2604.12319v1 Announce Type: new Abstract: Multimodal semantic segmentation has emerged as a powerful paradigm for enhancing scene understanding by leveraging complementary information from multi

Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning

arXiv:2604.11835v1 Announce Type: cross Abstract: Machine learning for tabular data remains constrained by poor schema generalization, a challenge rooted in the lack of semantic understanding of struc

safetyarxiv-cs-ai

CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning

arXiv:2604.10973v1 Announce Type: new Abstract: Reasoning over tabular data is a crucial capability for tasks like question answering and fact verification, as it requires models to comprehend both fr

tutorialsarxiv-cs-ai

Local Ai

Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models

arXiv:2604.10219v1 Announce Type: new Abstract: Multimodal Large Reasoning Models (MLRMs) have achieved remarkable strides in visual reasoning through test time compute scaling, yet long chain reasoni

local-aiarxiv-cs-ai

DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning

arXiv:2509.25866v2 Announce Type: replace Abstract: The 'thinking with images' paradigm represents a pivotal shift in the reasoning of Vision Language Models (VLMs), moving from text-dominant chain-of

Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning

arXiv:2604.11299v1 Announce Type: cross Abstract: In recent years, rapid advances in Multimodal Large Language Models (MLLMs) have increasingly stimulated research on ancient Chinese scripts. As the e

ForestPrune: High-ratio Visual Token Compression for Video Multimodal Large Language Models via Spatial-Temporal Forest Modeling

arXiv:2603.22911v2 Announce Type: replace-cross Abstract: Due to the great saving of computation and memory overhead, token compression has become a research hot-spot for MLLMs and achieved remarkable

researcharxiv-cs-ai

From UAV Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping

arXiv:2604.09907v1 Announce Type: cross Abstract: To improve crop genetics, high-throughput, effective and comprehensive phenotyping is a critical prerequisite. While such tasks were traditionally per

Progressive Multimodal Interaction Network for Reliable Quantification of Fish Feeding Intensity in Aquaculture

arXiv:2506.14170v3 Announce Type: replace-cross Abstract: Accurate quantification of fish feeding intensity is crucial for precision feeding in aquaculture, as it directly affects feed utilization and

SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions

arXiv:2307.01139v2 Announce Type: replace-cross Abstract: Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less

Local Ai

Tencent HY-World 2.0 appears to be dropping on April 15 — open-source multimodal 3D world generation from Tencent Hunyuan

local-air-stablediffusion

Tencent's HunyuanWorld is an open-source multimodal 3D world generation model from the Tencent Hunyuan team, featuring 360° immersive experiences via panoramic world proxies, mesh export capabilities

Investigating Multimodal Large Language Models to Support Usability Evaluation

arXiv:2508.16165v2 Announce Type: replace-cross Abstract: Usability evaluation is an essential method to support the design of effective and intuitive user interfaces (UIs). However, it commonly relie

applicationsarxiv-cs-ai

MONETA: Multimodal Industry Classification through Geographic Information with Multi Agent Systems

arXiv:2604.07956v2 Announce Type: replace Abstract: Industry classification schemes are integral parts of public and corporate databases as they classify businesses based on economic activity. Due to

Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement

arXiv:2604.09473v1 Announce Type: new Abstract: Fully immersive experiences that tightly integrate 6-DoF visual and auditory interaction are essential for virtual and augmented reality. While such exp

applicationsarxiv-cs-cv

Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Models

arXiv:2604.08815v1 Announce Type: new Abstract: Medical vision-language models (VLMs) show strong performance on radiology tasks but often produce fluent yet weakly grounded conclusions due to over-re

Agents

Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models

arXiv:2604.08545v1 Announce Type: new Abstract: The advent of agentic multimodal models has empowered systems to actively interact with external environments. However, current agents suffer from a pro

agentsarxiv-cs-cv

Draw-In-Mind: Rebalancing Designer-Painter Roles in Unified Multimodal Models Benefits Image Editing

arXiv:2509.01986v4 Announce Type: replace-cross Abstract: In recent years, integrating multimodal understanding and generation into a single unified model has emerged as a promising paradigm. While th

EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience

arXiv:2604.08063v1 Announce Type: new Abstract: Reconstructing visual stimuli from non-invasive electroencephalography (EEG) remains challenging due to its low spatial resolution and high noise, parti

Exploring Temporal Representation in Neural Processes for Multimodal Action Prediction

arXiv:2604.08418v1 Announce Type: new Abstract: Inspired by the human ability to understand and predict others, we study the applicability of Conditional Neural Processes (CNP) to the task of self-sup

tutorialsarxiv-cs-ro

Frailty Estimation in Elderly Oncology Patients Using Multimodal Wearable Data and Multi-Instance Learning

arXiv:2604.06985v1 Announce Type: cross Abstract: Frailty and functional decline strongly influence treatment tolerance and outcomes in older patients with cancer, yet assessment is typically limited

applicationsarxiv-cs-ai