Search: “multimodal”

SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models

arXiv:2606.25990v1 Announce Type: new Abstract: As multimodal conversational systems increasingly engage in spoken interaction, their ability to navigate paralinguistic social cues has become a critic

25 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity

DGX agent

arXiv:2606.25634v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable progress in single-image perception, yet their ability to reason about complex cross-view

model-releasesarxiv-cs-cv

25 Jun 2026

AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression

arXiv:2606.24286v1 Announce Type: new Abstract: Multimodal Large Language Models have achieved remarkable progress in short-form audio-video understanding, yet long-form audio-video comprehension rema

researcharxiv-cs-cl

24 Jun 2026

S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing

arXiv:2606.24441v1 Announce Type: new Abstract: We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose im

researcharxiv-cs-cv

24 Jun 2026

CLoE: Expert Consistency Learning for Robust Missing Modality Segmentation

arXiv:2603.09316v2 Announce Type: replace Abstract: Multimodal medical image segmentation often faces missing modalities at inference, which induces disagreement among modality experts and makes fusio

researcharxiv-cs-cv

Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs

model-releasesarxiv-cs-lg

arXiv:2606.20961v1 Announce Type: new Abstract: Continual adaptation is essential for multimodal large language models (MLLMs) deployed across evolving domains, but the state-of-the-art MR-LoRA method

T-IMPACT: A Severity-Aware Benchmark for Contextual Image-Text Manipulation

arXiv:2606.22339v1 Announce Type: new Abstract: Recent advances in vision-language models and generative editing systems have made it increasingly easy to produce persuasive multimodal misinformation

Three-Step Hierarchical Transformer for Multi-Pedestrian Trajectory Prediction

arXiv:2606.23058v1 Announce Type: new Abstract: Pedestrian trajectory prediction requires modeling temporal dynamics, multimodal cues, and social interactions in crowded environments. Existing methods

agentsarxiv-cs-cv

Modeling Complex Behaviors: Multi-Personality Composition and Dynamic Switching in Vision-Language Models

arXiv:2606.11074v1 Announce Type: cross Abstract: With the widespread deployment of Multimodal Large Language Models (MLLMs) in social interaction, understanding and controlling their behavior under c

10 Jun 2026

Soul Computing: A Theoretical Framework and Technical Architecture for Intelligent Agents with Independent Consciousness

arXiv:2606.10413v1 Announce Type: new Abstract: Breakthroughs in large language models and multimodal generation technologies have propelled the digital reconstruction of human mental traits, emotiona

researcharxiv-cs-ai

10 Jun 2026

Beyond Raw Signals: Undecoded Generative Latents as Privileged Synthetic Data

arXiv:2606.08336v1 Announce Type: new Abstract: While multimodal integration significantly improves computer vision models, deploying them incurs prohibitive inference costs and requires scarce, perfe

researcharxiv-cs-cv

ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China

arXiv:2606.08959v1 Announce Type: new Abstract: We introduce ChinaHeritaQA, a multimodal benchmark dataset for evaluating the cultural reasoning abilities of vision-language models (VLMs) on UNESCO Wo

KITE: A Tri-Modal Transformer Integrating Text, Images, and Knowledge Graphs for Fake News Detection

arXiv:2606.07651v1 Announce Type: cross Abstract: Traditional fake news detection methods are falling behind as multimodal misinformation grows more advanced, seamlessly blending deceptive text, manip

MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Models

arXiv:2606.07706v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated strong performance across multimodal tasks, yet their safety robustness remains an open challenge. Whi

PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflow

arXiv:2606.07549v1 Announce Type: new Abstract: Recent advances in Multimodal Large Language Models (MLLMs) and agent workflows have shown strong promise for computational pathology, yet reliable patc

safetyarxiv-cs-ai

Tutorials

PRISM: Topology-Aware Cross-Modal Imputation for Modality-Deficient Federated Graph Learning

arXiv:2606.09301v1 Announce Type: new Abstract: Multimodal federated graph learning (MM-FGL) aims to collaboratively learn from decentralized graphs with text and images. However, real-world clients m

tutorialsarxiv-cs-lg

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

arXiv:2606.08708v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective paradigm for improving the reasoning capability of Large Vision-Language M

safetyarxiv-cs-cv

Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning

arXiv:2606.09303v1 Announce Type: new Abstract: The rapid development of pretrained foundation models has enabled more general image segmentation. Multimodal large language models (MLLMs) have been wi

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

arXiv:2606.08063v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly un

When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding

arXiv:2606.08239v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains under

researcharxiv-cs-ai

SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows

arXiv:2602.09580v4 Announce Type: replace-cross Abstract: Real-world fine-tuning of dexterous manipulation policies remains challenging due to limited real-world interaction budgets and highly multimo

safetyarxiv-cs-lg

8 Jun 2026

When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing

arXiv:2606.07171v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) enable flexible instruction-driven image editing, but privacy risks arise when user images expose diverse and u

8 Jun 2026

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

arXiv:2606.05950v1 Announce Type: new Abstract: Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models. However, most existing methods remain con

6 Jun 2026

MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models

arXiv:2606.05177v1 Announce Type: new Abstract: Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and

5 Jun 2026

Resonant Minds: Closed-Loop Social Avatars with Theory of Mind

arXiv:2606.05896v1 Announce Type: new Abstract: Creating lifelike digital humans with genuine social intelligence requires unifying cognitive reasoning and multimodal generation within a coherent fram

agentsarxiv-cs-cv

5 Jun 2026

Local Ai

Vision Hopfield Memory Networks

arXiv:2603.25157v2 Announce Type: replace-cross Abstract: Recent vision and multimodal foundation backbones, such as Transformer families and state-space models like Mamba, have achieved remarkable pr

local-aiarxiv-cs-cv

5 Jun 2026

A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs

arXiv:2606.04596v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used for video understanding, yet their reliability under multi-video inputs remains poorly un

4 Jun 2026

Applications

Dynamic Content Moderation in Livestreams: Combining Supervised Classification with MLLM-Boosted Similarity Matching

arXiv:2512.03553v3 Announce Type: replace-cross Abstract: Content moderation remains a critical yet challenging task for large-scale user-generated video platforms, especially in livestreaming environ

applicationsarxiv-cs-ai

4 Jun 2026

KODA: Contrastive Representation Comparison and Alignment for Vision-Language Foundation Models

arXiv:2606.04180v1 Announce Type: new Abstract: Vision-language foundation models such as CLIP and SigLIP provide widely used representations for multimodal learning systems. While these models are ty

safetyarxiv-cs-lg

4 Jun 2026

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

arXiv:2606.03793v1 Announce Type: new Abstract: Multimodal Large Language Models integrate visual perception into language reasoning, introducing a continuous attack surface susceptible to adversarial

safetyarxiv-cs-cl

Google's new Gemma 4 12B model is designed to run on any laptop with 16GB of RAM

model-releasesars-technica

Google DeepMind released Gemma 4 12B, an open AI model that brings multimodal capabilities to everyday laptops by processing text, images, and audio natively without separate encoders. Small enough to

Let the Dynamics Flow: Stable Flow Matching Dynamical Systems

model-releasesarxiv-cs-ro

arXiv:2606.03834v1 Announce Type: new Abstract: Flow matching has recently emerged as a powerful approach for imitation learning, enabling scalable, expressive, and multimodal motion policies. However

Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents

arXiv:2606.03236v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have substantially advanced mobile agents, yet proactive mobile assistance remains challenging because agents m

Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detection

arXiv:2606.02812v1 Announce Type: new Abstract: Modeling patient trajectories from longitudinal electronic health records (EHRs) requires reasoning over sparse, noisy, and long-context multimodal sequ

agentsarxiv-cs-ai

A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision

arXiv:2606.01992v1 Announce Type: cross Abstract: Industrial anomaly detection has historically been a unimodal task. Recent multimodal vision-language models have produced systems that admit textual

Agent Skills Should Go Beyond Text: The Case for Visual Skills

arXiv:2606.01414v1 Announce Type: new Abstract: Reusable skills are a key mechanism for extending agent capabilities, allowing agents to accumulate experience and solve increasingly complex tasks. Yet

agentsarxiv-cs-cv

Beyond the Mouth: Upper-Face Affective Cues in Audiovisual Sentence Recognition under Acoustic Uncertainty

arXiv:2606.00670v1 Announce Type: cross Abstract: Face-to-face speech comprehension is inherently multimodal, integrating acoustic signals with visible articulation, facial expression, head motion, an

researcharxiv-cs-ai

Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding

arXiv:2606.00564v1 Announce Type: cross Abstract: While on-policy distillation offers dense supervision for training small reasoning models, its optimization dynamics in the multimodal domain remain u

safetyarxiv-cs-cl

Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays

arXiv:2509.15234v2 Announce Type: replace Abstract: Multimodal learning from paired medical images and clinical text is a central challenge in medical data-driven informatics, where effective cross-mo

Hardware

First technical Deepdive on M3 on the internet😎

First technical Deepdive on M3 on the internet😎 MiniMax-M3 combines 1M context, native multimodality, and MiniMax Sparse Attention. The next layer is serving it efficiently: KV-block-major sparse atte

hardwaretogether-ai--x

GLENS: Global Search via Learning from Solver Iterates with Diffusion Models

model-releasesarxiv-cs-lg

arXiv:2606.00366v1 Announce Type: new Abstract: We consider the problem of generating a large collection of initial guesses for local minima of multimodal non-convex continuous optimization problems.

Tutorials

LLMs Need Encoders for Semantic IDs Too

arXiv:2606.00324v1 Announce Type: cross Abstract: Multimodal LLMs use dedicated encoders to bridge non-language modalities (vision encoders for images, depth models for audio codec tokens) because raw

tutorialsarxiv-cs-ai

PaintBench: Deterministic Evaluation of Precise Visual Editing

model-releasesarxiv-cs-lg

arXiv:2606.00188v1 Announce Type: cross Abstract: While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle. To p

RescueBench: Can Embodied Agents Save Lives in the Wild ?

arXiv:2606.01848v1 Announce Type: new Abstract: Search-and-rescue (SAR) requires embodied agents to explore unfamiliar environments under multimodal uncertainty, perform multi-stage interactions, and

Sandboxed Coding Agents are Competitive Omni-modal Task Solvers

arXiv:2606.00579v1 Announce Type: new Abstract: As multimodal LLMs increasingly target video and audio, it is often assumed that such tasks require native omnimodal models. We show that this is not al

MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models

arXiv:2511.16940v3 Announce Type: replace Abstract: Modern Vision-Language Models (VLMs) pose significant individual-level privacy risks by linking fragmented multimodal data to identifiable individua

1 Jun 2026

Tutorials

Personalize Your Large Vision-language Models With In-context Prompt Tuning

arXiv:2605.31513v1 Announce Type: new Abstract: Large vision-language models (LVLMs) have demonstrated strong general multimodal capability and are increasingly deployed in downstream systems. This tr

tutorialsarxiv-cs-cv

1 Jun 2026

CCS: Clinical Consensus Selection for Radiology Report Generation