Search: “multimodal”

Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective

arXiv:2607.27660v1 Announce Type: cross Abstract: Submodular Information Measures (SIMs) have recently emerged as a powerful framework for representation learning and multimodal learning. In particula

31 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation

DGX agent

arXiv:2607.25489v1 Announce Type: new Abstract: Large language models and multimodal foundation models are enabling medical artificial intelligence (AI) systems to move beyond isolated prediction and

safetyarxiv-cs-cv

29 Jul 2026

FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding

arXiv:2607.25266v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have enabled long-form video understanding at a scale that was not previously possible. However, the density of

MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

arXiv:2607.25948v1 Announce Type: cross Abstract: Any-to-any models predict any modality from any combination of others within a single network, a formulation used in multimodal vision and vision-lang

Shieldstral

model-releasesarxiv-cs-cl

arXiv:2607.25857v1 Announce Type: new Abstract: We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7imes its size on text s

Agents

VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents

arXiv:2607.24748v1 Announce Type: cross Abstract: Visually-rich documents such as reports, slides, and manuals often distribute the evidence needed to answer a question across multiple pages, mixing t

agentsarxiv-cs-ai

Tutorials

BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion

arXiv:2607.24110v1 Announce Type: new Abstract: Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cros

tutorialsarxiv-cs-cv

Tutorials

Controlling Embedding Spaces with Text-Conditioned Transformations

arXiv:2607.22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classific

tutorialsarxiv-cs-ai

Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement

arXiv:2607.22770v1 Announce Type: cross Abstract: Although artificial intelligence (AI) has shown promising performance in several medical tasks, accurate dementia etiology diagnosis with AI remains c

safetyarxiv-cs-cv

Disentangling Semantic Attention from Structural Bias in the Attention Manifold

arXiv:2607.24017v1 Announce Type: cross Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws. Specifically, MLLM

Looking for Affect in Spontaneous Finnish Speech through Linguistic Interpretability

model-releasesarxiv-cs-cl

arXiv:2607.24155v1 Announce Type: new Abstract: Existing research on affect in speech has shown how acoustic surface characteristics and content-related linguistic aspects of speech both relate to per

MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models

arXiv:2607.22566v1 Announce Type: new Abstract: MedLoCoMo is a Medical Long-Context Memory benchmark for patient-specific clinical reasoning over multi-admission medical dialogue. Existing medical QA

Hardware

Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models

arXiv:2607.23445v1 Announce Type: cross Abstract: Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting au

hardwarearxiv-cs-cl

Revitalizing Public Urban Places through Cultural and Political Memory: A Technological Approach with LLMs and Augmented Reality

arXiv:2607.22613v1 Announce Type: cross Abstract: This paper explores the intersection of memory, place, and identity, examining how new technologies, particularly Apple Vision Pro, can illuminate thi

Continual Video-MLLM Adaptation over Evolving Domains

model-releasesarxiv-cs-cv

arXiv:2607.18716v1 Announce Type: new Abstract: Video multimodal large language models have shown strong capability in video understanding, yet their adaptation to sequentially evolving domains remain

23 Jul 2026

DRGBT-1K: A Large-scale High-quality Benchmark for Dynamic RGBT Tracking

model-releasesarxiv-cs-cv

arXiv:2607.19772v1 Announce Type: new Abstract: Dynamic RGBT (DRGBT) tracking aims to continuously localize a target when the available sensing modalities and observation platforms vary over time. Com

23 Jul 2026

Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions

arXiv:2607.19061v2 Announce Type: replace-cross Abstract: Hateful optical illusions expose a serious gap in current multimodal safety systems. On original-view hateful illusions, previous work shows t

23 Jul 2026

Hardware

Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit

arXiv:2607.13095v1 Announce Type: cross Abstract: We present a full-pipeline inference optimization for the MiMo-V2.5 model family, which combines Hybrid Sliding Window Attention (Hybrid SWA), sparse

hardwarearxiv-cs-ai

How Far Can Root Cause Analysis Go on Real-World Telemetry Data?

arXiv:2607.13548v1 Announce Type: new Abstract: Identifying root causes in production microservice failures requires reasoning over large-scale, multimodal telemetry spanning metrics, logs, and traces

Applications

Learning Speaker Identity Beyond Language and Modality Constraints: Insights from the POLY-SIM 2026 Challenge

arXiv:2607.13669v1 Announce Type: new Abstract: Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and

applicationsarxiv-cs-cv

Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows

arXiv:2607.13078v1 Announce Type: cross Abstract: LLMs are now proposed for fraud detection, scam investigation, content moderation, and other trust-and-safety workflows. Much of the public literature

Constructed Reality, Contested Priors: Decoupling and the Architecture of Cognitive Relapse Under the Free Energy Principle

arXiv:2607.11958v1 Announce Type: new Abstract: Under the free energy principle, a predictive system does not observe reality directly; it maintains a generative model of the world and experiences tha

researcharxiv-cs-lg

From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answering

arXiv:2511.11132v4 Announce Type: replace Abstract: Knowledge-based Visual Question Answering (KBVQA) necessitates external knowledge incorporation beyond cross-modal understanding. Existing KBVQA met

Gene Expression-Informed Jointly Controlled Generative Modeling for Precision Molecular Design

arXiv:2607.11978v1 Announce Type: cross Abstract: Precision molecular design aims to discover personalized drug candidates through joint control of multiple conditions, such as biological relevance an

UniVR: Thinking in Visual Space for Unified Visual Reasoning

model-releasesarxiv-cs-cv

arXiv:2607.12800v1 Announce Type: new Abstract: Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation in

Infinity-Parser2 Technical Report

arXiv:2607.07836v1 Announce Type: new Abstract: We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end

Omni-Sleep: A Sleep Foundation Model via Hierarchical Contrastive Learning of CNS--ANS Dynamic

arXiv:2607.07720v1 Announce Type: cross Abstract: Sleep physiology arises from the coordinated dynamics of the central nervous system (CNS) and autonomic nervous system (ANS), as reflected by multimod

SHAP-Weighted Cross-Modal Expert Fusion for Emotion and Sentiment Recognition: Evidence and Limits

arXiv:2607.08573v1 Announce Type: new Abstract: Multimodal emotion and sentiment recognition is commonly addressed by early fusion, which concatenates modalities before classification, or late fusion,

Tutorials

Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning

arXiv:2607.08572v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) often follow a fixed Think-then-Answer paradigm, which is inefficient in heterogeneous multitask settings becau

tutorialsarxiv-cs-cv

The Importance of Encoder Choice:A Tabular-Image Study

arXiv:2607.07756v1 Announce Type: new Abstract: Multimodal learning usually requires a dedicated encoder per modality. When a tabular modality is involved, prior work has been mostly using a plain MLP

researcharxiv-cs-lg

UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks

model-releasesarxiv-cs-cl

arXiv:2607.08768v1 Announce Type: new Abstract: The rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operati

CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training

arXiv:2607.07292v1 Announce Type: cross Abstract: Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consis

Explain Before You Answer: A Survey on Compositional Visual Reasoning

arXiv:2508.17298v3 Announce Type: replace-cross Abstract: Compositional visual reasoning has emerged as a key research frontier in multimodal AI, aiming to endow machines with the human-like ability t

Gradient-free Riemannian Langevin Sampler

arXiv:2607.07519v1 Announce Type: new Abstract: We address the problem of efficiently sampling multimodal probability distributions, where standard Markov Chain Monte Carlo methods often suffer from p

researcharxiv-cs-lg

SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts

arXiv:2607.06620v1 Announce Type: cross Abstract: Recent Multimodal Large Language Models (MLLMs) struggle to bridge the representational gap between 2D semantic understanding and 3D spatial geometry.

Agents

CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration

arXiv:2607.05465v1 Announce Type: cross Abstract: Complex image creation and editing often require more than a single generation or editing model. A user request may involve synthesizing images, local

agentsarxiv-cs-ai

8 Jul 2026

Gemma 4 Technical Report

arXiv:2607.02770v1 Announce Type: cross Abstract: We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute effi

Hierarchical Multi-to-Single-Modal Knowledge Distillation for Disruption Prediction in EAST

arXiv:2607.04241v1 Announce Type: cross Abstract: Plasma disruption is a critical threat to tokamak safety. Existing data-driven predictors mainly rely on time-series diagnostic signals, while visible

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

arXiv:2607.03865v1 Announce Type: cross Abstract: Generative models such as diffusion and flow matching have advanced robotic visuomotor policies by modeling multimodal action distributions, but their

Reliability-Aware CT-MRI Registration: A Quality Engineering Framework with Stability Analysis and Risk Classification

arXiv:2607.02585v1 Announce Type: new Abstract: Multimodal CT-MRI registration is central to image-guided radiotherapy, surgical navigation, and diagnostic workflows, but most pipelines report only ag

safetyarxiv-cs-cv

TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction

arXiv:2607.04484v1 Announce Type: new Abstract: Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal rea

VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning

arXiv:2607.02927v1 Announce Type: cross Abstract: Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (V

XS-VLA: Coupling Coarse-grained Spatial Distillation with Latent Flow Matching for Lightweight Robotic Control

model-releasesarxiv-cs-lg

arXiv:2607.04171v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong multimodal understanding and spatial grounding, but their computational cost limits real-time r

ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion

arXiv:2507.05624v2 Announce Type: replace Abstract: Multimodal emotion and intent recognition is essential for automated human-computer interaction, It aims to analyze users' speech, text, and visual

3 Jul 2026

ESC: Emotional Self-Correction for Reliable Vision-Language Models

arXiv:2607.02089v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, yet they remain vulnerable to unreliable reasoning. Ex

3 Jul 2026

ASPIRE: Agentic /Skills Discovery for Robotics

arXiv:2607.00272v1 Announce Type: cross Abstract: Traditional robot programming is challenging: it requires orchestrating multimodal perception, managing physical contact dynamics, and handling divers

2 Jul 2026

Explainability in mulimodal deep transformation models for stroke outcome prediction

arXiv:2504.06299v2 Announce Type: replace-cross Abstract: Multimodal prediction models based on imaging and clinical data are increasingly used for clinical decision support, yet their interpretabilit

2 Jul 2026

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping