Search: “multimodal”

Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution

arXiv:2607.26596v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual understanding within a unified tran

30 Jul 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications

DGX agent

arXiv:2607.27145v1 Announce Type: new Abstract: As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitorin

safetyarxiv-cs-cv

30 Jul 2026

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

arXiv:2607.25933v1 Announce Type: cross Abstract: Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practic

Tutorials

HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document

arXiv:2607.24861v1 Announce Type: cross Abstract: Question answering (QA) over complex documents requires models to retrieve and integrate evidence distributed across distant document regions and moda

tutorialsarxiv-cs-ai

Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

model-releasesarxiv-cs-cl

arXiv:2607.24904v1 Announce Type: cross Abstract: Standard vision-language models (VLMs) suffer from Moravec's paradox: they excel at complex offline visual reasoning but struggle with simple streamin

MyMentorLLM: A psychotherapy GenAI environment with multimodal voice/text patients, trainees and experts for deliberate practice

arXiv:2607.25667v1 Announce Type: cross Abstract: Psychotherapists need repeated training and supervision by experts; however, scalability is problematic. Here we present MyMentorLLM, a multimodal voi

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

arXiv:2607.24957v1 Announce Type: new Abstract: We introduce PerceptionBench, a benchmark specifically designed to evaluate the atomic visual perception capabilities of Multimodal Large Language Model

Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering

arXiv:2607.25422v1 Announce Type: new Abstract: Knowledge-intensive multimodal question answering (KI-MMQA) sits at the intersection of three expensive primitives: long visual token sequences, dense r

SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models

arXiv:2607.25818v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs), such as Qwen2.5-VL and InternVL3, generate large numbers of vision tokens for high-resolution inputs, l

Group Preference Collapse in Personalized Multimodal Large Language Models

arXiv:2607.22603v1 Announce Type: new Abstract: Personalized multimodal large language models (MLLMs) aim to generate user-specific responses, but existing methods mainly rely on profile-level informa

28 Jul 2026

Safety

MIME: Multimodal Interactive Motion Encoder

arXiv:2607.22702v1 Announce Type: new Abstract: Text-motion representation learning has advanced rapidly, with growing interest in multi person interactions for animation, AR/VR, and embodied AI. Thes

safetyarxiv-cs-cv

28 Jul 2026

Task-Conditional Faithfulness Auditing of Multimodal LLMs for Grid Diagnosis

arXiv:2607.24539v1 Announce Type: new Abstract: Multimodal large language models (LLMs) can combine topology, measurements, and incident text for grid diagnosis, yet answer accuracy does not establish

28 Jul 2026

Geo3R: Mitigating Spatial Reasoning Hallucination in Multimodal Large Language Models

arXiv:2607.21085v1 Announce Type: new Abstract: Despite remarkable progress in visual understanding, Multimodal Large Language Models (MLLMs) remain prone to hallucinations when reasoning about spatia

Glad to see Google sharing data on how Gemini is being used. Especially interesting is that the usefulness of multimodal AI for manual labor…

model-releasesethan-mollick--x

Glad to see Google sharing data on how Gemini is being used. Especially interesting is that the usefulness of multimodal AI for manual labor may be greater than expected. https://blog.google/innovatio

HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models

arXiv:2607.21105v1 Announce Type: new Abstract: Although Multimodal Large Language Models have achieved strong performance across a wide range of vision-language tasks, they still suffer from hallucin

Safety

QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detection

arXiv:2603.05964v3 Announce Type: replace Abstract: Quantizing open-vocabulary object detection (OVOD) models reduces their memory and computational costs, but extremely low-bit quantization severely

safetyarxiv-cs-cv

Tutorials

Quality-Aware Multimodal Fusion Reveals Implicit Identity in Valence-Arousal Features

arXiv:2607.21347v1 Announce Type: new Abstract: Conventional face recognition relies on static appearance cues and degrades in unconstrained settings with expression variation, occlusion, and poor lig

tutorialsarxiv-cs-cv

An Intelligent-Cloud Edge Multimodal Interaction System for Robots

arXiv:2607.14675v2 Announce Type: replace-cross Abstract: Robust human-robot interaction in complex environments requires accurate gesture perception, semantic scene understanding, and reliable task p

MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

arXiv:2607.19235v1 Announce Type: cross Abstract: Theory of Mind (ToM), the ability to infer other's beliefs, intentions, and states of knowledge, is central to social interaction, yet remains challen

Applications

Singapore-based Ropedia, which captures real-world human experience via video and converts it into model-ready multimodal datasets, raised a $22M pre-Series A (Kyt Dotson/SiliconANGLE)

Kyt Dotson / SiliconANGLE: Singapore-based Ropedia, which captures real-world human experience via video and converts it into model-ready multimodal datasets, raised a 22M pre-Series A — Singapore-bas

applicationstechmeme

Team RAS in 11th ABAW Competition: Multimodal Ambivalence Recognition Approach

arXiv:2607.14702v2 Announce Type: replace Abstract: Automatic recognition of ambivalence and hesitancy is challenging because these states may be expressed through inconsistent linguistic, acoustic, f

Advancing Multimodal Judge Models through a Capability-Oriented Benchmark and MCTS-Driven Data Generation

arXiv:2603.00546v2 Announce Type: replace Abstract: Using Multimodal Large Language Models (MLLMs) as judges to achieve precise and consistent evaluations has gradually become an emerging paradigm acr

16 Jul 2026

Towards Enhancing 3D Spatial Reasoning in Medical Multimodal Large Language Models

arXiv:2607.13860v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in 2D medical image understanding, their extension to 3D volumetric

16 Jul 2026

Do You Remember? Toward Memory-Centric Multimodal AI

arXiv:2607.11919v1 Announce Type: cross Abstract: Human memory is reconstructive, not a faithful recording. Current multimodal LLMs (MLLMs) lack this capability: they process images through a frozen v

Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

arXiv:2509.22415v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved strong vision-language performance, yet their token-level visual evidence remains diffi

GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents

arXiv:2607.11192v2 Announce Type: replace Abstract: A large share of day-to-day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, constr

Never too late for any company to get in on open models. Inkling is a MoE transformer with 975B (41B active). Native multimodal support. Con…

Never too late for any company to get in on open models. Inkling is a MoE transformer with 975B (41B active). Native multimodal support. Context lengths: 64K and 256K. Will be testing and providing th

researchdair-ai--x

This is for pretraining based on nemotron nvp4 figures, mfu etc Large context, multimodal, long RL we are seeing now could make it a multipl…

model-releasesemad-mostaque--x

This is for pretraining based on nemotron nvp4 figures, mfu etc Large context, multimodal, long RL we are seeing now could make it a multiple of this Worth noting how good the lite model is, similar t

Local Ai

TSCA-Net: Temporal-Spatial Clique Attention for Interpretable Multimodal Pedestrian Trajectory Prediction

arXiv:2607.11939v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction in crowded environments remains challenging due to the multimodal uncertainty of human motion and the variable

local-aiarxiv-cs-cv

CT-CLIP Representations for Multimodal Lung Cancer Survival Prediction

arXiv:2607.08503v1 Announce Type: new Abstract: Accurate prognosis prediction is important for treatment planning in lung cancer, but deep learning-driven survival modelling is often limited by the sc

Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

arXiv:2603.12478v2 Announce Type: replace Abstract: Multimodal instruction tuning is often compute-inefficient because training budgets are spread across large mixed image-video pools whose utility is

Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer

arXiv:2607.08227v1 Announce Type: new Abstract: Photorealistic Style Transfer (PST) aims to transfer the color and tonal style of a reference to a content image while strictly preserving its structura

UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery

arXiv:2607.08267v1 Announce Type: new Abstract: Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in comple

AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer's Disease Diagnosis

arXiv:2607.07091v1 Announce Type: cross Abstract: In longitudinal Alzheimer's disease (AD) diagnosis support, clinical and imaging information is often collected at irregular visits. Integrating these

Safety

EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

arXiv:2602.23802v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture th

safetyarxiv-cs-ai

Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

arXiv:2607.06651v1 Announce Type: new Abstract: Federated learning (FL) over mobile and edge devices increasingly involves multimodal models in which clients differ in both sensing capability and comp

researcharxiv-cs-lg

From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics

arXiv:2603.26772v2 Announce Type: replace-cross Abstract: Automated semantic annotation of broadcast television content presents distinctive challenges, combining structured audiovisual composition, d

Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG

arXiv:2607.05438v1 Announce Type: cross Abstract: Multimodal retrieval-augmented generation (RAG) grounds a generator in evidence drawn from heterogeneous modalities -- text, tables, and images. The d

8 Jul 2026

Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems

arXiv:2607.06094v1 Announce Type: new Abstract: Faults on a cyber-physical system (CPS) are too rare and unrepresentative to characterise, or even to select a model on, so detection must instead model

researcharxiv-cs-lg

8 Jul 2026

ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models

arXiv:2502.09696v3 Announce Type: replace Abstract: Large Multimodal Models (LMMs) exhibit shortfalls when interpreting images and, by some measures, have poorer spatial cognition than young children

8 Jul 2026

AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes

model-releasesarxiv-cs-cl

arXiv:2607.04410v1 Announce Type: new Abstract: We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder sub

An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble

arXiv:2607.02692v1 Announce Type: new Abstract: Glaucoma is a progressive eye disease that can lead to irreversible vision loss if not detected at an early stage. Conventional diagnostic procedures ar

BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

arXiv:2607.03184v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-res

CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection

arXiv:2607.02930v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full-parameter retraining is computationally expensive as real-world knowled

Applications

Deep Learning for Semen Analysis in Male Infertility: Computer Vision, Multimodal Fusion, and Clinical Translation

arXiv:2607.05311v1 Announce Type: new Abstract: Male infertility contributes substantially to the global infertility burden, and sperm analysis remains central to diagnosis, treatment planning, and as

applicationsarxiv-cs-cv

DOSE-I: A Multimodal Biosignal Dataset of Procedural Sedation for Endoscopy -- Technical Report

arXiv:2607.02570v1 Announce Type: cross Abstract: In this document, we describe characteristics and technical details of the multimodal biosignal dataset DOSE-I of procedural sedation for endoscopy pu

Echoes of Unrest: A Multimodal NLP Framework for Early Warning of Fake News and Violence-Driven Mob Activity

arXiv:2607.02734v1 Announce Type: cross Abstract: Rapid growth in social media has transformed global communication by enabling fast information exchange, but it has also accelerated the spread of mis

EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

arXiv:2607.03934v1 Announce Type: new Abstract: Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-relate