Search: “multimodal”

arXiv:2607.11939v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction in crowded environments remains challenging due to the multimodal uncertainty of human motion and the variable

local-aiarxiv-cs-cv

15 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

CT-CLIP Representations for Multimodal Lung Cancer Survival Prediction

DGX agent

arXiv:2607.08503v1 Announce Type: new Abstract: Accurate prognosis prediction is important for treatment planning in lung cancer, but deep learning-driven survival modelling is often limited by the sc

researcharxiv-cs-cv

10 Jul 2026

Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

arXiv:2603.12478v2 Announce Type: replace Abstract: Multimodal instruction tuning is often compute-inefficient because training budgets are spread across large mixed image-video pools whose utility is

10 Jul 2026

Multimodal 3D LUT Generation via StatLUT with Statistical Features for Photorealistic Style Transfer

arXiv:2607.08227v1 Announce Type: new Abstract: Photorealistic Style Transfer (PST) aims to transfer the color and tonal style of a reference to a content image while strictly preserving its structura

10 Jul 2026

UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery

arXiv:2607.08267v1 Announce Type: new Abstract: Unmanned aerial vehicles (UAVs) increasingly rely on visual grounding capabilities to localize task-relevant targets from diverse instructions in comple

10 Jul 2026

AT-Attn: Temporal-Aware Cross-Attention for Longitudinal Multimodal Alzheimer's Disease Diagnosis

arXiv:2607.07091v1 Announce Type: cross Abstract: In longitudinal Alzheimer's disease (AD) diagnosis support, clinical and imaging information is often collected at irregular visits. Integrating these

EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

arXiv:2602.23802v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture th

Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

arXiv:2607.06651v1 Announce Type: new Abstract: Federated learning (FL) over mobile and edge devices increasingly involves multimodal models in which clients differ in both sensing capability and comp

researcharxiv-cs-lg

From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics

arXiv:2603.26772v2 Announce Type: replace-cross Abstract: Automated semantic annotation of broadcast television content presents distinctive challenges, combining structured audiovisual composition, d

Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG

arXiv:2607.05438v1 Announce Type: cross Abstract: Multimodal retrieval-augmented generation (RAG) grounds a generator in evidence drawn from heterogeneous modalities -- text, tables, and images. The d

8 Jul 2026

Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems

arXiv:2607.06094v1 Announce Type: new Abstract: Faults on a cyber-physical system (CPS) are too rare and unrepresentative to characterise, or even to select a model on, so detection must instead model

researcharxiv-cs-lg

8 Jul 2026

ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models

arXiv:2502.09696v3 Announce Type: replace Abstract: Large Multimodal Models (LMMs) exhibit shortfalls when interpreting images and, by some measures, have poorer spatial cognition than young children

8 Jul 2026

AI Wizards at EXIST 2026: Hierarchical Soft-Label Learning for Multimodal Sexism Identification in Memes

model-releasesarxiv-cs-cl

arXiv:2607.04410v1 Announce Type: new Abstract: We present the AI Wizards submission to EXIST 2026 for multimodal sexism identification in memes. The task is composed of three, increasingly harder sub

An Automated Multimodal Glaucoma Detection Framework Using ViT and a Stacking-Based Ensemble

arXiv:2607.02692v1 Announce Type: new Abstract: Glaucoma is a progressive eye disease that can lead to irreversible vision loss if not detected at an early stage. Conventional diagnostic procedures ar

BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

arXiv:2607.03184v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-res

CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection

arXiv:2607.02930v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full-parameter retraining is computationally expensive as real-world knowled

Applications

Deep Learning for Semen Analysis in Male Infertility: Computer Vision, Multimodal Fusion, and Clinical Translation

arXiv:2607.05311v1 Announce Type: new Abstract: Male infertility contributes substantially to the global infertility burden, and sperm analysis remains central to diagnosis, treatment planning, and as

applicationsarxiv-cs-cv

DOSE-I: A Multimodal Biosignal Dataset of Procedural Sedation for Endoscopy -- Technical Report

arXiv:2607.02570v1 Announce Type: cross Abstract: In this document, we describe characteristics and technical details of the multimodal biosignal dataset DOSE-I of procedural sedation for endoscopy pu

Echoes of Unrest: A Multimodal NLP Framework for Early Warning of Fake News and Violence-Driven Mob Activity

arXiv:2607.02734v1 Announce Type: cross Abstract: Rapid growth in social media has transformed global communication by enabling fast information exchange, but it has also accelerated the spread of mis

EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

arXiv:2607.03934v1 Announce Type: new Abstract: Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-relate

EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation

arXiv:2508.16170v2 Announce Type: replace-cross Abstract: MultiModal Recommendation (MMR) systems have emerged as a promising solution for improving recommendation quality by leveraging rich item-side

Applications

GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

arXiv:2508.03750v2 Announce Type: replace-cross Abstract: Early and accurate glaucoma detection is critical to prevent irreversible vision loss, yet existing AI methods often rely on unimodal inputs a

applicationsarxiv-cs-cv

SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning

model-releasesarxiv-cs-cl

arXiv:2603.23483v2 Announce Type: replace-cross Abstract: Agentic multimodal large language models (MLLMs) (e.g., OpenAI o3 and Gemini Agentic Vision) achieve remarkable reasoning capabilities through

TexTailor: Inference-Time Textual Guidance Tailoring for Multimodal Diffusion Transformers

arXiv:2601.02211v2 Announce Type: replace Abstract: Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and

The P^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

arXiv:2505.15379v2 Announce Type: replace Abstract: We present the P^3 dataset, a large-scale multimodal benchmark for building vectorization, constructed from aerial LiDAR point clouds, high-resoluti

Transferability Between Understanding and Generation in Unified Multimodal Models

arXiv:2607.04423v1 Announce Type: cross Abstract: Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains und

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

arXiv:2511.20272v2 Announce Type: replace Abstract: While Multimodal Large Language Models (MLLMs) have become adept at recognizing objects, they often lack the intuitive, human-like understanding of

Agents

Web-CogReasoner: Towards Multimodal Knowledge-Induced Cognitive Reasoning for Web Agents

arXiv:2508.01858v3 Announce Type: replace-cross Abstract: Multimodal large-scale models have significantly advanced the development of web agents, enabling perception and interaction with digital envi

agentsarxiv-cs-ai

WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling

arXiv:2607.03461v1 Announce Type: new Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in

MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support

arXiv:2607.01814v1 Announce Type: new Abstract: Traditional Chinese Medicine (TCM) diagnosis, particularly through tongue inspection, faces persistent challenges in subjectivity and reproducibility. T

3 Jul 2026

Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners

arXiv:2607.00125v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable abilities when analyzing images, yet translating these capabilities to few-shot im

2 Jul 2026

Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference

arXiv:2606.31903v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation. Existing acce

Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios

arXiv:2601.21173v2 Announce Type: replace-cross Abstract: With the rapid development of industrial intelligence and unmanned inspection, reliable perception and safety assessment for AI systems in com

On Optimizing Multimodal Jailbreaks for Spoken Language Models

arXiv:2603.19127v2 Announce Type: replace Abstract: As Spoken Language Models (SLMs) integrate speech and text modalities, they inherit the safety vulnerabilities of their LLM backbone while introduci

safetyarxiv-cs-lg

SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search

arXiv:2606.31504v1 Announce Type: new Abstract: We present SimpleSearch-VL, an efficient, reliable, and practical framework for multimodal agentic search. Its core idea is to improve the agent's own s

UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

arXiv:2606.31451v1 Announce Type: cross Abstract: Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities. However, existing res

Can AI Draw Science? A Benchmark for Evaluating Scientific Figure Generation by Text-to-Image and Multimodal Models

arXiv:2606.28406v1 Announce Type: cross Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design sch

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

arXiv:2606.29805v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are prone to hallucination as their generation preferences are insufficiently calibrated to visual evidence, ca

safetyarxiv-cs-cv

GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

arXiv:2606.28514v1 Announce Type: new Abstract: Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents. Existing benchmarks show that these m

MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory

model-releasesarxiv-cs-lg

arXiv:2606.29788v1 Announce Type: new Abstract: When a multimodal AI agent is asked to forget a fact, current memory systems usually delete the text entry and report success. We find that the fact can

Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement

arXiv:2403.06728v2 Announce Type: replace Abstract: Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. The performance o

safetyarxiv-cs-cv

Reliability-Prioritized Fine-Grained Generation in Multimodal Large

arXiv:2606.29573v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theo

SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

arXiv:2510.12784v2 Announce Type: replace-cross Abstract: Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding

safetyarxiv-cs-cl

X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining

arXiv:2606.14752v2 Announce Type: replace-cross Abstract: Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing act

AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration

arXiv:2606.28049v1 Announce Type: new Abstract: In recent years, multimodal large language models (MLLMs) have shown strong potential for embodied intelligence, yet their ability to maintain geometric

Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering

arXiv:2606.16494v2 Announce Type: replace-cross Abstract: Knowledge-based visual question answering (KB-VQA) lets vision-language systems answer questions that exceed their parametric knowledge by con

MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy

arXiv:2606.27652v1 Announce Type: new Abstract: We find that explicit reasoning does not necessarily translate into better multimodal emotion recognition (MER) accuracy, even though it makes predictio

Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting

arXiv:2602.01588v3 Announce Type: replace-cross Abstract: Multimodal time series forecasting is crucial in real-world applications, where decisions depend on both numerical data and contextual signals