Search: “multimodal”

Feature Extraction in the Remote Sensing Data Value Chain: A Systematic Review of Methods and Applications

arXiv:2510.18935v3 Announce Type: replace Abstract: Earth observation involves collecting, analyzing, and processing an ever-growing mass of data. This planetary data is crucial for addressing relevan

tutorialsarxiv-cs-cv

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding

DGX agent

arXiv:2601.14724v3 Announce Type: replace-cross Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated significant improvement in offline video understanding. Howe

hardwarearxiv-cs-cl

17 Apr 2026

RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework

arXiv:2604.15308v1 Announce Type: new Abstract: High-level autonomous driving requires motion planners capable of modeling multimodal future uncertainties while remaining robust in closed-loop interac

ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold

arXiv:2604.13392v1 Announce Type: new Abstract: Tabular data remains prevalent in high-stakes domains such as healthcare and finance, where predictive models are expected to provide both high accuracy

To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs

arXiv:2603.18373v2 Announce Type: replace Abstract: When VLMs answer correctly, do they genuinely rely on visual information or exploit language shortcuts? We introduce the Tri-Layer Diagnostic Framew

Towards Design Compositing

arXiv:2604.14605v1 Announce Type: new Abstract: Graphic design creation involves harmoniously assembling multimodal components such as images, text, logos, and other visual assets collected from diver

Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions

model-releasesarxiv-cs-cl

arXiv:2405.19088v3 Announce Type: replace Abstract: Recent advancements in large multimodal language models have demonstrated remarkable proficiency across a wide range of tasks. Yet, these models sti

Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET

arXiv:2406.12632v3 Announce Type: replace-cross Abstract: Positron emission tomography (PET) provides molecular biomarkers for Alzheimer's disease and related dementias (ADRD) and is increasingly used

fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding

model-releasesarxiv-cs-cl

arXiv:2511.21760v3 Announce Type: replace Abstract: Recent advances in multimodal large language models (LLMs) have enabled unified reasoning across images, audio, and video, but extending such capabi

Hardware

Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Model

arXiv:2603.28554v2 Announce Type: replace Abstract: Visual document understanding typically requires separate retrieval and generation models, doubling memory and system complexity. We present Hydra,

hardwarearxiv-cs-cv

Hardware

MaMe & MaRe: Matrix-Based Token Merging and Restoration for Efficient Visual Perception and Synthesis

arXiv:2604.13432v1 Announce Type: new Abstract: Token compression is crucial for mitigating the quadratic complexity of self-attention mechanisms in Vision Transformers (ViTs), which often involve num

hardwarearxiv-cs-cv

Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges

model-releasesarxiv-cs-lg

arXiv:2604.13602v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) and related alignment paradigms have become central to steering large language models (LLMs) and multi

See&Say: Vision Language Guided Safe Zone Detection for Autonomous Package Delivery Drones

arXiv:2604.13292v1 Announce Type: new Abstract: Autonomous drone delivery systems are rapidly advancing, but ensuring safe and reliable package drop-offs remains highly challenging in cluttered urban

SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs

arXiv:2604.13710v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) exhibit strong reasoning and world knowledge, yet adapting them for retrieval remains challenging. Existing app

Local Ai

UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding

arXiv:2604.14113v1 Announce Type: cross Abstract: GUI grounding, which localizes interface elements from screenshots given natural language queries, remains challenging for small icons and dense layou

local-aiarxiv-cs-cl

UNBOX: Unveiling Black-box visual models with Natural-language

arXiv:2603.08639v2 Announce Type: replace Abstract: Ensuring trustworthiness in open-world visual recognition requires models that are interpretable, fair, and robust to distribution shifts. Yet moder

Agentic Discovery with Active Hypothesis Exploration for Visual Recognition

arXiv:2604.12999v1 Announce Type: new Abstract: We introduce HypoExplore, an agentic framework that formulates neural architecture discovery for visual recognition as a hypothesis-driven scientific in

agentsarxiv-cs-cv

Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving

arXiv:2510.00919v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for exper

Boosting Visual Instruction Tuning with Self-Supervised Guidance

arXiv:2604.12966v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) perform well on many vision-language tasks but often struggle with vision-centric problems that require fine-gr

Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

arXiv:2604.12896v1 Announce Type: new Abstract: Multimodal language models (MLLMs) are increasingly paired with vision tools (e.g., depth, flow, correspondence) to enhance visual reasoning. However, d

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment

arXiv:2604.12813v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) have shown promising performance on video quality assessment (VQA) tasks. However, adapting them to new

From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception

arXiv:2604.12508v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding, they frequently falter in fine

HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Models

arXiv:2604.12115v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong multimodal performance, but still suffer from hallucinations caused by unstable visual grounding and

Visual Preference Optimization with Rubric Rewards

arXiv:2604.13029v1 Announce Type: cross Abstract: The effectiveness of Direct Preference Optimization (DPO) depends on preference data that reflect the quality differences that matter in multimodal ta

Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis

arXiv:2604.10233v1 Announce Type: cross Abstract: 3D medical image analysis is of great importance in disease diagnosis and treatment. Recently, multimodal large language models (MLLMs) have exhibited

BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs

arXiv:2604.10528v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) demonstrate remarkable zero-shot recognition capabilities across a diverse spectrum of multimodal tasks, it yet rema

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games

arXiv:2604.11741v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer

agentsarxiv-cs-ai

Counting to Four is still a Chore for VLMs

arXiv:2604.10039v1 Announce Type: new Abstract: Vision--language models (VLMs) have achieved impressive performance on complex multimodal reasoning tasks, yet they still fail on simple grounding skill

Descriptor-Injected Cross-Modal Learning: A Systematic Exploration of Audio-MIDI Alignment via Spectral and Melodic Features

arXiv:2604.10283v1 Announce Type: cross Abstract: Cross-modal retrieval between audio recordings and symbolic music representations (MIDI) remains challenging because continuous waveforms and discrete

safetyarxiv-cs-lg

Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate

arXiv:2604.11258v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) in healthcare suffer from severe confirmation bias, often hallucinating visual details to support initial, pote

safetyarxiv-cs-cl

Applications

Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms

arXiv:2604.09799v1 Announce Type: cross Abstract: Human activity recognition (HAR) has become a key component of intelligent systems for healthcare monitoring, assistive living, smart environments, an

applicationsarxiv-cs-ai

GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agents

arXiv:2603.24329v2 Announce Type: replace-cross Abstract: Multimodal LLMs are increasingly deployed as perceptual backbones for autonomous agents in 3D environments, from robotics to virtual worlds. T

agentsarxiv-cs-ai

Applications

Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language

arXiv:2604.11600v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress but continue to struggle with geometric reasoning, primarily due to the perce

applicationsarxiv-cs-cv

Improving the Reasoning of Multi-Image Grounding in MLLMs via Reinforcement Learning

arXiv:2507.00748v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) perform well in single-image visual grounding but struggle with real-world tasks that demand cross-image re

tutorialsarxiv-cs-cv

Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs

arXiv:2603.27494v2 Announce Type: replace-cross Abstract: To enhance the perception and reasoning capabilities of multimodal large language models in complex visual scenes, recent research has introdu

agentsarxiv-cs-ai

Learning to Forget -- Hierarchical Episodic Memory for Lifelong Robot Deployment

arXiv:2604.11306v1 Announce Type: cross Abstract: Robots must verbalize their past experiences when users ask 'Where did you put my keys?' or 'Why did the task fail?' Yet maintaining life-long episodi

LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loops

arXiv:2506.14493v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown great promise but require substantial computational resources during inference. Attackers can ex

researcharxiv-cs-cl

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

arXiv:2604.10024v1 Announce Type: cross Abstract: Long video summarization presents significant challenges for current multimodal large language models (MLLMs), particularly in maintaining temporal fi

MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languages

model-releasesarxiv-cs-cl

arXiv:2512.01512v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved great success in Speech-to-Text Translation (S2TT) tasks. However, current research is constr

MCGA: A Multi-task Classical Chinese Literary Genre Audio Corpus

arXiv:2601.09270v3 Announce Type: replace Abstract: With the rapid advancement of Multimodal Large Language Models (MLLMs), their potential has gained significant attention in Chinese Classical Studie

researcharxiv-cs-cl

MGA: Memory-Driven GUI Agent for Observation-Centric Interaction

arXiv:2510.24168v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have significantly advanced GUI agents, yet long-horizon automation remains constrained by two critical bot

safetyarxiv-cs-ai

MLLM-as-a-Judge Exhibits Model Preference Bias

arXiv:2604.11589v1 Announce Type: new Abstract: Automatic evaluation using multimodal large language models (MLLMs), commonly referred to as MLLM-as-a-Judge, has been widely used to measure model perf

Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization

arXiv:2604.11259v1 Announce Type: new Abstract: Mobile GUI agents powered by Multimodal Large Language Models (MLLMs) can execute complex tasks on mobile devices. Despite this progress, most existing

NeuroPath: Practically Adopting Motor Imagery Decoding through EEG Signals

arXiv:2604.09654v1 Announce Type: cross Abstract: Motor Imagery (MI) is an emerging Brain-Computer Interface (BCI) paradigm where a person imagines body movements without physical action. By decoding

OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Video

arXiv:2604.11102v1 Announce Type: new Abstract: Current multimodal large language models (MLLMs) have demonstrated remarkable capabilities in short-form video understanding, yet translating long-form

Applications

POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

arXiv:2604.11627v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable capabilities in cross-modal understanding and generation. However, the ra

applicationsarxiv-cs-cv

Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning

arXiv:2604.11407v1 Announce Type: cross Abstract: We revisit retrieval-augmented generation (RAG) by embedding retrieval control directly into generation. Instead of treating retrieval as an external

Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utility

arXiv:2602.03402v3 Announce Type: replace Abstract: Vision language models (VLMs) extend the reasoning capabilities of large language models (LLMs) to cross-modal settings, yet remain highly vulnerabl

safetyarxiv-cs-ai