Search: “multimodal”

UNBOX: Unveiling Black-box visual models with Natural-language

arXiv:2603.08639v2 Announce Type: replace Abstract: Ensuring trustworthiness in open-world visual recognition requires models that are interpretable, fair, and robust to distribution shifts. Yet moder

safetyarxiv-cs-cv

16 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agentic Discovery with Active Hypothesis Exploration for Visual Recognition

DGX agent

arXiv:2604.12999v1 Announce Type: new Abstract: We introduce HypoExplore, an agentic framework that formulates neural architecture discovery for visual recognition as a hypothesis-driven scientific in

agentsarxiv-cs-cv

15 Apr 2026

Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving

arXiv:2510.00919v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for exper

Boosting Visual Instruction Tuning with Self-Supervised Guidance

arXiv:2604.12966v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) perform well on many vision-language tasks but often struggle with vision-centric problems that require fine-gr

Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

arXiv:2604.12896v1 Announce Type: new Abstract: Multimodal language models (MLLMs) are increasingly paired with vision tools (e.g., depth, flow, correspondence) to enhance visual reasoning. However, d

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment

arXiv:2604.12813v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) have shown promising performance on video quality assessment (VQA) tasks. However, adapting them to new

From Attenuation to Attention: Variational Information Flow Manipulation for Fine-Grained Visual Perception

arXiv:2604.12508v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding, they frequently falter in fine

HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Models

arXiv:2604.12115v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong multimodal performance, but still suffer from hallucinations caused by unstable visual grounding and

Visual Preference Optimization with Rubric Rewards

arXiv:2604.13029v1 Announce Type: cross Abstract: The effectiveness of Direct Preference Optimization (DPO) depends on preference data that reflect the quality differences that matter in multimodal ta

Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis

arXiv:2604.10233v1 Announce Type: cross Abstract: 3D medical image analysis is of great importance in disease diagnosis and treatment. Recently, multimodal large language models (MLLMs) have exhibited

BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs

arXiv:2604.10528v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) demonstrate remarkable zero-shot recognition capabilities across a diverse spectrum of multimodal tasks, it yet rema

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games

arXiv:2604.11741v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer

agentsarxiv-cs-ai

Counting to Four is still a Chore for VLMs

arXiv:2604.10039v1 Announce Type: new Abstract: Vision--language models (VLMs) have achieved impressive performance on complex multimodal reasoning tasks, yet they still fail on simple grounding skill

Descriptor-Injected Cross-Modal Learning: A Systematic Exploration of Audio-MIDI Alignment via Spectral and Melodic Features

arXiv:2604.10283v1 Announce Type: cross Abstract: Cross-modal retrieval between audio recordings and symbolic music representations (MIDI) remains challenging because continuous waveforms and discrete

safetyarxiv-cs-lg

Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debate

arXiv:2604.11258v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) in healthcare suffer from severe confirmation bias, often hallucinating visual details to support initial, pote

safetyarxiv-cs-cl

Applications

Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms

arXiv:2604.09799v1 Announce Type: cross Abstract: Human activity recognition (HAR) has become a key component of intelligent systems for healthcare monitoring, assistive living, smart environments, an

applicationsarxiv-cs-ai

Local Ai

Forget about VAEs? SenseNova's NEO-unify achieves 31.5 PSNR without an encoder – Native Image Gen is coming.

local-air-stablediffusion

SenseNova's NEO-unify is an encoder-free unified multimodal model built on a Mixture-of-Transformers (MoT) backbone that eliminates the need for traditional VAE encoders in image generation. The 2B NE

GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agents

arXiv:2603.24329v2 Announce Type: replace-cross Abstract: Multimodal LLMs are increasingly deployed as perceptual backbones for autonomous agents in 3D environments, from robotics to virtual worlds. T

agentsarxiv-cs-ai

Applications

Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language

arXiv:2604.11600v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress but continue to struggle with geometric reasoning, primarily due to the perce

applicationsarxiv-cs-cv

Improving the Reasoning of Multi-Image Grounding in MLLMs via Reinforcement Learning

arXiv:2507.00748v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) perform well in single-image visual grounding but struggle with real-world tasks that demand cross-image re

tutorialsarxiv-cs-cv

Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs

arXiv:2603.27494v2 Announce Type: replace-cross Abstract: To enhance the perception and reasoning capabilities of multimodal large language models in complex visual scenes, recent research has introdu

agentsarxiv-cs-ai

Learning to Forget -- Hierarchical Episodic Memory for Lifelong Robot Deployment

arXiv:2604.11306v1 Announce Type: cross Abstract: Robots must verbalize their past experiences when users ask 'Where did you put my keys?' or 'Why did the task fail?' Yet maintaining life-long episodi

tutorialsarxiv-cs-ai

LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loops

arXiv:2506.14493v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown great promise but require substantial computational resources during inference. Attackers can ex

researcharxiv-cs-cl

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

arXiv:2604.10024v1 Announce Type: cross Abstract: Long video summarization presents significant challenges for current multimodal large language models (MLLMs), particularly in maintaining temporal fi

MCAT: Scaling Many-to-Many Speech-to-Text Translation with MLLMs to 70 Languages

model-releasesarxiv-cs-cl

arXiv:2512.01512v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved great success in Speech-to-Text Translation (S2TT) tasks. However, current research is constr

MCGA: A Multi-task Classical Chinese Literary Genre Audio Corpus

arXiv:2601.09270v3 Announce Type: replace Abstract: With the rapid advancement of Multimodal Large Language Models (MLLMs), their potential has gained significant attention in Chinese Classical Studie

researcharxiv-cs-cl

MGA: Memory-Driven GUI Agent for Observation-Centric Interaction

arXiv:2510.24168v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have significantly advanced GUI agents, yet long-horizon automation remains constrained by two critical bot

MLLM-as-a-Judge Exhibits Model Preference Bias

arXiv:2604.11589v1 Announce Type: new Abstract: Automatic evaluation using multimodal large language models (MLLMs), commonly referred to as MLLM-as-a-Judge, has been widely used to measure model perf

Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization

arXiv:2604.11259v1 Announce Type: new Abstract: Mobile GUI agents powered by Multimodal Large Language Models (MLLMs) can execute complex tasks on mobile devices. Despite this progress, most existing

NeuroPath: Practically Adopting Motor Imagery Decoding through EEG Signals

arXiv:2604.09654v1 Announce Type: cross Abstract: Motor Imagery (MI) is an emerging Brain-Computer Interface (BCI) paradigm where a person imagines body movements without physical action. By decoding

tutorialsarxiv-cs-ai

OmniScript: Towards Audio-Visual Script Generation for Long-Form Cinematic Video

arXiv:2604.11102v1 Announce Type: new Abstract: Current multimodal large language models (MLLMs) have demonstrated remarkable capabilities in short-form video understanding, yet translating long-form

Applications

POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

arXiv:2604.11627v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable capabilities in cross-modal understanding and generation. However, the ra

applicationsarxiv-cs-cv

Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning

arXiv:2604.11407v1 Announce Type: cross Abstract: We revisit retrieval-augmented generation (RAG) by embedding retrieval control directly into generation. Instead of treating retrieval as an external

tutorialsarxiv-cs-ai

Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utility

arXiv:2602.03402v3 Announce Type: replace Abstract: Vision language models (VLMs) extend the reasoning capabilities of large language models (LLMs) to cross-modal settings, yet remain highly vulnerabl

RL makes MLLMs see better than SFT

arXiv:2510.16333v2 Announce Type: replace Abstract: A dominant assumption in Multimodal Language Model (MLLM) research is that its performance is largely inherited from the LLM backbone, given its imm

Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding

arXiv:2604.11244v1 Announce Type: new Abstract: Advances in Multimodal Large Language Models (MLLMs) are transforming video captioning from a descriptive endpoint into a semantic interface for both vi

Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understanding

arXiv:2604.11122v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated immense potential in Earth observation. However, the massive visual tokens generated when p

Sign Language Recognition in the Age of LLMs

model-releasesarxiv-cs-cl

arXiv:2604.11225v1 Announce Type: cross Abstract: Recent Vision Language Models (VLMs) have demonstrated strong performance across a wide range of multimodal reasoning tasks. This raises the question

TacMan-Turbo: Proactive Tactile Control for Robust and Efficient Articulated Object Manipulation

arXiv:2508.02204v4 Announce Type: replace Abstract: Adept manipulation of articulated objects is essential for robots to operate successfully in human environments. Such manipulation requires both eff

tutorialsarxiv-cs-ro

UNIGEOCLIP: Unified Geospatial Contrastive Learning

arXiv:2604.11668v1 Announce Type: new Abstract: The growing availability of co-located geospatial data spanning aerial imagery, street-level views, elevation models, text, and geographic coordinates o

safetyarxiv-cs-cv

VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments

arXiv:2506.02387v3 Announce Type: replace Abstract: Recent advancements in Vision Language Models (VLMs) have expanded their capabilities to interactive agent tasks, yet existing benchmarks remain lim

Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations

arXiv:2506.09067v2 Announce Type: replace-cross Abstract: Generative medical vision-language models~(Med-VLMs) are primarily designed to generate complex textual information~(e.g., diagnostic reports)

Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models

arXiv:2503.14075v3 Announce Type: replace-cross Abstract: Large vision-language models (VLMs) have demonstrated remarkable capabilities in open-world multimodal understanding, yet their high computati

researcharxiv-cs-cl

Learning Vision-Language-Action World Models for Autonomous Driving

arXiv:2604.09059v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving by integrating perception, reasoning, and

Local Ai

llama4 108b

This Reddit thread on r/ollama discusses running Meta's Llama 4 Maverick — a ~108B parameter model — locally using Ollama. Llama 4 models are natively multimodal AI models supporting text and image un

local-air-ollama

Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch

arXiv:2604.09100v1 Announce Type: new Abstract: We propose a multimodal, physically grounded approach for metric-scale amodal object reconstruction and pose estimation under severe hand occlusion. Unl

tutorialsarxiv-cs-cv

PinpointQA: A Dataset and Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos

arXiv:2604.08991v1 Announce Type: cross Abstract: Small object-centric spatial understanding in indoor videos remains a significant challenge for multimodal large language models (MLLMs), despite its

Streaming Video Instruction Tuning

arXiv:2512.21334v2 Announce Type: replace Abstract: We present Streamo, a real-time streaming video LLM that serves as a general-purpose interactive assistant. Unlike existing online video models that