Search: “multimodal”

VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation

model-releasesarxiv-cs-ai

arXiv:2609.37709v1 Announce Type: cross Abstract: Recent multimodal image generation models can take multiple images and textual instructions as input, enabling reference-based generation guided not o

30 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Visual Parallel Search: Learning to Search High-Resolution Images with Parallel Tile Inspection and Adaptive Zoom

DGX agent

arXiv:2609.37002v1 Announce Type: cross Abstract: High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to an

model-releasesarxiv-cs-ai

30 Sept 2026

Local Ai

Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack

arXiv:2609.37096v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. In

local-aiarxiv-cs-cv

30 Sept 2026

CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation

model-releasesarxiv-cs-ro

arXiv:2609.33807v1 Announce Type: new Abstract: How well can general-purpose multimodal models turn visual understanding and reasoning into embodied manipulation via executable code? We introduce Code

Constrained Flow Policy Updates: A Generalized Schrodinger Bridge View

arXiv:2609.32952v1 Announce Type: new Abstract: Online safe reinforcement learning (RL) seeks policies that maximize reward while satisfying safety constraints. Reward and safety can induce multimodal

Counterfactual Attention Policy Distillation for Temporal Video Grounding

arXiv:2609.34581v1 Announce Type: new Abstract: Temporal video grounding is a key capability of advanced Multimodal Large Language Models (MLLMs) for the thorough understanding of video events, which

Data Processing for Offline Evaluation in Recommender Systems: a Survey

arXiv:2609.31696v1 Announce Type: cross Abstract: Offline evaluation is the dominant experimental paradigm in recommender systems research, enabling reproducible and cost-effective comparisons on hist

researcharxiv-cs-lg

Diffusion Reward Models

arXiv:2609.33803v1 Announce Type: new Abstract: Reward models underpin the alignment of large language models, yet the dominant designs reduce each prompt--response pair to a point estimate or to a di

ECG-Scroll: A Long-Horizon, Streaming Benchmark and Agent Environment for Interpretation of Ambulatory Electrocardiograms

model-releasesarxiv-cs-lg

arXiv:2609.33117v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can now interpret a standard ten-second, twelve-lead electrocardiogram (ECG) with clinically grounded, reward-v

Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction

arXiv:2609.32353v1 Announce Type: new Abstract: Visual token reduction is an effective way to accelerate multimodal large language models (MLLMs), but performance deteriorates rapidly under extremely

FSS-UBrain: Multi-region Few-Shot Brain Tumor MRI Segmentation

arXiv:2609.32273v1 Announce Type: new Abstract: Accurate delineation of whole tumor (WT), tumor core (TC), and enhancing tumor (ET) from multimodal magnetic resonance imaging remains challenging under

InfoEdit: Probing Global Layout Reasoning in Infographic Editing

arXiv:2609.33286v1 Announce Type: new Abstract: Multimodal foundation models edit natural photographs at production quality, yet the same models struggle with structured visual content such as infogra

K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings

arXiv:2609.34082v1 Announce Type: cross Abstract: Interpreting architecture, engineering, and construction (AEC) drawings is hard for general Multimodal Large Language Models (MLLMs) and vision-langua

Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

arXiv:2609.35767v1 Announce Type: new Abstract: Unified multimodal models can both look at and render images, so in principle they can repair their own generations: diagnose what an image gets wrong,

Learning Transferable Sensor Models via Language-Informed Pretraining

arXiv:2603.11950v2 Announce Type: replace-cross Abstract: Multimodal language models have demonstrated strong semantic understanding and reasoning over physiological and behavioral signals captured fr

Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

arXiv:2609.35536v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can explain deepfake verdicts in natural language, but such explanations are not necessarily visually grounded

MetaSampling: Making Frame Samplers Efficient for Long-Video Question Answering

arXiv:2609.33998v1 Announce Type: new Abstract: Frame selection is an important component of long-video question answering (VQA) with Multimodal Large Language Models (MLLMs). Existing frame-selection

NavHarness: Towards Lifelong Embodied Navigation

arXiv:2609.34276v1 Announce Type: new Abstract: Frontier models can now perform well on individual embodied navigation tasks through multi-round multimodal reasoning with simple tools. Across successi

agentsarxiv-cs-ro

One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs

arXiv:2609.33833v1 Announce Type: new Abstract: Adversarial attacks have long posed a fundamental threat to machine learning systems. As multimodal large language models (MLLMs) rapidly evolve and bec

OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories

model-releasesarxiv-cs-lg

arXiv:2609.32810v1 Announce Type: cross Abstract: Multidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions, yet bench

OPERA: A Unified Omnimodal Progressive Spatio-Temporal Reasoning Agent for Referring Video Segmentation

arXiv:2609.33338v1 Announce Type: new Abstract: Referring video segmentation with heterogeneous multimodal queries---spanning text, audio, and reference images---demands both robust cross-modal unders

agentsarxiv-cs-cv

P4Q: Co-designing Token Pruning and Quantization for Vision-Language Model Acceleration

arXiv:2609.34867v1 Announce Type: new Abstract: Vision language models have achieved strong performance across a wide range of multimodal applications, yet their substantial computational and memory c

Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence

arXiv:2609.34563v1 Announce Type: new Abstract: Latent visual reasoning (LVR) enables multimodal large language models (MLLMs) to perform intermediate computation in continuous latent tokens rather th

Local Ai

Revisit to Segment: Working Memory Distillation for Reasoning Segmentation

arXiv:2609.34863v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have approached image segmentation by reasoning about visual content and predicting target locations. Their gen

local-aiarxiv-cs-cv

RoboFFT: Finetuning generative robot policy via online reinforcement learning with forward process

arXiv:2609.32236v1 Announce Type: new Abstract: Generative models, such as diffusion and flow-based models, have shown strong promise for robot policy learning by capturing complex and multimodal acti

safetyarxiv-cs-ro

Robot Manipulation with GPT-6-Astra: Body Knowledge, Experience Reuse, Emergent Skills, and Sim2Real Transfer

arXiv:2609.31770v1 Announce Type: cross Abstract: General-purpose multimodal agents can write robot-control programs, but repeated exploration and model-mediated action selection can make execution sl

agentsarxiv-cs-cv

SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding

arXiv:2609.33518v1 Announce Type: new Abstract: Recent 3D large multimodal models (3D-LMMs) rely on a visual bottleneck to compress complex 3D scene evidence into a limited number of visual tokens com

Schrodinger--Follmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis

arXiv:2609.33239v1 Announce Type: cross Abstract: Diffusion policies represent multimodal action distributions, but an advantage-weighted update does not specify how to sample from the resulting targe

Seeing and Solving Are Not Enough for Vision-Language Models

arXiv:2609.33694v1 Announce Type: new Abstract: Vision-language models (VLMs) answer visual questions by combining visual information extraction with downstream problem solving. We investigate a funda

SIMANF: Sample Free Learning of Unnormalized Distributions via Simulated Annealing in Normalizing Flows

arXiv:2609.32279v1 Announce Type: new Abstract: Efficiently learning and sampling from high dimensional, multimodal unnormalized distributions without target samples remains a challenging problem. Alt

researcharxiv-cs-lg

Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding

arXiv:2609.35497v1 Announce Type: new Abstract: Long video understanding relies on video memory to overcome the context limits of multimodal large language models. Existing methods follow a build-then

agentsarxiv-cs-cv

SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization

arXiv:2609.34884v1 Announce Type: new Abstract: Post-training quantization reduces the deployment cost of vision-language models (VLMs), but preserving multimodal capabilities at low bit widths remain

Synergizing Discriminative Exemplars and Self-Refined Experience for MLLM-based In-Context Learning in Medical Diagnosis

arXiv:2603.27737v2 Announce Type: replace Abstract: General Multimodal Large Language Models (MLLMs) often underperform in capturing domain-specific nuances in medical diagnosis, trailing behind fully

The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding

arXiv:2609.31747v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) must balance local detail against scene context when interpreting ultra-high-resolution (UHR) remote sensing (R

Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity

arXiv:2510.15508v2 Announce Type: replace Abstract: In this study, we propose an enhancement to the similarity computation mechanism in multimodal contrastive pretraining frameworks such as CLIP. Prio

researcharxiv-cs-lg

VLM-Guided Experience Replay

arXiv:2602.01915v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) have enabled powerful semantic and multimodal reasoning capabiliti

agentsarxiv-cs-lg

ZonoGPT: Towards An Abstract Domain for Verifying Large GPT Models

arXiv:2609.34457v1 Announce Type: new Abstract: Transformer-based models are widely used for reasoning, coding, and multimodal agentic tasks. To provide formal assurance of desirable behaviors, such a

Local Ai

Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding

arXiv:2609.31135v1 Announce Type: cross Abstract: Spatio-Temporal Video Grounding (STVG) aims to localize the spatio-temporal tube in a video corresponding to a natural language query. While recent me

local-aiarxiv-cs-ai

28 Sept 2026

Applications

Provable Speech Attributes Conversion via Latent Independence

arXiv:2510.05191v3 Announce Type: replace-cross Abstract: Conditional generation and disentangled representation learning are central to controlled generation across audio, vision, and multimodal doma

applicationsarxiv-cs-ai

28 Sept 2026

SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion

arXiv:2609.30703v1 Announce Type: cross Abstract: Spatial misregistration and cross-modal discrepancies often cause ghosting, structural blurring, and content imbalance in RGB-T fusion. Existing metho

safetyarxiv-cs-ai

28 Sept 2026

internlm/Intern-Decision 4B and 0.8B

model-releasesr-localllama

https://huggingface.co/internlm/Intern-Decision-0.8B Intern-Decision-4B Demo | Model Weights | GitHub Intern-Decision-4B is a multimodal structured decision model fine-tuned from Qwen3.5-4B. It accept

26 Sept 2026

ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts

arXiv:2609.29349v1 Announce Type: cross Abstract: ArGuard is a shared task on harmful content detection in Arabic memes and LLM prompts. It includes two tracks: Track A focuses on multimodal hate dete

safetyarxiv-cs-ai

CrossScale-GLIO: Topology-Preserving Vision-Language Alignment of MRI and Whole-Slide Histopathology for Diffuse Glioma

arXiv:2609.28524v1 Announce Type: cross Abstract: Magnetic resonance imaging and histopathology observe the same glioma at radically different scales. We present CrossScale-GLIO, a visual multimodal f

safetyarxiv-cs-ai

DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs

model-releasesarxiv-cs-ai

arXiv:2609.28570v1 Announce Type: new Abstract: Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven.

Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents

model-releasesarxiv-cs-lg

arXiv:2609.28564v1 Announce Type: cross Abstract: Agentic video-generation systems close a loop between a generator and a verifier: an LLM plans shots, calls a text-to-video model, and a multimodal ju

Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation

arXiv:2609.28949v1 Announce Type: new Abstract: Few-shot segmentation (FSS) aims to segment unseen object categories with a few (e.g., one or five) labeled examples, enabling efficient adaptation to n

MoVISA: Multi-Token Reasoning for Video Object Segmentation

arXiv:2609.28956v1 Announce Type: new Abstract: Recent advances in video object segmentation with Multimodal Large Language Model (MLLM) reasoning have demonstrated the effectiveness of using a single

PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation

arXiv:2609.29760v1 Announce Type: new Abstract: Humans typically rely on vision, touch, hearing, and proprioception to perceive contact and adapt their actions during manipulation. Providing robots wi

safetyarxiv-cs-ro