Search: “multimodal”

VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understanding

arXiv:2605.05848v2 Announce Type: replace-cross Abstract: Video large multimodal models increasingly face a scalability bottleneck: long videos produce excessively long visual-token sequences, which s

safetyarxiv-cs-ai

11 May 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

SenseNova U1 ComfyUI Node: 8-step LoRA support and GGUF VRAM/RAM optimization tips

DGX agent

I'd like to search for information about this specific Reddit post to provide you with an accurate summary based on actual content rather than assumptions. SenseNova-U1 is SenseTime's unified multimod

local-air-stablediffusion

9 May 2026

How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?

arXiv:2602.02924v2 Announce Type: replace Abstract: Diffusion policy sampling enables reinforcement learning (RL) to represent multimodal action distributions beyond suboptimal unimodal Gaussian polic

safetyarxiv-cs-lg

Physical Adversarial Clothing Evades Visible-Thermal Detectors via Non-Overlapping RGB-T Pattern

arXiv:2605.04675v1 Announce Type: new Abstract: Visible-thermal (RGB-T) object detection is a crucial technology for applications such as autonomous driving, where multimodal fusion enhances performan

Technical highlights: CogViT Vision Encoder - Built with dual-teacher distillation: SigLIP2 for semantics, DINOv3 for texture. A two-stage r…

Technical highlights: CogViT Vision Encoder - Built with dual-teacher distillation: SigLIP2 for semantics, DINOv3 for texture. A two-stage recipe, masked modeling, then contrastive pretraining, with Q

agentszhipu-ai--x

When Relations Break: Analyzing Relation Hallucination in Vision-Language Model Under Rotation and Noise

arXiv:2605.05045v1 Announce Type: cross Abstract: Vision-language models (VLMs) achieve strong multimodal performance but remain prone to relation hallucination, which requires accurate reasoning over

researcharxiv-cs-cl

Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems

arXiv:2605.01758v1 Announce Type: new Abstract: Large multimodal model-based Multi-Agent Systems (MASs) enable collaborative complex problem solving through specialized agents. However, MASs are vulne

safetyarxiv-cs-ai

CellxPert: Inference-Time MCMC Steering of a Multi-Omics Single-Cell Foundation Model for In-Silico Perturbation

arXiv:2605.00930v1 Announce Type: cross Abstract: In this work, we introduce CellxPert, a scalable multimodal foundation model that unifies single-cell and spatial multi-omics within a common represen

researcharxiv-cs-ai

Conditional Diffusion Sampling

arXiv:2605.04013v1 Announce Type: cross Abstract: Sampling from unnormalized multimodal distributions with limited density evaluations remains a fundamental challenge in machine learning and natural s

researcharxiv-cs-lg

CoVSpec: Efficient Device-Edge Co-Inference for Vision-Language Models via Speculative Decoding

arXiv:2605.02218v1 Announce Type: new Abstract: Vision-language models (VLMs) have demonstrated strong capabilities in multimodal perception and reasoning. However, deploying large VLMs on mobile devi

researcharxiv-cs-ai

DataEvolver: Let Your Data Build and Improve Itself via Goal-Driven Loop Agents

arXiv:2605.01789v1 Announce Type: new Abstract: Constructing controllable visual data is a major bottleneck for image editing and multimodal understanding. Useful supervision is rarely produced by a s

Local Ai

SenseNova U1 Infographic Test: Capabilities in Image-Based Reasoning

local-air-stablediffusion

SenseNova U1 is an open-source unified multimodal model that integrates understanding, reasoning, and generation within a single architecture, with particular strength in creating complex infographics

Task-Aware Scanning Parameter Configuration for Robotic Inspection Using Vision Language Embeddings and Hyperdimensional Computing

arXiv:2605.03909v1 Announce Type: cross Abstract: Robotic laser profiling is widely used for dimensional verification and surface inspection, yet measurement fidelity is often dominated by sensor conf

Valley3: Scaling Omni Foundation Models for E-commerce

arXiv:2605.01278v1 Announce Type: new Abstract: In this work, we present Valley3, an omni multimodal large language model (MLLM) developed for diverse global e-commerce tasks, with unified understandi

VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models

arXiv:2605.01449v1 Announce Type: cross Abstract: Universal adversarial attacks on aligned multimodal large language models are increasingly reported with attack success rates in the 60-80% range, sug

Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs

arXiv:2605.01324v1 Announce Type: new Abstract: Although reinforcement learning (RL) has significantly advanced reasoning capabilities in large multimodal language models (MLLMs), its efficacy remains

safetyarxiv-cs-cv

EduGage: Methods and Dataset for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning

arXiv:2605.01238v1 Announce Type: cross Abstract: Engagement, which links to attentional, emotional, and cognitive dimensions, plays an important role in learning. In online and video-based learning e

researcharxiv-cs-cv

Efficient Reasoning with Hidden Thinking

model-releasesarxiv-cs-cl

arXiv:2501.19201v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities in Multimodal Large Language Mod

ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue

model-releasesarxiv-cs-ro

arXiv:2605.01371v1 Announce Type: new Abstract: The rapid advancement of Multimodal Large Language Models (MLLMs) has empowered Unmanned Aerial Vehicle (UAV) with exceptional capabilities in spatial r

Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

arXiv:2601.22709v3 Announce Type: replace Abstract: Vision-Language Models (VLMs) achieve strong multimodal performance but are costly to deploy, and post-training quantization often causes significan

High-Speed, Scalable Sensor Readout for Dexterous Robotic Hands via Shift-Register Multiplexing

arXiv:2605.01434v1 Announce Type: new Abstract: Dexterous robotic hands require high-speed multimodal sensing across many degrees of freedom, yet existing readout architectures often impose trade-offs

researcharxiv-cs-ro

Hardware

How to Build In-Vehicle AI Agents with NVIDIA: From Cloud to Car

The automotive cockpit is undergoing a fundamental shift from rule-based interfaces to agentic, multimodal AI systems capable of reasoning, planning, and acting. This is enabled by an agentic AI pipel

hardwarenvidia-developer

Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression

arXiv:2605.01402v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) struggle with numerical regression under long-tailed target distributions. Token-level supervised fine-tuning (

safetyarxiv-cs-cl

Omni-NegCLIP: Enhancing CLIP with Front-Layer Contrastive Fine-Tuning for Comprehensive Negation Understanding

arXiv:2603.29258v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) have demonstrated strong capabilities across a wide range of multimodal tasks. However, recent studies have shown that

researcharxiv-cs-cv

Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

arXiv:2603.27259v2 Announce Type: replace Abstract: Long video understanding (LVU) remains a core challenge in multimodal learning. Although recent vision-language models (VLMs) have made notable prog

Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes

arXiv:2510.24332v3 Announce Type: replace-cross Abstract: Purpose: Surgical scene understanding is key to advancing computer-aided and intelligent surgical systems. Current approaches predominantly re

Video Active Perception: Effective Inference-Time Long-Form Video Understanding with Vision-Language Models

arXiv:2605.01662v1 Announce Type: new Abstract: Large vision-language models (VLMs) have advanced multimodal tasks such as video question answering (QA). However, VLMs face the challenge of selecting

Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs

model-releasesarxiv-cs-lg

arXiv:2605.02735v1 Announce Type: new Abstract: Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models, enabling high-dimensional visual evide

X2SAM: Any Segmentation in Images and Videos

arXiv:2605.00891v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong image-level visual understanding and reasoning, yet their pixel-level perception acros

Applications

Being-H0.7: A Latent World-Action Model from Egocentric Videos

arXiv:2605.00078v1 Announce Type: cross Abstract: Visual-Language-Action models (VLAs) have advanced generalist robot control by mapping multimodal observations and language instructions directly to a

applicationsarxiv-cs-cv

FreeRet: MLLMs as Training-Free Retrievers

arXiv:2509.24621v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) are emerging as versatile foundations for mixed-modality retrieval. Yet, they often require heavy post-hoc

safetyarxiv-cs-cv

Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration

arXiv:2605.00370v1 Announce Type: new Abstract: Centralized multimodal learning commonly compresses language, acoustic, and visual signals into a single fused representation for prediction. While effe

agentsarxiv-cs-lg

Applications

MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Video

arXiv:2605.00495v1 Announce Type: cross Abstract: Recent advances in multimodal generation have enabled high-quality audio generation from silent videos. Practical applications, such as sound producti

applicationsarxiv-cs-cv

Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

arXiv:2605.00814v1 Announce Type: new Abstract: While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a 'Visual Signal Dilution' p

Thinking with Geometry: Active Geometry Integration for Spatial Reasoning

arXiv:2602.06037v4 Announce Type: replace Abstract: Recent progress in spatial reasoning with Multimodal Large Language Models (MLLMs) increasingly leverages geometric priors from 3D encoders. However

VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understanding

arXiv:2603.22285v2 Announce Type: replace Abstract: Long video understanding remains challenging for multimodal large language models (MLLMs) due to limited context windows, which necessitate identify

researcharxiv-cs-cv

Applications

A Pattern Language for Resilient Visual Agents

arXiv:2604.28001v1 Announce Type: new Abstract: Integrating multimodal foundation models into enterprise ecosystems presents a fundamental software architecture challenge. Architects must balance comp

applicationsarxiv-cs-ai

AutoSurfer -- Teaching Web Agents through Comprehensive Surfing, Learning, and Modeling

arXiv:2604.27253v1 Announce Type: new Abstract: Recent advances in multimodal large language models (LLMs) have revolutionized web agents that can automate complex tasks on websites. However, their ac

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

arXiv:2506.07180v3 Announce Type: replace-cross Abstract: As video large language models (Video-LLMs) become increasingly integrated into real-world applications that demand grounded multimodal reason

OR-VSKC: Resolving Visual-Semantic Knowledge Conflicts in Operating Rooms with Synthetic Data-Guided Alignment

arXiv:2506.22500v2 Announce Type: replace-cross Abstract: Automated identification of surgical safety risks is critical for improving patient outcomes; however, Multimodal Large Language Models (MLLMs

Delineating Knowledge Boundaries for Honest Large Vision-Language Models

arXiv:2604.26419v1 Announce Type: cross Abstract: Large Vision-Language Models (VLMs) have achieved remarkable multimodal performance yet remain prone to factual hallucinations, particularly in long-t

researcharxiv-cs-ai

FlowS: One-Step Motion Prediction via Local Transport Conditioning

model-releasesarxiv-cs-ro

arXiv:2604.26065v1 Announce Type: new Abstract: Generative motion prediction must satisfy three simultaneous requirements for real-world autonomy: high accuracy, diverse multimodal futures, and strict

Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learning

arXiv:2601.13942v2 Announce Type: replace-cross Abstract: Large Multimodal Models (LMMs) have achieved remarkable success in visual understanding, yet they struggle with knowledge-intensive queries in

safetyarxiv-cs-ai

Thinking with Drafting: Optical Decompression via Logical Reconstruction

model-releasesarxiv-cs-cl

arXiv:2602.11731v2 Announce Type: replace Abstract: Existing multimodal large language models have achieved high-fidelity visual perception and exploratory visual generation. However, a precision para

Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation

arXiv:2604.26946v1 Announce Type: new Abstract: Breakthrough progress in vision-based navigation through unknown environments has been achieved by using multimodal large language models (MLLMs). These

What Google Cloud announced in AI this month

model-releasesgoogle-cloud-ai

Editor’s note: Want to keep up with the latest from Google Cloud? Check back here for a monthly recap of our latest updates, announcements, resources, events, learning opportunities, and more. We host

DeepSeek has began grayscale testing for DeepSeek with Vision

model-releasesr-localllama

DeepSeek V4 is undergoing limited grayscale testing with a new interface featuring Fast, Expert, and Vision modes . The Vision version represents the multimodal component of the upcoming DeepSeek V4 r

29 Apr 2026

Highlights: 👉 1M context for long-horizon agentic workflows 👉 Stronger agentic coding across frontend, repo-level, and terminal-based task…