Search: “multimodal”

arXiv:2511.17855v2 Announce Type: replace Abstract: Robots must learn from both what people do and what they say, but either modality alone is often incomplete: physical corrections are grounded but a

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning

DGX agent

arXiv:2507.01908v3 Announce Type: replace Abstract: Instruction-based image editing (IIE) has advanced rapidly with the success of diffusion models. However, existing efforts primarily focus on simple

model-releasesarxiv-cs-cv

14 May 2026

Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering

arXiv:2605.12961v1 Announce Type: new Abstract: Image clustering aims to partition unlabeled image datasets into distinct groups. A core aspect of this task is constructing and leveraging prior knowle

Tutorials

Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education

arXiv:2605.12988v1 Announce Type: new Abstract: Students learning algorithms often need support as they interpret traces, debug reasoning errors, and apply procedures across unfamiliar problem instanc

tutorialsarxiv-cs-ai

Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs

arXiv:2605.13737v1 Announce Type: new Abstract: When an omnimodal large language model accepts a question whose textual premise contradicts what it actually sees or hears, does the failure lie in perc

StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos

arXiv:2512.01707v3 Announce Type: replace-cross Abstract: Streaming video understanding requires models not only to process temporally incoming frames, but also to anticipate user intention for realis

Towards Unified Surgical Scene Understanding:Bridging Reasoning and Grounding via MLLMs

arXiv:2605.13530v1 Announce Type: cross Abstract: Surgical scene understanding is a cornerstone of computer-assisted intervention. While recent advances, particularly in surgical image segmentation, h

applicationsarxiv-cs-ai

Agents

Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context

arXiv:2605.13831v1 Announce Type: new Abstract: Long-context modeling is becoming a core capability of modern large vision-language models (LVLMs), enabling sustained context management across long-do

agentsarxiv-cs-cv

When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning

arXiv:2601.14104v2 Announce Type: replace-cross Abstract: Backdoor attacks can cause reinforcement learning (RL) policies to behave normally under clean inputs while executing malicious behaviors when

Aligning Flow Map Policies with Optimal Q-Guidance

arXiv:2605.12416v1 Announce Type: new Abstract: Generative policies based on expressive model classes, such as diffusion and flow matching, are well-suited to complex control problems with highly mult

safetyarxiv-cs-lg

Allegory of the Cave: Measurement-Grounded Vision-Language Learning

arXiv:2605.11727v1 Announce Type: cross Abstract: Vision-language models typically reason over post-ISP RGB images, although RGB rendering can clip, suppress, or quantize sensor evidence before infere

B4DL: A Benchmark for 4D LiDAR LLM in Spatio-Temporal Understanding

arXiv:2508.05269v2 Announce Type: replace Abstract: Understanding dynamic outdoor environments requires capturing complex object interactions and their evolution over time. LiDAR-based 4D point clouds

Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner

arXiv:2510.03206v2 Announce Type: replace-cross Abstract: Diffusion language models, especially masked discrete diffusion models, have achieved great success recently. While there are some theoretical

applicationsarxiv-cs-cl

Design Your Ad: Personalized Advertising Image and Text Generation with Unified Autoregressive Models

arXiv:2605.12138v1 Announce Type: cross Abstract: Generating realistic and user-preferred advertisements is a key challenge in e-commerce. Existing approaches utilize multiple independent models drive

researcharxiv-cs-cl

Detecting In-Person Conversations in Noisy Real-World Environments with Smartwatch Audio and Motion Sensing

arXiv:2507.12002v2 Announce Type: replace Abstract: Social interactions play a crucial role in shaping human behavior, relationships, and societies. It encompasses various forms of communication, such

applicationsarxiv-cs-lg

GeoR-Bench: Evaluating Geoscience Visual Reasoning

arXiv:2605.11541v1 Announce Type: new Abstract: Geoscience intelligence is expected to understand, reason about, and predict earth system changes to support human decision-making in critical domains s

Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation

arXiv:2605.11208v1 Announce Type: new Abstract: Automated, clinician-grade assessment reports for surgical procedures could reduce documentation burden and provide objective feedback, yet remain chall

Hide to See: Reasoning-prefix Masking for Visual-anchored Thinking in VLM Distillation

arXiv:2605.11651v1 Announce Type: cross Abstract: Recent think-answer approaches in VLMs, such as Qwen3-VL-Thinking, boost reasoning performance by leveraging intermediate thinking steps before the fi

applicationsarxiv-cs-cl

Large-Small Model Collaboration for Farmland Semantic Change Detection

arXiv:2605.12282v1 Announce Type: new Abstract: Farmland Semantic Change Detection (SCD) is essential for cultivated land protection, yet existing benchmarks and models remain insufficient for fine-gr

MambaNetBurst: Direct Byte-level Network Traffic Classification without Tokenization or Pretraining

arXiv:2605.11034v1 Announce Type: cross Abstract: We present MambaNetBurst, a compact tokenizer-free byte-level sequence classifier for network burst classification based on a Mamba-2 backbone. In con

researcharxiv-cs-lg

OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation

arXiv:2605.12480v1 Announce Type: new Abstract: Recent advances in joint audio-video generation have been remarkable, yet real-world applications demand strong per-modality fidelity, cross-modal align

ReasonEdit: Editing Vision-Language Models using Human Reasoning

arXiv:2602.02408v4 Announce Type: replace Abstract: Model editing aims to correct errors in large, pretrained models without altering unrelated behaviors. While some recent works have edited vision-la

researcharxiv-cs-cv

Reconstructing Sepsis Trajectories from Clinical Case Reports using LLMs: the Textual Time Series Corpus for Sepsis

arXiv:2504.12326v3 Announce Type: replace Abstract: Clinical case reports and discharge summaries may be the most complete and accurate summarization of patient encounters, yet they are finalized, i.e

REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View

arXiv:2605.11824v1 Announce Type: new Abstract: A realistic view of the vehicle's surroundings is generally offered by camera sensors, which is crucial for environmental perception. Affordable radar s

ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction

arXiv:2605.11212v1 Announce Type: new Abstract: Computer-use agents~(CUAs) rely on visual observations of graphical user interfaces, where each screenshot is encoded into a large number of visual toke

researcharxiv-cs-cl

ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes

arXiv:2605.11680v1 Announce Type: new Abstract: We introduce ShapeCodeBench, a synthetic benchmark for perception-to-program reconstruction: given a rendered raster image, a model must emit an executa

Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling

arXiv:2605.05922v2 Announce Type: replace Abstract: Recent advances in generative video models are increasingly driven by post-training and test-time scaling, both of which critically depend on the qu

researcharxiv-cs-cv

Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Survey

arXiv:2304.10891v2 Announce Type: replace-cross Abstract: Transformer-based models are becoming a central paradigm in autonomous driving because they can capture long-range spatial dependencies, multi

Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision

arXiv:2605.11782v1 Announce Type: new Abstract: Visual impairment affects hundreds of millions of people worldwide, severely limiting their ability to navigate urban environments safely and independen

A Visuo-Tactile Data Collection System with Haptic Feedback for Coarse-to-Fine Imitation Learning

arXiv:2605.08757v1 Announce Type: new Abstract: We present a visuo-tactile data-collection system that generates temporally structured, contact-rich demonstrations for imitation learning. Conventional

researcharxiv-cs-ro

Active Testing of Large Language Models via Approximate Neyman Allocation

arXiv:2605.10075v1 Announce Type: new Abstract: Large language models (LLMs) require reliable evaluation from pre-training to test-time scaling, making evaluation a recurring rather than one-off cost.

researcharxiv-cs-ai

AllocMV: Optimal Resource Allocation for Music Video Generation via Structured Persistent State

arXiv:2605.10723v1 Announce Type: cross Abstract: Generating long-horizon music videos (MVs) is frequently constrained by prohibitive computational costs and difficulty maintaining cross-shot consiste

researcharxiv-cs-ai

Attention Grounded Enhancement for Visual Document Retrieval

arXiv:2511.13415v2 Announce Type: replace-cross Abstract: Visual document retrieval requires understanding heterogeneous and multi-modal content to satisfy implicit information needs. Recent advances

BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation

arXiv:2605.10845v1 Announce Type: cross Abstract: As global cross-lingual communication intensifies, language barriers in visually rich documents such as PDFs remain a practical bottleneck. Existing d

Local Ai

Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval

arXiv:2605.08421v1 Announce Type: new Abstract: Visual Document Retrieval (VDR) models mostly rely on late interaction architectures, in which documents are represented by a set of local patch embeddi

local-aiarxiv-cs-cv

Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization

arXiv:2605.10764v1 Announce Type: cross Abstract: Recent studies show that gradient-based universal image jailbreaks on vision-language models (VLMs) exhibit little or no cross-model transferability,

safetyarxiv-cs-ai

Built Environment Reasoning from Remote Sensing Imagery Using Large Vision--Language Models

arXiv:2605.08404v1 Announce Type: cross Abstract: This work investigates the use of large language models (LLMs) for tasks in smart cities. The core idea is to leverage remote sensing imagery to chara

Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts

arXiv:2503.05066v5 Announce Type: replace-cross Abstract: The Mixture of Experts (MoE) is an effective architecture for scaling large language models by leveraging sparse expert activation to balance

CoLA-Flow Policy: Temporally Coherent Imitation Learning via Continuous Latent Action Flow Matching for Robotic Manipulation

arXiv:2601.23087v3 Announce Type: replace Abstract: Learning long-horizon robotic manipulation requires jointly achieving expressive behavior modeling, real-time inference, and stable execution, which

safetyarxiv-cs-ro

Count Anything at Any Granularity

arXiv:2605.10887v1 Announce Type: new Abstract: Open-world object counting remains brittle: despite rapid advances in vision-language models (VLMs), reliably counting the objects a user intends is far

applicationsarxiv-cs-cv

Agents

Cross-Modal RGB-D Fusion Transformer for 6D Pose Estimation of Non-Cooperative Spacecraft with Stereo-Derived Depth

arXiv:2605.08592v1 Announce Type: new Abstract: On-orbit servicing and active debris removal involving non-cooperative spacecraft require reliable pose estimation to supply accurate position and orien

agentsarxiv-cs-cv

CrossVL: Complexity-Aware Feature Routing and Paired Curriculum for Cross-View Vision-Language Detection

arXiv:2605.09802v1 Announce Type: cross Abstract: Vision-language models (VLMs) enable text-guided object detection but degrade severely under cross-view scenarios where ground and aerial viewpoints d

researcharxiv-cs-ai

Data-driven transport modelling without overfit

arXiv:2605.08801v1 Announce Type: new Abstract: Macroscopic transport modelling aims to predict traffic flows after proposed public policy interventions, such as a new road or railway section or a tem

safetyarxiv-cs-lg

EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding

arXiv:2605.09874v1 Announce Type: cross Abstract: Next-generation visual assistants, such as smart glasses, embodied agents, and always-on life-logging systems, must reason over an entire day or more

Energy-based models for diagnostic reconstruction and analysis in a laboratory plasma device

arXiv:2605.08645v1 Announce Type: cross Abstract: Energy-based models (EBMs) provide a powerful and flexible way of learning a joint probability distribution over data by constructing an energy surfac

researcharxiv-cs-lg

ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning

arXiv:2605.09982v1 Announce Type: new Abstract: Recent advancements in Vision-Language Models (VLMs) enable large language models (LLMs) to process high-resolution images, significantly improving real

applicationsarxiv-cs-cv

Local Ai

Event Fields: Learning Latent Event Structure for Waveform Foundation Models

arXiv:2605.08685v1 Announce Type: cross Abstract: We propose a new class of waveform foundation models that departs from conventional sequence based representations by modeling physiological time seri

local-aiarxiv-cs-ai

EventTSF: Event-Aware Non-Stationary Time Series Forecasting

arXiv:2508.13434v2 Announce Type: replace-cross Abstract: Time series forecasting is vital in diverse sectors such as energy and transportation, where non-stationary dynamics are deeply intertwined wi

applicationsarxiv-cs-ai