Search: “multimodal”

GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs

arXiv:2606.04184v1 Announce Type: new Abstract: True general intelligence requires not only a model of the physical world but also a social world model: the capacity to infer how individual mental sta

4 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series

DGX agent

arXiv:2606.03301v1 Announce Type: new Abstract: We introduce SagaQA, a long-form video benchmark for multi-hop reasoning over full-length TV series. Existing video reasoning benchmarks often emphasize

model-releasesarxiv-cs-cl

3 Jun 2026

Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation

arXiv:2606.03963v1 Announce Type: cross Abstract: Deep reinforcement learning has shown strong potential for enabling autonomous robots to learn complex navigational tasks. However, its practical use

safetyarxiv-cs-ai

3 Jun 2026

TimeOmni-VL: Unified Models for Time Series Understanding and Generation

arXiv:2602.17149v2 Announce Type: replace-cross Abstract: Recent time series modeling faces a sharp divide between numerical generation and semantic understanding, with research showing that generatio

3 Jun 2026

Local Ai

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606.03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text. Yet, it remains unc

local-aiarxiv-cs-cl

3 Jun 2026

Tutorials

CLIP-like Model as a Foundational Density Ratio Estimator

arXiv:2506.22881v3 Announce Type: replace Abstract: Density ratio estimation is a core concept in statistical machine learning because it provides a unified mechanism for tasks such as importance weig

tutorialsarxiv-cs-cv

Flow-Transformed Implicit Processes for Function-Space Variational Inference

arXiv:2606.01954v1 Announce Type: new Abstract: Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space m

FlowTime: Towards Continuous Generative Watch Time Prediction via Flow-based Personalized Priors

arXiv:2606.01352v1 Announce Type: new Abstract: Watch time has emerged as a pivotal metric for optimizing deep user engagement in short-video recommender systems. However, current methods of watch tim

MAVL: A Multilingual Audio-Video Lyrics Dataset for Animated Song Translation

arXiv:2505.18614v5 Announce Type: replace Abstract: Lyrics translation requires both accurate semantic transfer and preservation of musical rhythm, syllabic structure, and poetic style. In animated mu

Agents

MUSCLE-NET: Predicted-Multiscale-Aware Network for Pedestrian Trajectory Forecasting

arXiv:2606.00471v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction is essential for safe navigation in autonomous driving and intelligent transportation systems. Despite substan

agentsarxiv-cs-cv

Reason, Retrieve, Re-rank: A Zero-Shot Reasoning-Aware Framework for Composed Video Retrieval

arXiv:2606.00910v1 Announce Type: new Abstract: Composed Video Retrieval (CoVR) seeks the target video that results from applying a free-form textual modification to a reference video. We address the

Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs

arXiv:2606.00592v1 Announce Type: new Abstract: Effective visual communication stems from the harmony of multiple design principles, such as readability, contrast, alignment, overlap, and coherence, w

Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval

arXiv:2606.01615v1 Announce Type: new Abstract: Video-language models are pivotal for tasks such as moment retrieval and highlight detection, yet they often struggle to capture the dynamic, non-linear

safetyarxiv-cs-cv

TVIR: Building Deep Research Agents Towards Text--Visual Interleaved Report Generation

arXiv:2606.02320v1 Announce Type: new Abstract: Deep Research Agents have shown strong capability in multi-step information retrieval, reasoning, and long-form report generation, but existing benchmar

Mitigating Stethoscope-Induced Shortcuts in Respiratory Sound Classification under Federated Domain Generalization with Causality-Inspired Interventions

arXiv:2605.29862v1 Announce Type: cross Abstract: AI-driven respiratory sound classification (RSC) is promising for automated pulmonary disease detection, yet multi-site deployment is hindered by inte

Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition

arXiv:2505.05968v3 Announce Type: replace Abstract: Offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts, particularly stemming from the h

safetyarxiv-cs-lg

Agents

Supercharging Thermal Gaussian Splatting with Depth Estimation

arXiv:2605.30328v1 Announce Type: new Abstract: Efficient and robust 3D scene representation is crucial in autonomous driving, robotics, and related fields. While RGB images provide valuable content f

agentsarxiv-cs-cv

When, why, and how do diffusion posterior samplers fail? A finite-sample lens

arXiv:2605.30330v1 Announce Type: new Abstract: Diffusion models have excellent capacity to model complex distributions of natural data, which has made them a popular and effective choice for posterio

X-GS: An Extensible Framework for Perceiving and Thinking via 3D Gaussian Splatting

arXiv:2603.09632v3 Announce Type: replace-cross Abstract: 3D Gaussian Splatting (3DGS) has emerged as a powerful technique for novel view synthesis, subsequently extending into numerous spatial AI app

researcharxiv-cs-cl

CyberJurors: A Multi-Agent Simulation Task for E-Commerce Disputes Verdict

arXiv:2605.28369v1 Announce Type: new Abstract: E-commerce platforms have begun recruiting crowdsourced jurors to adjudicate massive volumes of transaction disputes. Unlike formal legal judgment, E-co

Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning

arXiv:2512.02019v3 Announce Type: replace-cross Abstract: Diffusion models excel at sampling from complex, unnormalized distributions. In this work, we extend Maximum Entropy Reinforcement Learning (M

safetyarxiv-cs-ai

Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation

arXiv:2605.28239v1 Announce Type: new Abstract: Semi-supervised referring expression segmentation (SS-RES) aims to achieve precise pixel-level language grounding under limited annotation, yet suffers

Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects

arXiv:2603.21165v2 Announce Type: replace Abstract: Bangla culture is richly expressed through region, dialect, history, food, politics, media, and everyday visual life, yet it remains underrepresente

Applications

SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter

arXiv:2605.28084v1 Announce Type: cross Abstract: Laughter is a complex social signal that conveys communicative intent beyond amusement. While prior work has focused on isolated laughter analysis tas

applicationsarxiv-cs-ai

Text-Only Data Synthesis for Vision Language Model Training

arXiv:2503.22655v2 Announce Type: replace Abstract: Training vision-language models (VLMs) typically requires large-scale, high-quality image-text pairs, but collecting or synthesizing such data is co

UniMaia: Steering Chess Policies with Language for Human-like Play

arXiv:2605.27767v1 Announce Type: cross Abstract: Recent advances in large language models have enabled natural language to serve as a flexible interface for controlling complex systems, but often at

On the Robustness of Machine Unlearning for Vision-Language Models

arXiv:2605.26992v1 Announce Type: new Abstract: Vision-language models (VLMs) may memorize undesirable information from training data, motivating growing interest in machine unlearning. In this work,

27 May 2026

Synheart Capacity: A Theory-Driven Physiological Representation of Cognitive Capacity Dynamics from Wearable Signals

arXiv:2605.24416v1 Announce Type: new Abstract: Human cognitive performance is constrained by limited mental resources, yet continuous computational estimation of cognitive capacity dynamics remains a

26 May 2026

Terrain-Adaptive Grouser Wheel for Optimal Planetary Exploration: Design and Experimental Investigation

arXiv:2605.24311v1 Announce Type: new Abstract: Planetary rovers operating in extraterrestrial environments often encounter significant mobility challenges due to varying terrain features such as grad

researcharxiv-cs-ro

26 May 2026

Applications

Emotion Recognition in Sign Language Conversation

arXiv:2605.23328v1 Announce Type: new Abstract: Emotion Recognition in Conversation is a core component of affective computing, while current resources of sign language emotion datasets primarily focu

applicationsarxiv-cs-cl

25 May 2026

FAST-ME: Foundation-aware Adaptive Stopping for Motion Estimation for Efficient IoT Video Analysis

arXiv:2605.23428v1 Announce Type: new Abstract: In modern multimedia systems, efficient video processing is critical, especially in resource-constrained environments such as IoT-based camera networks,

25 May 2026

GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning

arXiv:2605.22558v1 Announce Type: new Abstract: Spatio-temporal reasoning in vision-language models requires visual representations that preserve physical geometry rather than merely semantic appearan

22 May 2026

Applications

PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought

arXiv:2605.22013v1 Announce Type: new Abstract: Understanding 3D point clouds through language remains a fundamental challenge in computer graphics and visual computing, due to the irregular structure

applicationsarxiv-cs-cv

22 May 2026

Revisiting Integration of Image and Metadata for DICOM Series Classification: Cross-Attention and Dictionary Learning

arXiv:2602.23833v2 Announce Type: replace-cross Abstract: Automated identification of DICOM image series is essential for large-scale medical image analysis, quality control, protocol harmonization, a

22 May 2026

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning

arXiv:2605.20247v1 Announce Type: cross Abstract: Catastrophic forgetting remains a major obstacle to continual learning in large language models (LLMs) and vision--language models (VLMs). Although Mi

21 May 2026

extit{Stochastic} MeanFlow Policies: One-Step Generative Control with Entropic Mirror Descent

arXiv:2605.21282v1 Announce Type: new Abstract: Online off-policy reinforcement learning (RL) is shaped by two coupled choices: the policy class and the update rule. Gaussian policies are fast and hav

safetyarxiv-cs-lg

21 May 2026

VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence

arXiv:2605.20676v1 Announce Type: new Abstract: Establishing a clear link between model predictions and the visual evidence that supports them is critical for transparency and reliability in multimoda

21 May 2026

Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond

arXiv:2502.04575v3 Announce Type: replace-cross Abstract: Given an unnormalized probability density piproptoe^{-V}, estimating its normalizing constant Z=int_{R^d}e^{-V(x)}dx or free energy F=-log Z i

CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing

arXiv:2605.19484v1 Announce Type: cross Abstract: While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workfl

Hardware

Hyrax: An Extensible Framework for Rapid ML Experimentation and Unsupervised Discovery in the Era of Rubin, Roman, and Euclid

arXiv:2605.18959v1 Announce Type: cross Abstract: The NSF-DOE Vera C. Rubin Observatory, Roman Space Telescope, Euclid, and other next-generation surveys will deliver imaging, spectroscopic, and time-

hardwarearxiv-cs-lg

M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models

arXiv:2605.18774v1 Announce Type: cross Abstract: In long, multi-page industrial documents, retrieval-augmented generation (RAG) depends heavily on whether chunk boundaries follow the document's true

OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments

arXiv:2605.18758v1 Announce Type: cross Abstract: Current benchmarks for graphical user interface (GUI) agents predominantly rely on static screenshots. However, real-world smartphone interaction rout

A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability

arXiv:2605.18750v1 Announce Type: cross Abstract: Pipeline parallelism is a key technique for scaling large-model training, but modern workloads exhibit runtime variability in computation and communic

Local Ai

ALIGN: A Vision-Language Framework for High-Accuracy Accident Location Inference through Geo-Spatial Neural Reasoning

arXiv:2511.06316v3 Announce Type: replace Abstract: In low- and middle-income countries, public safety and urban planning initiatives frequently face a critical shortage of accurate, location-specific

local-aiarxiv-cs-ai

Designing streetscapes from street-view imagery using diffusion models

arXiv:2605.17527v1 Announce Type: new Abstract: Street-view imagery (SVI) is widely used to quantify key indicators of urban environment, such as green- ery, sky, or road view indices. However, existi

DriveSafe: A Framework for Risk Detection and Safety Suggestions in Driving Scenarios

arXiv:2605.16892v1 Announce Type: cross Abstract: Comprehensive situational awareness is essential for autonomous vehicles operating in safety-critical environments, as it enables the identification a

FAM-HRI: Foundation-Model Assisted Multi-Modal Human-Robot Interaction Combining Gaze and Speech

model-releasesarxiv-cs-ro

arXiv:2503.16492v3 Announce Type: replace-cross Abstract: ffective Human-Robot Interaction (HRI) is crucial for enhancing accessibility and usability in real-world robotics applications. However, exis

Learning Native Continuation for Action Chunking Flow Policies

arXiv:2602.12978v2 Announce Type: replace-cross Abstract: Action chunking enables Vision Language Action (VLA) models to run in real time, but naive chunked execution often exhibits discontinuities at

safetyarxiv-cs-ai