Search: “multimodal”

Chronos: A Physics-Informed Full-History Framework for Non-Markovian Long-Horizon Manipulation

arXiv:2606.30318v1 Announce Type: new Abstract: General-purpose robot policies should be modeled as dynamical systems, yet many VLA and generative imitation policies still rely on present observations

safetyarxiv-cs-ro

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

CMTFormer: Marrying Transformer with Hierarchical Information Interaction for RGB-Event Object Detection

DGX agent

arXiv:2606.29136v1 Announce Type: cross Abstract: Event cameras capture sparse brightness changes with high temporal resolution and high dynamic range, compensating for the deficiencies of the convent

safetyarxiv-cs-ai

30 Jun 2026

Delayed Bidirectional Alignment via Disentangled Audio Semantics for Audio-Visual Segmentation

arXiv:2512.20117v2 Announce Type: replace Abstract: Audio-Visual Segmentation (AVS) aims to localize sound-producing objects at the pixel level by integrating auditory and visual cues. However, existi

Digitizing Coaching Intelligence: An Agentic Framework for Holistic Athlete Profiling using VLM and RAG

arXiv:2606.28570v1 Announce Type: cross Abstract: Athlete assessment is a critical process for tracking physical progress and identifying elite talent. However, during mass recruitment drives, traditi

Hardware

DreamForge-World 0.1 Preview: A Low-Compute Real-Time Controllable World Model

arXiv:2606.30292v1 Announce Type: cross Abstract: We present DreamForge-World 0.1 Preview, a preview foundational world model for real-time interactive world simulation. The system adapts the LongLive

hardwarearxiv-cs-cv

EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal

arXiv:2512.21545v2 Announce Type: replace Abstract: Object removal must prevent the masked target from reappearing and reconstruct the occluded background with structural and contextual fidelity, rath

Eval-Actions: Fine-Grained Execution Quality Evaluation for Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2601.18723v2 Announce Type: replace Abstract: Although Vision--Action (VA) and Vision--Language--Action (VLA) policies have advanced robotic manipulation, their evaluation remains dominated by b

Applications

Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model

arXiv:2606.29384v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have become an important paradigm of embodied AI. However, existing VLA models typically assume well-lit and stable

applicationsarxiv-cs-cv

ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images

arXiv:2606.28920v1 Announce Type: new Abstract: Remote sensing visual grounding (RSVG) aims to locate specific objects in high-resolution RS imagery using free-form natural language descriptions. Whil

local-aiarxiv-cs-cv

Flow Matching in Feature Space for Stochastic World Modeling

arXiv:2606.29059v1 Announce Type: cross Abstract: World modeling requires forecasting uncertain futures while preserving information useful for downstream perception. Existing visual world models ofte

From Failure Taxonomy to Intervention: A Diagnostic Methodology for Industry-Scale AVLM in Video and Live-Streaming Platform Moderation

model-releasesarxiv-cs-lg

arXiv:2606.30059v1 Announce Type: new Abstract: Industry-scale video and live-streaming moderation imposes requirements that are difficult to satisfy with generic pretrained public models or external

Keypose Exploration: Efficient Automatic Trajectory Labelling and Cross-Embodiment Policy Transfer

arXiv:2606.29028v1 Announce Type: new Abstract: Keypose-based manipulation decomposes tasks into critical waypoints to simplify policy learning for long-horizon tasks, but existing approaches rely on

safetyarxiv-cs-ro

Tutorials

LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

arXiv:2606.28445v1 Announce Type: cross Abstract: Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous speech offers a non-invasive screening modal

tutorialsarxiv-cs-ai

Low-cost concept-based localized explanations: How far can we get with training-free approaches?

arXiv:2606.29069v1 Announce Type: new Abstract: Concept-based Explainable AI (C-XAI) seeks human-understandable explanations grounded in semantic concepts, yet validation is limited by the scarcity of

local-aiarxiv-cs-ai

Mitigating Batch Effects in Histopathology via Language-Mediated Robust Embedding Generation

arXiv:2606.28697v1 Announce Type: cross Abstract: Pathology foundation models (PFMs) have demonstrated strong potential across clinical and scientific applications, yet their performance is often hind

researcharxiv-cs-cl

Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning

arXiv:2606.29425v1 Announce Type: new Abstract: Existing multi-agent debate frameworks suffer from two critical limitations: they rely on static architectures where agent roles and coordination patter

local-aiarxiv-cs-ai

More time to build with Step 3.7 Flash: in partnership with @StepFun_ai, we’re extending the free usage period in Nous Portal by an addition…

More time to build with Step 3.7 Flash: in partnership with @StepFun_ai, we’re extending the free usage period in Nous Portal by an additional 15 days! Media Step 3.7 Flash is now free for 30 days via

agentsnous-research--x

MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs

arXiv:2606.30026v1 Announce Type: cross Abstract: Audiovisual arts encompass diverse creative disciplines, including cinema, visual arts, stage performance, and game design, where artistic meaning ari

PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

arXiv:2606.28344v1 Announce Type: cross Abstract: Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems

agentsarxiv-cs-ai

PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science

arXiv:2508.17117v3 Announce Type: replace-cross Abstract: Existing plant-disease datasets target classification and detection, leaving vision-language models unable to support interactive, reasoning-b

Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

arXiv:2606.29464v1 Announce Type: cross Abstract: Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently trai

safetyarxiv-cs-ai

SADL: What to Ignore? A Benchmark for Subject-Aware Distractor Localization

model-releasesarxiv-cs-cv

arXiv:2606.30393v1 Announce Type: new Abstract: Photographs frequently contain visual distractors besides foregrounds and backgrounds of the intended subject, competing for attention and weakening com

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

arXiv:2603.29139v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visuali

Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution

arXiv:2606.28971v1 Announce Type: new Abstract: Real-world image restoration (IR) remains challenging due to complex and coupled degradations. While recent agentic IR frameworks leverage Large Languag

agentsarxiv-cs-cv

Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation

model-releasesarxiv-cs-cv

arXiv:2606.30244v1 Announce Type: new Abstract: Referring Remote Sensing Image Segmentation (RRSIS) seeks to localize and segment the target object or region specified by a natural language expression

SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis

arXiv:2606.29586v1 Announce Type: cross Abstract: Vision-language foundation models have shown strong potential in medical image analysis. Although foundation models for ultrasound imaging have recent

local-aiarxiv-cs-ai

Steerable Visual Representations

arXiv:2604.02327v2 Announce Type: replace-cross Abstract: Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tas

researcharxiv-cs-ai

Hardware

SwarmX: Agentic Scheduling for Low-Latency Agentic Systems

arXiv:2606.21401v2 Announce Type: replace-cross Abstract: Agentic AI applications compose multiple model calls and tool executions, creating new scheduling challenges for GPU-CPU clusters. Their infer

hardwarearxiv-cs-ai

SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios

arXiv:2511.17649v4 Announce Type: replace-cross Abstract: Tangible control interfaces (TCIs), such as appliance panels, remotes, elevators, and embedded GUIs, are a fundamental component of everyday h

The NTNU System at the S&I Challenge 2025 SLA Open Track

model-releasesarxiv-cs-cl

arXiv:2506.05121v3 Announce Type: replace Abstract: A recent line of research on spoken language assessment (SLA) employs neural models such as BERT and wav2vec 2.0 (W2V) to evaluate speaking proficie

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is …

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents ob

local-aijim-fan--x

Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task

arXiv:2512.10359v1 Announce Type: cross Abstract: Video Question Answering (VideoQA) task serves as a critical playground for evaluating whether foundation models can effectively perceive, understand,

agentsarxiv-cs-ai

TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment

arXiv:2606.30313v1 Announce Type: new Abstract: Longitudinal glioblastoma response assessment requires comparing subtle tumor changes across MRI time points using structured clinical criteria such as

researcharxiv-cs-cv

UniCA: Bi-directional Cross-Attention with Positive Similarity Loss for Robust Multi-Modal Retrieval

model-releasesarxiv-cs-cv

arXiv:2606.28350v1 Announce Type: cross Abstract: Multi-modal retrieval has become increasingly critical for handling the growing volume of integrated visual-textual data in real-world applications, b

Vivid-VR: Distilling Concepts from Text-to-Video Diffusion Transformer for Photorealistic Video Restoration

arXiv:2508.14483v4 Announce Type: replace Abstract: We present Vivid-VR, a DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to c

We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then in…

model-releasesgoogle-ai--x

We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the other—all at a fraction of the

When Stopping Fails: Rethinking Minimal Risk Conditions through Human-Interactive Autonomous Driving for Safe Transportation Systems

arXiv:2606.29115v1 Announce Type: cross Abstract: Autonomous vehicles (AVs) are increasingly deployed in urban environments, yet their safety frameworks remain primarily designed around collision avoi

safetyarxiv-cs-ai

A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges

arXiv:2606.27380v1 Announce Type: new Abstract: Automated coaching for oral presentations sits at the intersection of computer-assisted pronunciation training (CAPT), prosody modeling, and speech synt

researcharxiv-cs-cl

An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models

arXiv:2604.00784v2 Announce Type: replace Abstract: Surgical video understanding is a crucial prerequisite for advancing Computer-Assisted Surgery. While vision-language models (VLMs) have recently be

researcharxiv-cs-cv

Benchmarking Multi-Modal Graph-based Social Media Popularity Prediction

arXiv:2606.27539v1 Announce Type: cross Abstract: Social media popularity prediction aims to forecast the future reach or influence of online content from early-stage observations. Accurate prediction

From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection

arXiv:2606.27973v1 Announce Type: cross Abstract: Speech-based cognitive impairment detection offers a noninvasive, accessible alternative to costly biomarker assays, yet transformer-based models rema

GRAFT: Biological Graph and Hypergraph Benchmarks for Linked Gene Expression and Phenotypic Trait Prediction in Arabidopsis thaliana

arXiv:2606.27413v1 Announce Type: cross Abstract: Understanding which genes control which traits in an organism remains one of the central challenges in biology. Despite significant advances in data c

GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

arXiv:2511.11266v4 Announce Type: replace Abstract: Vision-language models have recently emerged as promising planners for autonomous driving, where success hinges on topology-aware reasoning over spa

agentsarxiv-cs-cv

Hippocampus-DETR: An Explicit Memory Object Detection Framework Based on Hippocampus Modeling

arXiv:2606.27831v1 Announce Type: cross Abstract: This paper addresses the lack of explicit memory mechanisms in current object detection models and proposes Hippocampus-DETR, a novel detection framew

researcharxiv-cs-ai

Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge

arXiv:2606.27527v1 Announce Type: cross Abstract: Large Language Models (LLMs) possess broad conceptual knowledge acquired through large-scale text pretraining, yet their potential to supervise models

researcharxiv-cs-ai

ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On

arXiv:2606.27773v1 Announce Type: new Abstract: Image-based virtual try-on has emerged as a compelling task in e-commerce and augmented reality, yet existing methods struggle to simultaneously preserv

Pair Nova 2 Lite with Claude for cost-optimized document processing

model-releasesaws-ml-blog

In this post, we show how pairing Amazon Nova 2 Lite with Anthropic’s Claude Sonnet 4.6 delivers an efficient solution for digitizing scanned documents at scale. We built a two-model pipeline on Amazo

Parameter Efficient Hybrid Transformer (PEHT) for Network Traffic Prediction via Dynamic Urban Congestion Integration

arXiv:2606.28274v1 Announce Type: cross Abstract: Accurate network traffic prediction is a critical element for efficient resource allocation in dynamic urban cellular networks. However, prediction re