Search: “multimodal”

Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling

arXiv:2607.01612v1 Announce Type: new Abstract: Training large language models (LLMs) with reinforcement learning (RL) has significantly advanced their performance on reasoning and question-answering

3 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios

DGX agent

arXiv:2607.02343v1 Announce Type: cross Abstract: Humans can selectively attend to a target sound and estimate its direction in complex scenarios, whereas such selective localization remains challengi

tutorialsarxiv-cs-ai

3 Jul 2026

We are pleased to present our latest research at #ICML2026, “Bridging Spherical Black-Box Optimizers” https://arxiv.org/abs/2606.25761 When …

We are pleased to present our latest research at #ICML2026, “Bridging Spherical Black-Box Optimizers” https://arxiv.org/abs/2606.25761 When optimizing through simulators, external APIs, or in reinforc

researchdavid-ha--x

3 Jul 2026

AI Native Games: A Survey and Roadmap

arXiv:2607.00527v1 Announce Type: new Abstract: Generative AI now enables games to produce dialogue, quests, characters, images, and worlds at runtime. Yet generation alone does not make a game AI-nat

BrainFIBRE: A Foundation Model via Information Decomposition for Brain Microstructure

arXiv:2607.00573v1 Announce Type: new Abstract: Diffusion MRI probes brain microstructure with particular sensitivity to early cerebrovascular and neurodegenerative changes. Neurite Orientation Disper

ClinRAG-GRAPH: Clinical-prior Retrieval-Augmented Graph Model with Domain Adversarial Learning for Breast pCR Prediction

arXiv:2607.00798v1 Announce Type: new Abstract: Neoadjuvant chemotherapy (NAC) response prediction is clinically important for treatment stratification in breast cancer. However, robust pre-treatment

DroneIQA-VLE: Multi-Task Drone Image Quality Assessment via Vision-Language Ensemble

arXiv:2607.00416v1 Announce Type: new Abstract: We present DroneIQA-VLE, our solution to the ICME 2026 Drone-IQA Grand Challenge on Target-aware Image Quality Assessment for Low-altitude UAV Images. T

GaussianFusion: Unified 3D Gaussian Representation for Multi-Modal Fusion Perception

arXiv:2607.00746v1 Announce Type: cross Abstract: The bird's-eye view (BEV) representation enables multi-sensor features to be fused within a unified space, serving as the primary approach for achievi

Histopathology Multi-modal Embedding for Pathology Composed Retrieval

arXiv:2502.07221v4 Announce Type: replace Abstract: To overcome the black-box nature of predictive AI and the hallucination risks of generative models, retrieval-based models offer an interpretable, e

NEW paper from NVIDIA. They discuss robot programming that compounds experience instead of throwing it away. Traditional robot programming f…

NEW paper from NVIDIA. They discuss robot programming that compounds experience instead of throwing it away. Traditional robot programming forces you to orchestrate perception, contact dynamics, diver

safetydair-ai--x

OmniFall: From Staged Through Synthetic to Wild, A Unified Multi-Domain Dataset for Robust Fall Detection

arXiv:2505.19889v3 Announce Type: replace Abstract: Visual fall detection models are usually trained on small, staged datasets. Their real-world utility remains unclear; such data lacks diversity and

Local Ai

Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning

arXiv:2607.01191v1 Announce Type: new Abstract: Fine-grained visual reasoning remains challenging for vision-language models, especially when small but critical visual cues are buried in high-resoluti

local-aiarxiv-cs-cv

Local Ai

Segmenting, Fast and Slow: Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing

arXiv:2607.00124v1 Announce Type: new Abstract: Object-centric models inspired by DETR have become the dominant paradigm for open-vocabulary video instance segmentation (OV-VIS). While recent efforts

local-aiarxiv-cs-cv

SleepLM: Natural-Language Intelligence for Human Sleep

arXiv:2602.23605v2 Announce Type: replace Abstract: We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural lan

Valdi: Value Diffusion World Models

arXiv:2607.00917v1 Announce Type: cross Abstract: World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive e

Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs

arXiv:2607.00302v1 Announce Type: new Abstract: Touch supplies the physical grounding needed to perceive intrinsic material properties, such as friction and compliance, that vision alone often cannot

Agents

Agentic AI Enhances Physician Trust in Clinical Decision Making

arXiv:2606.30658v1 Announce Type: cross Abstract: Medical AI has shifted from reasoning to agentic AI, a new paradigm that autonomously invokes external tools during reasoning, rendering intermediate

agentsarxiv-cs-ai

ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models

arXiv:2606.30682v1 Announce Type: cross Abstract: Recent advances in language--audio retrieval have been largely driven by contrastive dual-encoder architectures that align audio and text in a shared

AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

arXiv:2606.30811v1 Announce Type: new Abstract: Audio-video generation has recently gained unprecedented research attention, aiming to synthesize high-quality sounding video content with fine-grained

Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding

arXiv:2606.31169v1 Announce Type: new Abstract: Existing AI-assisted oracle bone inscription (OBI) visual recognition and understanding studies mainly focus on character-level, ignoring the long-form

Bridging Video Understanding and Generation in a Unified Framework

arXiv:2606.31326v1 Announce Type: new Abstract: Recently, unified image generation and understanding have been extensively explored. However, extending such unified modeling paradigms to the video dom

ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents

arXiv:2606.31174v1 Announce Type: new Abstract: Production large language-model (LLM) agents are increasingly deployed not as lone problem-solvers but as managers: a main model creates specialized sub

Applications

CryoACE: An Atom-centric Framework for Accurate and Automated Model Building in Cryo-EM

arXiv:2606.31332v1 Announce Type: new Abstract: Protein automodeling from cryo-EM density maps faces unique challenges in enforcing physicochemical validity and managing conformational heterogeneity.

applicationsarxiv-cs-ai

Agents

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

arXiv:2606.31980v1 Announce Type: new Abstract: Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a mul

agentsarxiv-cs-cl

Dual Sparse Aggregation Transformer for Multispectral Object Detection

arXiv:2606.31015v1 Announce Type: new Abstract: Transformer-based approaches have obtained excellent performance in multispectral object detection tasks due to their ability to model long-range depend

Editing Everything Everywhere All at Once

arXiv:2606.31278v1 Announce Type: new Abstract: Editing multiple elements of an image in a single forward pass is a practical alternative to multi-turn image manipulation, offering improved efficiency

Applications

EgoCogNav: Cognition-aware Human Egocentric Navigation

arXiv:2511.17581v3 Announce Type: replace-cross Abstract: Modeling the cognitive and experiential factors of human navigation is central to deepening our understanding of human-environment interaction

applicationsarxiv-cs-cv

ENPIRE -> ASPIRE, our 2nd work in the series for Physical AutoResearch. We are building the components for robot self-improvement, one /skil…

ENPIRE -> ASPIRE, our 2nd work in the series for Physical AutoResearch. We are building the components for robot self-improvement, one /skill at a time. Today, we give robots a /skills library that se

researchjim-fan--x

FROST: Training-Free Few-Shot Segmentation with Frozen Features and Nonparametric Statistics

arXiv:2606.31136v1 Announce Type: new Abstract: Few-shot segmentation asks a model to delineate a target class in a query image from only a handful of annotated examples, a setting most acute in remot

Gemini Omni Flash: Image to Video https://links.comfy.org/4y3e93O

Gemini Omni Flash is a model capability that generates video content from still images, likely integrated into or demonstrated through ComfyUI's node-based interface. This feature represents advanceme

model-releasescomfyui--x

Gemini Omni Flash: Video Edit https://links.comfy.org/445S9aM

Gemini Omni Flash: Video Edit is a ComfyUI workflow or tool that enables video editing capabilities using Google's Gemini Omni Flash model, likely allowing users to perform AI-assisted video editing t

model-releasescomfyui--x

Human-Agent Collaborative Paper-to-Page Crafting

arXiv:2510.19600v2 Announce Type: replace-cross Abstract: In the quest for scientific progress, communicating research is as vital as the discovery itself. Yet, researchers are often sidetracked by th

M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

arXiv:2601.08758v4 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning, a

MIRTH: Mutual-Information Reasoning with Temporal Hubs for Vision-Language-Action Agents

arXiv:2606.31167v1 Announce Type: cross Abstract: VLA models have emerged as a powerful paradigm for transferring semantic knowledge from web-scale data to physical robotic control. However, current s

PGUDA: Pressure-Guided Unsupervised Domain Adaptation with Cross-Modal Knowledge Distillation for sEMG-Based Gesture Recognition

arXiv:2606.31349v1 Announce Type: cross Abstract: Surface electromyography (sEMG)-based gesture recognition has emerged as a promising technology for natural human-computer interaction. However, its p

Physics-informed Conditional Normalizing Flows for Angles-only Cislunar Orbit Determination

arXiv:2606.30936v1 Announce Type: cross Abstract: Generative Astrodynamics is advanced in this work by extending generative modelling to an orbit determination problem in the cislunar environment. The

REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent

arXiv:2511.17442v3 Announce Type: replace-cross Abstract: Foundation Models (FMs) are increasingly integrated into remote sensing (RS) pipelines. These models include unimodal vision encoders and mult

Rethinking Foundation Model Collaboration: Enhancing Specialized Models through Proxy Task Reasoning

arXiv:2606.31157v1 Announce Type: new Abstract: Foundation models are increasingly integrated into embodied intelligence systems, but directly assigning them structured prediction tasks requires preci

UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization

arXiv:2606.31732v1 Announce Type: new Abstract: Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise al

Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation

arXiv:2606.30698v1 Announce Type: new Abstract: Robotic-assisted endovascular interventions demand accurate, stable, and context-aware guidewire navigation in complex and patient-specific vascular ana

safetyarxiv-cs-ro

When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models

arXiv:2604.03316v2 Announce Type: replace Abstract: Attention sinks are defined as tokens that attract disproportionate attention. While these have been studied in single modality transformers, their

Tutorials

World Narrative Model for Highly Controllable Video Generation: A Paradigm Shift from Pixel Sampling to Physical World Orchestration

arXiv:2606.31946v1 Announce Type: new Abstract: The fundamental obstacle to industrial grade video generation is the lack of controllability: existing models treat video as a pixel distribution sampli

tutorialsarxiv-cs-cv

Xiaomi-GUI-0 Technical Report

arXiv:2606.31410v1 Announce Type: new Abstract: Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions s

A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents

model-releasesarxiv-cs-cl

arXiv:2606.29719v1 Announce Type: cross Abstract: Measurements of proprietary LLM evaluators can become invalid within weeks -- we document one case and provide the diagnostic framework to detect it.

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606.29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs). However, directly applyi

BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language

arXiv:2606.30319v1 Announce Type: new Abstract: Modeling the bidirectional correspondence between external sensory stimuli and internal neural activity has emerged as a critical frontier in neuroscien

BrepLLM: Enabling Large Language Models to Understand Boundary Representations

arXiv:2512.16413v2 Announce Type: replace Abstract: Current token-sequence-based Large Language Models (LLMs) struggle to directly process 3D Boundary Representation (B-rep) models that contain comple

Bringing speed and strong cost performance to the market with Gemini Omni Flash and Nano Banana 2 Lite

model-releasesgoogle-cloud-ai

Great creative happens when your tools move at the speed of your ideas. To help you create rich, reliable experiences while reducing regeneration time and costs, we’re adding two new models to Gemini