Search: “multimodal”

DreamOmni3: Scribble-based Editing and Generation

arXiv:2512.22525v2 Announce Type: replace Abstract: Recently unified generation and editing models have achieved remarkable success with their impressive performance. These models rely mainly on text

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving

DGX agent

arXiv:2608.10413v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for end-to-end autonomous driving by enabling unified reasoning across

model-releasesarxiv-cs-cv

12 Aug 2026

DuplexWorld: Can voice agents help you get through the day?

arXiv:2608.10716v1 Announce Type: cross Abstract: Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing

agentsarxiv-cs-ai

GraFine: Retrieval-Time Refinement for Efficient Graph RAG over Corpus Graphs

arXiv:2601.18579v2 Announce Type: replace-cross Abstract: Graph RAG on corpus graphs enhances retrieval by leveraging intermediate node content as contextual clues to uncover unretrieved oracle nodes.

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

arXiv:2608.09991v1 Announce Type: cross Abstract: Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging durin

Multiclass Sentiment Analysis for Identifying Political Viewpoints

arXiv:2608.11049v1 Announce Type: cross Abstract: The rapid growth of social media has created vast amounts of political discourse, which provides valuable opportunities to analyze public opinions and

SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception

arXiv:2608.10497v1 Announce Type: new Abstract: While foundation models have significantly advanced human recognition across diverse modalities, they predominantly rely on static, geometric feature ex

safetyarxiv-cs-cv

Situation Graph Prediction for User Perspective Modeling

arXiv:2602.13319v2 Announce Type: replace Abstract: Perspective-aware AI requires modeling evolving internal states---goals, emotions, contexts---not merely preferences. Progress is limited by a data

Surfacing the Unsaid: CUE-Bench for Affective Stance in Chinese Discourse

arXiv:2608.10810v1 Announce Type: cross Abstract: Emotion understanding in discourse requires reasoning beyond surface sentiment because speakers often convey affect through indirect, implicit, polite

Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

arXiv:2608.11191v1 Announce Type: cross Abstract: GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ab

The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset

arXiv:2608.10839v1 Announce Type: new Abstract: This preprint presents the results of the fourth GENEA Challenge, a large-scale human evaluation of five speech-driven gesture-generation systems traine

safetyarxiv-cs-cv

TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs

arXiv:2608.00422v2 Announce Type: replace Abstract: Large language models (LLMs) can generate fluent reasoning traces that nevertheless lead to incorrect answers, making response-level uncertainty est

TRACE-GS: On-Policy Trajectory Distillation with Privileged Geometric Conditioning for Sparse-View 3DGS Restoration

arXiv:2608.10286v1 Announce Type: new Abstract: We present TRACE-GS, an on-policy trajectory distillation framework that leverages privileged geometric conditioning at training time, thereby adapting

safetyarxiv-cs-cv

Local Ai

VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics

arXiv:2608.11201v1 Announce Type: new Abstract: Recent advances in video generation models have significantly improved the realism of synthetic videos, blurring the boundary between generated and auth

local-aiarxiv-cs-cv

Local Ai

When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs

arXiv:2608.10489v1 Announce Type: new Abstract: Abundant visual information strengthens vision-language model (VLM) perception, yet massive visual tokens raise inference costs. Existing visual token p

local-aiarxiv-cs-cv

Applications

BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoning

arXiv:2608.07742v1 Announce Type: new Abstract: Visual-language models (VLMs) frequently struggle with robustness issues in real-world situations due to low- or varying-quality input images. In this p

applicationsarxiv-cs-cv

CircuitReason-1k: Benchmarking Long-Horizon Visual-to-Symbolic Reasoning inElectrical Circuits

arXiv:2608.09374v1 Announce Type: new Abstract: Electrical circuit analysis requires more than recognizing components in an image. A solver must ground symbols and labels, recover latent topology, sel

Describe-to-Score: A text-guided framework for image complexity assessment

arXiv:2509.16609v2 Announce Type: replace Abstract: Accurately assessing image complexity (IC) is essential for many vision tasks, yet existing approaches rely almost exclusively on visual features an

Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards

arXiv:2608.07535v1 Announce Type: cross Abstract: Multi-modal large language models (MLLMs) integrate heterogeneous modalities through modality alignment and fusion, enabling stronger understanding an

safetyarxiv-cs-ai

Frequency-Domain Dual-Branch Fusion for Medical Visual Question Answering

arXiv:2608.08307v1 Announce Type: cross Abstract: Medical Visual Question Answering (VQA) requires aligning subtle visual evidence, including lesion texture, boundary sharpness, and diffuse density ch

Goal-oriented Navigation Instruction Generation with Tour Video Priors

arXiv:2608.08596v1 Announce Type: new Abstract: Navigation Instruction Generation (NIG) aims to produce step-by-step natural language instructions for navigation guidance. Existing studies primarily t

Learning Physical Interaction: A Survey of Tactile- and Force-aware Robot Learning

arXiv:2608.07558v1 Announce Type: cross Abstract: Physically grounded robot intelligence requires robots to perceive, reason about, and regulate their interactions with the physical world. This capabi

SpikeWorld: Fast-State Adaptation for Frozen Spiking World Models

arXiv:2608.07712v1 Announce Type: cross Abstract: A predictive model receives a self-supervised signal whenever the consequence of an action is observed. Using that signal after deployment is difficul

VIGIL: Tackling Hallucination Detection in Image Recontextualization

arXiv:2602.14633v2 Announce Type: replace Abstract: We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a benchmark dataset and framework that provides a fine-grained categoriz

VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling

arXiv:2608.08630v1 Announce Type: new Abstract: Vision Language Models (VLMs) face significant challenges with ultra-long, interleaved image-text sequences due to the quadratic complexity of self-atte

What to Edit Next: Visually Aligned Image-Editing Follow-Up Suggestions in Conversational Systems

arXiv:2608.07565v1 Announce Type: cross Abstract: Conversational assistants increasingly recommend follow-up edits to help users continue a task. Existing systems primarily target text-only interactio

ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models

arXiv:2608.08060v1 Announce Type: new Abstract: Fine-tuning vision-language models such as CLIP typically requires backpropagation (BP) through the full model, which is infeasible when only forward-pa

Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding

arXiv:2608.06532v1 Announce Type: new Abstract: LVLMs are increasingly used to read financial charts, tables, and documents, where a single misread figure can move a decision and the most authoritativ

safetyarxiv-cs-cl

Local Ai

ControlRef: Efficient Layout-Guided Multi-Instance Generation via Anchored 4D-RoPE

arXiv:2608.06878v1 Announce Type: new Abstract: Layout-guided multi-instance generation is essential for controllable image synthesis in Multi-Modal Diffusion Transformers (MM-DiTs). However, integrat

local-aiarxiv-cs-cv

DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding

arXiv:2608.07067v1 Announce Type: new Abstract: Long-document understanding requires locating sparse and heterogeneous evidence across hundreds of pages, yet existing systems remain limited by static

Georeferencing Non-Gazetteered Place Names using Biological Specimen Records

arXiv:2608.06884v1 Announce Type: cross Abstract: Biological specimen records collected by natural history institutions constitute a rich source of temporal geographic knowledge, capturing biodiversit

How Should I Pick a Foundation Model for My Robot? In Favor of a Community Evaluation Framework for Social Robots

arXiv:2608.06898v1 Announce Type: cross Abstract: Researchers who seek to build social robot applications on foundation models are faced with a difficult question: how should we pick a model? Public l

safetyarxiv-cs-cl

International Transfer of Stochastic Cortical Self-Reconstruction

arXiv:2608.07092v1 Announce Type: cross Abstract: Stochastic cortical self-reconstruction (SCSR) enables personalized mapping of gray matter atrophy, a hallmark of neurodegenerative disorders such as

Latent Fact-Checking: Detecting Misinformation through Activation Engineering

model-releasesarxiv-cs-cl

arXiv:2608.06417v1 Announce Type: cross Abstract: The proliferation of misinformation online has driven demand for scalable detection systems. While most existing approaches rely on surface-level ling

Tutorials

LoRAScan: Detecting Backdoor Prompts in Low-Rank Adapters for Large Language Models via Down-Projection Activation Spikes

arXiv:2608.06795v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) enables efficient specialization and distribution of large language models through compact adapters. However, untrusted ada

tutorialsarxiv-cs-ai

Mobile Interaction for Assessing Fatigue, Sleep, and Activity in Neurodegenerative and Chronic Diseases

arXiv:2608.06380v1 Announce Type: cross Abstract: Fatigue, sleep, or disturbances in daily activities are common symptoms among patients with neurodegenerative disorders (NDD) and immune-mediated infl

PsychoAgent: An Affect-Sensitive Cognitive Architecture for Conflict-Aware Memory in LLM Agents

arXiv:2608.07438v1 Announce Type: new Abstract: Human-like cognition does not select past experience by topical similarity alone: affective significance and unresolved conflict also shape what becomes

SkillProx: Self-Evolving Agent Skills via Proximal Textual Gradient Descent

arXiv:2608.07449v1 Announce Type: new Abstract: LLM agents increasingly adapt to recurring tasks by accumulating procedural knowledge in skills. These skills are lightweight, reusable textual artifact

agentsarxiv-cs-ai

SLED: Scalable Location Encoding via Distillation

arXiv:2608.06612v1 Announce Type: cross Abstract: The plethora of readily available geospatial data offers exciting opportunities to learn high quality representations of the planet, but the sheer siz

Stockmark-Nemotron-3-Nano-Omni-JapanDocReader: Structured Document Parsing via Capability Injection and Forgetting Control

model-releasesarxiv-cs-cl

arXiv:2608.06758v1 Announce Type: new Abstract: We present Stockmark-Nemotron-3-Nano-Omni-JapanDocReader, a Japanese document understanding model built from Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products

arXiv:2606.15485v2 Announce Type: replace-cross Abstract: Agentic AI systems act autonomously, use tools, adapt to context, and operate in complex real-world environments. However, these same characte

agentsarxiv-cs-ai

Applications

Towards a Theoretical Understanding of Two Tower Recommendation Models

arXiv:2403.00802v2 Announce Type: replace-cross Abstract: Production-grade recommender systems rely heavily on a large-scale corpus used by online media services, including Netflix, Pinterest, and Ama

applicationsarxiv-cs-ai

UAV3DCrop: Benchmarking 3D Reconstruction in Repeated Multi-Angle UAV Crop Surveys

arXiv:2608.06404v1 Announce Type: new Abstract: Accurate 3D crop monitoring underpins data-driven precision agriculture by enabling field-scale analysis of plant structure, growth dynamics, and manage

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

arXiv:2608.07051v1 Announce Type: new Abstract: Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous op

MoCA: Implicit Social Context Analysis

model-releasesarxiv-cs-cl

arXiv:2608.05825v1 Announce Type: new Abstract: Human social communication, such as affection and intent, is often conveyed in highly implicit ways, where underlying meanings are expressed through ind

7 Aug 2026

EmpaAva: An Open-source Agentic 3D-Avatar Empathetic Live Chatbot

arXiv:2608.04709v1 Announce Type: new Abstract: This paper presents EmpaAva, to our knowledge the first open-source, agentic 3D-avatar empathetic chatbot, which carries empathetic response generation

agentsarxiv-cs-cl

6 Aug 2026

SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding

arXiv:2608.05137v1 Announce Type: new Abstract: Understanding 3D scenes is fundamental to embodied intelligence, requiring joint reasoning over heterogeneous information from multiple modalities, incl

6 Aug 2026

Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation

arXiv:2608.02791v1 Announce Type: new Abstract: MLLM-based segmentation faces a core segmentation trilemma: high segmentation performance, preserved dialogue ability, and fast inference. Embedding-pre