Search: “multimodal”

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tuned

researcharxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Adversarial Calibration Attack on Autonomous Vehicles

DGX agent

arXiv:2608.28778v1 Announce Type: cross Abstract: Autonomous vehicles (AVs) rely on accurate camera-LiDAR calibration for multimodal sensor fusion. In practice, calibration can drift due to vibration,

model-releasesarxiv-cs-cv

1 Sept 2026

Beyond Static Visual Tokens: Structured Sequential Visual Chain-of-Thought Reasoning

arXiv:2603.26737v2 Announce Type: replace-cross Abstract: Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual acce

Beyond Visual Boundaries: Rethinking Scene Segmentation for Movie RAG

arXiv:2608.28699v1 Announce Type: new Abstract: Understanding long-form video remains a fundamental challenge for multimodal large language models (MLLMs). Sparse frame sampling fails to capture fine-

researcharxiv-cs-cv

DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering

arXiv:2608.29037v1 Announce Type: cross Abstract: Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimoda

Drift Calibration in Geometric Eye Tracking Systems

arXiv:2608.29739v1 Announce Type: new Abstract: Geometric eye trackers can provide the spatial accuracy required for gaze-based interaction and multimodal studies, but their measurements remain sensit

EmoLASP: Emotion Recognition with Language Models and Answer Set Programming

arXiv:2608.29035v1 Announce Type: new Abstract: Emotion recognition in conversations is increasingly tackled with language models, but these models can be unstable and expensive to fine-tune or to pro

Gurukul AI: An Interactive AI-Driven Educational Platform for Indian Education System

arXiv:2608.28611v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) like ChatGPT and LLaMA have transformed AI-driven education, but these systems are predominantly train

HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving

arXiv:2608.30672v1 Announce Type: new Abstract: Recent advances in large language models and multimodal models have pushed remote sensing (RS) processing from simple perception models to agentic syste

Tutorials

Instruction Distillation: Text Instructions as Visual Examples

arXiv:2608.28696v1 Announce Type: new Abstract: Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved i

tutorialsarxiv-cs-cv

Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey

arXiv:2508.13073v3 Announce Type: replace-cross Abstract: Robotic manipulation, a key frontier in robotics and embodied AI, requires precise motor control and multimodal understanding, yet traditional

agentsarxiv-cs-cv

MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation

arXiv:2608.29304v1 Announce Type: new Abstract: Translating electroencephalography (EEG) into functional magnetic resonance imaging (fMRI) is important for medical neuroimaging, clinical brain-state m

researcharxiv-cs-lg

No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpus

arXiv:2608.28875v1 Announce Type: cross Abstract: Supplying context at inference time to a large multimodal model is an inexpensive lever for adapting speech transcription to a domain, and earlier res

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background c

PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents

arXiv:2608.30760v1 Announce Type: new Abstract: Recent studies have shown that multimodal large language models (MLLMs) can serve as embodied agents, translating language instructions and visual obser

safetyarxiv-cs-lg

RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generation

arXiv:2608.30727v1 Announce Type: cross Abstract: Small-object detection under long-tailed data distributions is a fundamental yet challenging problem in multimedia. Railway Foreign Object Detection (

safetyarxiv-cs-ai

Reactivating Test-Time Scaling for Plane Geometry Problem Solving

arXiv:2608.30156v1 Announce Type: new Abstract: Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise

ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering

arXiv:2608.28707v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in Visual Question Answering (VQA), yet they continue to struggle with questi

researcharxiv-cs-cl

Review Before Trust: Source-Grounded Integrity Gates for AI-Assisted Personal Health Records

arXiv:2608.29965v1 Announce Type: new Abstract: Large language models can convert medical documents into structured data, but plausible output may still be unsupported by the source. Persisting such o

safetyarxiv-cs-ai

RLG-TPV: Radar- and LiDAR-Guided Tri-Perspective View Fusion for Camera-Radar 3D Object Detection

arXiv:2608.29194v1 Announce Type: new Abstract: Tri-Perspective View (TPV) representations describe 3D scene structure through top, side, and front feature planes, but existing TPV lifting is primaril

researcharxiv-cs-cv

Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

arXiv:2602.08136v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic i

safetyarxiv-cs-ai

SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos

arXiv:2608.30716v1 Announce Type: new Abstract: Recent advances in Large Multimodal Models (LMMs) have greatly improved video understanding, yet their ability to reason about human-centered social sit

TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film

arXiv:2608.30068v1 Announce Type: new Abstract: Films communicate through deliberate creative choices, including lighting, color, composition, editing, dialogue, music, and sound. Humans naturally int

The Race between Agentic AI Capabilities and Data Quality Control in Online Surveys

arXiv:2608.28597v1 Announce Type: new Abstract: Online surveys are a foundational data collection instrument in a variety of fields, with attention checks serving as critical guardians of response qua

agentsarxiv-cs-ai

VocalAffectBench: Evaluating Vocal Emotion Recognition in AI Audio Models

arXiv:2608.28932v1 Announce Type: new Abstract: Voice products increasingly need affective cues that are present in speech but absent from transcripts. We introduce VocalAffectBench, a public, test-on

Benchmarking General Mobile Assistants in Challenging Real-World Scenarios

arXiv:2608.27477v1 Announce Type: cross Abstract: Graphical user interfaces have emerged as an important environment for evaluating autonomous AI agents on multimodal interactive tasks. Existing bench

CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning

model-releasesarxiv-cs-ro

arXiv:2608.28266v1 Announce Type: new Abstract: Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still la

Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents

arXiv:2608.27866v1 Announce Type: new Abstract: Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal La

safetyarxiv-cs-cv

Local Ai

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

arXiv:2608.28192v1 Announce Type: new Abstract: Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval.

local-aiarxiv-cs-cv

Synthetic Linguistic Agency: How an Embodied Mortal Agent Learns Linguistic Affordances through Consequential Social Experience

arXiv:2608.27843v1 Announce Type: new Abstract: Contemporary language models can converse fluently and influence human decisions, yet their exchanges do not enter a continuing, vulnerable life of thei

agentsarxiv-cs-cl

Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection

arXiv:2509.24192v2 Announce Type: replace Abstract: Vision-language models (VLMs) have advanced multimodal perception, demonstrated by open-vocabulary object detection with simple language queries. St

AdaThinking-E: One-Token Entropy Regulation for Adaptive Thinking

arXiv:2608.26141v1 Announce Type: new Abstract: Multimodal large language models have demonstrated strong document reasoning capabilities by incorporating explicit thinking processes. While this capab

Beyond Accuracy: A Qualitative Analysis of Vision-Language Models for Hate Speech Detection in Memes

arXiv:2608.26143v1 Announce Type: cross Abstract: Memes have turned out to be a powerful tool through which individuals share their ideas concerning contemporary social and political problems. Their a

CODE: Cross-Modal Calibration and Dynamic Suppression for Open World Object Detection

arXiv:2608.27214v1 Announce Type: new Abstract: Open World Object Detection (OWOD) built on multimodal foundation models often suffers from semantic ambiguity caused by unidirectional text-to-vision m

Diffusion Policies for Short-Horizon Planning in Robot Crowd Navigation

model-releasesarxiv-cs-lg

arXiv:2608.27158v1 Announce Type: new Abstract: Robot crowd navigation requires safe and efficient decision-making under dense, dynamic, and multimodal human--robot interactions. Existing reinforcemen

Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Prevention

arXiv:2608.26168v1 Announce Type: cross Abstract: The lifecycle of hallucination in LLMs is a concept that enables building solid frameworks on the control and reliability of LLMs in high-stakes envir

Local Ai

Mapping Written Words to Spoken Words in a Different Language Using Only Visual Grounding

arXiv:2608.26925v1 Announce Type: new Abstract: In many low-resource settings, even just eliciting speech for data collection is difficult. One promising approach has been to ask speakers to describe

local-aiarxiv-cs-cl

Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Paper

arXiv:2608.26596v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly capable scientific assistants, yet they remain far from fully autonomous research. This transi

RCMN: Understanding Misleadingness in Influential Public Discourse

arXiv:2608.27358v1 Announce Type: cross Abstract: Influential public discourse shapes public beliefs and can also mislead, not only through what is stated, but also through how information is framed,

Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models

arXiv:2608.27065v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has recently emerged as an effective post-training paradigm that improves policy optimization through dense token-lev

safetyarxiv-cs-cv

AWM: Answerable Working Memory for Long-Document VQA Agents

arXiv:2608.25618v1 Announce Type: new Abstract: Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working

agentsarxiv-cs-cl

Tutorials

Fast Generative Grasping via Lie Group-Constrained MeanFlow

arXiv:2608.26076v1 Announce Type: new Abstract: Grasp synthesis is a core task in robotic manipulation, for which the solution typically forms a multimodal distribution rather than a point estimate. G

tutorialsarxiv-cs-ro

How Robust Are Automated Fact-Checking Systems? A Cross-Benchmark Evaluation

model-releasesarxiv-cs-lg

arXiv:2608.25934v1 Announce Type: cross Abstract: Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a

Local Ai

Planetary Prediction Engine: Autonomous Geospatial Prediction via Intelligent Data Selection and Foundation Model Embeddings

arXiv:2608.26088v1 Announce Type: cross Abstract: Addressing critical global challenges, from food security and disaster risk to disease outbreaks and socio-economic vulnerability, demands high-fideli

local-aiarxiv-cs-lg

When Does Context Routing Help? A Systematic Study of Multi-Modal Fusion in Time Series Forecasting

model-releasesarxiv-cs-lg

arXiv:2608.25128v1 Announce Type: new Abstract: Multi-modal time series forecasting methods integrate auxiliary context into temporal predictions through increasingly sophisticated fusion mechanisms.

TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal

26 Aug 2026

Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation

arXiv:2608.24366v1 Announce Type: new Abstract: End-to-end multimodal driving has progressed rapidly by fusing camera and LiDAR streams. Existing pipelines remain fragile under asymmetric sensor degra

26 Aug 2026

VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference

arXiv:2608.24063v1 Announce Type: cross Abstract: While Vision Large Language Models (VLLMs) have achieved remarkable success in multimodal reasoning, their long-context inference remains prohibitivel