Search: “multimodal”

Announcing the Google Gen AI SDK for Kotlin 1.0: Idiomatic multiplatform access to Gemini

model-releasesgoogle-cloud-ai

Integrating modern generative AI capabilities into Kotlin applications shouldn't require juggling raw HTTP clients or bridging disparate Java libraries. Today, we're excited to announce the 1.0 releas

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness

DGX agent

arXiv:2609.02512v1 Announce Type: new Abstract: Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular amongst users, companies, and aestheticians. This raises the q

model-releasesarxiv-cs-cv

3 Sept 2026

DiDrive: A Risk-Aware Hierarchical Diffusion Framework for Safe Offline Reinforcement Learning in Autonomous Driving

model-releasesarxiv-cs-lg

arXiv:2609.01609v1 Announce Type: new Abstract: While diffusion models effectively capture multimodal behavioral priors for autonomous driving, offline reinforcement learning (RL) policies remain susc

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

arXiv:2609.02344v1 Announce Type: cross Abstract: Existing cell embedding methods predominantly rely on transcriptomic or proteomic measurements and represent each cell as a holistic entity, thereby o

local-aiarxiv-cs-ai

Testing orangesouth/MinimaxH3CinematicRealism and vpakarinen/better-human-motion-h3-lora 'dean winchester vs sub zero' pruned fp8 model no turbo lora 32steps

model-releasesr-stablediffusion

https://huggingface.co/orangesouth/MinimaxH3CinematicRealism/tree/main https://huggingface.co/vpakarinen/better-human-motion-h3-lora/tree/main prompt integrated_multimodal_description: dy, [Shot 1] Ph

Towards Zero-Shot Transfer Across Embodiments For Driving VLAs

arXiv:2609.02341v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) have shown strong potential in autonomous driving by leveraging multimodal pretraining for instruction following, v

agentsarxiv-cs-cv

Video2Reaction: Training Foundation Video Models to Predict Audience Reaction

arXiv:2609.01816v1 Announce Type: new Abstract: We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed

A Glance Is All You Need: Single-Pass Fine-Grained Image Captioning with SimLoss

arXiv:2609.00591v1 Announce Type: new Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level ca

local-aiarxiv-cs-cv

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieva

ExBind: A Controlled Diagnostic Benchmark for Visual-to-Executable Correspondence

arXiv:2609.01344v1 Announce Type: new Abstract: Multimodal coding and editing systems must map a visible or semantic referent to the exact executable object that can be edited. A wrong reference may s

Jailbreaking Text-to-Image Models Through Cracks: Navigating Heterogeneous Safety Filters via Multi-Agent Debate

arXiv:2609.01168v1 Announce Type: new Abstract: Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by he

Risk-Aware Decision-Making for Autonomous Overtaking: A World Model-Based Mixture-of-Experts Framework

arXiv:2609.00385v1 Announce Type: cross Abstract: Autonomous highway overtaking demands foresighted decision-making to handle complex interactions, stochastic traffic evolution, and temporal risk accu

local-aiarxiv-cs-ai

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inpu

researcharxiv-cs-cl

UI-Venus-2 Technical Report

arXiv:2609.00028v1 Announce Type: new Abstract: Multimodal GUI agents have emerged as a promising paradigm for digital task automation, yet transitioning from benchmark-oriented models to dependable r

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tuned

researcharxiv-cs-cl

Adversarial Calibration Attack on Autonomous Vehicles

arXiv:2608.28778v1 Announce Type: cross Abstract: Autonomous vehicles (AVs) rely on accurate camera-LiDAR calibration for multimodal sensor fusion. In practice, calibration can drift due to vibration,

Beyond Static Visual Tokens: Structured Sequential Visual Chain-of-Thought Reasoning

arXiv:2603.26737v2 Announce Type: replace-cross Abstract: Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual acce

researcharxiv-cs-ai

Beyond Visual Boundaries: Rethinking Scene Segmentation for Movie RAG

arXiv:2608.28699v1 Announce Type: new Abstract: Understanding long-form video remains a fundamental challenge for multimodal large language models (MLLMs). Sparse frame sampling fails to capture fine-

researcharxiv-cs-cv

DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering

arXiv:2608.29037v1 Announce Type: cross Abstract: Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimoda

Drift Calibration in Geometric Eye Tracking Systems

arXiv:2608.29739v1 Announce Type: new Abstract: Geometric eye trackers can provide the spatial accuracy required for gaze-based interaction and multimodal studies, but their measurements remain sensit

EmoLASP: Emotion Recognition with Language Models and Answer Set Programming

arXiv:2608.29035v1 Announce Type: new Abstract: Emotion recognition in conversations is increasingly tackled with language models, but these models can be unstable and expensive to fine-tune or to pro

Gurukul AI: An Interactive AI-Driven Educational Platform for Indian Education System

arXiv:2608.28611v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) like ChatGPT and LLaMA have transformed AI-driven education, but these systems are predominantly train

HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving

arXiv:2608.30672v1 Announce Type: new Abstract: Recent advances in large language models and multimodal models have pushed remote sensing (RS) processing from simple perception models to agentic syste

Tutorials

Instruction Distillation: Text Instructions as Visual Examples

arXiv:2608.28696v1 Announce Type: new Abstract: Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved i

tutorialsarxiv-cs-cv

Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey

arXiv:2508.13073v3 Announce Type: replace-cross Abstract: Robotic manipulation, a key frontier in robotics and embodied AI, requires precise motor control and multimodal understanding, yet traditional

agentsarxiv-cs-cv

MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation

arXiv:2608.29304v1 Announce Type: new Abstract: Translating electroencephalography (EEG) into functional magnetic resonance imaging (fMRI) is important for medical neuroimaging, clinical brain-state m

researcharxiv-cs-lg

No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpus

arXiv:2608.28875v1 Announce Type: cross Abstract: Supplying context at inference time to a large multimodal model is an inexpensive lever for adapting speech transcription to a domain, and earlier res

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background c

researcharxiv-cs-ai

PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents

arXiv:2608.30760v1 Announce Type: new Abstract: Recent studies have shown that multimodal large language models (MLLMs) can serve as embodied agents, translating language instructions and visual obser

safetyarxiv-cs-lg

RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generation

arXiv:2608.30727v1 Announce Type: cross Abstract: Small-object detection under long-tailed data distributions is a fundamental yet challenging problem in multimedia. Railway Foreign Object Detection (

Reactivating Test-Time Scaling for Plane Geometry Problem Solving

arXiv:2608.30156v1 Announce Type: new Abstract: Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise

ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering

arXiv:2608.28707v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in Visual Question Answering (VQA), yet they continue to struggle with questi

researcharxiv-cs-cl

Review Before Trust: Source-Grounded Integrity Gates for AI-Assisted Personal Health Records

arXiv:2608.29965v1 Announce Type: new Abstract: Large language models can convert medical documents into structured data, but plausible output may still be unsupported by the source. Persisting such o

RLG-TPV: Radar- and LiDAR-Guided Tri-Perspective View Fusion for Camera-Radar 3D Object Detection

arXiv:2608.29194v1 Announce Type: new Abstract: Tri-Perspective View (TPV) representations describe 3D scene structure through top, side, and front feature planes, but existing TPV lifting is primaril

researcharxiv-cs-cv

Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

arXiv:2602.08136v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic i

SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos

arXiv:2608.30716v1 Announce Type: new Abstract: Recent advances in Large Multimodal Models (LMMs) have greatly improved video understanding, yet their ability to reason about human-centered social sit

TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film

arXiv:2608.30068v1 Announce Type: new Abstract: Films communicate through deliberate creative choices, including lighting, color, composition, editing, dialogue, music, and sound. Humans naturally int

The Race between Agentic AI Capabilities and Data Quality Control in Online Surveys

arXiv:2608.28597v1 Announce Type: new Abstract: Online surveys are a foundational data collection instrument in a variety of fields, with attention checks serving as critical guardians of response qua

agentsarxiv-cs-ai

VocalAffectBench: Evaluating Vocal Emotion Recognition in AI Audio Models

arXiv:2608.28932v1 Announce Type: new Abstract: Voice products increasingly need affective cues that are present in speech but absent from transcripts. We introduce VocalAffectBench, a public, test-on

Benchmarking General Mobile Assistants in Challenging Real-World Scenarios

arXiv:2608.27477v1 Announce Type: cross Abstract: Graphical user interfaces have emerged as an important environment for evaluating autonomous AI agents on multimodal interactive tasks. Existing bench

CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning

model-releasesarxiv-cs-ro

arXiv:2608.28266v1 Announce Type: new Abstract: Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still la

Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents

arXiv:2608.27866v1 Announce Type: new Abstract: Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal La

safetyarxiv-cs-cv

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

arXiv:2608.28192v1 Announce Type: new Abstract: Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval.

local-aiarxiv-cs-cv

Synthetic Linguistic Agency: How an Embodied Mortal Agent Learns Linguistic Affordances through Consequential Social Experience

arXiv:2608.27843v1 Announce Type: new Abstract: Contemporary language models can converse fluently and influence human decisions, yet their exchanges do not enter a continuing, vulnerable life of thei

agentsarxiv-cs-cl

Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection

arXiv:2509.24192v2 Announce Type: replace Abstract: Vision-language models (VLMs) have advanced multimodal perception, demonstrated by open-vocabulary object detection with simple language queries. St

an underrated part of glm-5.3 is its low hallucination rate claude fable 5 is over 2x higher, while gpt-5.6 luna is over 3x higher

model-releasestogether-ai--x

GLM‑5.3 demonstrates a notably lower hallucination rate compared with other contemporary large language models. According to a report by Together AI at TogetherCompute (published August 30 2026), Clau

30 Aug 2026

AdaThinking-E: One-Token Entropy Regulation for Adaptive Thinking

arXiv:2608.26141v1 Announce Type: new Abstract: Multimodal large language models have demonstrated strong document reasoning capabilities by incorporating explicit thinking processes. While this capab

28 Aug 2026

Beyond Accuracy: A Qualitative Analysis of Vision-Language Models for Hate Speech Detection in Memes

arXiv:2608.26143v1 Announce Type: cross Abstract: Memes have turned out to be a powerful tool through which individuals share their ideas concerning contemporary social and political problems. Their a