Search: “multimodal”

AVCap: Reinforcing Audio-Video Joint Caption with Detail-Aware Reward

arXiv:2608.06930v1 Announce Type: new Abstract: Detailed audio-video joint captioning is essential for multimodal video understanding and generation. However, prior works are constrained by three main

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

CAS2UML: A Handwritten Sketch-to-PlantUML Dataset for Class and Activity Diagrams

DGX agent

arXiv:2608.07036v1 Announce Type: cross Abstract: Automated UML generation from sketches and images is gaining renewed attention with the rise of large language models and multimodal AI. However, repr

researcharxiv-cs-cv

10 Aug 2026

Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning

arXiv:2608.06938v1 Announce Type: cross Abstract: The visual reasoning ability of multimodal large language models (MLLMs) is crucial for downstream applications, particularly counter-commonsense reas

I Seek You in Videos: Identity-Conditioned Queries for Person-Centric Video Reasoning

arXiv:2608.07417v1 Announce Type: cross Abstract: Real-world video reasoning often involves multimodal, multi-source inputs, whereas existing video reasoning tasks typically assume a simplified video-

LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agents

arXiv:2608.06948v1 Announce Type: new Abstract: AI models are becoming increasingly adept at understanding and processing spatial information, thereby facilitating agentic problem-solving in spatial t

Native Long Video Understanding Models locally?

model-releasesr-localllama

I've been building a personal project and wanted to check with the community on multi-modal inputs since I can't find a lot of material around this online. Ultimately I'm trying to build something tha

Playing with physics is so cool in Minimax H3

model-releasesr-stablediffusion

Prompt: 'integrated_multimodal_description: [Shot 1] Live-action, ultra-realistic first-person footage at night on a rainy city street, filmed with authentic handheld smartphone qualities. The phone i

ReGraph: Learning to Generate Recipe Graphs from Food Images

arXiv:2608.06917v1 Announce Type: new Abstract: Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transf

RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs

arXiv:2608.07088v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) encode images as long visual token sequences, making prefilling and KV-cache storage expensive. Existing trai

Together Serverless Inference gives developers a high-throughput, managed production path for running Muse Glimmer across agentic and multim…

Together Serverless Inference gives developers a high-throughput, managed production path for running Muse Glimmer across agentic and multimodal workloads. Start building: https://www.together.ai/mode

agentstogether-ai--x

Toward a Causal Data Management Ecosystem for Decision Making and Agentic AI

arXiv:2608.07214v1 Announce Type: cross Abstract: Modern AI is no longer a single model but an ecosystem: classical ML predictors, deep and multimodal models, large language models, and agents, each t

agentsarxiv-cs-ai

Local Ai

What Characters Minimax H3 knows - Part 2 - Videogames

local-air-stablediffusion

Here is the second Edition, Videogames. Workflow is the same as in the first Part, its pretty simple: This is the Prompt: Brad Pitt integrated_multimodal_description: [Shot 1] Live-action, contemporar

A Six-Dimensional Taxonomy of Post-Training Adaptation Techniques with Applications in AI Governance

model-releasesarxiv-cs-lg

arXiv:2608.06246v1 Announce Type: new Abstract: Post-training adaptation has become central to modern machine learning practice and includes techniques such as retraining, fine-tuning, parameter-effic

Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs

arXiv:2608.05592v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong progress in video understanding, yet it remains challenging because the token limitation m

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

arXiv:2603.08652v2 Announce Type: replace Abstract: Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integra

Decoupling Perception from Description: Computation-Grounded Representation Alignment between Multivariate Time Series and Language

model-releasesarxiv-cs-lg

arXiv:2608.05238v1 Announce Type: new Abstract: Training multimodal models to align time series with language runs into a self-supervision trap. The usual recipe asks an LLM to read a series and write

DistMedVL: Distributional Vision-Language Alignment for Uncertainty-Aware Medical Image Segmentation

arXiv:2608.05683v1 Announce Type: cross Abstract: Cross-modal alignment of visual and textual representations is fundamental to multimodal medical image understanding, yet remains hindered by uncertai

Nonvisual Classification of Ground-Condition by Artificial Proprioception in an Amoeba-Inspired Autonomous Walking Robot

arXiv:2608.05684v1 Announce Type: cross Abstract: Nonvisual classification of ground condition based on a multimodal sensing approach was investigated for an amoeba-inspired autonomous walking robot.

agentsarxiv-cs-ai

One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

arXiv:2608.05707v1 Announce Type: new Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows. Sinc

StepReflect: Structured UI Transition Reflection for Mobile GUI Agents

arXiv:2608.05587v1 Announce Type: new Abstract: Autonomous mobile GUI agents require accurate action reflection for reliable long-horizon execution. Existing approaches rely on open-ended multimodal r

The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images

arXiv:2608.06270v1 Announce Type: new Abstract: The 'thinking-with-images' paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations o

Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders

arXiv:2608.04586v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT). However, when processing multilingual

Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting

arXiv:2607.15890v2 Announce Type: replace Abstract: Perceiving multimodal cues and forecasting fine-grained actions from an egocentric (Ego) perspective is vital for applications like robot manipulati

MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding

arXiv:2608.04587v1 Announce Type: new Abstract: Long-form video understanding requires locating sparse, question-relevant evidence in long, multimodal videos. Real-world video distributions differ in

agentsarxiv-cs-cv

Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos

model-releasesarxiv-cs-cl

arXiv:2608.04939v1 Announce Type: new Abstract: Social media videos often communicate meanings that go beyond their visible actions, captions, or speech. A mundane clip may become humorous, ironic, or

RUTA: Principled Visual Token Allocation via Rate-Utility Optimization

arXiv:2608.04132v1 Announce Type: new Abstract: High-resolution images and long videos provide vision-language models with rich context for multimodal reasoning and fine-grained perception, but the re

SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for Unpaired RGB+Thermal 3D Reconstruction

arXiv:2603.18774v2 Announce Type: replace Abstract: Foundational feed-forward visual geometry models enable accurate and efficient camera pose estimation and scene reconstruction by learning strong sc

The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering

arXiv:2608.04589v1 Announce Type: cross Abstract: EgoCross is a cross-domain egocentric video question answering benchmark designed to evaluate whether multimodal large language models can generalize

Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding

arXiv:2607.11844v2 Announce Type: replace Abstract: Recent Multimodal Large Language Models (MLLMs) achieve strong performance on single-view video understanding benchmarks. However, sports videos inv

ChartAnno: Evaluating MLLMs for Chart Annotation Generation

arXiv:2608.03464v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made significant progress in chart understanding, generation, and editing, but their ability to annotate e

Local Ai

CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment

arXiv:2608.03247v1 Announce Type: cross Abstract: Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, d

local-aiarxiv-cs-cl

DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning

arXiv:2608.03292v1 Announce Type: new Abstract: Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneo

HERO: Hierarchical Evidential Reasoning Optimization for Radiology Report Generation via Reason-then-Summarize

arXiv:2601.03321v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have substantially advanced Radiology Report Generation (RRG), yet aligning them through reinforcemen

Learning Molecular Representations from Cellular Phenotypes with Structure Preservation

arXiv:2608.02688v1 Announce Type: cross Abstract: Phenotypic drug discovery enables the discovery of functional relationships between molecular structures and cellular responses. However, existing mul

Mistral introduces Shieldstral to provide lightweight policy-aware moderation for AI models

model-releasessiliconangle

French artificial intelligence startup Mistral AI SAS today introduced a lightweight multimodal safety artificial intelligence open-weight model that can classify outputs for AI models that outperform

Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding

arXiv:2608.02980v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) have achieved remarkable success on images and short videos, yet scaling them to long videos remains challenging due to f

UHP Detection: LVLMs have their Unique Hallucination Pattern in the Consistency Space

arXiv:2608.03817v1 Announce Type: cross Abstract: Large vision--language models (LVLMs) demonstrate strong multimodal reasoning capabilities but remain prone to hallucination, where model predictions

A Benchmark Dataset for MLLM-Generated Image Detection: GPT Image2 & Nano Banana2

arXiv:2608.01258v1 Announce Type: new Abstract: The realism of images generated by multimodal large language models (MLLMs), such as GPT Image2 and Nano Banana2, has improved rapidly in recent years.

ARMOR: A Robust Self-Supervised Framework for Root Cause Analysis in Microservices under Missing Modality

arXiv:2603.25538v3 Announce Type: replace Abstract: Automated incident management is critical for microservice reliability. While recent unified frameworks leverage multimodal data for joint optimizat

safetyarxiv-cs-lg

Attention-Steered Vision-Language Models for Sign Language Translation

arXiv:2608.00235v1 Announce Type: new Abstract: Vision-language models (VLMs) have emerged as a powerful framework for multimodal video understanding. However, they remain limited in the sign language

Beyond Accuracy: Auditing Spatial Provenance in Visual Token Pruning for OCR-Critical MLLM Inference

arXiv:2608.00077v1 Announce Type: new Abstract: Visual-token pruning is usually judged by answer quality at a fixed retention budget. For text-rich multimodal large language models (MLLMs), this proto

CAPEval: A Decoupled Caption Evaluation across Understanding and Generation

arXiv:2608.02589v1 Announce Type: new Abstract: Captions serve as a primary supervision signal for both multimodal understanding and text-to-image generation. However, previous evaluations treat the c

Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception

arXiv:2608.01055v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-obj

CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation

model-releasesarxiv-cs-cl

arXiv:2608.01942v1 Announce Type: cross Abstract: Text-to-video (T2V) generation models have advanced rapidly, yet their ability to represent diverse cultural contexts remains underexplored. Existing

DiffuseAgent-MI: Distributionally-Grounded,Tool-Integrated Self-Evolving Agents for Faithful Visual Reasoning

arXiv:2608.00540v1 Announce Type: new Abstract: Tool-integrated vision-language agents have made remarkable progress on compositional and multi-step visual reasoning. Yet their outputs frequently exhi

safetyarxiv-cs-cv

DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable

arXiv:2608.00548v1 Announce Type: new Abstract: Recent image-generation models and multimodal agents can produce high-quality visuals for increasingly complex visual communication tasks. Yet their ras

Local Ai

Dynamic Resolution Routing for Efficient Egocentric Grounding

arXiv:2608.01638v1 Announce Type: new Abstract: Egocentric visual grounding requires high-resolution inputs to localize small objects. However, scaling Multimodal Large Language Models to this domain

local-aiarxiv-cs-cv

Tutorials

Learning Where to Look and How to Judge: Resolution-agnostic Image Quality Assessment with Quality-aware Saliency

arXiv:2608.01730v1 Announce Type: new Abstract: No-reference image quality assessment (NR IQA) has recently benefited from deep and multimodal models, yet many SOTA systems still violate at least one

tutorialsarxiv-cs-cv