Search: “multimodal”

What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering

arXiv:2608.14841v1 Announce Type: new Abstract: Long-document visual question answering (VQA) over documents of tens to hundreds of pages mixing text, tables, charts, and figures typically follows ret

18 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Qwen Live EP2 - Agent First: Multimodal Gets to Work - Streamed live 11 hours ago

DGX agent

Possibly best way to spend current hour before grabbing 27B model(Instead of opening duplicate repeated 27B posts here). I tried to grab summary(thought of including in this thread) of this video usin

model-releasesr-localllama

14 Aug 2026

Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification

arXiv:2608.11472v1 Announce Type: new Abstract: Pancreatic cancer is among the most lethal malignancies; risk stratification of intraductal papillary mucinous neoplasms (IPMNs) offers a crucial opport

researcharxiv-cs-cv

13 Aug 2026

Qwen-MusicAVQA-7B: A Multimodal Model for Music Audio-Visual QA

arXiv:2608.11329v1 Announce Type: cross Abstract: A common approach to adding audio to a vision-language model is to train or adapt a large omni-modal system. We show that a lightweight alternative ca

13 Aug 2026

Agentic Anomaly Detection with ORCA-Style Dynamic Inductive Bias Adaptation in Multimodal Wearable Time Series Data

arXiv:2608.08859v1 Announce Type: cross Abstract: Wireless Body Area Networks (WBANs) generate multivariate physiological time series that are highly nonstationary and must often be processed under st

Local Ai

Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance

arXiv:2608.09482v1 Announce Type: cross Abstract: All-in-one image restoration is a unified low-level vision task that aims to effectively recover high-quality images from inputs degraded by various t

local-aiarxiv-cs-ai

CFD-Guided Detection of Concept Drift in Multimodal Physiologic Signals

arXiv:2608.07759v1 Announce Type: cross Abstract: Cardiovascular AI models can classify clean elec- trocardiogram (ECG) signals, but real wearable signals change because of motion, breathing, posture,

From Speech to Interaction: Analyzing Multimodal Systems in Cocktail-Party Scenarios

arXiv:2608.08510v1 Announce Type: new Abstract: Humans have the remarkable ability to engage in spontaneous informal conversations and selectively attend to individual speakers while filtering out com

researcharxiv-cs-cl

LF{}^{2}AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models

arXiv:2503.06211v3 Announce Type: replace-cross Abstract: Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as aud

safetyarxiv-cs-ai

Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework

arXiv:2608.09529v1 Announce Type: new Abstract: As an important subfield of cross-modal generation, synthesizing static visual content in the form of images from audio, namely audio-to-image (A2I) gen

safetyarxiv-cs-cv

Chat UIs with native audio input for multimodal models?

model-releasesr-localllama

I've been running Gemma 4 E4B with oMLX and I can't find any chat interfaces that directly send the audio file to the model instead of running the audio through a separate STT layer. I can confirm the

10 Aug 2026

KVAE: Family of Tokenizers for Multimodal Generative Models

arXiv:2608.05798v1 Announce Type: new Abstract: Latent diffusion modeling (LDM), a prominent paradigm, utilizes tokenizers to map input signal to compressed representation. This dependency positions t

researcharxiv-cs-cv

7 Aug 2026

LoDA: A Level of Detection Aware Method and a Multimodal Sensing Benchmark for Object Level Change Detection

arXiv:2608.05356v1 Announce Type: new Abstract: High-definition 3D LiDAR maps are important for autonomous driving and smart-city services, which require reliable detection of object-level changes in

7 Aug 2026

OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction

arXiv:2608.05539v1 Announce Type: new Abstract: Recent vision-language models (VLMs) can generate executable CAD programs from images, but existing methods mainly target coarse, general-purpose 3D obj

7 Aug 2026

Multimodal Spatiotemporal Atmospheric Data Assimilation with Latent Flow-matching

arXiv:2608.05103v1 Announce Type: new Abstract: Data assimilation (DA) uses Bayesian inference to update the state of a numerical forecast model with observed data. In this study, we propose a fundame

researcharxiv-cs-lg

6 Aug 2026

OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routing

arXiv:2608.04434v1 Announce Type: new Abstract: Recent large language models (LLMs) have demonstrated remarkable progress in constraint-aware navigation, maze reasoning, and graph reasoning. However,

6 Aug 2026

TRNet: Topography-Guided Frequency Rectification and Structure-Aware Decoding for Multimodal Paddy Rice Segmentation

arXiv:2608.04154v1 Announce Type: cross Abstract: Mapping paddy rice from very-high-resolution imagery in mountainous and hilly regions is difficult because terrain alters optical appearance and incre

6 Aug 2026

Minimax-Optimal Semiparametric Contextual Dynamic Pricing with Multimodal Revenue

arXiv:2608.03142v1 Announce Type: cross Abstract: We study contextual dynamic pricing with arbitrary covariate sequences and bounded, possibly nonbinary purchase quantities. Demand follows a semiparam

5 Aug 2026

Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents

arXiv:2608.03327v1 Announce Type: new Abstract: Hybrid computer-use agents can act through screenshots or call text tools. We find that having a tool available does not settle which way the effect goe

5 Aug 2026

When Do Fewer Visual Tokens Accelerate Multimodal Inference? A Break-Even Study Across Decision Locations and Hardware

arXiv:2608.03649v1 Announce Type: new Abstract: Fewer visual tokens do not guarantee lower end-to-end latency. We evaluate break-even with a reproducible protocol that accounts for decision overhead,

5 Aug 2026

Evaluating VLMs on Multimodal Aristotelian Persuasion Tasks

arXiv:2608.01238v1 Announce Type: new Abstract: Vision Language Models (VLMs) have demonstrated exceptional performance across various tasks. However, they have not yet been thoroughly evaluated on mo

4 Aug 2026

FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models

arXiv:2607.29602v1 Announce Type: cross Abstract: Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are alre

3 Aug 2026

Visual Credit Audit for Multimodal Spatial Reasoning

arXiv:2607.27069v1 Announce Type: new Abstract: Closed yes/no spatial benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts. Under a fixed forced-choi

30 Jul 2026

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

arXiv:2607.24424v1 Announce Type: new Abstract: Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features ca

28 Jul 2026

Tutorials

Multimodal Semantic-Probabilistic Objectness for Open World Object Detection

arXiv:2607.23981v1 Announce Type: cross Abstract: Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incremental

tutorialsarxiv-cs-ai

28 Jul 2026

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

28 Jul 2026

Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

arXiv:2607.21780v1 Announce Type: new Abstract: Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into the

27 Jul 2026

Local Ai

FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence

Introducing FLUX 3. One multi-modal model for Image, Video, Audio and Action-Prediction. Creations are truer to life in every kind of style. Blog Post : https://bfl.ai/blog/flux-3 submitted by /u/pmtt

local-air-localllama

24 Jul 2026

ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program

arXiv:2607.19947v1 Announce Type: new Abstract: Electronic Theater Programs (ETPs) serve as critical promotional media in the performing arts, comprising a multi-page collection of heterogeneous visua

23 Jul 2026

FM^2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

arXiv:2607.13386v1 Announce Type: new Abstract: Building foundation models for medical imaging requires pooling data across institutions, yet privacy regulations prohibit centralized aggregation. Exis

16 Jul 2026

Multimodal Empirical Bayes Variational Autoencoders for Joint Longitudinal and Time-to-Event Modeling

arXiv:2607.13984v1 Announce Type: cross Abstract: Longitudinal tumor measurements, dropout information, and genetic covariates provide complementary information about treatment response, but integrati

researcharxiv-cs-lg

16 Jul 2026

Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

arXiv:2607.08317v1 Announce Type: new Abstract: Modern AI models achieve strong performance on many established benchmarks, yet they still fail on tasks that humans find almost trivial, such as manipu

10 Jul 2026

MatBind: A Shared Embedding Space for Multimodal Materials Characterization

arXiv:2607.08470v1 Announce Type: new Abstract: Fully characterizing a crystalline material requires integrating heterogeneous data sources -- atomic structures, diffraction patterns, electronic densi

safetyarxiv-cs-lg

10 Jul 2026

BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning

arXiv:2607.07361v1 Announce Type: new Abstract: Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods a

researcharxiv-cs-cv

9 Jul 2026

CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

arXiv:2607.06534v1 Announce Type: new Abstract: Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the abilit

8 Jul 2026

LEGATO 2: Toward Multimodal Sheet Music Recognition and Understanding

arXiv:2607.05769v1 Announce Type: cross Abstract: We propose a novel pipeline, Legato 2, for extracting symbolic notation and semantic knowledge from images of sheet music. Legato 2 features the first

8 Jul 2026

Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards

arXiv:2607.04727v1 Announce Type: cross Abstract: Automatic data visualization generation has advanced rapidly with multi-modal large language models, yet existing efforts largely focus on static char

Explainable AI for Screening Abuse-Related Trauma in Bangladeshi Children: A Training-Free Multimodal Framework Evaluated on Noise-Aware Synthetic Data

arXiv:2607.04010v1 Announce Type: new Abstract: Bangladesh has an estimated 1.17 mental-health professionals per 100,000 population and only six child psychiatrists nationwide. No Bengali-language, cu

Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

model-releasesarxiv-cs-lg

arXiv:2607.04647v1 Announce Type: cross Abstract: Scalable Bayesian inference for generalized linear mixed models (GLMMs) provides uncertainty-aware analysis of correlated longitudinal data, but exist

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

arXiv:2607.03644v1 Announce Type: cross Abstract: Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, rada

OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layout

arXiv:2607.03261v1 Announce Type: new Abstract: Recent large language models (LLMs) have demonstrated remarkable progress in 3D spatial reasoning, spatial grounding, and fine-grained geometric underst

Agents

SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy

arXiv:2607.04378v1 Announce Type: new Abstract: Surgical automation is being increasingly studied, yet bridging visual scene understanding with autonomous action planning remains a fundamental challen

agentsarxiv-cs-ro

ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation

arXiv:2607.03657v1 Announce Type: cross Abstract: Gloss-free Sign Language Translation (SLT) translates sign language videos into spoken-language sentences without gloss annotations, avoiding costly l

PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation

arXiv:2607.01883v1 Announce Type: new Abstract: Code is the medium through which large language models generate structured artifacts: charts, scientific figures, vector graphics, CAD models, 3D scenes

3 Jul 2026

From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary

arXiv:2506.17294v3 Announce Type: replace-cross Abstract: The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding research area, offering adv

1 Jul 2026

Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning

arXiv:2510.12957v4 Announce Type: replace-cross Abstract: We treat the internals of generative models as mechanistic objects rather than black boxes. We introduce extbf{Attribution Graphs} (AGs), whic

30 Jun 2026

Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction

arXiv:2606.29949v1 Announce Type: cross Abstract: H&E-stained whole-slide images offer cohort-scale availability and rich spatial context but lack molecular specificity, whereas bulk RNA-seq provides

safetyarxiv-cs-ai

30 Jun 2026

EVLA: An Electro-Aware Multimodal Assistant for Physically-Grounded Driving Reasoning and Control

arXiv:2606.28938v1 Announce Type: new Abstract: Modern vision-language models (VLMs) for driving assistants typically treat vehicle dynamics as a black box, resulting in decisions that lack awareness