Search: “multimodal”

Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training

arXiv:2609.15051v1 Announce Type: cross Abstract: Training prompts in online reinforcement learning (RL) differ substantially in how informative they are for the current policy: some are already satur

safetyarxiv-cs-ai

15 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization

DGX agent

arXiv:2609.14469v1 Announce Type: new Abstract: Accurate 3D localization of unmanned aerial vehicles (UAVs) remains challenging for existing multimodal approaches due to sparse LiDAR geometry, modalit

safetyarxiv-cs-cv

15 Sept 2026

Sensory Precision Inference for Multimodal Arbitration under Uncertainty

arXiv:2609.15065v1 Announce Type: new Abstract: Autonomous agents operating on multisensory data cannot assume that all sensory modalities remain consistently informative. In real environments, sensor

agentsarxiv-cs-lg

15 Sept 2026

Speak to the City: Multimodal Resolution for Outside-the-Vehicle References

arXiv:2609.14691v1 Announce Type: cross Abstract: As autonomous vehicles and Extended Reality (XR) headsets enable novel in-car interactions, seamlessly querying physical landmarks, known as Outside-t

agentsarxiv-cs-cl

15 Sept 2026

Beyond the Query: Do Retrieval Signals Improve Adaptive Multimodal RAG Routing?

arXiv:2609.12437v1 Announce Type: new Abstract: Adaptive RAG often uses retrieval-time signals to decide whether another retrieval, reranking, or multimodal step should run. We ask whether these signa

researcharxiv-cs-lg

Applications

Detecting and Explaining Fake News Short Videos with Multimodal Content and Real-World Evidence

arXiv:2609.12678v1 Announce Type: new Abstract: Short-video platforms have become a primary news source for the public, which has also enabled the widespread dissemination of fake news videos. We stud

applicationsarxiv-cs-cv

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

arXiv:2606.14957v4 Announce Type: replace Abstract: Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic an

RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images

arXiv:2609.12497v1 Announce Type: new Abstract: Infrared-visible image fusion facilitates robust multimodal perception by integrating complementary textural nuances from visible sensors with thermal s

Local Ai

Exploring Multimodal Prompt for Visualization Authoring with Large Language Models

arXiv:2504.13700v2 Announce Type: replace-cross Abstract: Recent advances in large language models (LLMs) have shown great potential in automating the process of visualization authoring through simple

local-aiarxiv-cs-ai

12 Sept 2026

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv:2609.11243v1 Announce Type: new Abstract: Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These

12 Sept 2026

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

arXiv:2609.11282v1 Announce Type: new Abstract: Multimodal forecasting models that combine time series with text annotations promise richer prediction through textual context, but how do we know wheth

12 Sept 2026

BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset

arXiv:2609.08029v1 Announce Type: cross Abstract: Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-r

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

arXiv:2603.25565v2 Announce Type: replace Abstract: Current Large Multimodal Models (LMMs) in Earth observation are predominantly evaluated on planar optical tasks and often neglect the vertical dimen

Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Models

arXiv:2509.19994v3 Announce Type: replace Abstract: Multimodal pre-trained models (e.g., ImageBind), which align distinct data modalities into a shared embedding space, have shown remarkable success a

safetyarxiv-cs-cv

NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

arXiv:2609.09140v1 Announce Type: new Abstract: The digitization of healthcare has generated vast, longitudinal, and multimodal patient records over a lifetime, yet fully exploiting these data to repr

safetyarxiv-cs-lg

Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching

arXiv:2609.06343v1 Announce Type: new Abstract: Multimodal image matching is a fundamental task for multi-source information fusion. However, geometric distortions and nonlinear radiometric difference

RAH-VLA: Resolution-Adaptive Hierarchical Vision-Language Alignment for Multimodal Remote Sensing Understanding

arXiv:2512.23243v3 Announce Type: replace Abstract: Multimodal vision-language modeling has emerged as a promising paradigm for remote sensing (RS) image understanding. However, existing methods are l

safetyarxiv-cs-cv

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

arXiv:2609.07823v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains undere

Towards Unified Multimodal Graph Foundation Model: A Bridge-Router-Adapter Based Approach

arXiv:2609.06668v1 Announce Type: new Abstract: Multimodal graphs couple node attributes in different modalities, such as text and images, with relational structure, enabling topological structure and

researcharxiv-cs-lg

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

arXiv:2609.04438v1 Announce Type: new Abstract: Long-horizon multimodal agents should remember not only what happened but also who participated. This capability depends on linking recurring faces, voi

7 Sept 2026

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

arXiv:2609.04574v1 Announce Type: cross Abstract: Multimodal Graph Neural Networks have become standard for recommendation by augmenting sparse interaction data with content features. Yet current arch

safetyarxiv-cs-lg

7 Sept 2026

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

arXiv:2609.03526v1 Announce Type: new Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cult

4 Sept 2026

GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs

arXiv:2609.03892v1 Announce Type: cross Abstract: 3D spatial reasoning underpins understanding and acting in the physical world, yet it remains unreliable in current multimodal large language models (

researcharxiv-cs-ai

4 Sept 2026

MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models

arXiv:2609.01772v1 Announce Type: new Abstract: Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vis

3 Sept 2026

Multimodal Language Models as Text-to-Image Model Evaluators

arXiv:2505.00759v3 Announce Type: replace-cross Abstract: The steady improvements of text-to-image (T2I) generative models lead to slow deprecation of automatic evaluation benchmarks that rely on stat

3 Sept 2026

CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs

arXiv:2603.11915v2 Announce Type: replace Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Multimoda

2 Sept 2026

Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflict

arXiv:2609.00550v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly provided with contextual evidence in heterogeneous forms: as a text passage, as a rendered ima

researcharxiv-cs-cl

2 Sept 2026

MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions

arXiv:2608.30903v1 Announce Type: new Abstract: Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work h

1 Sept 2026

Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference

arXiv:2608.28726v1 Announce Type: new Abstract: The remarkable performance of multimodal large language models (MLLMs) comes at the cost of substantial computational overhead, posing significant chall

1 Sept 2026

Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models

arXiv:2608.27531v1 Announce Type: cross Abstract: The safety of large vision-language models is increasingly stress-tested by multimodal jailbreaks, yet existing attacks remain largely static at the m

31 Aug 2026

OmniFusion: Simultaneous Multilingual Multimodal Translations via Modular Fusion

arXiv:2512.00234v3 Announce Type: replace Abstract: There has been significant progress in open-source text-only translation large language models (LLMs) with better language coverage and quality. How

researcharxiv-cs-cl

31 Aug 2026

LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning

arXiv:2608.26820v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) exhibit strong generalization, visual instruction tuning for downstream tasks inevitably causes catastrop

28 Aug 2026

Modality Maturity Index: A benchmark for assessing multimodal capabilities of omni models

arXiv:2608.26317v1 Announce Type: cross Abstract: Frontier language models are increasingly marketed as omni systems that can perceive and respond across modalities. Existing evaluation frameworks, ho

28 Aug 2026

Hardware

MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs

arXiv:2606.17118v2 Announce Type: replace Abstract: Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compress

hardwarearxiv-cs-lg

27 Aug 2026

Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control

arXiv:2602.17068v2 Announce Type: replace Abstract: Human-centric traffic signal control in corridor networks must increasingly account for multimodal travelers, particularly high-occupancy public tra

agentsarxiv-cs-lg

27 Aug 2026

Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios

arXiv:2608.25529v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown strong performance in video understanding. However, their ability to follow instructions in this dom

27 Aug 2026

Applications

A Multimodal Foundation Model for Longitudinal Patient Representation and Scalable Insight Generation in Oncology

arXiv:2608.24688v1 Announce Type: new Abstract: Precision oncology necessitates a longitudinal model of patient state that captures cancer evolution and treatment over time, integrating multimodal obs

applicationsarxiv-cs-lg

Interested in designing and building agent harnesses for long-horizon, multimodal tasks? Nomic is hiring a Harness Engineer. We're post-trai…

Interested in designing and building agent harnesses for long-horizon, multimodal tasks? Nomic is hiring a Harness Engineer. We're post-training models and an agent harness that can perform most tasks

agentsnomic-ai--x

LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training

arXiv:2608.24845v1 Announce Type: cross Abstract: We present LAION-BVD, a large-scale open video dataset for multimodal learning, which contains 1.3B platform-specific video URLs collected from Common

researcharxiv-cs-ai

Applications

Wontopos Tablet 2: Measuring Multilingual and Multimodal Memory Retrieval Without Lexical Matching

arXiv:2608.23920v1 Announce Type: cross Abstract: We measure tablet-2, a production long-term memory engine for language models, on the text benchmarks the field already uses and on cross-lingual retr

applicationsarxiv-cs-cv

FinAcumen: Financial Multimodal Reasoning via Self-Evolving Experience Memory Harness

arXiv:2606.17642v3 Announce Type: replace Abstract: Financial multimodal reasoning requires agents to coordinate numerical computation, retrieval, visual interpretation, and temporal grounding across

agentsarxiv-cs-ai

FOVEA: Focused On-Demand Visual Evidence Adaptation for Cache-Friendly Multimodal Speculative Decoding

arXiv:2608.22883v1 Announce Type: new Abstract: Multimodal speculative decoding accelerates vision-language models by allowing a lightweight draft model to propose candidate tokens for parallel verifi

MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents

arXiv:2608.21810v1 Announce Type: cross Abstract: Clinical decision-making is inherently experience-driven: physicians progressively refine their reasoning by synthesizing patient history, multimodal

agentsarxiv-cs-cv

Semantics or Structure? Auditing Text Sensitivity in Multimodal Time-Series Forecasting

arXiv:2608.22321v1 Announce Type: new Abstract: Multimodal time-series forecasting has emerged as a promising paradigm in which natural-language context is expected to improve predictive performance.

VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

arXiv:2608.21883v1 Announce Type: new Abstract: Multimodal large reasoning models often rely on long Chain-of-Thought (CoT) traces in which a substantial fraction of tokens, such as repeated visual de

safetyarxiv-cs-cv

Volumetric Radiology AI in the Era of Multimodal Large Language Models

arXiv:2608.20549v1 Announce Type: new Abstract: Advances in multimodal large language models (MLLMs) are extending radiological artificial intelligence (AI) beyond task-specific image analysis toward

safetyarxiv-cs-ai

24 Aug 2026

Industry

Ox Alpha, a 'stealth model' from an unknown AI lab with a 1M-token multimodal context and capacity for 100T tokens/day, goes viral after launching on OpenRouter (Rohail Saleem/Wccftech)

Rohail Saleem / Wccftech: Ox Alpha, a “stealth model” from an unknown AI lab with a 1M-token multimodal context and capacity for 100T tokens/day, goes viral after launching on OpenRouter — When an unk

industrytechmeme

22 Aug 2026

Question-Guided Evidence Acquisition for Multimodal Visual Question Answering

arXiv:2608.19739v1 Announce Type: cross Abstract: Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them u