AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
21 May 2026

Multimodal Fusion for Sim2real Transfer in Visual Reinforcement Learning

ApplicationsDGX agent

arXiv:2507.09180v4 Announce Type: replace Abstract: Depth information is robust to scene appearance variations and inherently carries 3D spatial details. Thus, a visual backbone based on the vision tr

Multimodal Optimal Transport for Training-free Temporal Segmentation in Surgical Robotics

Model ReleasesDGX agent

arXiv:2602.24138v2 Announce Type: replace Abstract: Automated recognition of surgical phases and steps is a fundamental capability for intraoperative decision support, workflow automation, and skill a

SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining

ResearchDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2605.21075v1 Announce Type: new Abstract: Earth observation (EO) foundation models (FMs) are increasingly trained on multisensor data, spanning multispectral imagery (MSI), synthetic aperture ra

20 May 2026

CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering

Model ReleasesDGX agent

arXiv:2605.19075v1 Announce Type: cross Abstract: Grounded multi-video question answering over real-world news events requires systems to surface query-relevant evidence across heterogeneous video arc

MetaEarth-MM: Unified Multimodal Remote Sensing Image Generation with Scene-centered Joint Modeling

ResearchDGX agent

arXiv:2605.20090v1 Announce Type: new Abstract: Multi-modal remote sensing images are vital for Earth observation, yet complete paired observations are often scarce in practice. Existing generative me

Multimodal evaluators: MLLM-as-a-judge for image-to-text tasks in Strands Evals

IndustryDGX agent

If you’re building visual shopping, image or document understanding, or chart analysis, you need a way to verify whether your model’s response is actually grounded in the source image. A text-only eva

Multimodal system for skin cancer detection

ApplicationsDGX agent

arXiv:2601.14822v2 Announce Type: replace-cross Abstract: Melanoma detection is vital for early diagnosis and effective treatment. While deep learning models on dermoscopic images have shown promise,

PlantTraitNet: An Uncertainty-Aware Multimodal Framework for Global-Scale Plant Trait Inference from Citizen Science Data

Model ReleasesDGX agent

arXiv:2511.06943v3 Announce Type: replace-cross Abstract: Global plant maps of plant traits, such as leaf nitrogen or plant height, are essential for understanding ecosystem processes, including the c

RoomPilot: Controllable Indoor Scene Synthesis via Multimodal Semantic Parsing

ResearchDGX agent

arXiv:2512.11234v2 Announce Type: replace Abstract: Generating controllable indoor scenes is fundamental to applications in game development, architectural visualization, and embodied AI. However, exi

19 May 2026

Dimension-Uniform Discretization Analysis of Preconditioned Annealed Langevin Dynamics for Multimodal Gaussian Mixtures

ResearchDGX agent

arXiv:2605.16473v1 Announce Type: cross Abstract: Obtaining stable diffusion-based samplers in high- and infinite-dimensional settings is challenging because errors can accumulate across high-frequenc

EGI: A Multimodal Emotional AI Framework for Enhancing Scrum Master Real-time Self-Awareness

ResearchDGX agent

arXiv:2605.17684v1 Announce Type: new Abstract: While increasing research focuses on the emotional well-being of agile team members, a significant gap remains in emotion monitoring studies for Scrum M

MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents

Local AiDGX agent

arXiv:2605.18652v1 Announce Type: new Abstract: Recent GUI agents have made substantial progress in visual grounding and action prediction, yet they remain brittle in long-horizon tasks that require m

Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization

Local AiDGX agent

arXiv:2601.21458v2 Announce Type: replace Abstract: Modern deepfakes have evolved into localized and intermittent manipulations that require fine-grained temporal localization to mitigate severe digit

18 May 2026

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation

SafetyDGX agent

arXiv:2605.15532v1 Announce Type: cross Abstract: Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities, yet the prompts driving this process are typically

15 May 2026

GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation

Local AiDGX agent

arXiv:2605.14406v1 Announce Type: cross Abstract: Large-scale pretraining on Earth observation imagery has yielded powerful representations of the natural and built environment. However, most existing

Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation

SafetyDGX agent

arXiv:2602.02994v2 Announce Type: replace Abstract: Reinforcement learning has emerged as a principled post-training paradigm for Temporal Video Grounding (TVG) due to its on-policy optimization, yet

12 May 2026

Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model

Model ReleasesDGX agent

arXiv:2605.10739v1 Announce Type: cross Abstract: We introduce SMART-HC-VQA, a Sentinel-2-based visual question answering dataset derived from the IARPA SMART Heavy Construction dataset, designed for

High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models

ResearchDGX agent

arXiv:2512.21815v2 Announce Type: replace Abstract: Vision-language models (VLMs) achieve remarkable performance but remain vulnerable to adversarial attacks. Entropy, as a measure of model uncertaint

Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare

Model ReleasesDGX agent

arXiv:2605.08445v1 Announce Type: new Abstract: AI models are increasingly deployed in live clinical environments where they must perform reliably across complex, high-stakes workflows that standard t

11 May 2026

Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding

ResearchDGX agent

arXiv:2605.06679v1 Announce Type: new Abstract: Vision-Language Models (VLMs) are frequently undermined by object hallucination, generating content that contradicts visual reality, due to an over-reli

6 May 2026

Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models

ResearchDGX agent

arXiv:2605.02912v1 Announce Type: new Abstract: Video Anomaly Detection (VAD) has traditionally been framed as binary classification or outlier detection, providing neither interpretable reasoning nor

5 May 2026

Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

SafetyDGX agent

arXiv:2605.01896v1 Announce Type: new Abstract: Emerging multi-modal world models attempt to jointly generate videos across diverse modalities (e.g., RGB, depth, and mask), yet they fail to fully expl

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

Model ReleasesDGX agent

arXiv:2605.02782v1 Announce Type: cross Abstract: Automatic speech recognition (ASR) systems remain brittle on dysarthric and other atypical speech. Recent audio-language models raise the possibility

1 May 2026

CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understanding

ResearchDGX agent

arXiv:2602.12652v2 Announce Type: replace Abstract: Clouds are a common phenomenon that distorts optical satellite imagery, which poses a challenge for remote sensing. However, in the literature cloud

Simulating clinical interventions with a generative multimodal model of human physiology

ResearchDGX agent

arXiv:2604.27899v1 Announce Type: new Abstract: Understanding how human health changes over time, and why responses to interventions vary between individuals, remains a central challenge in medicine.

30 Apr 2026

A Multimodal Depth-Aware Method For Embodied Reference Understanding

ResearchDGX agent

arXiv:2510.08278v3 Announce Type: replace Abstract: Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While

Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI

Model ReleasesDGX agent

arXiv:2604.26382v1 Announce Type: cross Abstract: Most enterprise document AI today is a pipeline. Parse, index, retrieve, generate. Each of those stages has been studied to death on its own -- what's

Multimodal LLMs are not all you need for Pediatric Speech Language Pathology

Model ReleasesDGX agent

arXiv:2604.26568v1 Announce Type: new Abstract: Speech Sound Disorders (SSD) affect roughly five percent of children, yet speech-language pathologists face severe staffing shortages and unmanageable c

ViTaPEs: Visuotactile Position Encodings for Cross-Modal Alignment in Multimodal Transformers

SafetyDGX agent

arXiv:2505.20032v3 Announce Type: replace Abstract: Tactile sensing provides local essential information that is complementary to visual perception, such as texture, compliance, and force. Despite rec

29 Apr 2026

Variational Neural Belief Parameterizations for Robust Dexterous Grasping under Multimodal Uncertainty

Model ReleasesDGX agent

arXiv:2604.25897v1 Announce Type: cross Abstract: Contact variability, sensing uncertainty, and external disturbances make grasp execution stochastic. Expected-quality objectives ignore tail outcomes

28 Apr 2026

Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model

Model ReleasesDGX agent

arXiv:2508.06206v4 Announce Type: replace-cross Abstract: Affordance grounding focuses on predicting the specific regions of objects that are associated with the actions to be performed by robots. It

ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents

Model ReleasesDGX agent

arXiv:2604.23781v1 Announce Type: new Abstract: Language-model agents are increasingly used as persistent coworkers that assist users across multiple working days. During such workflows, the surroundi

DriVerse: Navigation World Model for Driving Simulation via Multimodal Trajectory Prompting and Motion Alignment

Model ReleasesDGX agent

arXiv:2504.18576v2 Announce Type: replace Abstract: This paper presents DriVerse, a generative model for simulating navigation-driven driving scenes from a single image and a future trajectory. Previo

Open-Vocabulary Semantic Segmentation Network Integrating Object-Level Label and Scene-Level Semantic Features for Multimodal Remote Sensing Images

ApplicationsDGX agent

arXiv:2604.24125v1 Announce Type: new Abstract: Semantic segmentation of multi-modal remote sensing imagery plays a pivotal role in land use/land cover (LULC) mapping, environmental monitoring, and pr

CheXmix: Unified Generative Pretraining for Vision Language Models in Medical Imaging

ResearchDGX agent

arXiv:2604.22989v1 Announce Type: cross Abstract: Recent medical multimodal foundation models are built as multimodal LLMs (MLLMs) by connecting a CLIP-pretrained vision encoder to an LLM using LLaVA-

24 Apr 2026

Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery

Model ReleasesDGX agent

arXiv:2604.21102v1 Announce Type: cross Abstract: We present a novel framework for automatically evaluating building conditions nationwide in the United States by leveraging large language models (LLM

Multimodal Bayesian Network for Robust Assessment of Casualties in Autonomous Triage

AgentsDGX agent

arXiv:2512.18908v2 Announce Type: replace Abstract: Mass Casualty Incidents can overwhelm emergency medical systems and resulting delays or errors in the assessment of casualties can lead to preventab

23 Apr 2026

3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models

ResearchDGX agent

arXiv:2604.05687v2 Announce Type: replace Abstract: Reconstructing 3D scenes from smoke-degraded multi-view images is particularly difficult because smoke introduces strong scattering effects, view-de

AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce

ResearchDGX agent

arXiv:2604.20135v1 Announce Type: new Abstract: Multimodal representation is crucial for E-commerce tasks such as identical product retrieval. Large representation models (e.g., VLM2Vec) demonstrate s

Hybrid Latent Reasoning with Decoupled Policy Optimization

SafetyDGX agent

arXiv:2604.20328v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning significantly elevates the complex problem-solving capabilities of multimodal large language models (MLLMs). However, a

21 Apr 2026

Mitigating Multimodal Hallucination via Phase-wise Self-reward

ResearchDGX agent

arXiv:2604.17982v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) still struggle with vision hallucination, where generated responses are inconsistent with the visual input. Exist

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows

Model ReleasesDGX agent

arXiv:2505.19897v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have extended their impact beyond Natural Language Processing, substantially fostering the development of interdi

TriTS: Time Series Forecasting from a Multimodal Perspective

Model ReleasesDGX agent

arXiv:2604.16748v1 Announce Type: new Abstract: Time series forecasting plays a pivotal role in critical sectors such as finance, energy, transportation, and meteorology. However, Long-term Time Serie

What's Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previews

Model ReleasesDGX agent

arXiv:2601.05563v2 Announce Type: replace Abstract: Even when factually correct, social-media news previews (image-headline pairs) can induce interpretation drift: by selectively omitting crucial cont

20 Apr 2026

FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation

Model ReleasesDGX agent

arXiv:2604.16298v1 Announce Type: new Abstract: UAV vision-language navigation (VLN) requires an agent to navigate complex 3D environments from an egocentric perspective while following ambiguous mult

neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing

Model ReleasesDGX agent

arXiv:2604.16170v1 Announce Type: new Abstract: We introduce neuralCAD-Edit, the first benchmark for editing 3D CAD models collected from expert CAD engineers. Instead of text conditioning as in prior

17 Apr 2026

ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agents

Model ReleasesDGX agent

arXiv:2603.20340v3 Announce Type: replace-cross Abstract: Self-generated skills for web agents are often unstable and can even hurt performance relative to direct acting. We argue that the key bottlen

14 Apr 2026

Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts

Model ReleasesDGX agent

arXiv:2604.10200v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) become integral to educational decision-making, ensuring their fairness is paramount. However, current text-centric eva

EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content

Model ReleasesDGX agent

arXiv:2604.05005v2 Announce Type: replace-cross Abstract: Large language models are increasingly used as educational assistants, yet evaluation of their educational capabilities remains concentrated o

Multimodal Dataset Normalization and Perceptual Validation for Music-Taste Correspondences

SafetyDGX agent

arXiv:2604.10632v1 Announce Type: cross Abstract: Collecting large, aligned cross-modal datasets for music-flavor research is difficult because perceptual experiments are costly and small by design. W

Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation

Model ReleasesDGX agent

arXiv:2504.14988v3 Announce Type: replace Abstract: Recent advancements in Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal perception capabilities, garnering significant a

10 Apr 2026

STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training

ResearchDGX agent

arXiv:2604.06836v2 Announce Type: new Abstract: Quantization is an effective way to reduce the memory cost of large-scale model training. However, most existing methods adopt fixed-precision policies,

Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation

SafetyDGX agent

arXiv:2604.06205v1 Announce Type: cross Abstract: The growth of online platforms and user content requires strong content moderation systems that can handle complex inputs from various media types. Wh

15 Sept 2026

Proximal-Only Transmission Matrix Recovery of an Arbitrarily Deformed Graded-Index Multimode Fiber

ResearchDGX agent

arXiv:2609.14869v1 Announce Type: cross Abstract: The multimode fiber is among the thinnest imaging conduits available, carrying hundreds to thousands of spatial modes through a cross-section comparab

23 Sept 2026

Qwen3.8-Omni: Towards Native Omni-Modal Agents

Model ReleasesDGX agent

arXiv:2609.25611v1 Announce Type: new Abstract: We introduce Qwen3.8-Omni-Flash, a natively multimodal agentic model for real-world multimodal productivity. Compared with previous omni models, which p

27 Jul 2026

Twins: Learn to Predict Unified Representations with Focal Loss

SafetyDGX agent

arXiv:2607.22531v1 Announce Type: new Abstract: Unified multimodal models seek a shared visual token space that supports both multimodal understanding and image generation. Discrete methods unify the

11 Apr 2026

All Models

ConceptsDGX agent

Auto-generated index of all models mentioned across the wiki.

16 Sept 2026

Affect-Prototype Guided Fusion for Open-Vocabulary Incomplete Multi-modal Emotion Recognition

ApplicationsDGX agent

arXiv:2609.16962v1 Announce Type: new Abstract: Open-vocabulary multimodal emotion recognition (OV-MER) aims to generate open natural-language emotion labels from multimodal affective cues. In real-wo

15 Oct 2026

From Stages to States: Unified Iterative Reasoning for MLLMs

ResearchDGX agent

arXiv:2610.15620v1 Announce Type: new Abstract: Multimodal Chain-of-Thought (MCoT) substantially enhances the complex reasoning capabilities of Multimodal Large Language Models (MLLMs) by explicitly m

26 Aug 2026

WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report

Model ReleasesDGX agent

arXiv:2608.24053v1 Announce Type: new Abstract: Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space f

← Previous
1…4748495051…169
Next →