AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
11 May 2026

EggHand: A Multimodal Foundation Model for Egocentric Hand Pose Forecasting

ResearchDGX agent

arXiv:2605.07642v1 Announce Type: new Abstract: Forecasting future 3D hand pose sequences from egocentric video is essential for understanding human intention and enabling embodied applications such a

LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation

Model ReleasesDGX agent

arXiv:2605.07640v1 Announce Type: cross Abstract: Remote sensing lithology interpretation is fundamental to geological surveys, mineral exploration, and regional geological mapping. Unlike general lan

OmicsLM: A Multimodal Large Language Model for Multi-Sample Omics Reasoning

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2605.06728v1 Announce Type: cross Abstract: Interpreting transcriptomic data is one of the most common analytical tasks in modern biology. Yet most current models either consume expression profi

Think Before You Drive: World Model-Inspired Multimodal Grounding for Autonomous Vehicles

Local AiDGX agent

arXiv:2512.03454v4 Announce Type: replace-cross Abstract: Interpreting natural-language commands to localize target objects is critical for autonomous driving (AD). Existing visual grounding (VG) meth

7 May 2026

Densification and forecasting of Sentinel-2 time series from multimodal SAR and Optical satellite data using deep generative models

ResearchDGX agent

arXiv:2605.04239v1 Announce Type: new Abstract: Optical satellite image time series are extensively used in many Earth observation applications, including agriculture, climate monitoring, and land sur

MIRAGE: Retrieval and Generation of Multimodal Images and Texts for Medical Education

ApplicationsDGX agent

arXiv:2605.04772v1 Announce Type: new Abstract: Access to diverse, well-annotated medical images with interactive learning tools is fundamental for training practitioners in medicine and related field

6 May 2026

A-CODE: Fully Atomic Protein Co-Design with Unified Multimodal Diffusion

ResearchDGX agent

arXiv:2605.03360v1 Announce Type: cross Abstract: We present A-CODE, a fully atomic unified one-stage protein co-design model that simultaneously refines discrete atom types and continuous atom coordi

RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models

Model ReleasesDGX agent

arXiv:2605.03821v1 Announce Type: new Abstract: Existing robot video world models are typically trained with low-level objectives such as reconstruction and perceptual similarity, which are poorly ali

5 May 2026

Bolek: A Multimodal Language Model for Molecular Reasoning

SafetyDGX agent

arXiv:2605.02745v1 Announce Type: new Abstract: Molecular property models increasingly support high-stakes drug-discovery decisions, but their outputs are often difficult to audit: classical predictor

Multimodal Confidence Modeling in Audio-Visual Quality Assessment

ApplicationsDGX agent

arXiv:2605.01219v1 Announce Type: cross Abstract: Audio-visual quality assessment (AVQA) is essential for streaming, teleconferencing, and immersive media. In realistic streaming scenarios, distortion

Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching

Model ReleasesDGX agent

arXiv:2303.10971v2 Announce Type: replace Abstract: The matching of 3D shapes has been extensively studied for shapes represented as surface meshes, as well as for shapes represented as point clouds.

4 May 2026

Hypergraph and Latent ODE Learning for Multimodal Root Cause Localization in Microservices

Model ReleasesDGX agent

arXiv:2605.00351v1 Announce Type: new Abstract: Root cause localization in cloud native microservice systems requires modeling complex service dependencies, irregular temporal dynamics, and heterogene

Introducing WARM-VR: Benchmark Dataset for Multimodal Wearable Affect Recognition in Virtual Reality

Model ReleasesDGX agent

arXiv:2605.00184v1 Announce Type: new Abstract: With the growing integration of human-computer interaction into everyday life, advances in machine learning have enabled systems to better perceive and

Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysis

ResearchDGX agent

arXiv:2512.01116v3 Announce Type: replace Abstract: The integration of histology images and gene profiles has shown great promise for improving survival prediction in cancer. However, current approach

1 May 2026

GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance

Model ReleasesDGX agent

arXiv:2503.12844v2 Announce Type: replace Abstract: For people affected by blindness and low vision (BLV), safe and independent navigation remains a major challenge, impacting over 2.2 billion individ

Iterative Multimodal Retrieval-Augmented Generation for Medical Question Answering

Model ReleasesDGX agent

arXiv:2604.27724v1 Announce Type: new Abstract: Medical retrieval-augmented generation (RAG) systems typically operate on text chunks extracted from biomedical literature, discarding the rich visual c

Learning Generalizable Multimodal Representations for Software Vulnerability Detection

Model ReleasesDGX agent

arXiv:2604.25711v2 Announce Type: replace-cross Abstract: Source code and its accompanying comments are complementary yet naturally aligned modalities-code encodes structural logic while comments capt

Linguistically Informed Multimodal Fusion for Vietnamese Scene-Text Image Captioning: Dataset, Graph Framework, and Phonological Attention

SafetyDGX agent

arXiv:2604.27712v1 Announce Type: cross Abstract: Scene-text image captioning requires fusing three information streams -- visual features, OCR-detected text, and linguistic knowledge -- to generate d

VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking

Model ReleasesDGX agent

arXiv:2601.08611v2 Announce Type: replace-cross Abstract: The growing scale of online misinformation urgently demands Automated Fact-Checking (AFC). Existing benchmarks for evaluating AFC systems, how

30 Apr 2026

A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

SafetyDGX agent

arXiv:2604.26379v1 Announce Type: new Abstract: Reliable seizure detection in mouse models is essential for preclinical epilepsy research, yet manual review of synchronized video-EEG recordings is lab

FASH-iCNN: Making Editorial Fashion Identity Inspectable Through Multimodal CNN Probing

ResearchDGX agent

arXiv:2604.26186v1 Announce Type: new Abstract: Fashion AI systems routinely encode the aesthetic logic of specific houses, editors, and historical moments without disclosing it. We present FASH-iCNN,

29 Apr 2026

Curriculum-guided multimodal representation learning enables generalizable prediction of nanomaterial-protein interactions

ResearchDGX agent

arXiv:2507.14245v2 Announce Type: replace Abstract: Nanomaterial-protein interactions (NPI) are pivotal to realizing the therapeutic and diagnostic potential of nanomaterials. Although AI promises to

Multimodal Contextualized Support for Enhancing Video Retrieval System

ResearchDGX agent

arXiv:2412.07584v2 Announce Type: replace Abstract: Current video retrieval systems, especially those used in competitions, primarily focus on querying individual keyframes or images rather than encod

The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents

Model ReleasesDGX agent

arXiv:2604.25299v1 Announce Type: new Abstract: Diffusion models have achieved success in high-fidelity data synthesis, yet their capacity for more complex, structured reasoning like text following ta

28 Apr 2026

DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery

Model ReleasesDGX agent

arXiv:2604.24029v1 Announce Type: cross Abstract: Identifying species in biology among tens of thousands of visually similar taxa while discovering unknown species in open-world environments remains a

Hierarchical Prototype-based Domain Priors for Multiple Instance Learning in Multimodal Histopathology Analysis

SafetyDGX agent

arXiv:2604.23982v1 Announce Type: new Abstract: Digital pathology has fundamentally altered diagnostic workflows by enabling the computational analysis of gigapixel Whole Slide Images (WSIs), yet effe

One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition

ResearchDGX agent

arXiv:2604.23173v1 Announce Type: new Abstract: Video Situation Recognition (VidSitu) addresses the challenging problem of 'who did what to whom, with what, how, and where' in a video. It tests thorou

27 Apr 2026

AromaGen: Interactive Generation of Rich Olfactory Experiences with Multimodal Language Models

Model ReleasesDGX agent

arXiv:2604.01650v2 Announce Type: replace-cross Abstract: Smell's deep connection with food, memory, and social experience has long motivated researchers to bring olfaction into interactive systems. Y

MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2604.22492v1 Announce Type: cross Abstract: Understanding social dominance in animal behavior is critical for neuroscience and behavioral studies. In this work, we explore the capability of Mult

Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models

Model ReleasesDGX agent

arXiv:2510.07632v2 Announce Type: replace Abstract: Frontier AI models have achieved remarkable progress, yet recent studies suggest they struggle with compositional reasoning, often performing at or

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents

Model ReleasesDGX agent

arXiv:2602.07038v2 Announce Type: replace-cross Abstract: Key Information Extraction (KIE) from real-world documents remains challenging due to substantial variations in layout structures, visual qual

24 Apr 2026

Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection

ResearchDGX agent

arXiv:2601.02438v3 Announce Type: replace-cross Abstract: Software vulnerability detection can be formulated as a binary classification problem that determines whether a given code snippet contains se

Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation

SafetyDGX agent

arXiv:2603.00110v2 Announce Type: replace Abstract: The scarcity of large-scale robotic data has motivated the repurposing of foundation models from other modalities for policy learning. In this work,

22 Apr 2026

OmniGen2: Towards Instruction-Aligned Multimodal Generation

Model ReleasesDGX agent

arXiv:2506.18871v4 Announce Type: replace-cross Abstract: In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generatio

21 Apr 2026

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

Model ReleasesDGX agent

arXiv:2604.18109v1 Announce Type: new Abstract: This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover t

Multimodal In-context Learning for ASR of Low-resource Languages

Model ReleasesDGX agent

arXiv:2601.05707v2 Announce Type: replace Abstract: Automatic speech recognition (ASR) still covers only a small fraction of the world's languages, mainly due to supervised data scarcity. In-context l

SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models

Model ReleasesDGX agent

arXiv:2604.16883v1 Announce Type: new Abstract: In long-context decoding for LLMs and LMMs, attention becomes increasingly memory-bound because each decoding step must load a large amount of KV-cache

STEP-PD: Stage-Aware and Explainable Parkinson's Disease Severity Classification Using Multimodal Clinical Assessments

ResearchDGX agent

arXiv:2604.17611v1 Announce Type: new Abstract: Parkinson's disease (PD) is a progressive disorder in which symptom burden and functional impairment evolve over time, making severity staging essential

17 Apr 2026

CAVERS: Multimodal SLAM Data from a Natural Karstic Cave with Ground Truth Motion Capture

Model ReleasesDGX agent

arXiv:2604.15052v1 Announce Type: new Abstract: Autonomous robots operating in natural karstic caves face perception and navigation challenges that are qualitatively distinct from those encountered in

16 Apr 2026

A Multimodal Clinically Informed Coarse-to-Fine Framework for Longitudinal CT Registration in Proton Therapy

ResearchDGX agent

arXiv:2604.13397v1 Announce Type: new Abstract: Proton therapy offers superior organ-at-risk sparing but is highly sensitive to anatomical changes, making accurate deformable image registration (DIR)

AIM-CoT: Active Information-driven Multimodal Chain-of-Thought for Vision-Language Reasoning

ResearchDGX agent

arXiv:2509.25699v2 Announce Type: replace Abstract: Interleaved-Modal Chain-of-Thought (I-MCoT) advances vision-language reasoning, such as Visual Question Answering (VQA). This paradigm integrates sp

15 Apr 2026

M2HRI: An LLM-Driven Multimodal Multi-Agent Framework for Personalized Human-Robot Interaction

AgentsDGX agent

arXiv:2604.11975v1 Announce Type: new Abstract: Multi-robot systems hold significant promise for social environments such as homes and hospitals, yet existing multi-robot works treat robots as functio

ReefMapGS: Enabling Large-Scale Underwater Reconstruction by Closing the Loop Between Multimodal SLAM and Gaussian Splatting

ResearchDGX agent

arXiv:2604.11992v1 Announce Type: cross Abstract: 3D Gaussian Splatting is a powerful visual representation, providing high-quality and efficient 3D scene reconstruction, but it is crucially dependent

14 Apr 2026

A Mamba-Based Multimodal Network for Multiscale Blast-Induced Rapid Structural Damage Assessment

SafetyDGX agent

arXiv:2604.11709v1 Announce Type: new Abstract: Accurate and rapid structural damage assessment (SDA) is crucial for post-disaster management, helping responders prioritise resources, plan rescues, an

Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting

Model ReleasesDGX agent

arXiv:2603.13941v2 Announce Type: replace Abstract: Growing waste streams and the transition to a circular economy require efficient automated waste sorting. In industrial settings, materials move on

Can LLMs Reason About Attention? Towards Zero-Shot Analysis of Multimodal Classroom Behavior

HardwareDGX agent

arXiv:2604.03401v2 Announce Type: replace-cross Abstract: Understanding student engagement usually requires time-consuming manual observation or invasive recording that raises privacy concerns. We pre

VisText-Mosquito: A Unified Multimodal Dataset for Visual Detection, Segmentation, and Textual Explanation on Mosquito Breeding Sites

ResearchDGX agent

arXiv:2506.14629v3 Announce Type: replace-cross Abstract: Mosquito-borne diseases pose a major global health risk, requiring early detection and proactive control of breeding sites to prevent outbreak

13 Apr 2026

Multimodal Anomaly Detection for Human-Robot Interaction

SafetyDGX agent

arXiv:2604.09326v1 Announce Type: cross Abstract: Ensuring safety and reliability in human-robot interaction (HRI) requires the timely detection of unexpected events that could lead to system failures

On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment

SafetyDGX agent

arXiv:2604.08579v1 Announce Type: cross Abstract: We study cross-modal alignment between independently pretrained vision (DINOv2) and language (all-MiniLM-L6-v2) encoders using the functional map fram

10 Apr 2026

ChangeBridge: Spatiotemporal Image Generation with Multimodal Controls for Remote Sensing

ResearchDGX agent

arXiv:2507.04678v3 Announce Type: replace Abstract: Spatiotemporal image generation is a highly meaningful task, which can generate future scenes conditioned on given observations. However, existing c

Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models

Local AiDGX agent

arXiv:2604.06912v1 Announce Type: cross Abstract: MLLMs require high-resolution visual inputs for fine-grained tasks like document understanding and dense scene perception. However, current global res

SyncBreaker:Stage-Aware Multimodal Adversarial Attacks on Audio-Driven Talking Head Generation

ResearchDGX agent

arXiv:2604.08405v1 Announce Type: new Abstract: Diffusion-based audio-driven talking-head generation enables realistic portrait animation, but also introduces risks of misuse, such as fraud and misinf

What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning

Model ReleasesDGX agent

arXiv:2604.06995v1 Announce Type: new Abstract: Existing Graphical User Interface (GUI) reasoning tasks remain challenging, particularly in UI understanding. Current methods typically rely on direct s

Zatom-1: A Multimodal Flow Foundation Model for 3D Molecules and Materials

ResearchDGX agent

arXiv:2602.22251v3 Announce Type: replace-cross Abstract: General-purpose 3D chemical modeling encompasses molecules and materials, requiring both generative and predictive capabilities. However, most

16 Oct 2026

Beyond Generation: Using Multimodal LLMs as Detectors of LLM-Generated Deepfake Facial Images

Model ReleasesDGX agent

arXiv:2610.21760v1 Announce Type: new Abstract: As image generation models become increasingly capable of producing highly realistic content, traditional forensic techniques are struggling to keep up.

CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness

AgentsDGX agent

arXiv:2610.21910v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance, they are increasingly expected to perform spatiotemporal reasoning over motion, actions, and view changes beyo

FuseAdapt: Parameter-Efficient Multimodal Fusion for Semantic Segmentation with Missing Modalities

Model ReleasesDGX agent

arXiv:2610.21779v1 Announce Type: new Abstract: Multi-modal semantic segmentation benefits from complementary sensors, but this benefit is fragile when some modalities are degraded or missing at test

Multimodal AI Detection for Math-Heavy Scientific Text

Model ReleasesDGX agent

arXiv:2610.21062v1 Announce Type: new Abstract: As large language models become increasingly capable of generating, and assisting with, scientific writing, there is a need for greater transparency and

15 Oct 2026

FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity

AgentsDGX agent

arXiv:2610.19155v1 Announce Type: new Abstract: Human demonstrations offer a scalable alternative to costly robot data collection for dexterous manipulation, but most human datasets lack force measure

Instruction Mining From Images: Constructing a Synthetic Dataset for Multimodal Learning ILSP DCVLR Competition Submission

ResearchDGX agent

arXiv:2610.19932v1 Announce Type: new Abstract: This work presents a compact and reproducible pipeline for synthesizing and curating image-based instruction-response pairs for DCVLR. We draw from the

← Previous
1…4344454647…169
Next →