AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
1 Jul 2026

LLM-Powered Interactive Robotic Action Synthesis from Multimodal Speech, Gestures, and Music

ResearchDGX agent

arXiv:2606.31158v1 Announce Type: cross Abstract: The quest for intuitive and natural human-robot interaction (HRI) remains a significant challenge in robotics. Traditional methods often rely on rigid

Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

TutorialsDGX agent

arXiv:2506.23102v2 Announce Type: replace-cross Abstract: Current CT report generation frameworks predominantly rely on global feature representations, often failing to capture region-specific details

Robust Autonomous UAV Landing on Maritime Platforms via Multimodal Agentic AI and Active Wave Compensation

AgentsDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2606.31613v1 Announce Type: new Abstract: Autonomous aerial inspection of marine infrastructure is frequently compromised by stochastic sea states, introducing risks of high-kinetic impacts, pos

30 Jun 2026

Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning

SafetyDGX agent

arXiv:2606.29812v1 Announce Type: new Abstract: Inductive biases steer learning toward generalizable solutions by encoding task structure. In this work, we identify a crucial missing bias in MLLMs: cr

Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification

Local AiDGX agent

arXiv:2606.29023v1 Announce Type: cross Abstract: Spatio-temporal grounding in long videos requires precise temporal localization and robust object tracking conditioned on natural-language queries. Wh

Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI

ResearchDGX agent

arXiv:2606.28991v1 Announce Type: new Abstract: Cardiac magnetic resonance imaging (CMR) routinely records structured acquisition metadata, yet most CMR foundation models rely primarily on image-only

LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion

Model ReleasesDGX agent

arXiv:2606.29900v1 Announce Type: cross Abstract: Personality recognition in asynchronous video interviews (AVIs) has become increasingly important due to their widespread adoption in modern recruitme

Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework

Model ReleasesDGX agent

arXiv:2606.29808v1 Announce Type: cross Abstract: Chart data extraction, which reverse-engineers data tables from chart images, is essential for reproducibility, analysis, retrieval, and redesign. Exi

ManimAgent: Self-Evolving Multimodal Agents for Visual Education

AgentsDGX agent

arXiv:2606.30296v1 Announce Type: new Abstract: Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode:

Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models

Local AiDGX agent

arXiv:2606.28369v1 Announce Type: cross Abstract: Semantic search and recommendation of similar documents, such as news and reports about unusual environmental events (e.g., a dead whale washed ashore

NIVA: A Multimodal Foundation Model for Actionable Earth System Intelligence

TutorialsDGX agent

arXiv:2606.28546v1 Announce Type: new Abstract: Recent advances in AI-driven weather and climate modeling have improved forecast skill while reducing computational cost. However, existing data-driven

OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data

ResearchDGX agent

arXiv:2606.30019v1 Announce Type: new Abstract: Music-driven dance video generation aims to synthesize expressive human motion that is temporally aligned with music while maintaining high visual fidel

29 Jun 2026

Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

ResearchDGX agent

arXiv:2606.28104v1 Announce Type: new Abstract: Vision-based assessment can provide convenient and cost-effective evaluation in Traditional Chinese Medicine (TCM) rehabilitation training, where action

Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videos

Model ReleasesDGX agent

arXiv:2606.27484v1 Announce Type: new Abstract: Autism spectrum disorder (ASD) affects 1 in 31 US children, yet median age at diagnosis exceeds four years. Artificial intelligence pipelines that provi

Multimodal Evaluator Preference Collapse: Cross-Modal Coupling in Self-Evolving Agents

Model ReleasesDGX agent

arXiv:2606.16682v3 Announce Type: replace-cross Abstract: When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge. We show that Evaluator Preferen

ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2603.19466v2 Announce Type: replace Abstract: Effective collaboration begins with knowing when to ask for help. For example, when trying to identify an occluded object, a human would ask someone

ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

AgentsDGX agent

arXiv:2606.27974v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fi

26 Jun 2026

BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning

AgentsDGX agent

arXiv:2602.22284v3 Announce Type: replace Abstract: Recent advancements in deep learning have actively addressed complex challenges within the Computer-Aided Design (CAD) domain.However, most existing

Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars

Model ReleasesDGX agent

arXiv:2606.26107v1 Announce Type: cross Abstract: Sign language communication systems, that integrate emotional expression remain underexplored, particularly for low-resource languages. This pilot stu

25 Jun 2026

An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

Model ReleasesDGX agent

arXiv:2606.25174v1 Announce Type: cross Abstract: Field-scale retrieval of surface soil moisture (SM), leaf area index (LAI), and plant height (PH) is essential for precision agriculture, yet it remai

SAC^2-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition

SafetyDGX agent

arXiv:2606.25542v1 Announce Type: new Abstract: Micro-expression recognition (MER) is challenging due to subtle facial movements, limited data, and the ambiguous relationship between Action Units (AUs

24 Jun 2026

3D Masked Autoencoders are Robust Learners of Volumetric and Multimodal Cellular Representations for Microscopy

Local AiDGX agent

arXiv:2606.23964v1 Announce Type: cross Abstract: Self-supervised learning in fluorescence microscopy often relies on 2D projections, despite the inherently three-dimensional nature of cells. We prese

E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis

Local AiDGX agent

arXiv:2606.23888v1 Announce Type: cross Abstract: While Vision-Language Models (VLMs) show great promise in volumetric medical report generation, they frequently suffer from visual hallucinations and

MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

Model ReleasesDGX agent

arXiv:2606.24155v1 Announce Type: new Abstract: Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a red

SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots

ResearchDGX agent

arXiv:2606.24350v1 Announce Type: new Abstract: Legged robots rely on accurate ground interaction awareness to traverse variable terrains, such as slippery surfaces. Existing slip detection methods of

23 Jun 2026

GO: The Great Outdoors Multimodal Dataset

AgentsDGX agent

arXiv:2501.19274v2 Announce Type: replace Abstract: The Great Outdoors (GO) dataset is a multi-modal annotated data resource aimed at advancing ground robotics research in unstructured environments. E

MMGist: A Comprehensive Multimodal Benchmark for 2027

Model ReleasesDGX agent

arXiv:2606.22437v1 Announce Type: new Abstract: We conduct a systematic study of 18 widely used vision-language benchmarks and identify three major issues: 1) many items do not rely on visual cues and

Retrieval-Augmented Multimodal Learning for Enzyme-Substrate Interaction Prediction Under Low-Homology Shift

ResearchDGX agent

arXiv:2606.22823v1 Announce Type: new Abstract: Enzyme substrate interaction (ESI) prediction is a fundamental computational task for biocatalyst discovery and reaction screening in large biochemical

SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

SafetyDGX agent

arXiv:2606.22144v1 Announce Type: new Abstract: Gastrointestinal cancers represent a growing health burden in the South Asian region, driven largely by rapid changes in socio-economic conditions & lif

Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation

SafetyDGX agent

arXiv:2606.19120v2 Announce Type: replace-cross Abstract: On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned

When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents

Model ReleasesDGX agent

arXiv:2606.22864v1 Announce Type: new Abstract: Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for

When Does a Video-Language Model Stop Watching? Reward Strength Controls the Formation and Reversal of Visual Shortcuts in Multimodal RLVR

ResearchDGX agent

arXiv:2606.22043v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is increasingly applied to large vision-language models (LVLMs), yet outcome-only optimization c

22 Jun 2026

Embed the world: Multimodal AI for searchable aerial imagery at scale

Model ReleasesDGX agent

In this post, we walk through the problem space, our architecture on Amazon Bedrock and Amazon OpenSearch Serverless, the evaluation methodology we built on OpenStreetMap ground truth, four experiment

11 Jun 2026

Context-Aware Multimodal Claim Verification in Spoken Dialogues

Model ReleasesDGX agent

arXiv:2606.11420v1 Announce Type: new Abstract: Every day, millions absorb claims from podcasts and streams that no fact-checker ever sees. Spoken misinformation is built through conversation, where c

ParseFixer: An Agentic Framework for Document Parsing via Selective Multimodal Correction

Model ReleasesDGX agent

arXiv:2606.11977v1 Announce Type: new Abstract: In this report, we present our third-place solution for the DataMFM Challenge Track 1: Document Parsing. This track requires models to recover structure

Sustainability assessment using multimodal AI agents

AgentsDGX agent

arXiv:2507.17012v2 Announce Type: replace Abstract: Reducing the rapidly growing environmental impact of the computing industry requires assessing the emissions of electronics at scale. However, a tra

10 Jun 2026

ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations

SafetyDGX agent

arXiv:2606.11188v1 Announce Type: new Abstract: This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-

Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

Model ReleasesDGX agent

arXiv:2606.10819v1 Announce Type: cross Abstract: RS-MLLMs enable natural-language understanding and spatial reasoning over earth observation imagery. However, existing models support only a narrow ra

MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents

Model ReleasesDGX agent

arXiv:2606.10742v1 Announce Type: cross Abstract: External memory has become a core component of modern web agents, enabling long-horizon reasoning through the retrieval of past experiences. However,

9 Jun 2026

Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

ResearchDGX agent

arXiv:2606.09770v1 Announce Type: cross Abstract: Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topog

MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework

ResearchDGX agent

arXiv:2606.07654v1 Announce Type: cross Abstract: Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language

When Vision Misleads, Let Location Speak: A Worldwide Image Geo-Localization Method via Location Attention Mechanism and Large Multimodal Models

Local AiDGX agent

arXiv:2606.08918v1 Announce Type: new Abstract: Worldwide image geo-localization aims to determine the capture location of an image on a global scale. Existing methods often mislocalize images by matc

8 Jun 2026

A robust PPG foundation model using multimodal physiological supervision

TutorialsDGX agent

arXiv:2606.07365v1 Announce Type: cross Abstract: Photoplethysmography (PPG), a non-invasive measure of changes in blood volume, is widely used in both wearable devices and clinical settings. Recent P

5 Jun 2026

BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding

ResearchDGX agent

arXiv:2606.05515v1 Announce Type: new Abstract: Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the na

4 Jun 2026

LMM-IR: Large-Scale Netlist-Aware Multimodal Framework for Static IR-Drop Prediction

ResearchDGX agent

arXiv:2511.12581v2 Announce Type: replace Abstract: Static IR drop analysis is a fundamental and critical task in the field of chip design. Nevertheless, this process can be quite time-consuming, pote

Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM

ResearchDGX agent

arXiv:2606.04719v1 Announce Type: new Abstract: The Transformer's quadratic complexity with input length imposes an unsustainable computational load on large language models (LLMs). In contrast, the S

3 Jun 2026

Auditing Engagement Incentives in the Kidfluencer Ecosystem: A Multimodal Weak Supervision Approach

Model ReleasesDGX agent

arXiv:2606.03173v1 Announce Type: cross Abstract: The rise of `kidfluencers' on YouTube has raised ethical concerns about child digital labor and exploitation. While emerging legislation attempts to r

Benchmarking Visual State Tracking in Multimodal Video Understanding

Model ReleasesDGX agent

arXiv:2606.03920v1 Announce Type: new Abstract: Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacit

Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding

ResearchDGX agent

arXiv:2606.03604v1 Announce Type: new Abstract: When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author i

CP-Agent: Context-Aware Multimodal Reasoning for Cellular Morphological Profiling under Chemical Perturbations

SafetyDGX agent

arXiv:2606.03435v1 Announce Type: new Abstract: Cell Painting combines multiplexed fluorescent staining, high-content imaging, and quantitative analysis to generate high-dimensional phenotypic readout

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

ResearchDGX agent

arXiv:2509.22468v2 Announce Type: replace-cross Abstract: High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce. W

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset

Local AiDGX agent

arXiv:2606.02956v1 Announce Type: new Abstract: Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We prese

2 Jun 2026

Brain-Atlas-Guided Generative Counterfactual Attention for Explainable Cognitive Decline Diagnosis Using Multimodal Connectomes

ResearchDGX agent

arXiv:2606.01237v1 Announce Type: new Abstract: Mild cognitive impairment (MCI) and subjective cognitive decline (SCD) are closely associated with the early Alzheimer's disease continuum, where accura

Ego-METAS: Egocentric online Multimodal Energy-efficient Temporal Action Segmentation benchmark

Model ReleasesDGX agent

arXiv:2606.02246v1 Announce Type: new Abstract: To operate in the physical world, embodied agents must perceive their environment in an 'always-on' fashion, selectively accessing the most informative

MMTalker: Multiresolution 3D Talking Head Synthesis with Multimodal Feature Fusion

ResearchDGX agent

arXiv:2604.02941v2 Announce Type: replace Abstract: Speech-driven three-dimensional (3D) facial animation synthesis aims to build a mapping from one-dimensional (1D) speech signals to time-varying 3D

ObjEmbed: Towards Universal Multimodal Object Embeddings

SafetyDGX agent

arXiv:2602.01753v3 Announce Type: replace Abstract: Aligning objects with corresponding textual descriptions is a fundamental challenge and a realistic requirement in vision-language understanding. Wh

Property Prediction of Stacked Bilayer Materials: A Multimodal Learning Approach

ApplicationsDGX agent

arXiv:2606.01012v1 Announce Type: new Abstract: AI for materials science is a critical topic within AI for science, aiming to accelerate materials discovery and produce accurate property predictions.

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

Model ReleasesDGX agent

arXiv:2601.04946v3 Announce Type: replace-cross Abstract: Automatic metrics are widely used to evaluate text-to-image models, often replacing human judgment in benchmarking, model selection, and large

SCL: Towards Domain Generalization via Single-Temporal Multimodal Contrastive Learning for Remote Sensing Change Detection

TutorialsDGX agent

arXiv:2404.11326v5 Announce Type: replace Abstract: In recent years, change detection and anomaly detection models based on CNN and transformer have achieved remarkable success across various datasets

1 Jun 2026

Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval

ResearchDGX agent

arXiv:2605.31229v1 Announce Type: cross Abstract: While retrieval is a core function of vision-language models, continually updating these models for retrieval tasks remains critically underexplored.

← Previous
1…4142434445…169
Next →