AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
7 Jul 2026

Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

Local AiDGX agent

arXiv:2607.04636v1 Announce Type: new Abstract: Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance

GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Text

ResearchDGX agent

arXiv:2312.15320v3 Announce Type: replace-cross Abstract: Individuals with suspected rare genetic disorders often undergo multiple clinical evaluations, imaging studies, laboratory tests, and genetic

H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation

SafetyDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2607.02592v1 Announce Type: new Abstract: On-policy distillation (OPD) has recently emerged as an effective post-training paradigm by providing supervision on student-generated trajectories. How

Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology

ResearchDGX agent

arXiv:2607.04020v1 Announce Type: new Abstract: Uterine diseases represent an important category of gynecologic pathology and require accurate histopathological assessment for diagnosis and treatment

PhenoNEST: A Neuro-Symbolic Framework for Ontology-Aware Multimodal Plant Phenotyping and Trait Discovery

ResearchDGX agent

arXiv:2607.03245v1 Announce Type: new Abstract: High-throughput plant phenotyping generates valuable data that often remains trapped in unstructured text and isolated RGB images. To bridge this semant

Triple-Phase Multimodal Knowledge Aggregation Framework for Microbial Keratitis Subtype Diagnosis on Slit-Lamp Photography

Model ReleasesDGX agent

arXiv:2607.03740v1 Announce Type: cross Abstract: Microbial keratitis requires rapid pathogen identification to guide treatment, but culture- and PCR-based diagnostics are slow and resource-intensive.

UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

Model ReleasesDGX agent

arXiv:2603.24690v2 Announce Type: replace Abstract: In-context learning (ICL) enables fast task adaptation from demonstrations without per-task parameter updates but remains highly sensitive to exampl

3 Jul 2026

Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

ResearchDGX agent

arXiv:2509.11606v4 Announce Type: replace-cross Abstract: Cardiovascular diseases (CVDs) are the leading cause of death worldwide, accounting for approximately 17.9 million deaths each year. Early det

2 Jul 2026

Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility

ResearchDGX agent

arXiv:2607.00228v1 Announce Type: cross Abstract: Modern time-domain surveys such as the Zwicky Transient Facility (ZTF) generate hundreds of thousands of alerts each night, making real-time decisions

Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach

ResearchDGX agent

arXiv:2607.01115v1 Announce Type: cross Abstract: University stakeholders often face difficulties in accessing timely and reliable information, especially in developing countries, where there are very

1 Jul 2026

CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation

ResearchDGX agent

arXiv:2606.32012v1 Announce Type: cross Abstract: Uncertainty estimation has been a long-standing challenge in AI models; it amounts to 'knowing what you don't know,' and metacognition is notoriously

GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

ResearchDGX agent

arXiv:2511.00810v4 Announce Type: replace-cross Abstract: Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regi

IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editing

SafetyDGX agent

arXiv:2606.13368v2 Announce Type: replace Abstract: Computer-Aided Design is pivotal in modern manufacturing, yet existing automated methods predominantly rely on open-loop, one-shot generation, creat

30 Jun 2026

Multimodal Mathematical Reasoning with Diverse Solving Perspective

Model ReleasesDGX agent

arXiv:2507.02804v2 Announce Type: replace Abstract: Recent progress in large-scale reinforcement learning (RL) has notably enhanced the reasoning capabilities of large language models (LLMs), especial

Multimodal Representation Alignment for Cross-modal Information Retrieval

SafetyDGX agent

arXiv:2506.08774v2 Announce Type: replace-cross Abstract: Different machine learning models can represent the same underlying concept in different ways. This variability is particularly valuable for i

See, Think, Learn: A Self-Taught Multimodal Reasoner

TutorialsDGX agent

arXiv:2512.02456v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have achieved remarkable progress in integrating visual perception with language understanding. However, effecti

StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley

Model ReleasesDGX agent

arXiv:2507.07445v3 Announce Type: replace Abstract: Autonomous agents navigating human society must master both production activities and social interactions, yet existing benchmarks rarely evaluate t

Traffic-CBM: A Structurally Interpretable Multimodal Framework for Encrypted Traffic Classification

TutorialsDGX agent

arXiv:2606.29909v1 Announce Type: new Abstract: Encrypted traffic classification has achieved strong performance, but its decision process remains difficult to interpret. Existing methods usually comb

29 Jun 2026

Dialogue to Detection: A Multimodal Hybrid NLP Pipeline for Insurance Fraud Detection

AgentsDGX agent

arXiv:2606.28002v1 Announce Type: cross Abstract: Insurance fraud imposes substantial financial losses and operational inefficiencies, raising premiums and impacting trust among legitimate policyholde

DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection

Model ReleasesDGX agent

arXiv:2606.27499v1 Announce Type: cross Abstract: Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when a

HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering

ResearchDGX agent

arXiv:2603.18558v2 Announce Type: replace-cross Abstract: Long-form video question answering requires reasoning over extended temporal contexts, making frame selection a critical bottleneck for multi-

26 Jun 2026

Methane-Plume Segmentation From Hyperspectral Satellite Imagery Via Multimodal Deep Learning

ApplicationsDGX agent

arXiv:2606.26416v1 Announce Type: new Abstract: Efficient detection of methane plumes is crucial for understanding and mitigating global warming, as accurately identifying and segmenting them in earth

Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling

SafetyDGX agent

arXiv:2606.26922v1 Announce Type: cross Abstract: Continuous driver monitoring in automated vehicles requires low-latency inference while avoiding unsafe decisions under uncertain driver states. Large

25 Jun 2026

Cross-Attention Multimodal Learning for Predicting Response to Neoadjuvant Imatinib in Gastrointestinal Stromal Tumors: A Multicenter Retrospective Study

ResearchDGX agent

arXiv:2606.25579v1 Announce Type: cross Abstract: Background: Response to neoadjuvant imatinib in gastrointestinal stromal tumors (GISTs) is highly variable and cannot be reliably predicted using curr

MIL-LC: A Robust Magnetometer-Inertial-LiDAR Fusion Multimodal Localization Framework

AgentsDGX agent

arXiv:2606.25796v1 Announce Type: new Abstract: Localization in challenging environments, such as GNSS-denied, geometrically repetitive, or textureless scenes commonly found in offices, hotels, and un

23 Jun 2026

A Physics-Informed, Behavior-Aware Digital Twin for Robust Multimodal Forecasting of Core Body Temperature in Precision Livestock Farming

ApplicationsDGX agent

arXiv:2604.04098v2 Announce Type: replace Abstract: Precision livestock farming requires accurate and timely heat stress prediction to ensure animal welfare and optimize farm management. This study pr

Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving

Model ReleasesDGX agent

arXiv:2606.22913v1 Announce Type: new Abstract: Recent Vision-Language-Action (VLA) models have advanced end-to-end autonomous driving by incorporating reasoning for better interpretability and planni

Morphology-Aware Multimodal Representation Learning for Insect Phylogenetic Reconstruction

Model ReleasesDGX agent

arXiv:2606.22077v1 Announce Type: new Abstract: Morphological traits provide important evidence for phylogenetic reconstruction and evolutionary relationship analysis. Recent image-based approaches ha

Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion

ResearchDGX agent

arXiv:2606.21135v1 Announce Type: new Abstract: Human motion generation has been widely studied across diverse input modalities, text, music, and video, and recent efforts have unified these into sing

Predicting Immune Biomarkers with MultiModal Mixture-of-Expert Pathology Foundation Models Empowers Precision Oncology

ResearchDGX agent

arXiv:2606.18123v2 Announce Type: replace Abstract: Predicting immune biomarkers associated with the tumor immune microenvironment (TIME) is critical for advancing precision oncology, yet existing app

11 Jun 2026

DeceptionX: Explainable Deception Detection with Multimodal Large Language Models

ApplicationsDGX agent

arXiv:2606.11385v1 Announce Type: new Abstract: Deception detection is a critical and highly challenging task within affective computing and behavioral analysis. Existing deep learning methods typical

Decoding Multimodal Cues: Unveiling the Implicit Meaning Behind Hateful Videos

ResearchDGX agent

arXiv:2606.11953v1 Announce Type: new Abstract: Hateful videos have become prevalent on online platforms, highlighting an urgent need for effective detection. However, existing studies primarily focus

10 Jun 2026

Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

AgentsDGX agent

arXiv:2606.11176v1 Announce Type: cross Abstract: Data tells stories that shape society; the data journalist's job is to turn raw information into stories non-experts can trust. A high-quality news fe

9 Jun 2026

AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning

Model ReleasesDGX agent

arXiv:2603.14342v2 Announce Type: replace-cross Abstract: Modern agricultural data is sourced from diverse platforms and spans multiple spatial scales, ranging from ground-level close-up photography t

Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

SafetyDGX agent

arXiv:2606.08081v1 Announce Type: cross Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in s

mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models

SafetyDGX agent

arXiv:2606.07531v1 Announce Type: cross Abstract: We introduce mllm-shap, an open-source Python framework designed to extend Shapley Value (SV) explainability from text-only Large Language Models to M

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

Model ReleasesDGX agent

arXiv:2606.08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large mul

SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM

Local AiDGX agent

arXiv:2511.14143v2 Announce Type: replace-cross Abstract: Video Moment Retrieval is a task in video understanding that aims to localize a specific temporal segment in an untrimmed video based on a nat

8 Jun 2026

Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generation

ApplicationsDGX agent

arXiv:2606.06869v1 Announce Type: new Abstract: Aim: Existing AI-assisted traditional Chinese medicine diagnostic tools suffer from opaque reasoning processes, passive interaction, and limited treatme

From Vision to Text: A Compact Multimodal Approach for Robust, Cross-Domain Presentation Attack Detection on ID Cards

Model ReleasesDGX agent

arXiv:2606.06966v1 Announce Type: new Abstract: Cross-domain shifts challenge Presentation Attack Detection (PAD) on ID Cards, given the restricted data available due to privacy concerns. This work pr

Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization

SafetyDGX agent

arXiv:2606.07000v1 Announce Type: new Abstract: Recent post-training methods, particularly Reinforcement Learning with Verifiable Rewards (RLVR), have significantly enhanced the reasoning ability of L

5 Jun 2026

Biomazon: A Multimodal Dataset for 3D Forest Structure and Biomass Modeling in the Amazon Basin

Model ReleasesDGX agent

arXiv:2606.05368v1 Announce Type: new Abstract: Accurate, spatially explicit characterization of tropical forest structure is essential for carbon accounting and ecosystem monitoring, yet most ML pipe

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

Model ReleasesDGX agent

arXiv:2505.05026v5 Announce Type: replace Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept. While recent

ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation

SafetyDGX agent

arXiv:2606.05718v1 Announce Type: new Abstract: On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher. In m

4 Jun 2026

ViewMask-1-to-3: Multi-View Consistent Image Generation via Multimodal Discrete Diffusion Models

ResearchDGX agent

arXiv:2512.14099v3 Announce Type: replace Abstract: Motivated by discrete diffusion's success in language-vision modeling, we explore its potential for multi-view generation, a task dominated by conti

3 Jun 2026

Enhancing Protein-Protein Interaction Prediction with Hierarchical Motif-based Multimodal Protein Embedding

ResearchDGX agent

arXiv:2606.02629v1 Announce Type: cross Abstract: Protein-protein interactions (PPIs) are essential for many biological processes. However, existing PPI prediction approaches suffer from two major lim

Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records

Model ReleasesDGX agent

arXiv:2606.02747v1 Announce Type: cross Abstract: Planning records define restrictions over geographic areas, but their source documents often provide only indirect spatial evidence rather than machin

2 Jun 2026

Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion

SafetyDGX agent

arXiv:2602.23694v3 Announce Type: replace-cross Abstract: Human operators are still frequently exposed to hazardous environments such as disaster zones and industrial facilities, where intuitive and r

Longitudinal Multimodal Sensing of Physical Activity and Well-Being in Older Adults

SafetyDGX agent

arXiv:2606.00345v1 Announce Type: new Abstract: Wearable and mobile sensing technologies enable continuous monitoring of human behavior and health in real-world settings. However, predictive modeling

ProductWebGen: Benchmarking Multimodal Product Webpage Generation

Model ReleasesDGX agent

arXiv:2606.01022v1 Announce Type: cross Abstract: Crafting a product display webpage from a source product image, along with layout and visual content instructions, holds significant practical value f

Response-Aware Multimodal Learning for Post-Treatment Visual Acuity Forecasting

Local AiDGX agent

arXiv:2606.00588v1 Announce Type: new Abstract: Long-term visual acuity (VA) outcomes after anti-VEGF therapy are central to patient counseling, expectation setting, and follow-up planning in diabetic

Towards Automated Discovery: A Review of Generative Models, Multimodal Learning and Closed-Loop Workflows in Inverse Materials Design

TutorialsDGX agent

arXiv:2606.02507v1 Announce Type: cross Abstract: Inverse materials design is shifting materials discovery from forward prediction to targeted proposal of candidates that satisfy objectives under phys

1 Jun 2026

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

TutorialsDGX agent

arXiv:2605.30912v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers.

Best-Arm Identification-Based Trust Region Selection for Bayesian Optimization on Multimodal Functions

Local AiDGX agent

arXiv:2605.31050v1 Announce Type: new Abstract: Gaussian process-based Bayesian optimization (BO) is a popular approach for expensive black-box optimization, but its performance often degrades on comp

UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis

ResearchDGX agent

arXiv:2510.15710v3 Announce Type: replace Abstract: Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central

Your Multimodal Speech Model Says I Have a Face for Radio

Model ReleasesDGX agent

arXiv:2605.30472v1 Announce Type: new Abstract: As large neural models have become better at language tasks, researchers are increasingly building multi- and omnimodal models that handle more modaliti

29 May 2026

Archon: A Unified Multimodal Model for Holistic Digital Human Generation

ResearchDGX agent

arXiv:2605.30311v1 Announce Type: cross Abstract: Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visu

Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance

SafetyDGX agent

arXiv:2411.14279v2 Announce Type: replace-cross Abstract: Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. However, despite showing promising per

OmniCD: A Foundational Framework for Remote Sensing Image Change Detection Guided by Multimodal Semantics

ResearchDGX agent

arXiv:2605.30168v1 Announce Type: new Abstract: Change detection (CD) in remote sensing is vital for applications such as urban monitoring and disaster assessment, yet traditional methods struggle wit

UniNote: A Unified Embedding Model for Multimodal Representation and Ranking

Local AiDGX agent

arXiv:2605.29287v1 Announce Type: cross Abstract: Item-to-Item (I2I) retrieval is a fundamental part of modern content platforms, supporting critical industrial workflows from recommendation engines t

← Previous
1…3536373839…169
Next →