AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
1 Jun 2026

ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment

SafetyDGX agent

arXiv:2605.30965v1 Announce Type: cross Abstract: Recent advancements in text-guided audio generation have yielded promising results in diverse domains, including sound effects, speech, and music. How

Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search

Model ReleasesDGX agent

arXiv:2605.30917v1 Announce Type: cross Abstract: As large-scale visual-document corpora such as arXiv papers and enterprise PDFs continue to grow, visual-document retrieval has gained increasing atte

TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation

Model Releases
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
DGX agent

arXiv:2605.30673v1 Announce Type: new Abstract: Classroom videos contain observable teaching practices, but their pedagogical and visual signals are rarely organized in forms suitable for model evalua

29 May 2026

COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings

ResearchDGX agent

arXiv:2605.29628v1 Announce Type: cross Abstract: Contrastive Language-Audio Pretraining (CLAP) models are widely used for audio understanding and support modality-agnostic condition swapping in many

Lightweight Multimodal LLM-Enabled Cost-Effective Defect Grading of Power Transmission Equipment

ResearchDGX agent

arXiv:2605.28822v1 Announce Type: new Abstract: Defect grading of power transmission equipment (DGPTE) is crucial to the stability of electric energy transmission. Although existing machine learning m

Multimodal LLMs See Sentiment

Model ReleasesDGX agent

arXiv:2508.16873v3 Announce Type: replace Abstract: Understanding how visual content conveys sentiment is increasingly important in a digital landscape dominated by imagery. However, sentiment percept

Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2605.29299v1 Announce Type: cross Abstract: Evaluations of dental vision-language models remain fragmented across datasets, task definitions and metrics, and often ignore their computational cos

ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control

SafetyDGX agent

arXiv:2605.29425v1 Announce Type: new Abstract: Reinforcement learning (RL) has shown promise in traffic signal control (TSC). However, its reliance on predefined states limits responsiveness to obser

Run Step 3.7 Flash on NVIDIA GPUs with Enterprise-Ready Multimodal AI

HardwareDGX agent

Step 3.7 Flash is a 198B-parameter Mixture-of-Experts vision-language model designed for enterprise-scale production workloads, featuring native image and video input, a 256k context window, and confi

28 May 2026

Adaptive Multimodal Agents-Based Framework for Automatic Workflow Execution

AgentsDGX agent

arXiv:2605.28607v1 Announce Type: new Abstract: Modern information systems require autonomous agents capable of navigating complex workflows, yet current methodologies often struggle with the transiti

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

SafetyDGX agent

arXiv:2605.28774v1 Announce Type: new Abstract: Vision-language models with extended reasoning succeed on complex problems, but many real-world problems require external tools that internal reasoning

Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance

Local AiDGX agent

arXiv:2602.03491v2 Announce Type: replace-cross Abstract: Reasoning over table images remains challenging for Large Vision-Language Models (LVLMs) due to complex layouts and tightly coupled structure-

FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales

Model ReleasesDGX agent

arXiv:2605.28174v1 Announce Type: cross Abstract: Foundation models offer a promising route to transferable remote sensing representations, but many current approaches depend on very large pretraining

FLUID: From Ephemeral IDs to Multimodal Semantic Codes for Industrial-Scale Livestreaming Recommendation

ApplicationsDGX agent

arXiv:2605.21832v2 Announce Type: replace Abstract: Modern recommender systems rely heavily on ID-based collaborative filtering: each item is represented by a unique ID embedding that accumulates coll

Semantic-Aware Interpretable Multimodal Music Auto-Tagging

ResearchDGX agent

arXiv:2505.17233v3 Announce Type: replace Abstract: Music auto-tagging is essential for organizing and discovering music in extensive digital libraries. While foundation models achieve exceptional per

When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?

SafetyDGX agent

arXiv:2605.27932v1 Announce Type: cross Abstract: Think-with-image reasoning is emerging as a new inference paradigm for large vision-language models, but its safety implications remain poorly underst

27 May 2026

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

TutorialsDGX agent

arXiv:2605.27310v1 Announce Type: new Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry nee

Particle-Lund Multimodality in Jet Taggers

ResearchDGX agent

arXiv:2605.26821v1 Announce Type: cross Abstract: The Lund plane offers a physics-motivated, hierarchical representation of QCD radiation within jets, while transformer-based taggers have reached stat

QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents

AgentsDGX agent

arXiv:2605.27068v1 Announce Type: cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM)

26 May 2026

FairJudge: Abstention-Aware Multimodal Judges for Fairness and Alignment Evaluation in Text-to-Image Models

SafetyDGX agent

arXiv:2510.22827v3 Announce Type: replace-cross Abstract: Evaluating text-to-image (T2I) systems requires judging not only whether an image matches a prompt, but also whether socially salient attribut

ORACAL: A Robust and Explainable Multimodal Framework for Smart Contract Vulnerability Detection with Causal Graph Enrichment

Model ReleasesDGX agent

arXiv:2603.28128v2 Announce Type: replace Abstract: Although Graph Neural Networks (GNNs) have shown promise for smart contract vulnerability detection, they still face significant limitations. Homoge

25 May 2026

ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2605.23694v1 Announce Type: new Abstract: Chart descriptions are essential for accessibility, cross-modal retrieval, and assisting readers in extracting insights from complex visualizations. As

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Model ReleasesDGX agent

arXiv:2510.08945v3 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduc

FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence

AgentsDGX agent

arXiv:2605.22868v1 Announce Type: new Abstract: Autonomous systems and smart-industry deployments increasingly split computation across near-sensor, edge, and cloud resources, where tight energy, late

Inconsistency-aware Multimodal Schrodinger Bridge for Deepfake Localization

ResearchDGX agent

arXiv:2605.23113v1 Announce Type: new Abstract: Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under singl

Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems

Model ReleasesDGX agent

arXiv:2605.22891v1 Announce Type: new Abstract: Evaluation in scientific reconstruction is dominated by pointwise metrics - RMSE, MAE, per-event resolution - under the implicit assumption that lower e

22 May 2026

Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models

Model ReleasesDGX agent

arXiv:2605.22732v1 Announce Type: cross Abstract: We investigate whether acoustic emotion recognition models can serve as proxies for the Pathos dimension in political speech analysis, as operationali

Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?

ResearchDGX agent

arXiv:2605.22170v1 Announce Type: new Abstract: In recent years, several Speech Language Models (SLMs) that represent speech and written text jointly have been presented. The question then emerges abo

Towards Selection of Large Multimodal Models as Engines for Burned-in Protected Health Information Detection in Medical Images

Model ReleasesDGX agent

arXiv:2511.02014v2 Announce Type: replace Abstract: The detection of Protected Health Information (PHI) in medical imaging is critical for safeguarding patient privacy and ensuring compliance with reg

20 May 2026

EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

Model ReleasesDGX agent

arXiv:2605.19004v1 Announce Type: new Abstract: Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensin

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction

Model ReleasesDGX agent

arXiv:2605.19765v1 Announce Type: new Abstract: Existing affective-computing, social-signal-processing, and meeting corpora capture important parts of human interaction, but they rarely support analys

How does longer temporal context enhance multimodal narrative video processing in the brain?

SafetyDGX agent

arXiv:2602.07570v2 Announce Type: replace-cross Abstract: Understanding how humans and artificial intelligence systems process complex narrative videos is a fundamental challenge at the intersection o

StampFormer: A Physics-Guided Material-Geometry-Coupled Multimodal Model for Rapid Prediction of Physical Fields in Sheet Metal Stamping

ResearchDGX agent

arXiv:2605.18835v1 Announce Type: new Abstract: Traditional sheet metal forming relies on time-consuming and expensive Finite Element Analysis (FEA) for design validation, a process that significantly

19 May 2026

CAM-VFD: Cross-Attention Multimodal Video Forgery Detection

ResearchDGX agent

arXiv:2605.17133v1 Announce Type: cross Abstract: The rapid advancement of Deepfake technologies and video manipulation tools poses a critical challenge to multimedia forensics, judicial evidence inte

F2IND-IT! -- Multimodal Fuzzy Fake Indian News Detection using Images and Text

ResearchDGX agent

arXiv:2605.17115v1 Announce Type: new Abstract: Biased manipulation of facts across regional and national media outlets complicates misinformation detection in diverse landscapes like India. This pape

Gesture First, LLM-Assisted Voice Complement: Exploring Multimodal Robot 'Puppeteer' Teleoperation Via Virtual Counterpart in Augmented Reality

ResearchDGX agent

arXiv:2506.13189v3 Announce Type: replace-cross Abstract: Robot teleoperation via augmented reality (AR) offers a promising path toward more intuitive human-robot interaction (HRI). We present a head-

M^2FedAQI: Multimodal Federated Learning for Air Quality Prediction on Heterogeneous Edge Devices

Model ReleasesDGX agent

arXiv:2605.16375v1 Announce Type: new Abstract: Accurate air quality prediction is essential for public health, environmental monitoring, and industrial safety. However, most existing approaches rely

Multimodal Cultural Heritage Knowledge Graph Extension with Language and Vision Models

Model ReleasesDGX agent

arXiv:2605.17669v1 Announce Type: new Abstract: The preservation and interpretation of cultural heritage increasingly rely on digital technologies, among which Knowledge Graphs (KGs) stand out for the

On Improving Multimodal Pedestrian Trajectory Prediction with CVAE: A Study on Benchmark and Robot Data

Model ReleasesDGX agent

arXiv:2605.18262v1 Announce Type: new Abstract: Accurate pedestrian trajectory prediction is crucial for autonomous systems operating in complex environments, such as modular buses and delivery robots

Ranking-Aware Calibration for Reliable Multimodal Reinforcement Learning

SafetyDGX agent

arXiv:2605.16999v1 Announce Type: new Abstract: Reinforcement learning post-training has substantially improved the reasoning accuracy of vision-language models, yet the resulting policies remain poor

18 May 2026

MAgSeg: Segmentation of Agricultural Landscapes in High-Resolution Satellite Imagery using Multimodal Large Language Models

SafetyDGX agent

arXiv:2605.16179v1 Announce Type: new Abstract: Agricultural landscape segmentation in the Global South is challenging as it is characterized by fragmented plots, high intra-class variance, and a scar

15 May 2026

MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

Model ReleasesDGX agent

arXiv:2510.23477v2 Announce Type: replace Abstract: Effective math tutoring requires not only solving problems but also diagnosing students' difficulties and guiding them step by step. While multimoda

Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation

ResearchDGX agent

arXiv:2508.05008v2 Announce Type: replace Abstract: Vision-Language Models (VLMs), such as CLIP, have demonstrated remarkable zero-shot capabilities in various computer vision tasks. However, their ap

ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

AgentsDGX agent

arXiv:2605.14113v1 Announce Type: cross Abstract: While interpretable prototype networks offer compelling case-based reasoning for clinical diagnostics, their raw continuous outputs lack the semantic

TeachAnything: A Multimodal Crowdsourcing Platform for Training Embodied AI Agents in Symmetrical Reality

AgentsDGX agent

arXiv:2605.14556v1 Announce Type: new Abstract: Symmetrical Reality (SR) is emerging as a future trend for human-agent coexistence, placing higher demands on agents to acquire human-like intelligence.

14 May 2026

Learning to See What You Need: Gaze Attention for Multimodal Large Language Models

Local AiDGX agent

arXiv:2605.13080v1 Announce Type: new Abstract: When humans describe a visual scene, they do not process the entire image uniformly; instead, they selectively fixate on regions relevant to their inten

Multitask Multimodal Fusion with Tabular Foundation Models for Peak and Durability Prediction of Pertussis Booster Response

ResearchDGX agent

arXiv:2605.12852v1 Announce Type: new Abstract: Pertussis booster vaccination produces immune responses that vary widely across individuals in both peak magnitude and long-term durability. These two p

VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence

ResearchDGX agent

arXiv:2605.13798v1 Announce Type: new Abstract: Cross-modal 3D medical image analysis requires voxelwise representations that remain anatomically consistent across imaging contrasts, scanners, and acq

12 May 2026

DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis

ApplicationsDGX agent

arXiv:2605.08138v1 Announce Type: new Abstract: Synthetic data has emerged as a crucial solution to the data scarcity bottleneck in large language models (LLMs), particularly for specialized domains a

EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding

Model ReleasesDGX agent

arXiv:2605.08847v1 Announce Type: new Abstract: In the context of today's high-pressure, aging society, the demand for large-scale emotional models capable of providing empathetic support is more crit

Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning

SafetyDGX agent

arXiv:2605.09395v1 Announce Type: new Abstract: In this paper, we propose the first VLnderline{extbf{M}} nderline{extbf{a}}gentic nderline{extbf{r}}easoning framework for few-nderline{extbf{s}}hot mul

EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving

Model ReleasesDGX agent

arXiv:2605.10556v1 Announce Type: new Abstract: As large language models span dense, mixture-of-experts, and state-space architectures and are deployed on heterogeneous accelerators under increasingly

FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

Model ReleasesDGX agent

arXiv:2605.08820v1 Announce Type: cross Abstract: Artificial Intelligence (AI)-generated images have become increasingly realistic and readily adaptable to concrete real-world claims, creating new cha

iPay: Integrated Payment Action Recognition via Multimodal Networks and Adaptive Spatial Prior Learning

Local AiDGX agent

arXiv:2605.10732v1 Announce Type: cross Abstract: Automated transit payment analysis is vital for scalable fare auditing and passenger analytics, yet practice still relies on limited manual inspection

Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology

Model ReleasesDGX agent

arXiv:2605.09152v1 Announce Type: new Abstract: Deciphering animal intent is a fundamental challenge in computational ethology, largely because of semantic aliasing, the phenomenon where identical ext

Neuroscience-Inspired Analyses of Visual Interestingness in Multimodal Transformers

ResearchDGX agent

arXiv:2605.08188v1 Announce Type: cross Abstract: Human attention is the gateway to conscious perception, memory and decision-making. However, its role in modern transformer models remains largely une

Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoning

SafetyDGX agent

arXiv:2605.09614v1 Announce Type: new Abstract: Long chain-of-thought (CoT) reasoning improves large vision--language models, but visual information often fades during generation, limiting long-horizo

The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence?

SafetyDGX agent

arXiv:2605.09352v1 Announce Type: new Abstract: Understanding why independently trained neural networks from different modalities converge toward shared representations, and where this convergence lea

UM-Text: A Unified Multimodal Model for Image Understanding and Visual Text Editing

ResearchDGX agent

arXiv:2601.08321v3 Announce Type: replace Abstract: With the rapid advancement of image generation, visual text editing using natural language instructions has received increasing attention. The main

11 May 2026

Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment

TutorialsDGX agent

arXiv:2605.06969v1 Announce Type: new Abstract: Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance percepti

← Previous
1…4243444546…169
Next →