AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
7 Sept 2026

BioSync: Transformer-Based Cross-Modal Fusion for a Multimodal Physiological Digital Biomarker

Model ReleasesDGX agent

arXiv:2609.04504v1 Announce Type: new Abstract: Cardiac, neural, behavioral, and speech measurements from wearable and mobile devices provide partial, noise-sensitive views of physiological state. Bio

Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction

Local AiDGX agent

arXiv:2609.05361v1 Announce Type: new Abstract: Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and robots in real-world environments. This paper introduce

Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection

Model ReleasesDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2603.05623v2 Announce Type: replace-cross Abstract: Camera-LiDAR fusion is widely used in autonomous driving to enable accurate 3D object detection. However, bird's-eye view (BEV) fusion detecto

Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

Model ReleasesDGX agent

arXiv:2609.04801v1 Announce Type: new Abstract: Contextualized visual personalization can retrieve a true record yet apply it to the wrong visual subject. We formalize when a record may condition an a

4 Sept 2026

Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents

Model ReleasesDGX agent

arXiv:2609.03438v1 Announce Type: new Abstract: Graphical user interface (GUI) agents are increasingly used to execute natural-language instructions on user interfaces, yet real users may issue infeas

Landmark-Based Discrimination of Injury-Associated Athlete-Sessions from Minute-Resolution Multimodal Football Monitoring Data

ResearchDGX agent

arXiv:2609.03790v1 Announce Type: new Abstract: Athlete monitoring data may be recorded minute by minute throughout a match or training session, while injury information may only indicate whether the

2 Sept 2026

Agentic Multimodal Models for Environmental Hyperspectral Unmixing

AgentsDGX agent

arXiv:2609.01289v1 Announce Type: new Abstract: Hyperspectral unmixing is a key task in remote sensing that aims to decompose mixed pixels in hyperspectral images into their constituent material signa

Differentially Private Paired Table-Image Multimodal Synthesis

SafetyDGX agent

arXiv:2609.00708v1 Announce Type: cross Abstract: Differentially private (DP) synthesis has been extensively studied for tabular and image data separately, yet many real-world datasets contain images

1 Sept 2026

Cross-Regional Grapevine Cold Hardiness Prediction via Learned Multimodal Latent Representations

ResearchDGX agent

arXiv:2608.31097v1 Announce Type: new Abstract: Accurate daily predictions of cold hardiness in woody plants are critical in regions where freezing temperatures can damage dormant buds and reduce seas

MIRAGE-CAD: Construction-Mediated Multimodal Generation of Executable CAD Programs

ResearchDGX agent

arXiv:2608.28669v1 Announce Type: new Abstract: Recovering an executable parametric CAD program from an observed object is fundamentally ambiguous, because the same final geometry can result from diff

Parametric Multimodal User Memory: Storing What Captions Cannot Carry

AgentsDGX agent

arXiv:2608.28609v1 Announce Type: cross Abstract: A personalized agent needs a user memory: a persistent model of who its user is. Today it is almost always text -- transcripts and captions retrieved

SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks

AgentsDGX agent

arXiv:2606.15872v2 Announce Type: replace Abstract: Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of

28 Aug 2026

Relaxation-Aware Multimodal Sensing of Soft Gripper Driven by Structure-Perception-Learning

ResearchDGX agent

arXiv:2608.26622v1 Announce Type: new Abstract: Achieving stable, sustained grasping with soft robotic hands remains a fundamental challenge. Compliance enables safe and adaptive contact, yet the intr

26 Aug 2026

SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception

Model ReleasesDGX agent

arXiv:2608.24325v1 Announce Type: new Abstract: Reliable underwater perception requires complementary sensing under variable visibility. Optical cameras capture appearance and semantics but degrade ra

25 Aug 2026

LetzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents

Model ReleasesDGX agent

arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-li

Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules

ResearchDGX agent

arXiv:2608.22642v1 Announce Type: cross Abstract: Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and i

Multimodal-Language-Model-Driven Interaction and Companionship for Service Robots in Elderly-Care Facilities

SafetyDGX agent

arXiv:2608.21387v1 Announce Type: new Abstract: Service robots are increasingly deployed in elderly-care facilities to alleviate caregiver workload and enhance the quality of daily care. However, most

Toward a Foundation Plug-and-Play Prior for Computed Tomography Reconstruction via a Multimodal Diffusion Model

ResearchDGX agent

arXiv:2608.23190v1 Announce Type: new Abstract: Computed tomography (CT) throughput is limited by scan time, which grows with both the number of projections acquired and the detector integration time

19 Aug 2026

Expressivity In Multimodal Contrastive Learning

ResearchDGX agent

arXiv:2608.17203v1 Announce Type: cross Abstract: Contrastive learning has become a cornerstone of modern representation learning, powering CLIP-style models that underpin text-to-image generation, vi

Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal Policies

SafetyDGX agent

arXiv:2608.17484v1 Announce Type: new Abstract: Frozen vision-language-action (VLA) policies are increasingly improved at test time by sampling additional policy behaviors or introducing external demo

18 Aug 2026

Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance

ResearchDGX agent

arXiv:2608.14651v1 Announce Type: new Abstract: Effective disaster risk communication is a foundational humanitarian challenge, yet current emergency infrastructure fails to meet the needs of individu

MIRROR: Multimodal Intelligent Radiology Reasoning and Observation Reporter

ResearchDGX agent

arXiv:2608.16709v1 Announce Type: cross Abstract: A radiologist reading a model's output faces two problems. The model returns a number and no reason, and any system that turns that number into readab

Multi-Agent Closed-Loop Reasoning for Organic Structure Elucidation from Multimodal Spectra

AgentsDGX agent

arXiv:2608.14720v1 Announce Type: cross Abstract: Following the molecular discovery and synthesis revolutions, scalable automated structure elucidation from routine spectroscopic data remains an outst

PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Models

Model ReleasesDGX agent

arXiv:2608.14741v1 Announce Type: cross Abstract: We introduce PolyComp, a procedurally generated and verified benchmark that stresses visual recognition and compositional spatial reasoning. In each p

What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering

ResearchDGX agent

arXiv:2608.14841v1 Announce Type: new Abstract: Long-document visual question answering (VQA) over documents of tens to hundreds of pages mixing text, tables, charts, and figures typically follows ret

14 Aug 2026

Qwen Live EP2 - Agent First: Multimodal Gets to Work - Streamed live 11 hours ago

Model ReleasesDGX agent

Possibly best way to spend current hour before grabbing 27B model(Instead of opening duplicate repeated 27B posts here). I tried to grab summary(thought of including in this thread) of this video usin

13 Aug 2026

Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification

ResearchDGX agent

arXiv:2608.11472v1 Announce Type: new Abstract: Pancreatic cancer is among the most lethal malignancies; risk stratification of intraductal papillary mucinous neoplasms (IPMNs) offers a crucial opport

Qwen-MusicAVQA-7B: A Multimodal Model for Music Audio-Visual QA

Model ReleasesDGX agent

arXiv:2608.11329v1 Announce Type: cross Abstract: A common approach to adding audio to a vision-language model is to train or adapt a large omni-modal system. We show that a lightweight alternative ca

11 Aug 2026

Agentic Anomaly Detection with ORCA-Style Dynamic Inductive Bias Adaptation in Multimodal Wearable Time Series Data

Model ReleasesDGX agent

arXiv:2608.08859v1 Announce Type: cross Abstract: Wireless Body Area Networks (WBANs) generate multivariate physiological time series that are highly nonstationary and must often be processed under st

Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance

Local AiDGX agent

arXiv:2608.09482v1 Announce Type: cross Abstract: All-in-one image restoration is a unified low-level vision task that aims to effectively recover high-quality images from inputs degraded by various t

CFD-Guided Detection of Concept Drift in Multimodal Physiologic Signals

ResearchDGX agent

arXiv:2608.07759v1 Announce Type: cross Abstract: Cardiovascular AI models can classify clean elec- trocardiogram (ECG) signals, but real wearable signals change because of motion, breathing, posture,

From Speech to Interaction: Analyzing Multimodal Systems in Cocktail-Party Scenarios

ResearchDGX agent

arXiv:2608.08510v1 Announce Type: new Abstract: Humans have the remarkable ability to engage in spontaneous informal conversations and selectively attend to individual speakers while filtering out com

LF{}^{2}AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models

SafetyDGX agent

arXiv:2503.06211v3 Announce Type: replace-cross Abstract: Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as aud

Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework

SafetyDGX agent

arXiv:2608.09529v1 Announce Type: new Abstract: As an important subfield of cross-modal generation, synthesizing static visual content in the form of images from audio, namely audio-to-image (A2I) gen

10 Aug 2026

Chat UIs with native audio input for multimodal models?

Model ReleasesDGX agent

I've been running Gemma 4 E4B with oMLX and I can't find any chat interfaces that directly send the audio file to the model instead of running the audio through a separate STT layer. I can confirm the

7 Aug 2026

KVAE: Family of Tokenizers for Multimodal Generative Models

ResearchDGX agent

arXiv:2608.05798v1 Announce Type: new Abstract: Latent diffusion modeling (LDM), a prominent paradigm, utilizes tokenizers to map input signal to compressed representation. This dependency positions t

LoDA: A Level of Detection Aware Method and a Multimodal Sensing Benchmark for Object Level Change Detection

Model ReleasesDGX agent

arXiv:2608.05356v1 Announce Type: new Abstract: High-definition 3D LiDAR maps are important for autonomous driving and smart-city services, which require reliable detection of object-level changes in

OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction

Model ReleasesDGX agent

arXiv:2608.05539v1 Announce Type: new Abstract: Recent vision-language models (VLMs) can generate executable CAD programs from images, but existing methods mainly target coarse, general-purpose 3D obj

6 Aug 2026

Multimodal Spatiotemporal Atmospheric Data Assimilation with Latent Flow-matching

ResearchDGX agent

arXiv:2608.05103v1 Announce Type: new Abstract: Data assimilation (DA) uses Bayesian inference to update the state of a numerical forecast model with observed data. In this study, we propose a fundame

OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routing

Model ReleasesDGX agent

arXiv:2608.04434v1 Announce Type: new Abstract: Recent large language models (LLMs) have demonstrated remarkable progress in constraint-aware navigation, maze reasoning, and graph reasoning. However,

TRNet: Topography-Guided Frequency Rectification and Structure-Aware Decoding for Multimodal Paddy Rice Segmentation

ResearchDGX agent

arXiv:2608.04154v1 Announce Type: cross Abstract: Mapping paddy rice from very-high-resolution imagery in mountainous and hilly regions is difficult because terrain alters optical appearance and incre

5 Aug 2026

Minimax-Optimal Semiparametric Contextual Dynamic Pricing with Multimodal Revenue

Model ReleasesDGX agent

arXiv:2608.03142v1 Announce Type: cross Abstract: We study contextual dynamic pricing with arbitrary covariate sequences and bounded, possibly nonbinary purchase quantities. Demand follows a semiparam

Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents

Model ReleasesDGX agent

arXiv:2608.03327v1 Announce Type: new Abstract: Hybrid computer-use agents can act through screenshots or call text tools. We find that having a tool available does not settle which way the effect goe

When Do Fewer Visual Tokens Accelerate Multimodal Inference? A Break-Even Study Across Decision Locations and Hardware

Model ReleasesDGX agent

arXiv:2608.03649v1 Announce Type: new Abstract: Fewer visual tokens do not guarantee lower end-to-end latency. We evaluate break-even with a reproducible protocol that accounts for decision overhead,

4 Aug 2026

Evaluating VLMs on Multimodal Aristotelian Persuasion Tasks

Model ReleasesDGX agent

arXiv:2608.01238v1 Announce Type: new Abstract: Vision Language Models (VLMs) have demonstrated exceptional performance across various tasks. However, they have not yet been thoroughly evaluated on mo

3 Aug 2026

FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models

Model ReleasesDGX agent

arXiv:2607.29602v1 Announce Type: cross Abstract: Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are alre

30 Jul 2026

Visual Credit Audit for Multimodal Spatial Reasoning

Model ReleasesDGX agent

arXiv:2607.27069v1 Announce Type: new Abstract: Closed yes/no spatial benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts. Under a fixed forced-choi

28 Jul 2026

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

Model ReleasesDGX agent

arXiv:2607.24424v1 Announce Type: new Abstract: Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features ca

Multimodal Semantic-Probabilistic Objectness for Open World Object Detection

TutorialsDGX agent

arXiv:2607.23981v1 Announce Type: cross Abstract: Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incremental

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

Model ReleasesDGX agent

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

27 Jul 2026

Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

Model ReleasesDGX agent

arXiv:2607.21780v1 Announce Type: new Abstract: Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into the

24 Jul 2026

FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence

Local AiDGX agent

Introducing FLUX 3. One multi-modal model for Image, Video, Audio and Action-Prediction. Creations are truer to life in every kind of style. Blog Post : https://bfl.ai/blog/flux-3 submitted by /u/pmtt

23 Jul 2026

ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program

Model ReleasesDGX agent

arXiv:2607.19947v1 Announce Type: new Abstract: Electronic Theater Programs (ETPs) serve as critical promotional media in the performing arts, comprising a multi-page collection of heterogeneous visua

16 Jul 2026

FM^2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

Model ReleasesDGX agent

arXiv:2607.13386v1 Announce Type: new Abstract: Building foundation models for medical imaging requires pooling data across institutions, yet privacy regulations prohibit centralized aggregation. Exis

Multimodal Empirical Bayes Variational Autoencoders for Joint Longitudinal and Time-to-Event Modeling

ResearchDGX agent

arXiv:2607.13984v1 Announce Type: cross Abstract: Longitudinal tumor measurements, dropout information, and genetic covariates provide complementary information about treatment response, but integrati

10 Jul 2026

Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

Model ReleasesDGX agent

arXiv:2607.08317v1 Announce Type: new Abstract: Modern AI models achieve strong performance on many established benchmarks, yet they still fail on tasks that humans find almost trivial, such as manipu

MatBind: A Shared Embedding Space for Multimodal Materials Characterization

SafetyDGX agent

arXiv:2607.08470v1 Announce Type: new Abstract: Fully characterizing a crystalline material requires integrating heterogeneous data sources -- atomic structures, diffraction patterns, electronic densi

9 Jul 2026

BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning

ResearchDGX agent

arXiv:2607.07361v1 Announce Type: new Abstract: Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods a

8 Jul 2026

CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

Model ReleasesDGX agent

arXiv:2607.06534v1 Announce Type: new Abstract: Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the abilit

LEGATO 2: Toward Multimodal Sheet Music Recognition and Understanding

ResearchDGX agent

arXiv:2607.05769v1 Announce Type: cross Abstract: We propose a novel pipeline, Legato 2, for extracting symbolic notation and semantic knowledge from images of sheet music. Legato 2 features the first

← Previous
1…4546474849…169
Next →