AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “multimodal”

GridTimelineEvolution
10,119 results
15 Oct 2026

Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters

ResearchDGX agent

arXiv:2610.19901v1 Announce Type: new Abstract: Adapting a single vision-language model to medical image parsing requires training choices that reflect the demands of each task. We study task-specific

RetailAgent: Structured Adverse Timing in Self-Conditioned Multimodal LLM Trading Agents

SafetyDGX agent

arXiv:2608.28399v2 Announce Type: replace Abstract: In financial markets, a sequential policy that reacts systematically to price movements may become predictable to other market participants. This pa

14 Oct 2026

Efficient Hybrid Hard--Soft Signal Temporal Logic MPC for Driving under Multimodal Uncertainty

Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Safety
DGX agent

arXiv:2610.15934v1 Announce Type: new Abstract: Motion planning for automated vehicles in interactive traffic has to simultaneously ensure safety, respect traffic rules, and maintain comfortable progr

Multimodal Fusion of RGB and Complementary Modalities for Semantic Segmentation

Model ReleasesDGX agent

arXiv:2610.17800v1 Announce Type: new Abstract: Multi-modal semantic segmentation adds an auxiliary sensing stream to RGB images (RGB-X), such as thermal, LiDAR, depth, event, polarization, or light f

Multimodal Sensor-Gated Mid-Air Launch and Vision-Based Recovery of a Child UAV on a Hovering Mothership Drone

AgentsDGX agent

arXiv:2610.15740v1 Announce Type: new Abstract: Launching a micro unmanned aerial vehicle (UAV) from a hovering carrier and recovering it in flight requires reliable confirmation that the child is sec

Parameter-Efficient Multimodal Brain Tumor Segmentation With Stage-Wise Shared Adapters

Model ReleasesDGX agent

arXiv:2610.18571v1 Announce Type: new Abstract: Different MRI modalities reveal different parts of a brain tumor. Segmentation models combine information from these modalities to outline tumor regions

12 Oct 2026

Decision 3.0: open, multimodal decision models from 0.6B to 27B. Ranks #1 on both Jev Decision Index boards

Model ReleasesDGX agent

With the vLLM Semantic Router team we released Decision 3.0 (d3), our new SOTA decision model family ranging from 0.6B to 27B, Apache-2.0, all reading text, images and video. How it works - Built on Q

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

ResearchDGX agent

arXiv:2610.13397v1 Announce Type: cross Abstract: We develop a model-free, noise-aware Bayesian clustering framework that constructs a martingale posterior over clusterings from a predictive Bayesian

9 Oct 2026

DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

Model ReleasesDGX agent

arXiv:2610.11993v1 Announce Type: cross Abstract: Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared w

Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers

ResearchDGX agent

arXiv:2610.11315v1 Announce Type: cross Abstract: In diffusion transformers, low-rank branches can mitigate 4-bit weight--activation (W4A4) post-training quantization (PTQ) loss by decomposing each we

EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams

ApplicationsDGX agent

arXiv:2610.12248v1 Announce Type: new Abstract: Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user's activity through first-pe

OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

AgentsDGX agent

arXiv:2610.12419v1 Announce Type: new Abstract: Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and

8 Oct 2026

Immiscible Diffusion Policy: Preserving Multimodal Robot Actions through Label-Free Noise Assignment

SafetyDGX agent

arXiv:2610.09369v1 Announce Type: cross Abstract: When diffusion policies were first introduced, they were expected to recover multi-modal action distributions. However, we find this expectation does

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

TutorialsDGX agent

arXiv:2610.09355v1 Announce Type: cross Abstract: Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalogra

PrismSSL: One Interface, Many Modalities; A Single-Interface Library for Multimodal Self-Supervised Learning

Model ReleasesDGX agent

arXiv:2511.17776v2 Announce Type: replace Abstract: We present PrismSSL, a Python library that unifies state-of-the-art self-supervised learning (SSL) methods across audio, vision, graphs, and cross-m

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

ResearchDGX agent

arXiv:2610.10409v1 Announce Type: cross Abstract: General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carr

6 Oct 2026

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning

Model ReleasesDGX agent

arXiv:2610.06571v1 Announce Type: new Abstract: Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences,

Conditional Flow Matching for Single-Neuron Electrophysiology: Capturing Multimodal Responses Across Stimuli

ResearchDGX agent

arXiv:2610.06520v1 Announce Type: new Abstract: Neurons of the brain exhibit a rich repertoire of electrophysiology dynamics with the same repeated stimulus eliciting very different voltage responses

v0.40.0-rc6: model: add multimodal embeddings (#18820)

Local AiDGX agent

Implements the EmbeddingGemma2Model architecture on the MLX runner: 24-layer bidirectional text encoder with PLE, shared gemma4 vision/audio towers, mean-pool + L2 output. /api/embed accepts per-item

2 Oct 2026

Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing

Model ReleasesDGX agent

arXiv:2610.00825v1 Announce Type: new Abstract: Dubbing quality control requires a reference-free judge that can determine whether a candidate text line matches a speaker's visible articulation in bot

Same Reward, Different Skills: When Multimodal RL Learns to Look

Model ReleasesDGX agent

arXiv:2610.01908v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With im

1 Oct 2026

Spherical Interpolation for Backward-Compatible Multimodal Representations

Local AiDGX agent

arXiv:2609.39836v1 Announce Type: new Abstract: Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natural

30 Sept 2026

FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning

Local AiDGX agent

arXiv:2609.37582v1 Announce Type: cross Abstract: Federated knowledge must remain usable by recipients with different modalities, private architectures, and tasks. We present FedSocket, which makes re

Multimodal Detection of Higher-Order Behavioral Constructs: Self-Compassion in Structured Reflective Interaction

TutorialsDGX agent

arXiv:2609.37148v1 Announce Type: cross Abstract: Many of the qualities that matter most in how people learn and grow, how someone regulates their emotions, reflects on a setback, or stays aware of ot

OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells

Model ReleasesDGX agent

arXiv:2609.37773v1 Announce Type: new Abstract: Artificial Intelligence Virtual Cells (AIVCs) are envisioned as scientific agents that simulate cellular responses, explain underlying mechanisms, and s

Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation

SafetyDGX agent

arXiv:2609.36995v1 Announce Type: cross Abstract: Few-step streaming audio--video generation requires both causal modeling and step distillation, yet standard training recipes face two context-related

VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict

ResearchDGX agent

arXiv:2609.37324v1 Announce Type: new Abstract: Conflicting emotional cues can be individually valid: a subdued voice may reflect a blocked goal while a smile satisfies a social obligation. Their inte

29 Sept 2026

AMBIT: Anticipatory Multimodal Body Recruitment for Bimanual Tracking on a Humanoid

ResearchDGX agent

arXiv:2609.33484v1 Announce Type: new Abstract: A humanoid with 5-DoF arms cannot track generic bimanual end-effector trajectories with its arms alone; pelvis and waist motion must be recruited, but w

Refinement Symmetry in Multimodal Transformers

ResearchDGX agent

arXiv:2609.32669v1 Announce Type: cross Abstract: Attention weights depend on token counts, which change with the representation of a signal. We study refinement symmetry: splitting a representation w

The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond

Model ReleasesDGX agent

arXiv:2609.33280v1 Announce Type: cross Abstract: Medical images are released with the reports that describe them, and protecting the image does not protect the report. This paper measures the text co

28 Sept 2026

DepthEvidence: Unifying Metric Depth Prediction and Geometric Reasoning in Multimodal Language Models

Model ReleasesDGX agent

arXiv:2609.31103v1 Announce Type: cross Abstract: Spatial reasoning with metric constraints requires linking objects to geometric measurements and preserving their numerical content during language re

25 Sept 2026

Accelerate multimodal RL training with SkyRL on Amazon SageMaker HyperPod

TutorialsDGX agent

Learn how to run SkyRL, an open-source reinforcement learning framework, on Amazon SageMaker HyperPod to post-train a Qwen3-VL-8B vision-language model with GRPO. This walkthrough covers building the

Multimodal Routing and Region Refinement for Language-Guided Medical Image Segmentation

Model ReleasesDGX agent

arXiv:2609.28860v1 Announce Type: new Abstract: Textual descriptions can reduce ambiguity in medical image segmentation by specifying the finding and location to be delineated. Existing text-guided me

Multimodal Thinking with Renderable Programs

Model ReleasesDGX agent

arXiv:2609.30130v1 Announce Type: cross Abstract: Current vision-language models (VLMs) excel at visual content understanding and text-based reasoning, yet their structure limits the advancement of in

OncoVision: Integrating Mammography and Clinical Data through Attention-Driven Multimodal AI for Enhanced Breast Cancer Diagnosis

ResearchDGX agent

arXiv:2511.19667v2 Announce Type: replace Abstract: OncoVision is a privileged-information training framework that uses mammography images and clinical features during training and performs inference

WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks

Model ReleasesDGX agent

arXiv:2507.00938v4 Announce Type: replace-cross Abstract: Foundation models now enable autonomous agents to interact with real-world websites, but existing benchmarks emphasize general-purpose browsin

24 Sept 2026

Building multimodal models for spatial reasoning

AgentsDGX agent

What is spatial reasoning? AI is moving beyond the digital world. The next generation of AI systems needs to understand and interact with physical environments, from robots to autonomous systems navig

Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

Model ReleasesDGX agent

arXiv:2609.27353v1 Announce Type: new Abstract: Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reprod

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

Model ReleasesDGX agent

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

23 Sept 2026

AIGC Video Detection based on the fusion of spatial-frequency-optical flow multimodal features

SafetyDGX agent

arXiv:2609.26274v1 Announce Type: new Abstract: The rapid evolution of generative AI (e.g., Sora, Hunyuan) makes it essential to develop effective detection strategies that can generalize across ever-

Designing Task-Induced Arousal: A Multimodal Stress Induction Method for Interactive Experiments

ResearchDGX agent

arXiv:2609.26156v1 Announce Type: cross Abstract: HCI and HRI studies often require short, repeatable arousal manipulations that can run while participants continue interacting with a device or robot.

Destination Support Restoration for Finite-Set Multimodal Trajectory Prediction

ResearchDGX agent

arXiv:2609.25942v1 Announce Type: new Abstract: Robots operating around pedestrians often reason over a finite set of predicted human futures. Repeated online updates can concentrate this limited pred

ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion

ResearchDGX agent

arXiv:2609.26648v1 Announce Type: cross Abstract: Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under chall

Text, Pixels, or Both? Evaluating Input Representations for Multimodal Document QA

ResearchDGX agent

arXiv:2609.22628v1 Announce Type: new Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We isol

Zephyron: Integrated Design and Analytical Evaluation of a Solar-Assisted Mobile Manipulator for Multimodal Environmental Reconnaissance and Distributed Visual Inference

SafetyDGX agent

arXiv:2609.25709v1 Announce Type: new Abstract: Environmental reconnaissance needs mobile platforms that carry sensors, preserve measurement context, and return interpretable evidence under limited en

22 Sept 2026

Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing

ResearchDGX agent

arXiv:2609.24791v1 Announce Type: cross Abstract: Challenging behaviors including aggression, self-injury, and property destruction are observed in 68% of autistic youth and pose risks to youth and ca

21 Sept 2026

Offline Multimodal Large Language Models for Decision Support in Air Operations

ResearchDGX agent

arXiv:2609.21390v1 Announce Type: new Abstract: Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In

Reading Less While Writing: A Closed-Form Bandwidth Dial for Streaming Multimodal Decoders

Model ReleasesDGX agent

arXiv:2609.20845v1 Announce Type: new Abstract: A decoder that turns video or audio into text conventionally consumes the entire input before emitting a word. Offline this is merely more than the task

Rectify, Don't Regret: On-Policy Closed-Loop Training for Multimodal Trajectory Prediction

SafetyDGX agent

arXiv:2603.23393v2 Announce Type: replace Abstract: Current trajectory prediction models are primarily trained in an open-loop manner, which often leads to covariate shift and compounding errors when

18 Sept 2026

Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment

ResearchDGX agent

arXiv:2609.20199v1 Announce Type: new Abstract: Cognitive workload reflects the mental effort required during task performance and is central to the design of adaptive human-machine systems. The use o

Understanding Role Switching in Human-AI Collaboration through Multimodal Behavioral Signals

ResearchDGX agent

arXiv:2509.20666v2 Announce Type: replace-cross Abstract: Human-AI collaboration often requires dividing complex tasks into complementary subtasks. As a task unfolds, users may want to shift which sub

16 Sept 2026

DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment

Model ReleasesDGX agent

arXiv:2601.18493v2 Announce Type: replace Abstract: Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-cen

14 Sept 2026

Intern-S2-397B (multimodal, reasoning, coding, and scientific agent capabilities)

Local AiDGX agent

Model: https://huggingface.co/internlm/Intern-S2-397B Collection: https://huggingface.co/collections/internlm/intern-s2 From Intern Large Models on 𝕏: https://x.com/intern_lm/status/209942518458737097

On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

Local AiDGX agent

arXiv:2609.11961v1 Announce Type: new Abstract: Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-pre

Physics as the label for measuring and correcting materials reasoning in multimodal models

Model ReleasesDGX agent

arXiv:2609.12181v1 Announce Type: new Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate ph

12 Sept 2026

Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36

Model ReleasesDGX agent

I find this new model at HF: 'Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding. Architecture Agnes-3.0-Flash

11 Sept 2026

AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

Local AiDGX agent

arXiv:2609.09212v1 Announce Type: cross Abstract: This paper presents an end-to-end evaluation framework for image-triggered command injection against computer-use agents (CUAs). The goal is to test w

An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks

Model ReleasesDGX agent

arXiv:2609.09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read

9 Sept 2026

A Multimodal Label Forecasting Method for Aperiodic Visuo-Motor Time Series

Model ReleasesDGX agent

arXiv:2609.07930v1 Announce Type: new Abstract: Deep learning models have been increasingly applied to Time Series Forecasting (TSF) in recent years. Transformer-based and MLP-based models have both b

KODAMA: Multimodal Digital Twin Reconstruction for Urban RF Propagation Modelling

ApplicationsDGX agent

arXiv:2609.07298v1 Announce Type: new Abstract: 3D reconstruction typically strives for geometric fidelity or visual plausibility. Radio frequency digital twins (RFDT) are instead judged by whether co

← Previous
1…4445464748…169
Next →