Search: “multimodal”

Stabilizing the Modality Gap: Null-Space Projected Continual Learning for Vision-Language Models

arXiv:2507.19839v2 Announce Type: replace-cross Abstract: The modality gap is intrinsic to multimodal alignment, where embeddings from different modalities remain partially separated in a shared repre

16 Oct 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

@AIatMeta Together Serverless Inference gives developers a high-throughput, managed production path for running Muse Spark Open across agent…

DGX agent

@AIatMeta Together Serverless Inference gives developers a high-throughput, managed production path for running Muse Spark Open across agentic coding and multimodal workloads. Start building: https://

agentstogether-ai--x

15 Oct 2026

Are Benchmarks Reliable? Toward Structural Diagnosis via Item-Level Capability Boundaries

arXiv:2609.33196v2 Announce Type: replace Abstract: Evaluating large language models (LLMs) relies heavily on benchmark scores, yet aggregate metrics can obscure whether individual items serve their i

CORE: Contrast-and-Reflect for Self-Supervised Prompt Optimization in Anomaly Synthesis

arXiv:2610.19610v1 Announce Type: new Abstract: Industrial anomaly detection (AD) is bottlenecked by the scarcity of anomalous samples, making high-fidelity anomaly synthesis crucial for improving dow

DarKnight: Retrieval-Guided Agentic Framework for Breaking Dark Web CAPTCHA Gates

arXiv:2610.19830v1 Announce Type: cross Abstract: Cyber Threat Intelligence (CTI) investigators who monitor dark web marketplaces for leaked credentials and breach listings must pass the diverse CAPTC

Game-Creation-Agents: Autonomous Game Production

arXiv:2610.16776v1 Announce Type: new Abstract: Game-Creation-Agents (GCA) turns a natural-language request into a runnable Phaser 4 browser game. The pipeline first builds a multimodal game design do

HiPPT: Physiology-Grounded Wearable PPG Representation Learning via Multi-Scope Text Alignment

arXiv:2610.14489v1 Announce Type: new Abstract: Reusable wearable PPG representations should capture local pulse morphology and broader physiological regularities. Existing pretraining objectives, how

nvidia/Nemotron-3.5-Embed-1B-BF16 · Hugging Face

model-releasesr-localllama

Nemotron-3.5-Embed-1B-BF16 is a versatile embedding model developed by NVIDIA for multimodal and multilingual question-answering retrieval. The model can embed a document page in the form of an image,

Repairing Speech Access After Text Updates with Frozen Affine Fields

arXiv:2610.20253v1 Announce Type: new Abstract: Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated

researcharxiv-cs-cl

Agents

ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoning

arXiv:2610.16315v1 Announce Type: new Abstract: Test-time agentic computation can improve multimodal large language models (MLLMs) through planning, tool use, iterative perception, and reflection, but

agentsarxiv-cs-ai

Agents

Self-Evolving Visual Agent via Margin Reward

arXiv:2610.15084v1 Announce Type: new Abstract: Visual agents that interleave reasoning with tool invocation have posed a promising direction for multimodal tasks. However, learning effective tool inv

agentsarxiv-cs-ai

Task-evoked eye movements as candidate digital biomarkers of depression severity in adolescents and young adults

arXiv:2610.18494v1 Announce Type: cross Abstract: Objective digital measures of depression severity could complement conventional symptom assessments and facilitate more timely clinical evaluation. He

v0.7.0

model-releasesllama-cpp-releases

Overview llama.cpp v0.7.0 adds the LiliCorr speculative decoding technique and MTP for GLM5-Next, a new GPU cache for MoE experts kept in host memory (--moe-cache-mib, up to ~2.2x decode speedup when

VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement

arXiv:2610.19361v1 Announce Type: cross Abstract: Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target th

safetyarxiv-cs-cl

Bottlenecked semantic compression by representation reconstruction

arXiv:2610.17412v1 Announce Type: new Abstract: Vision Transformers incur high computational costs when processing long visual token sequences. Existing reduction methods typically rank tokens indepen

Conditional Multi-Event Temporal Grounding in Long-Form Video

arXiv:2606.15320v2 Announce Type: replace Abstract: Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing ever

Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning

model-releasesarxiv-cs-lg

arXiv:2610.14605v1 Announce Type: cross Abstract: Current evaluations of multimodal unlearning typically focus on whether a model stops outputting target information. This surface-level criterion, how

EviScene: Beyond 'Is It AI?' for Agentic Image Factuality Assessment

arXiv:2610.14609v1 Announce Type: new Abstract: Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality A

Hardware

FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention

arXiv:2610.17901v1 Announce Type: new Abstract: Long-context training of large language and multimodal models couples structured attention masks (document, prefix-LM, share-question, sliding-window, b

hardwarearxiv-cs-lg

HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoning

arXiv:2610.14679v1 Announce Type: new Abstract: We introduce HumanPhys, a human-centric benchmark for real-world physical perception, and a training corpus emitted by the same pipeline. Each question

Omni-Modal LLMs Know More Than They Say

arXiv:2610.16135v1 Announce Type: cross Abstract: Omni-modal large language models (Omni-LLMs) extend LLMs to process heterogeneous modalities (e.g., vision, audio, and IMU signals) alongside language

safetyarxiv-cs-cv

ReTrace: Cross-Stage Failure Attribution for Self-Improving Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2610.15814v1 Announce Type: new Abstract: General-purpose multimodal foundation models are increasingly capable of directly controlling robots, yet improving their manipulation capabilities thro

SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning

arXiv:2610.14439v1 Announce Type: new Abstract: Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over tim

Sharper Frames, Worse Rooms: Viewpoint Coverage Dominates Image Quality in Feed-Forward 3D Reconstruction

arXiv:2610.15854v1 Announce Type: new Abstract: Grounded spatial reasoning about indoor scenes rests on a geometric reconstruction that the multimodal system does not itself produce. Feed-forward mult

SurgiCLAW: An Interpretable Atomic-Skill Agent for Precision Robotic Micromanipulation

arXiv:2610.15766v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) and vision--language--action models have provided robots with new capabilities for long-horizon task und

safetyarxiv-cs-ro

The Design Space of Tri-Modal Masked Diffusion Models

researchapple-ml-research

Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent work initializing and finetuning a base unimodal model for bi-modal generation. Diverging f

UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models

arXiv:2610.16945v1 Announce Type: new Abstract: Recent advances in multimodal foundation models, exemplified by GPT-6 Astra, raise the prospect of general-purpose embodied reasoning and planning. Usin

safetyarxiv-cs-cv

What Makes a Good Visual Representation for Spatial Understanding in MLLMs?

arXiv:2610.17844v1 Announce Type: new Abstract: Spatial understanding is a necessary step in moving multimodal large language models (MLLMs) toward reasoning about the 3D physical world. Their visual

Local Ai

What if Spotlight actually understood the meaning inside your files? DigUp is an open-source macOS menu bar app powered by EmbeddingGemma 2 …

local-aigeorgi-gerganov--x

What if Spotlight actually understood the meaning inside your files? DigUp is an open-source macOS menu bar app powered by EmbeddingGemma 2 that brings 100% local, multimodal semantic search to your M

13 Oct 2026

CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learning

arXiv:2603.05911v2 Announce Type: replace-cross Abstract: Medical image segmentation is undergoing a paradigm shift from conventional visual pattern matching to cognitive reasoning analysis. Although

Energy-Efficient Spiking Neural Networks for Sensing-Aided Beam Prediction

arXiv:2610.12507v1 Announce Type: cross Abstract: Sensing-aided beam prediction can reduce the beam-training overhead of millimeter-wave (mmWave) systems by leveraging environmental information from c

researcharxiv-cs-ai

Local Ai

Explore DNA with DeepMind’s AlphaGenome and in-browser MedGemma 1.5 4B via WebGPU

Hey everyone, Sharing an educational tool called AskCyph Health Explorer to explore how single-letter DNA mutations lead to disease, how they alter 3D anatomy, inheritance probabilities, and available

local-air-localllama

Representation Alignment from a Few Paired Data via Distilled Optimal Transport

arXiv:2610.12781v1 Announce Type: new Abstract: Aligning two independently pretrained unimodal encoders typically demands millions of paired examples. In this paper, we study the regime where only N s

Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

arXiv:2610.12747v1 Announce Type: cross Abstract: Ego-trajectory prediction in urban roadway work zones remains an underexplored topic in autonomous-driving research, with limited dedicated data and s

UniConFlow: A Unified Constrained Flow-Matching Framework for Certified Motion Planning

arXiv:2506.02955v3 Announce Type: replace-cross Abstract: Generative models have become increasingly powerful tools for robot motion generation, enabling flexible and multimodal trajectory generation

Benchmark Designers Should 'Train on the Test Set' to Expose Exploitable Non-Visual Shortcuts

arXiv:2511.04655v2 Announce Type: replace Abstract: Multimodal LLMs can answer many questions in vision-centric benchmarks without looking at the image, using linguistic priors and skewed answer distr

Local Ai

DVD: Dynamic Vector Decoding for Efficient MLLM-based Perception

arXiv:2610.12266v1 Announce Type: cross Abstract: Multimodal large language models have made remarkable progress in bridging vision and language, facilitating various perception tasks essential for hu

local-aiarxiv-cs-ai

FastJEV: Understanding Redundancy for Compact JEV Inference

arXiv:2610.11379v1 Announce Type: new Abstract: JEV models make multimodal decisions by directly scoring candidates. Although the common context is encoded once, candidate evaluation can still repeat

FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding

arXiv:2602.01683v2 Announce Type: replace-cross Abstract: Transitioning Multimodal Large Language Models (MLLMs) from offline to online streaming video understanding is essential for continuous percep

researcharxiv-cs-ai

From What to Which: Decoding Modifier Grounding in Frozen MLLMs

arXiv:2610.12305v1 Announce Type: new Abstract: As Multimodal Large Language Models (MLLMs) can describe increasingly complex visual scenes, token-level grounding becomes crucial. Yet, when an MLLM ge

GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

arXiv:2610.11129v1 Announce Type: new Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated G

Tutorials

Mid-Training Language Models on Raw Video

arXiv:2610.11019v1 Announce Type: cross Abstract: Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an exi

tutorialsarxiv-cs-ai

Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

arXiv:2610.11419v1 Announce Type: cross Abstract: Multimodal brain tumor segmentation typically leverages multiple MRI modalities, yet incomplete modality acquisition is common in clinical practice du

researcharxiv-cs-ai

OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning

arXiv:2610.12458v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that ex

Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models

arXiv:2610.12090v1 Announce Type: new Abstract: Latent reasoning enables vision-language-action (VLA) models to transform multimodal observations into task-relevant internal states before generating c

SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces

arXiv:2610.11366v1 Announce Type: new Abstract: Spatial coding agents significantly improve spatial reasoning in Multimodal Large Language Models (MLLMs) by using external tools to generate verified e

safetyarxiv-cs-lg

Agents

CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion

arXiv:2610.09330v1 Announce Type: new Abstract: Infrared and visible (IR-VIS) image fusion integrates complementary multimodal information into a single fused image to support downstream vision tasks.

agentsarxiv-cs-cv

8 Oct 2026

Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data