Search: “multimodal”

arXiv:2608.10016v1 Announce Type: cross Abstract: Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing mo

local-aiarxiv-cs-ai

12 Aug 2026

Local Ai

Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.08794v1 Announce Type: new Abstract: Omni-modal LLMs jointly process audio, video, and text, but long multimodal sequences incur substantial prefill and KV-cache costs. Existing omni-modal

local-aiarxiv-cs-ai

11 Aug 2026

EgoBrain: Synergizing Minds and Eyes For Human Action Understanding

arXiv:2506.01353v3 Announce Type: replace Abstract: The integration of brain-computer interfaces (BCIs), in particular electroencephalography (EEG), with artificial intelligence (AI) has shown tremend

11 Aug 2026

Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

arXiv:2608.09931v1 Announce Type: new Abstract: Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Dist

11 Aug 2026

An AI4AI Framework for Visual Token Pruning

arXiv:2608.07193v1 Announce Type: cross Abstract: Visual-token pruning can substantially reduce the inference cost of multimodal large language models (MLLMs), yet existing methods largely rely on fix

10 Aug 2026

Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models

arXiv:2608.06901v1 Announce Type: new Abstract: Vision-language models (VLMs) have achieved remarkable generalization across diverse multimodal tasks through large-scale pre-training, yet their rapidl

10 Aug 2026

Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery

arXiv:2608.06931v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory

10 Aug 2026

C^3PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Models

arXiv:2608.05381v1 Announce Type: new Abstract: Current Multimodal Large Language Models (MLLMs) can process diverse sensory inputs, yet their reasoning remains heavily biased toward a dominant modali

7 Aug 2026

ChronoVision: Temporal Reasoning via Latent State Reconstruction

arXiv:2608.05631v1 Announce Type: new Abstract: Multimodal large language models excel at passive perception but struggle with complex visual cognitive tasks requiring multi-step temporal reasoning. T

7 Aug 2026

A Model Merging Approach for Continual MLLM Unlearning

arXiv:2608.04548v1 Announce Type: cross Abstract: Multimodal large language model (MLLM) unlearning methods have been proposed to remove private, sensitive, or proprietary information from well-traine

GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction

arXiv:2608.04504v1 Announce Type: cross Abstract: Vision-language models excel in many multimodal tasks but remain prone to a subtle yet impactful failure mode: they tend to overestimate dominant visu

nvidias nemotron omni only loads its text half on a mac, so i wrote the vision and audio towers in mlx

model-releasesr-localllama

nvidias nemotron omni is open weights and it sees, hears and reasons. theres already a 4bit mlx quant on hugging face but only the text backbone loads with standard mlx tooling. the model card says it

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

arXiv:2608.05131v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (ML

Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs

arXiv:2608.03450v1 Announce Type: cross Abstract: Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based

5 Aug 2026

Applications

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

arXiv:2608.02833v1 Announce Type: cross Abstract: Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current

applicationsarxiv-cs-ai

5 Aug 2026

DiffImaginE: Imagine to Verify Entity Types with Diffusio

arXiv:2608.03025v1 Announce Type: new Abstract: Multimodal named entity recognition (MNER) determines whether each candidate span and entity-type hypothesis is supported by joint textual and visual ev

tutorialsarxiv-cs-ai

5 Aug 2026

ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs

model-releasesarxiv-cs-cl

arXiv:2608.01979v1 Announce Type: cross Abstract: Visual token pruning reduces the inference cost of multimodal large language models, but a fixed token ratio is poorly matched to text-rich inputs. In

Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving

arXiv:2608.00237v1 Announce Type: new Abstract: Vision-language models (VLMs) have recently emerged as a promising paradigm for end-to-end autonomous driving, enabling agents to map multimodal inputs

MIEScore: Human-Aligned Evaluation for Multi-Source Image Editing

arXiv:2608.02059v1 Announce Type: new Abstract: Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-

PixVL: Self-Supervised Training of Pixel-Level MLLMs via a Unified Mask--Text Consistency Cycle

arXiv:2608.01354v1 Announce Type: new Abstract: Recent studies develop pixel-level multimodal large language models (MLLMs) that support both Region Segmentation and Region Understanding, extending mu

tutorialsarxiv-cs-cv

Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning

arXiv:2608.01314v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) increasingly rely on long chain-of-thought reasoning for complex tasks. However, as reasoning sequences lengthe

Agents

AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis

arXiv:2607.15755v2 Announce Type: replace-cross Abstract: Conversational Speech Synthesis (CSS) aims to synthesize speech with human-like emotional expression and contextual consistency in user-agent

agentsarxiv-cs-ai

3 Aug 2026

Local Ai

SafeNexus: Discovering and Steering Modality-Universal Safety Neurons in MLLMs

arXiv:2607.28969v1 Announce Type: new Abstract: Although Large Language Models (LLMs) have demonstrated promising safety performance, extending them to Multimodal Large Language Models (MLLMs) exposes

local-aiarxiv-cs-cv

3 Aug 2026

ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding

arXiv:2607.28678v1 Announce Type: new Abstract: Multimodal agents operating in long-horizon environments must build and continually update multimedia memories to support entity-consistent, temporally

3 Aug 2026

Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs

arXiv:2607.27700v1 Announce Type: new Abstract: Large Vision-Language Models (VLMs) suffer from prohibitive inference overhead due to long sequences of visual tokens. However, existing visual token re

Large scale cross-regional remote sensing flood monitoring framework for operative mapping and impact analysis

arXiv:2607.28401v1 Announce Type: new Abstract: Effective flood monitoring is critical for minimizing the impacts of flood disasters on populations and infrastructure. Yet reliable remote sensing acro

Minimax-H3 video model released, open weights coming in the next few days

model-releasesr-localllama

https://x.com/MiniMax_AI/status/2083006198828417501?s=20 Quote from their article: Today, we're launching MiniMax H3, a general-purpose multimodal generation model. H3 understands unified context acro

MMHBench: A Multi-Perspective Benchmark for Mental Health Understanding in Long-Form Videos

arXiv:2607.27895v1 Announce Type: cross Abstract: Mental health understanding in long-form videos requires nuanced reasoning over observable behavior, interpersonal context, and latent psychological s

Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models

arXiv:2607.27304v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) generate chain-of-thought (CoT) reasoning before answering clinical questions, but whether this reasoning causal

Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA

arXiv:2607.27830v1 Announce Type: new Abstract: High-resolution visual question answering (HR-VQA) is often treated as a problem of insufficient evidence acquisition, where failing multimodal large la

Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective

arXiv:2607.27660v1 Announce Type: cross Abstract: Submodular Information Measures (SIMs) have recently emerged as a powerful framework for representation learning and multimodal learning. In particula

Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation

arXiv:2607.25489v1 Announce Type: new Abstract: Large language models and multimodal foundation models are enabling medical artificial intelligence (AI) systems to move beyond isolated prediction and

FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding

arXiv:2607.25266v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have enabled long-form video understanding at a scale that was not previously possible. However, the density of

MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

arXiv:2607.25948v1 Announce Type: cross Abstract: Any-to-any models predict any modality from any combination of others within a single network, a formulation used in multimodal vision and vision-lang

Shieldstral

model-releasesarxiv-cs-cl

arXiv:2607.25857v1 Announce Type: new Abstract: We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7imes its size on text s

Agents

VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents

arXiv:2607.24748v1 Announce Type: cross Abstract: Visually-rich documents such as reports, slides, and manuals often distribute the evidence needed to answer a question across multiple pages, mixing t

agentsarxiv-cs-ai

BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion

arXiv:2607.24110v1 Announce Type: new Abstract: Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cros

tutorialsarxiv-cs-cv

Controlling Embedding Spaces with Text-Conditioned Transformations

arXiv:2607.22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classific

tutorialsarxiv-cs-ai

Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement

arXiv:2607.22770v1 Announce Type: cross Abstract: Although artificial intelligence (AI) has shown promising performance in several medical tasks, accurate dementia etiology diagnosis with AI remains c

Disentangling Semantic Attention from Structural Bias in the Attention Manifold

arXiv:2607.24017v1 Announce Type: cross Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws. Specifically, MLLM

Looking for Affect in Spontaneous Finnish Speech through Linguistic Interpretability

model-releasesarxiv-cs-cl

arXiv:2607.24155v1 Announce Type: new Abstract: Existing research on affect in speech has shown how acoustic surface characteristics and content-related linguistic aspects of speech both relate to per

MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models

arXiv:2607.22566v1 Announce Type: new Abstract: MedLoCoMo is a Medical Long-Context Memory benchmark for patient-specific clinical reasoning over multi-admission medical dialogue. Existing medical QA

Hardware

Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models

arXiv:2607.23445v1 Announce Type: cross Abstract: Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting au

hardwarearxiv-cs-cl

Revitalizing Public Urban Places through Cultural and Political Memory: A Technological Approach with LLMs and Augmented Reality

arXiv:2607.22613v1 Announce Type: cross Abstract: This paper explores the intersection of memory, place, and identity, examining how new technologies, particularly Apple Vision Pro, can illuminate thi

swiss-ai/Apertus-v1.5 70B/8B

model-releasesr-localllama

https://huggingface.co/swiss-ai/Apertus-v1.5-70B https://huggingface.co/swiss-ai/Apertus-v1.5-8B Apertus 1.5 is a family of 8B and 70B parameter language models designed to advance the state of multil

24 Jul 2026

Continual Video-MLLM Adaptation over Evolving Domains

arXiv:2607.18716v1 Announce Type: new Abstract: Video multimodal large language models have shown strong capability in video understanding, yet their adaptation to sequentially evolving domains remain

23 Jul 2026

DRGBT-1K: A Large-scale High-quality Benchmark for Dynamic RGBT Tracking

arXiv:2607.19772v1 Announce Type: new Abstract: Dynamic RGBT (DRGBT) tracking aims to continuously localize a target when the available sensing modalities and observation platforms vary over time. Com

23 Jul 2026

Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions

arXiv:2607.19061v2 Announce Type: replace-cross Abstract: Hateful optical illusions expose a serious gap in current multimodal safety systems. On original-view hateful illusions, previous work shows t