Search: “multimodal”

LUT: Latent Utility Training for Visual Reasoning

arXiv:2608.00743v1 Announce Type: new Abstract: Multimodal large language models have advanced visual understanding, yet perception-intensive reasoning remains challenging. Recent latent visual reason

Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.01635v1 Announce Type: new Abstract: Despite the progress of multimodal large language models (MLLMs), they continue to exhibit deficiencies in visual perception. Following visual instructi

safetyarxiv-cs-cv

4 Aug 2026

Quick on the Uptake: Eliciting Implicit Intents from Human Demonstrations for Personalized Mobile-Use Agents

arXiv:2508.08645v3 Announce Type: replace Abstract: As multimodal large language models advance rapidly, the automation of mobile tasks has become increasingly feasible through the use of mobile-use a

safetyarxiv-cs-cl

Applications

ReMiX-MAE: Learning Missing-Channel Cross-Modal Representations from RGB-Only Clinical Facial Videos for Sympathetic-Mediated Pain Assessment

arXiv:2608.02561v1 Announce Type: new Abstract: Automated pain assessment in real clinics is limited by scarce clinically grounded facial video data with weak labels (often sequence-level self-report)

applicationsarxiv-cs-cv

Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs

arXiv:2608.01473v1 Announce Type: cross Abstract: Multimodal large language models (MLLM) for surgical scene understanding typically inject hundreds of dense visual tokens into a language model, leadi

safetyarxiv-cs-cl

Two Sides of the Same Coin: Co-Evolving Search for Cross-Task Attacks on Vision-Language Models

arXiv:2608.02137v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit strong generalization across multimodal tasks but remain vulnerable to adversarial perturbations. Existing attacks

researcharxiv-cs-cv

UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction

arXiv:2608.01298v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.

Tutorials

Unleashing the Power of Text: Text-Guided Flow Matching for Image Fusion under Complex Degradations

arXiv:2608.00530v1 Announce Type: new Abstract: Infrared-visible image fusion under realistic degradation scenarios is a challenging task, as degradations not only cause a loss of reliable modality-sp

tutorialsarxiv-cs-cv

Agents

VC-Tooler: Learning Compositional and Adaptive Visual Tool Use

arXiv:2608.02217v1 Announce Type: new Abstract: Agentic multimodal reasoning extends passive image understanding by allowing VLMs to actively acquire and refine visual evidence through visual tool int

agentsarxiv-cs-cv

What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs

model-releasesarxiv-cs-cl

arXiv:2608.00013v1 Announce Type: new Abstract: Choosing the right large language model (LLM) backbone is the most consequential decision when building a vision-language model (VLM), yet it remains fu

Why are Chinese models better* at Frontend than the western top labs?

model-releasesr-localllama

I use A LOT both openAI and Anthropic products. When I need some frontend work (pure web dev) (or answer that feel less verbose and more to the point) I use Anthropic. For multimodality openAI feels b

Differentially Private Nonparametric Modal Learning with Applications to Regression and Clustering

model-releasesarxiv-cs-lg

arXiv:2607.29675v1 Announce Type: cross Abstract: Density modes provide a localized and interpretable summary of multimodal distributions, but their estimation under rigorous differential privacy cons

Evidence-Grounded Constraint Checking in Construction Documents

arXiv:2607.29058v1 Announce Type: new Abstract: Professional-document review is a constraint-checking problem in which decisions depend on relations among text, geometry, pages, and document revisions

researcharxiv-cs-ai

Applications

FibVLA: An Efficient Temporal Vision-Language-Action Model with Fibonacci Sampling

arXiv:2607.29596v1 Announce Type: cross Abstract: Vision-language-action models (VLAs), which leverage the cognition of multimodal information to infer physical-world actions, provide a generalized so

applicationsarxiv-cs-cv

Is It Time for the Renaissance of Salient Object Detection in the Era of MLLMs?

arXiv:2607.29222v1 Announce Type: new Abstract: The zero-shot capabilities of multimodal large language models (MLLMs) are pushing salient object detection (SOD) beyond task-specific supervision. To d

SciFigPlag-Bench: A Benchmark for Provenance-Aware Scientific Figure Plagiarism Detection

arXiv:2607.29124v1 Announce Type: new Abstract: Scientific figures often encode the visual evidence behind scientific findings, yet figure plagiarism remains underexplored as a benchmarked multimodal

SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery

arXiv:2607.29347v1 Announce Type: cross Abstract: Modern neuroscience relies on integrating multi-scale, multimodal datasets to uncover the neural principles underlying intelligence. However, analytic

Technological Advances in Detecting and Managing Cognitive Impairment in Older Adults: Trends, Challenges, and Future Directions

arXiv:2607.28687v1 Announce Type: cross Abstract: As populations age, cognitive decline from mild cognitive impairment (MCI) to dementia is a defining health challenge of the coming decades, yet routi

researcharxiv-cs-ai

Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers

arXiv:2607.28611v1 Announce Type: new Abstract: Visual generation increasingly requires high-resolution images, long videos, and multimodal context, making the quadratic cost of full attention prohibi

Divergence Decoding: Training-Free Capability Fusion

model-releasesarxiv-cs-lg

arXiv:2607.27248v1 Announce Type: cross Abstract: While large language models excel in reasoning, these generalists often lack knowledge for specialized scientific domains. Conversely, domain models~(

LoMeVQA: A Comprehensive Benchmark for Longitudinal Medical VQA

arXiv:2607.27806v1 Announce Type: new Abstract: In clinical practice, patients often undergo multiple imaging examinations over successive visits, yielding longitudinal data. Modeling such temporal in

PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology

arXiv:2607.27378v1 Announce Type: new Abstract: Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-g

Scaling Vision-Language Models Is Not Enough to Mitigate Bias

arXiv:2607.28211v1 Announce Type: new Abstract: Vision-Language Models (VLMs) such as CLIP are now foundational to multimodal systems, yet their robustness to spurious correlations remains poorly unde

Agents

Together AI gives developers a high-throughput production path for running Inkling-Small on @NVIDIAAI Accelerated Infrastructure across mult…

Together AI gives developers a high-throughput production path for running Inkling-Small on @NVIDIAAI Accelerated Infrastructure across multimodal, coding, and agentic workloads. Start building: https

agentstogether-ai--x

Hardware

Together AI gives developers a high-throughput production path for running Inkling-Small on NVIDIA Accelerated Infrastructure across multimo…

Together AI gives developers a high-throughput production path for running Inkling-Small on NVIDIA Accelerated Infrastructure across multimodal, coding, and agentic workloads. Start building: https://

hardwaretogether-ai--x

SCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation

arXiv:2607.26885v1 Announce Type: new Abstract: Vision-language pre-training (VLP) serves as a cornerstone for medical multimodal representation learning. However, existing medical VLP frameworks are

SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions

arXiv:2603.23118v2 Announce Type: replace Abstract: Recent works have shown that multimodal large language models (MLLMs) are highly vulnerable to hidden-pattern visual illusions, where the hidden con

SpatialQ: Understanding 3D Gaussian Splatting Scene Quality via Visual-based MLLM

arXiv:2607.26595v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) has emerged as an effective representation for novel view synthesis and 3D scene reconstruction, creating an increasing dem

Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation

model-releasesarxiv-cs-cl

arXiv:2607.26555v1 Announce Type: new Abstract: Multimodal fake news detectors often generalize poorly across domains because they learn to trust unreliable evidence: domain-specific shortcuts amplifi

Chart-Supported or Model-Supplied? Examining MLLM-Generated Claims for Accessible Visualization

arXiv:2607.25021v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can connect visualization patterns to external causes, consequences, and domain knowledge, but the evidential b

Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding

arXiv:2607.24794v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their lo

safetyarxiv-cs-ai

Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs

model-releasesarxiv-cs-lg

arXiv:2607.25196v1 Announce Type: new Abstract: Contrastive decoding (CD) has been proposed as a training-free strategy for mitigating object hallucinations in multimodal large language models (MLLMs)

Seen, Said, or Forgotten? A Causal Audit of Visual KV Memory Across Dialog Turns

arXiv:2607.25467v1 Announce Type: cross Abstract: Stateful multimodal assistants encode an image once but may answer questions about it many turns later. Attention-guided visual-KV eviction assumes th

researcharxiv-cs-ai

A Scale-adaptive Vision Model Links C. elegans Neuronal Morphology to Behavior for Neurotoxicity Assessment

arXiv:2607.23183v1 Announce Type: cross Abstract: Neurological disorders are a leading cause of global disability and are increasingly linked to environmental chemical exposures. Yet neurotoxicity ass

Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents

arXiv:2607.22689v1 Announce Type: new Abstract: Graphical user interface (GUI) agents are systems powered by large multimodal models (LMMs). They perceive screen state and execute user instructions th

ConFusion: Continuous Fusion Space Learning for Fine-Grained Controllable Infrared and Visible Image Fusion

arXiv:2607.23600v1 Announce Type: new Abstract: Controllable infrared-visible image fusion aims to integrate complementary thermal and structural information with flexible region-aware modulation, pro

Data Pyramid for Embodied Manipulation

arXiv:2607.24744v1 Announce Type: cross Abstract: Multimodal foundation models learned to see and to speak by consuming the whole internet. Embodied agents admit no such shortcut, since they require d

Embodied GPT-5.1: Evidence of a World Model?

arXiv:2607.23899v1 Announce Type: cross Abstract: This exploratory study examines whether a large multimodal language model, GPT-5.1, can serve as the high-level controller of a physical mobile robot

Infrared Imaging Empowered by Artificial Intelligence for Pediatric Skeletal Triage: A Narrative Review and Future Perspectives

arXiv:2607.24727v1 Announce Type: new Abstract: Background. Pediatric musculoskeletal trauma represents up to 18% of pediatric ED visits, yet diagnosis still depends on ionizing radiography. Cumulativ

Hardware

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

arXiv:2607.23193v1 Announce Type: new Abstract: Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show

hardwarearxiv-cs-cv

Agents

Scaling GUI Agents with Visual State Transitions

arXiv:2607.24112v1 Announce Type: new Abstract: We introduce State Transition Pretraining (STP) as a new scaling axis for GUI agents. During the STP stage, we continually pretrain a unified multimodal

agentsarxiv-cs-ai

Hardware

Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests

arXiv:2607.22864v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) excel at visual interpretation but fail on spatial reasoning tasks that humans solve reliably. Existing bench

hardwarearxiv-cs-ai

Local Ai

UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models

arXiv:2607.23373v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) remain bottlenecked by massive computational footprints, precluding their deployment on resource-constrained edge d

local-aiarxiv-cs-cv

UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling

arXiv:2607.24157v1 Announce Type: new Abstract: Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled

researcharxiv-cs-cv

Adversarial Style Optimization: Enhancing VLM Jailbreaks by GRPO-based Stylistic Triggers Optimization

arXiv:2607.21619v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive performance, but their safety alignment remains vulnerable to jailbreak attacks. Exist

safetyarxiv-cs-cl

27 Jul 2026

Local Ai

FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs

arXiv:2607.21624v1 Announce Type: cross Abstract: Transformer-based models have enabled unprecedented capabilities across language, vision, and multimodal tasks. On-device fine-tuning of transformer m

local-aiarxiv-cs-lg

27 Jul 2026

GRACE: Gradient-Free Robot Action Generation via Combined Diffusion-MPPI Posterior Mean Estimation

arXiv:2607.21661v1 Announce Type: new Abstract: Diffusion policies generate multimodal robot action sequences from demonstrations, but steering them toward deployment-time constraints typically relies

safetyarxiv-cs-ro

27 Jul 2026

Tutorials

C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs