Search: “multimodal”

CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA

arXiv:2607.21155v1 Announce Type: cross Abstract: Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiri

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DINO-VPT: Hierarchical Visual Prompt Tuning for Joint Physical-Digital Face Anti-Spoofing

DGX agent

arXiv:2607.20900v1 Announce Type: new Abstract: With the increasing diversity of spoofing attacks, there is a growing demand for unified Face Anti-Spoofing (FAS) models capable of detecting both physi

model-releasesarxiv-cs-cv

24 Jul 2026

MIRROR: Learning from the Other View for Multi-Modal Reasoning

arXiv:2607.21552v1 Announce Type: new Abstract: Unlike large language models (LLMs) that exhibit strong reasoning capabilities, vision-language models (VLMs) struggle with visual reasoning, even on ge

UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging

arXiv:2607.21546v1 Announce Type: new Abstract: Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary informatio

ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?

arXiv:2607.20868v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse expert-level tasks, but they still struggle with fundamental ab

WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidance

arXiv:2511.12997v2 Announce Type: replace Abstract: Multimodal LLM-powered agents have recently demonstrated impressive capabilities in web navigation, enabling agents to complete complex browsing tas

Wireless TokenCom: RL-Based Tokenizer Agreement for Multi-User Wireless Token Communications

arXiv:2602.12338v2 Announce Type: replace Abstract: Token Communications (TokenCom) has recently emerged as an effective new paradigm, where tokens are the unified units of multimodal communications a

safetyarxiv-cs-lg

A Deep Learning Framework for Predicting Solar EUV Irradiance During Significant Flares

arXiv:2607.19597v1 Announce Type: cross Abstract: We present FlareEUV, a multimodal deep learning framework for predicting daily extreme ultraviolet (EUV) irradiance at 6.5 nm over three consecutive d

researcharxiv-cs-lg

ChronoStitch: Training-Free Composition of Visual KV Memories for Long-Horizon Temporal Reasoning

arXiv:2607.19547v1 Announce Type: new Abstract: Long-video question answering requires a model to preserve visual evidence over time without repeatedly reprocessing the same video. A practical approac

researcharxiv-cs-cv

Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection

arXiv:2512.16300v3 Announce Type: replace Abstract: Existing image forgery detection (IFD) methods either exploit low-level, semantics-agnostic artifacts or rely on multimodal large language models (M

D3VL: Understanding Driving Scenes from 3D Time Series Data and Video with Language Models

arXiv:2607.19528v1 Announce Type: cross Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have triggered the development of end-to-end MLLMs for autonomous driving. However, the ma

ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis

arXiv:2607.19341v1 Announce Type: new Abstract: Recent advances in multimodal generative models have enabled instruction-based image generation to move beyond semantic manipulation to knowledge-driven

Pathologist Attention-Aligned Report Generation for Prostate Histopathology

arXiv:2607.19624v1 Announce Type: new Abstract: The allocation of visual attention by pathologists during cancer diagnosis is a highly selective process that critically shapes the information extracte

PathReportEval: A Systematic Benchmark for Pathology Report Generation

arXiv:2607.18448v1 Announce Type: cross Abstract: Pathology report generation from whole-slide images (WSIs) is a rapidly growing multimodal learning problem, yet progress is difficult to measure beca

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

arXiv:2607.18763v1 Announce Type: new Abstract: Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete multimodal a

applicationsarxiv-cs-cv

Ropedia raises $22M to scale human-centric data collection for embodied AI

Singapore-based robotics data infrastructure firm Ropedia Pte. Ltd. today announced it raised 22 million in Pre-Series A funding to scale up its collection of real-world, multimodal interaction data t

applicationssiliconangle

Style over Substance: A Shortcut Audit of Emotion-Description Preference Evaluation

arXiv:2607.18508v1 Announce Type: new Abstract: Preference over model-generated emotion descriptions is emerging as a standard evaluation metric for multimodal emotion understanding, exemplified by th

Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning

arXiv:2607.19790v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved language-model reasoning, yet its extension to vision-language models r

researcharxiv-cs-cv

microsoft/Fara1.5-27B · Hugging Face

model-releasesr-localllama

Fara1.5-27B is a multimodal computer use agent (CUA) for web browsers, from Microsoft Research AI Frontiers. It observes the browser through screenshots and acts on the user's behalf by emitting struc

22 Jul 2026

This is a neat feature. I wrote an article a few weeks back about how I built this into my agent orchestrator: https://x.com/omarsar0/status…

This is a neat feature. I wrote an article a few weeks back about how I built this into my agent orchestrator: https://x.com/omarsar0/status/2073404610501329247?s=20 But I made it multimodal from the

model-releasesdair-ai--x

21 Jul 2026

Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Models

arXiv:2607.13500v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have achieved strong performance in multimodal understanding, yet remain challenging to deploy on resource-constrained edg

applicationsarxiv-cs-cv

Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents

arXiv:2511.18685v4 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) show promising results as decision-making engines for embodied agents operating in complex, physical enviro

Local Ai

GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding

arXiv:2607.13454v1 Announce Type: cross Abstract: Although multimodal large language models (MLLMs) have achieved remarkable progress, understanding 3D spatial relationships from 2D images remains a c

local-aiarxiv-cs-ai

Agents

Grounded world models in biological organisms and future embodied AI

arXiv:2607.13560v1 Announce Type: cross Abstract: Recent advances in generative and embodied AI have been driven by large-scale predictive learning over multimodal data. However, the resulting systems

agentsarxiv-cs-ai

Industrial Dexterity Benchmark: A Hardware-Software Benchmarking Platform for Industrial Dexterous Manipulation

model-releasesarxiv-cs-ro

arXiv:2607.14021v1 Announce Type: new Abstract: Dexterous manipulation remains a critical bottleneck in industrial automation; tasks such as cable routing, connector insertion, and precision assembly

Mind the Gap: Action Rebinding Attacks against Android GUI Agents

arXiv:2601.12349v3 Announce Type: replace-cross Abstract: Large multimodal model powered GUI agents are emerging as high-privilege operators on mobile platforms, entrusted to perceive screen content a

Causal Supervision of Attention for Affective Behaviour Analysis

arXiv:2607.12091v1 Announce Type: new Abstract: Affective Behaviour Analysis aims to enable machines to infer human affective states from behavioural signals, particularly facial expressions, in real-

applicationsarxiv-cs-cv

Hardware

Globalized Constrained Stein Variational Inference for Diverse Feasible Robot Motion Planning

arXiv:2607.12732v1 Announce Type: new Abstract: Robot motion planning is inherently multimodal, yet classical planners typically return only a single solution. Probabilistic formulations address this

hardwarearxiv-cs-ro

IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment

arXiv:2607.12375v1 Announce Type: cross Abstract: Image Quality Assessment (IQA) in open-world environments remains challenging due to limited generalization and interpretability. Recent approaches ba

tencent/Hy-Embodied-RxBrain-1.0 · Hugging Face

model-releasesr-localllama

Introduction RxBrain (Hy-Embodied-RxBrain-1.0) is a unified multimodal foundation model for embodied cognition — a single model that couples language reasoning with visual imagination to deliver three

Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings

arXiv:2607.12678v1 Announce Type: cross Abstract: Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and s

Tools

The best base model for training isn't about someone else's benchmarks. Excited to offer @thinkymachines' first open-weights model: Inkling.…

The best base model for training isn't about someone else's benchmarks. Excited to offer @thinkymachines' first open-weights model: Inkling. 975B MoE, multimodal, with Apache 2.0. A great foundation f

toolsfireworks-ai--x

The Capacity of Thought: Benchmarking Llama 3.2 in Semantic fMRI Neural Language Decoding and Improving the Huth Encoding-Model Baseline

model-releasesarxiv-cs-cl

arXiv:2607.12079v1 Announce Type: new Abstract: Decoding continuous language from fMRI signals remains a core challenge in non-invasive brain-computer interface research. We present two complementary

Industry

Grok 4.5 Review

Grok 4.5 is an AI language model developed by xAI, featuring improvements in reasoning, coding, and multimodal capabilities compared to earlier versions. The model demonstrates enhanced performance ac

industryelon-musk--x

11 Jul 2026

AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding

arXiv:2607.08745v1 Announce Type: new Abstract: Recent advances in Vision-Language Models, Large Language Models, and Multimodal Large Language Models have improved autonomous driving tasks such as sc

Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement

arXiv:2606.12886v2 Announce Type: replace-cross Abstract: Interleaved thinking, where a unified multimodal model alternates between textual reasoning and visual generation, has shown promise on spatia

Agents

LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action

arXiv:2607.08182v1 Announce Type: cross Abstract: Vision-language-action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic

agentsarxiv-cs-ai

Post-Training in End-to-End Autonomous Driving

arXiv:2607.08072v1 Announce Type: new Abstract: End-to-end models that map multimodal inputs directly to future trajectories/maneuvers have emerged as an increasingly prominent research paradigm in au

Local Ai

Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment

arXiv:2607.08029v1 Announce Type: new Abstract: The emergence of vision language models with fewer than 3 billion parameters has accelerated the implementation of on-device multimodal intelligence. Ho

local-aiarxiv-cs-lg

Local Ai

Activation Quantization of Vision Encoders Needs Prefixing Registers

arXiv:2510.04547v5 Announce Type: replace-cross Abstract: Large pretrained vision encoders are central to multimodal intelligence, powering applications from on-device vision processing to vision-lang

local-aiarxiv-cs-cv

Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcripts

arXiv:2607.06611v1 Announce Type: cross Abstract: Automatically recognizing the sentiment, positive or negative, from speech is a challenging task, requiring both the analysis of vocal inflections and

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

arXiv:2607.07179v1 Announce Type: new Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information

DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

arXiv:2603.15685v2 Announce Type: replace-cross Abstract: Omnimodal large language models (OmniLLMs) jointly process audio and visual streams, but the resulting long multimodal token sequences make in

HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework

arXiv:2602.23615v3 Announce Type: replace Abstract: Current Large Multimodal Models (LMMs) struggle with high-resolution visual inputs during the reasoning process, as the number of image tokens incre

Heterogeneity-Adaptive Diffusion Schrodinger Bridge for PET-Guided Whole-Body MRI Translation

arXiv:2607.07401v1 Announce Type: cross Abstract: While whole-body multimodal medical imaging scanners have been increasingly recognized for more effective medical applications, the excessive long acq

Incentivizing Temporal-Awareness in Egocentric Video Understanding Models

Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings where reasoning dep

safetyapple-ml-research

POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

model-releasesarxiv-cs-lg

arXiv:2607.06649v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and

SensorFM: Towards a general intelligence and interface for wearable health data

SensorFM is a foundation model that analyzes multimodal wearable sensor signals from devices like smartwatches and fitness trackers to generate insights into health and activities. The model was train

researchgoogle-research