Search: “multimodal”

COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention

arXiv:2607.28649v1 Announce Type: cross Abstract: COSI-Lab presents a multimodal, multi-sensor dataset of an interdisciplinary scientific workshop containing 32 academics at an international conferenc

3 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

HenTwin: A Multimodal Digital Twin Framework for Longitudinal Biological State Monitoring in Laying Hens

DGX agent

arXiv:2607.28652v1 Announce Type: cross Abstract: Early-life monitoring in laying hens remains constrained by fragmented single-modality sensing and the absence of formal system-level state representa

researcharxiv-cs-ai

3 Aug 2026

ReMoE: Report-Guided Mixture-of-Experts for Multimodal OCT/OCTA Anomaly Detection

arXiv:2607.29039v1 Announce Type: new Abstract: Multimodal medical anomaly detection identifies samples deviating from normal patterns, where scarce abnormal cases make normality modeling from normal

3 Aug 2026

AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes

model-releasesarxiv-cs-cl

arXiv:2607.27393v1 Announce Type: new Abstract: Hateful memes are a growing form of multimodal online harm, where hostile intent is often conveyed through the joint interpretation of images, text, cul

31 Jul 2026

Tools

Inkling-Small is now live on Together AI. @thinkymachines’ new open-weight multimodal model delivers similar performance to Inkling at one-q…

Inkling-Small is now live on Together AI. @thinkymachines’ new open-weight multimodal model delivers similar performance to Inkling at one-quarter the size, built for coding, agents, and general multi

toolstogether-ai--x

31 Jul 2026

Tutorials

Challenges and proposed solutions in modeling multimodal medical data: A systematic review

arXiv:2505.06945v5 Announce Type: replace Abstract: Multimodal data modeling has emerged as a powerful approach in clinical research, enabling the integration of diverse data types such as imaging, ge

tutorialsarxiv-cs-lg

30 Jul 2026

See2Think: Do Multimodal Models Really Use Intermediate Visual States?

arXiv:2607.26769v1 Announce Type: new Abstract: Multimodal large language models increasingly use sketches, annotations, tools, and intermediate images during reasoning, but it remains unclear whether

30 Jul 2026

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

arXiv:2607.26326v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models. How

30 Jul 2026

CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

arXiv:2607.25294v1 Announce Type: cross Abstract: Real-world tasks often require models to learn from task-specific context rather than relying only on pre-trained knowledge. While recent work has hig

29 Jul 2026

VetClaw: An Edge-Cloud Multimodal Agentic System for Veterinary Disease Screening

arXiv:2607.26042v1 Announce Type: new Abstract: We present VetClaw, an edge-cloud multimodal agentic system for early veterinary disease screening. VetClaw uses a camera module as an edge sensing devi

safetyarxiv-cs-cv

29 Jul 2026

A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting

arXiv:2607.23633v1 Announce Type: cross Abstract: Sky-image irradiance studies often compare forecasting systems in which the image encoder, temporal model, fusion block, target definition, and traini

Color Fundus Photography Analysis: Co-evolution of Data, Preprocessing, and Modeling toward Multimodal AI

arXiv:2607.23972v1 Announce Type: new Abstract: Color Fundus Photography (CFP) is a primary non-invasive imaging modality for large-scale screening of ophthalmic and systemic diseases. Existing survey

Farm-LightSeek: An Edge-centric Multimodal Agricultural IoT Data Analytics Framework with Lightweight LLMs

arXiv:2506.03168v2 Announce Type: replace Abstract: Amid the challenges posed by global population growth and climate change, traditional agricultural Internet of Things (IoT) systems is currently und

LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models

arXiv:2606.16586v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) remain unreliable on fine-grained visual perception, even when high-resolution inputs preserve the necessar

local-aiarxiv-cs-cv

microsoft/Mage-VL · Hugging Face - An Efficient Codec-Native Streaming Multimodal Foundation Model

model-releasesr-localllama

Mage-VL is a codec-native, proactive-streaming multimodal foundation model for image and video understanding, whose visual encoder is trained entirely from scratch at a compact 4B scale. It targets a

Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training

arXiv:2607.22794v1 Announce Type: cross Abstract: Automatic depression detection with deep learning has shown promise but often suffers from limited generalization due to domain shift arising from int

safetyarxiv-cs-ai

Quality Action Assurance: Multimodal Verification of Examiner Claims in VR OSCEs

arXiv:2607.19063v2 Announce Type: replace Abstract: Objective Structured Clinical Examinations (OSCEs) are the gold standard for assessing clinical competence, yet scoring remains vulnerable to examin

local-aiarxiv-cs-ai

RMS@CC-MMD 2026: Multimodal Misogyny Detection via Geometric Interaction and Multi-View Consensus

arXiv:2607.22709v1 Announce Type: cross Abstract: The proliferation of internet memes has introduced new complexities to automated content moderation, particularly in detecting misogyny. Memes often r

safetyarxiv-cs-ai

Generative and multimodal AI for materials prediction and design: Progress, challenges, and perspectives

arXiv:2607.21660v1 Announce Type: cross Abstract: Artificial intelligence (AI) is accelerating materials prediction and design by enabling efficient exploration of chemical and structural spaces, with

safetyarxiv-cs-lg

27 Jul 2026

Time-Reversed Imaging: A Multimodal Benchmark and Framework for Reconstructing Past Human-Environment Interactions

arXiv:2607.22352v1 Announce Type: new Abstract: We introduce time-reversed imaging, a new paradigm that infers what just happened in a scene from fading multimodal traces. Instead of extrapolating or

27 Jul 2026

Enhancing Explainable Cardiac Diagnosis with Guide-Grounded Multimodal LLMs

arXiv:2607.20814v1 Announce Type: new Abstract: The electrocardiogram (ECG) is a cornerstone of cardiac as- sessment, yet clinical deployment of deep learning models remains con- strained by limited i

safetyarxiv-cs-ai

24 Jul 2026

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

arXiv:2607.11007v2 Announce Type: replace Abstract: Few-shot multimodal classification commonly attaches a lightweight head, such as k-nearest neighbors, logistic regression, or a linear SVM, to a fro

researcharxiv-cs-lg

24 Jul 2026

Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models

model-releasesarxiv-cs-lg

arXiv:2607.21496v1 Announce Type: cross Abstract: Cognitive impairment (CI) is a growing public health concern. Early and accurate diagnosis is critical for enabling timely intervention and improving

24 Jul 2026

Tutorials

Advancing Multimodal Fusion on Heterogeneous Medical Data with Hybrid Geometry Attention

arXiv:2607.19086v1 Announce Type: new Abstract: Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, cli

tutorialsarxiv-cs-cv

Forecasting the Number of Harvest-ready Fruits of Sweet Peppers Using Multimodal Time-Series Data

arXiv:2607.19975v1 Announce Type: new Abstract: Accurate yield forecasting at the individual-plant level is critical for precision agriculture and supply-chain planning, yet public datasets capturing

MTVDiff: Multimodal Conditional Latent Diffusion for Enhanced Thermal-to-Visible Face Translation

arXiv:2607.19886v1 Announce Type: new Abstract: Thermal-to-visible face translation presents fundamental challenges including geometric discontinuities, semantic attribute mismatches, and identity deg

safetyarxiv-cs-cv

Agents

Silent Failures in Multimodal Agentic Search:A Diagnostic Taxonomy and Cross-Judge Evaluation

arXiv:2607.19793v1 Announce Type: new Abstract: Multimodal agentic search systems increasingly rely on external tools to answer knowledge-intensive visual questions. However, existing evaluations main

agentsarxiv-cs-ai

LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

arXiv:2606.13061v2 Announce Type: replace Abstract: Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. De

16 Jul 2026

Rethinking Multimodal Fusion for Time Series: Text Modalities Need Constrained Fusion

arXiv:2603.22372v3 Announce Type: replace-cross Abstract: Recent advances in multimodal learning have motivated the integration of auxiliary modalities such as text or vision into time series (TS) for

researcharxiv-cs-ai

16 Jul 2026

SIVA-RL: Sensitivity-Invariance Visual Alignment for Multimodal Reinforcement Learning

arXiv:2607.13931v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) drives multimodal reasoning, but answer-level correctness does not guarantee that a vision-languag

safetyarxiv-cs-cv

16 Jul 2026

A Calibrated Multimodal Ensemble for Ambivalence/Hesitancy Recognition: System Description and Private-Test Submission Strategy

arXiv:2607.12176v1 Announce Type: new Abstract: Ambivalence and hesitancy (A/H) undermine digital behaviour-change interventions, and recognizing them automatically from video is the goal of the ABAW

15 Jul 2026

Excited for our first general model Inkling -- open weights, 975B, natively multimodal (text, image, audio). Available on Tinker, HuggingFac…

researchsoumith-chintala--x

Excited for our first general model Inkling -- open weights, 975B, natively multimodal (text, image, audio). Available on Tinker, HuggingFace and partners. It is yours to personalize and use openly. I

15 Jul 2026

Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors

arXiv:2512.15748v2 Announce Type: replace-cross Abstract: Visual Species Recognition (VSR) is a fundamental task in scientific disciplines that require species-level identification, including ecology,

15 Jul 2026

Agents

I guess image input is the big capability of the models, and tool use can be a substitute for non-omni model output. Still, multimodal voice…

Ethan Mollick notes that image input represents the primary advanced capability of current AI models, and that tool‑use can effectively replace outputs from non‑omni models. He observes that multimoda

agentsethan-mollick--x

13 Jul 2026

FDRMFL: Multimodal Federated Feature Extraction Model Based on Information Maximization and Contrastive Learning

arXiv:2512.02076v2 Announce Type: replace-cross Abstract: We propose FDRMFL, a task-driven multimodal feature extraction framework for federated regression under non-IID data distributions. Extracting

applicationsarxiv-cs-ai

General Incomplete Multimodal Learning via Dynamic Quality Perception

arXiv:2607.06943v1 Announce Type: new Abstract: Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, wher

Latent graph encoding of multimodal neuroimaging features with generative AI architectures

arXiv:2607.07027v1 Announce Type: cross Abstract: While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frame

researcharxiv-cs-ai

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

arXiv:2607.06633v1 Announce Type: cross Abstract: In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or

researcharxiv-cs-ai

LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models

arXiv:2602.14147v2 Announce Type: replace Abstract: Diffusion language models (dLLMs) recently emerged as a promising alternative to auto-regressive LLMs. The latest works further extended it to multi

8 Jul 2026

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning

arXiv:2409.06067v3 Announce Type: replace Abstract: Previous studies on federated learning (FL) often encounter performance degradation due to data heterogeneity among different clients. In light of t

local-aiarxiv-cs-ai

8 Jul 2026

Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models

arXiv:2607.05716v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong perception and reasoning capabilities. However, most existing models focus on isolated

8 Jul 2026

ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval

arXiv:2606.20280v2 Announce Type: replace-cross Abstract: Leveraging Multimodal Large Language Models (MLLMs) via contrastive learning has become a mainstream paradigm for improving the performance of

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

arXiv:2607.03817v1 Announce Type: new Abstract: Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input

local-aiarxiv-cs-cv

MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training

arXiv:2602.06285v2 Announce Type: replace Abstract: Recent research in geospatial machine learning has demonstrated that models pretrained with self-supervised learning on Earth observation data can p

Purify then Guide: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing

arXiv:2505.09484v2 Announce Type: replace Abstract: Face Anti-Spoofing (FAS) is essential for the security of facial recognition systems in diverse scenarios such as payment processing and surveillanc

Tutorials

Multimodal prompting is clearly the future. How we interact with agents is evolving. I share a bit (including a video walkthrough) of how I …

This post discusses the evolution of multimodal prompting in AI agent interactions, highlighting how users can now communicate with AI systems using multiple input types beyond text. The author provid

tutorialsdair-ai--x

4 Jul 2026

InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

arXiv:2607.02010v1 Announce Type: new Abstract: Multimodal large language models must adapt to evolving tasks and domains, yet continual improvement under bounded deployment footprint remains difficul

3 Jul 2026

MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering

arXiv:2607.01420v1 Announce Type: cross Abstract: As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and