Knowledge catalogue
arXiv:2605.20713v1 Announce Type: new Abstract: Multimodal IE in social media is difficult because a post may attach multiple images that are weakly related, redundant, or even misleading with respect
arXiv:2605.16409v1 Announce Type: cross Abstract: Optical character recognition (OCR) and multilingual text understanding remain major failure modes of multimodal large language models (MLLMs), partic
arXiv:2605.17946v1 Announce Type: new Abstract: Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external too
arXiv:2605.14448v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have emerged as a powerful backbone for multimodal embeddings. Recent methods introduce chain-of-thought (CoT
arXiv:2605.11301v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have heterogeneous strengths across OCR, chart understanding, spatial reasoning, visual question answering, c
arXiv:2605.12031v1 Announce Type: cross Abstract: Multimodal deep learning has shown strong potential in medical applications by integrating heterogeneous data sources such as medical images and struc
arXiv:2508.17497v2 Announce Type: replace-cross Abstract: Multimodal representation learning has been largely driven by contrastive models such as CLIP, which learn a shared embedding space by alignin
arXiv:2508.04750v2 Announce Type: replace Abstract: In real-world applications, multimodal time-series forecasting faces a key challenge: textual information is often useful but unreliable. Auxiliary
In this post, we build a multimodal retrieval system for aerospace manufacturing documents using Amazon Nova Multimodal Embeddings on Amazon Bedrock and Amazon S3 Vectors. We evaluate the system on 26
arXiv:2605.04128v1 Announce Type: cross Abstract: We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing
GLM-5V-Turbo Tech Report: Toward a Native Foundation Model for Multimodal Agents This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcemen
arXiv:2605.03790v1 Announce Type: new Abstract: With advances in multimodal research and deep learning, Multimodal Large Language Models (MLLMs) have emerged as a powerful paradigm for a wide range of
arXiv:2605.00733v1 Announce Type: cross Abstract: Federated Multimodal Learning (FML) trains multimodal models across decentralized clients while keeping their image-text pairs private. However, joint
arXiv:2604.27389v1 Announce Type: cross Abstract: In recent years, Multimodal Large Language Models (MLLMs) have achieved remarkable progress on a wide range of multimodal benchmarks. Despite these ad
arXiv:2604.26247v1 Announce Type: cross Abstract: Multimodal recommendation improves user modeling by integrating collaborative signals with heterogeneous item content. In real applications, user inte
arXiv:2604.25122v1 Announce Type: new Abstract: We present M^3-VQA, a novel knowledge-based Visual Question Answering (VQA) benchmark, to enhance the evaluation of multimodal large language models (ML
arXiv:2604.23733v1 Announce Type: new Abstract: Asking inquisitive questions while reading, and looking for their answers, is an important part in human discourse comprehension, curiosity, and creativ
arXiv:2604.20486v1 Announce Type: new Abstract: Training multimodal agents via reinforcement learning for knowledge-intensive visual reasoning is fundamentally hindered by the extreme sparsity of outc
arXiv:2604.20146v1 Announce Type: cross Abstract: Grounded Multimodal Named Entity Recognition (GMNER) aims to extract named entities and localize their visual regions within image-text pairs, serving
arXiv:2604.19689v1 Announce Type: new Abstract: Understanding artworks requires multi-step reasoning over visual content and cultural, historical, and stylistic context. While recent multimodal large
arXiv:2604.17319v1 Announce Type: cross Abstract: Grounded Multimodal Named Entity Recognition (GMNER) aims to jointly identify named entity mentions in text, predict their semantic types, and ground
arXiv:2510.09474v2 Announce Type: replace Abstract: Modern conversational agents like ChatGPT and Alexa+ rely on predefined policies specifying metadata, response styles, and tool-usage rules. As thes
arXiv:2604.16214v1 Announce Type: new Abstract: Understanding affective dynamics in real-world social systems is fundamental to modeling and analyzing human-human interactions in complex environments.
arXiv:2604.14029v1 Announce Type: new Abstract: While Large Multimodal Models (LMMs) demonstrate impressive visual perception, they remain epistemically constrained by their static parametric knowledg
arXiv:2604.12335v1 Announce Type: new Abstract: Training multimodal large language models (MLLMs) for video understanding requires large-scale annotated data spanning diverse tasks such as object coun
arXiv:2604.12271v1 Announce Type: new Abstract: Multimodal attributed graphs (MAGs) combine multimodal node attributes with structured relations. However, existing methods usually perform shared messa
arXiv:2604.12890v1 Announce Type: cross Abstract: Multimodal deep search agents have shown great potential in solving complex tasks by iteratively collecting textual and visual evidence. However, mana
arXiv:2505.11237v3 Announce Type: replace-cross Abstract: Metaphorical imagination, the ability to connect seemingly unrelated concepts, is fundamental to human cognition and communication. While unde
arXiv:2512.18073v2 Announce Type: replace Abstract: Multimodal LLMs (MLLMs) are capable of performing complex data analysis, visual question answering, generation, and reasoning tasks. However, their
arXiv:2604.09552v1 Announce Type: cross Abstract: Engineering rulebooks and technical standards contain multimodal information like dense text, tables, and illustrations that are challenging for retri
arXiv:2604.10064v1 Announce Type: new Abstract: Multimodal Transformers serve as the backbone for state-of-the-art vision-language models, yet their quadratic attention complexity remains a critical b
arXiv:2604.08884v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have made significant strides in natural image understanding, their ability to perceive and reason over
arXiv:2509.23310v3 Announce Type: replace Abstract: Deep learning-based techniques for the analysis of multimodal remote sensing data have become popular due to their ability to effectively integrate
arXiv:2604.08068v1 Announce Type: new Abstract: Decoding visual information from electroencephalography (EEG) has recently achieved promising results, primarily focusing on reconstructing two-dimensio
arXiv:2604.05704v2 Announce Type: replace Abstract: Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, mult
arXiv:2610.21318v1 Announce Type: cross Abstract: Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model
arXiv:2610.20906v1 Announce Type: cross Abstract: Dementia affects millions of older adults worldwide, yet early detection remains difficult outside clinical settings. Traditional models primarily rel
arXiv:2610.22068v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can act unsafely when individually benign images and texts are interpreted jointly. Beyond implicit reasoning r
arXiv:2610.19613v1 Announce Type: new Abstract: As machine unlearning for multimodal large language models (MLLMs) rapidly advances, rigorous evaluation of its effectiveness becomes increasingly impor
arXiv:2610.14395v1 Announce Type: new Abstract: Multimodal prediction models add a data source, such as imaging or an assay, to clinical information, and health systems must decide whether that added
arXiv:2610.18587v1 Announce Type: cross Abstract: Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as speech and text. Ps
arXiv:2610.17856v1 Announce Type: new Abstract: Time-series analysis is increasingly studied in multimodal settings, yet most benchmarks pair temporal signals with text or with visualizations derived
arXiv:2601.19060v2 Announce Type: replace Abstract: Multimodal Retrieval-Augmented Generation (MM-RAG) augments visual question answering (VQA) with external knowledge, but many systems rely on modula
arXiv:2610.15745v1 Announce Type: new Abstract: Multimodal 3D occupancy prediction underpins safe autonomous driving through comprehensive scene understanding. However, existing methods have largely p
arXiv:2610.13783v1 Announce Type: new Abstract: Memes play a central role in modern online communication, especially in contexts of political conflict where visual and textual cues combine to convey p
arXiv:2610.11724v1 Announce Type: new Abstract: Multimodal meme understanding is increasingly used to analyze socially sensitive content, yet existing models often exhibit biased behavior when interpr
arXiv:2610.11787v1 Announce Type: new Abstract: Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods ofte
arXiv:2610.11228v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong potential for sequential recommendation through their ability to reason over complex m
arXiv:2610.11176v1 Announce Type: new Abstract: Multimodal remote sensing image registration is a crucial prerequisite for the collaborative processing and downstream application of remote sensing dat
arXiv:2610.09360v1 Announce Type: cross Abstract: Real-world documents distribute evidence across text, tables, figures, and captions within complex page layouts. Answering complex questions over such
arXiv:2610.06972v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual understanding and multimodal reasoning, yet they remain fun
arXiv:2610.06896v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) are increasingly positioned as general purpose visual reasoners as part of the quest for artificial ge
arXiv:2602.02140v2 Announce Type: replace Abstract: Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, whethe
arXiv:2606.16193v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representati
.@GoogleDeepMind EmbeddingGemma 2 is now available on Ollama. Made for consumer devices and is multimodal. ollama pull embeddinggemma-2 Meet EmbeddingGemma 2, our first natively multimodal open model
arXiv:2507.07297v2 Announce Type: replace Abstract: Recent advances in large vision-language models have led to impressive performance in visual question answering and multimodal reasoning. However, i
arXiv:2610.03991v1 Announce Type: new Abstract: Multimodal skin lesion classification combines clinical images with patient metadata to improve diagnostic accuracy. However, complete metadata availabl
arXiv:2610.06378v1 Announce Type: new Abstract: The rapid evolution of video generation has narrowed the perceptual gap between authentic and synthetic videos, making generalizable AI-generated video
arXiv:2610.05240v1 Announce Type: new Abstract: Time-series foundation models offer a unified approach to forecasting across heterogeneous domains. Textual context and auxiliary observations provide c
arXiv:2610.05453v1 Announce Type: cross Abstract: Multimodal models are increasingly shifting toward unified architectures that understand and generate text, images, and other modalities within a shar