Knowledge catalogue
arXiv:2604.16980v1 Announce Type: new Abstract: Background: Large language models (LLMs) are increasingly proposed for diagnostic support, but few evaluations use real-world multimodal inpatient data,
arXiv:2604.17274v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in various perception and reasoning tasks. Despite this success, ens
arXiv:2604.17054v1 Announce Type: new Abstract: Scalable Vector Graphics (SVGs) function both as visual images and as structured code that encode rich geometric and layout information, yet most method
arXiv:2604.16311v1 Announce Type: new Abstract: Automated Fact-Checking (AFC) relies on claim extraction as a first step, yet existing methods largely overlook the multimodal nature of today's misinfo
arXiv:2604.16734v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have recently demonstrated strong capabilities in understanding and generating responses from diverse visual in
arXiv:2604.18112v1 Announce Type: new Abstract: In recent years, multimodal multidomain fake news detection has garnered increasing attention. Nevertheless, this direction presents two significant cha
arXiv:2510.15253v3 Announce Type: replace Abstract: Document understanding is critical for applications from financial analysis to scientific discovery. Current approaches, whether OCR-based pipelines
arXiv:2604.15377v1 Announce Type: cross Abstract: Accurate and timely rainfall nowcasting is crucial for disaster mitigation and water resource management. Despite recent advances in deep learning, pr
arXiv:2604.15670v1 Announce Type: new Abstract: Reasoning segmentation has recently expanded from ground-level scenes to remote-sensing imagery, yet UAV data poses distinct challenges, including obliq
arXiv:2512.14098v3 Announce Type: replace Abstract: Any-to-Any models are an emerging class of multimodal models that accept combinations of text and multimodal data as input and generate them as outp
arXiv:2604.14951v1 Announce Type: cross Abstract: Tool learning with foundation models aims to endow AI systems with the ability to invoke external resources -- such as APIs, computational utilities,
arXiv:2604.14363v1 Announce Type: new Abstract: Multimodal language models systematically underperform on visual perception tasks, yet the structure underlying this failure remains poorly understood.
arXiv:2603.27064v2 Announce Type: replace-cross Abstract: Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a ca
arXiv:2604.14016v1 Announce Type: new Abstract: Multimodal Continual Instruction Tuning (MCIT) is essential for sequential task adaptation of Multimodal Large Language Models (MLLMs) but is severely r
arXiv:2604.13418v1 Announce Type: new Abstract: Motivated by the underspecified, multi-hop nature of search queries and the multimodal, heterogeneous, and often conflicting nature of real-world web re
This guide covers how to train and fine-tune multimodal embedding and reranker models using the Sentence Transformers library, enabling systems to work with both text and image data simultaneously. It
arXiv:2604.13403v1 Announce Type: new Abstract: In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations. Despite its success in large language models, the exte
arXiv:2603.08291v3 Announce Type: replace Abstract: Multimodal Mathematical Reasoning (MMR) has recently attracted increasing attention for its capability to solve mathematical problems involving both
arXiv:2604.11136v1 Announce Type: cross Abstract: Object-level spatial-temporal understanding is essential for video question answering, yet existing multimodal large language models (MLLMs) encode fr
arXiv:2604.00013v2 Announce Type: replace-cross Abstract: Multimodal sentiment analysis aims to integrate textual, acoustic, and visual information for deep emotional understanding. Despite the progre
arXiv:2601.07516v2 Announce Type: replace-cross Abstract: Vision-language models are increasingly employed as multimodal conversational agents (MCAs) for diverse conversational tasks. Recently, reinfo
arXiv:2604.09905v1 Announce Type: new Abstract: Emergency department triage relies heavily on both quantitative vital signs and qualitative clinical notes, yet multimodal machine learning models predi
arXiv:2604.10784v1 Announce Type: new Abstract: Recent advances in unified multimodal models (UMMs) have led to a proliferation of architectures capable of understanding, generating, and editing acros
arXiv:2604.08846v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been shown to be vulnerable to malicious queries that can elicit unsafe responses. Recent work uses prom
arXiv:2604.09253v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are powerful but remain vulnerable to multimodal jailbreak attacks. Existing attacks mainly rely on either explicit visu
arXiv:2604.08701v1 Announce Type: new Abstract: We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calib
arXiv:2603.18472v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) perform strongly on natural images, yet their ability to understand discrete visual symbols remains u
arXiv:2506.19420v2 Announce Type: replace Abstract: Multimodal sarcasm understanding is a high-order cognitive task. Although large language models (LLMs) have shown impressive performance on many dow
arXiv:2510.12184v2 Announce Type: replace Abstract: Recently, efficient Multimodal Large Language Models (MLLMs) have gained significant attention as a solution to their high computational complexity,
arXiv:2509.23322v2 Announce Type: replace Abstract: With the continuous expansion of Large Language Models (LLMs) and advances in reinforcement learning, LLMs have demonstrated exceptional reasoning c
arXiv:2604.06714v1 Announce Type: new Abstract: AI applications driven by multimodal large language models (MLLMs) are prone to hallucinations and pose considerable risks to human users. Crucially, su
arXiv:2610.21948v1 Announce Type: cross Abstract: As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes es
arXiv:2610.22240v1 Announce Type: new Abstract: Multimodal deep search is moving from short queries to long-horizon investigations that continually acquire evidence. However, training such agents rema
arXiv:2610.21266v1 Announce Type: cross Abstract: Continuous video and clinical records provide complementary information for understanding how patient behavior evolves over a hospital stay. A central
arXiv:2508.04136v4 Announce Type: replace-cross Abstract: Few-shot fine-grained visual classification (FGVC) aims to discriminate between subtly different categories with very limited data. While Mult
arXiv:2509.09658v3 Announce Type: replace Abstract: Hallucinations in multimodal large language models (MLLMs), where a model generates content inconsistent with the input image, pose significant risk
arXiv:2610.20538v1 Announce Type: new Abstract: Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burge
arXiv:2608.01008v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) can memorize and disclose sensitive information, motivating machine unlearning to remove designated knowled
arXiv:2610.19108v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitat
arXiv:2610.15227v1 Announce Type: new Abstract: Unified vision-language models (VLMs) perform both text-to-image (T2I) and image-to-text (I2T) generation with one shared Transformer backbone, yet how
arXiv:2610.18238v1 Announce Type: new Abstract: Vision-language models (VLMs) have been widely adopted for document parsing and optical character recognition (OCR), as they can accurately extract page
arXiv:2610.13956v1 Announce Type: new Abstract: Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting future representations rather than reconstructing observations. Variat
1M-token, multimodal decision models are here! But there is more to this one. You can classify, extract, and summarize with 1M tokens of text and image input. ScaleDown's Constrained Decoding Models r
arXiv:2610.12957v1 Announce Type: cross Abstract: Embodied agents must translate multimodal observations and user requests into executable actions while adapting to product constraints, changing goals
arXiv:2610.13527v1 Announce Type: cross Abstract: Multimodal language models (MLLMs) exhibit promising spatial reasoning capabilities, yet the internal mechanisms supporting them remain poorly underst
arXiv:2610.12981v1 Announce Type: cross Abstract: Visual interventions provide a training-free way to reduce hallucinations in multimodal large language models. However, an intervention may have littl
arXiv:2610.11918v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mai
arXiv:2610.12393v1 Announce Type: new Abstract: Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains
arXiv:2610.10400v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have made significant progress in visual understanding and generation. However, generating interleaved image--t
arXiv:2601.17645v2 Announce Type: replace-cross Abstract: Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examin
arXiv:2603.04364v2 Announce Type: replace-cross Abstract: Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet the
arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using pr
arXiv:2610.08216v1 Announce Type: new Abstract: Multimodal vision-language systems typically fuse image and text embeddings through classical operators such as concatenation, attention, bilinear pooli
arXiv:2610.07774v1 Announce Type: new Abstract: Vision-language models (VLMs) face compositional safety risks where harmful intent emerges from the interaction between visual and textual inputs. As mi
arXiv:2507.18104v3 Announce Type: replace Abstract: Brain encoding models predict time-resolved neural activity from computational representations of ongoing experience, providing a principled framewo
https://huggingface.co/google/embeddinggemma-2 EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combina
arXiv:2610.03574v1 Announce Type: new Abstract: We introduce HyperBrowseComp, a multilingual and multimodal browsing benchmark comprising 423 manually authored and human-validated questions across 13
arXiv:2610.02943v1 Announce Type: new Abstract: Multimodal 3D Human Pose Estimation (3D HPE) combines complementary information from RGB, LiDAR, and mmWave radar, but models trained on correlated obse
arXiv:2610.02876v1 Announce Type: new Abstract: A multimodal large language model that correctly reports a spatial fact does not necessarily use that fact in subsequent reasoning. To study this distin
arXiv:2508.21143v4 Announce Type: replace Abstract: Cognitive science research treats visual perception, the ability to understand and make sense of a visual input, as one of the early developmental s