Search: “multimodal”

MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training

arXiv:2504.09844v4 Announce Type: replace-cross Abstract: Modern frameworks for training large foundation models (LFMs) employ dataloaders in a data-parallel manner, with each loader processing a disj

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Multi-Plane HyperX: A Low-Latency and Cost-Effective Network for Large-Scale AI and HPC Systems

DGX agent

arXiv:2604.23519v1 Announce Type: cross Abstract: Multi-plane architectures have become increasingly prevalent in the Fat-Tree networks of AI data centers. By leveraging multiple ports on a single net

researcharxiv-cs-lg

28 Apr 2026

Multivariate Gaussian NeRF for Wide Field-of-View Ultrasound Reconstruction

arXiv:2604.24187v1 Announce Type: new Abstract: Wide Field-of-View (WFoV) reconstruction enhances 3D ultrasound imaging by providing valuable anatomical context for segmentation models and visualizati

researcharxiv-cs-cv

NeuroClaw Technical Report

model-releasesarxiv-cs-cv

arXiv:2604.24696v1 Announce Type: new Abstract: Agentic artificial intelligence systems promise to accelerate scientific workflows, but neuroimaging poses unique challenges: heterogeneous modalities (

'Noisier' Noise Contrastive Eestimation is (Almost) Maximum Likelihood

arXiv:2405.16730v2 Announce Type: replace-cross Abstract: Noise Contrastive Estimation (NCE) has fueled major breakthroughs in representation learning and generative modeling. Yet a long-standing chal

On the Complementarity of Quantum and Classical Features: Adaptive Hybrid Quantum-Classical Feature Fusion for Breast Cancer Classification

arXiv:2604.22903v1 Announce Type: cross Abstract: The integration of quantum machine learning with classical deep learning offers promising avenues for medical image analysis by mapping data into high

Orthogonal Representation Learning for Estimating Causal Quantities

arXiv:2502.04274v4 Announce Type: replace Abstract: End-to-end representation learning has become a powerful tool for estimating causal quantities from high-dimensional observational data, but its eff

safetyarxiv-cs-lg

Partition-of-Unity Gaussian Kolmogorov-Arnold Networks

arXiv:2604.23599v1 Announce Type: cross Abstract: Gaussian basis functions provide an efficient and flexible alternative to spline activations in KANs. In this work, we introduce the partition-of-unit

Robust Audio-Text Retrieval via Cross-Modal Attention and Hybrid Loss

arXiv:2604.23323v1 Announce Type: new Abstract: Audio-text retrieval enables semantic alignment between audio content and natural language queries, supporting applications in multimedia search, access

Scalable LLM-based Coding of Dialogue in Healthcare Simulation: Balancing Coding Performance, Processing Time, and Environmental Impact

arXiv:2604.23255v1 Announce Type: cross Abstract: Research shows that dialogue, the interactive process through which participants articulate their thinking, plays a central role in constructing share

See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection

arXiv:2604.24339v1 Announce Type: cross Abstract: Recent advances in Vision-Language Models (VLMs) have benefited from Reinforcement Learning (RL) for enhanced reasoning. However, existing methods sti

Hardware

Self-Rewarding Vision-Language Model via Reasoning Decomposition

arXiv:2508.19652v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) often suffer from visual hallucinations: generating things that are not consistent with visual inputs and language sho

hardwarearxiv-cs-cv

Tutorials

SMoES: Soft Modality-Guided Expert Specialization in MoE-VLMs

arXiv:2604.23996v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) has become a prevalent backbone for large vision-language models (VLMs), yet how modality-specific signals should guide expert

tutorialsarxiv-cs-cv

SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing

arXiv:2604.23392v1 Announce Type: new Abstract: Refereeing is vital in sports, where fair, accurate, and explainable decisions are fundamental. While intelligent assistant technologies are being widel

Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency

arXiv:2604.24380v1 Announce Type: new Abstract: While Large Vision Language Models (LVLMs) demonstrate impressive capabilities, their substantial computational and memory requirements pose deployment

Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey

arXiv:2505.15957v4 Announce Type: replace-cross Abstract: With advancements in large audio-language models (LALMs), which enhance large language models (LLMs) with auditory capabilities, these models

V-SEAM: Visual Semantic Editing and Attention Modulating for Causal Interpretability of Vision-Language Models

arXiv:2509.14837v2 Announce Type: replace Abstract: Recent advances in causal interpretability have extended from language models to vision-language models (VLMs), seeking to reveal their internal mec

VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models

model-releasesarxiv-cs-cv

arXiv:2510.08618v2 Announce Type: replace-cross Abstract: Omni-modal large language models (OLLMs) offer a promising end-to-end solution for slide-enhanced speech recognition due to their inherent mul

Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines

arXiv:2604.23001v1 Announce Type: cross Abstract: Despite remarkable progress in Vision--Language--Action (VLA) models, a central bottleneck remains underexamined: the data infrastructure that underli

Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms

arXiv:2604.23775v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models are emerging as a unified substrate for embodied intelligence. This shift raises a new class of safety challenges, s

safetyarxiv-cs-ro

Tutorials

ZID-Net: Zero-Inference Diffusion Prior Decoupling Network for Single Image Dehazing

arXiv:2604.23709v1 Announce Type: new Abstract: Single image dehazing is often constrained by a trade-off between restoration quality and computational efficiency. While efficient, CNN networks strugg

tutorialsarxiv-cs-cv

A systematic review of generative AI usage for IT project management

arXiv:2604.21958v1 Announce Type: cross Abstract: This paper aims to synthesize current knowledge on generative AI in IT project management using the PRISMA methodology to provide researchers with a c

An Integrated Framework for Explainable, Fair, and Observable Hospital Readmission Prediction: Development and Validation on MIMIC-IV

arXiv:2604.22535v1 Announce Type: new Abstract: Objective: To propose and retrospectively validate an integrated framework addressing three barriers to clinical translation of readmission prediction:

safetyarxiv-cs-lg

An Interdisciplinary and Cross-Task Review on Missing Data Imputation

arXiv:2511.01196v3 Announce Type: replace-cross Abstract: Missing data is a fundamental challenge in data science, significantly hindering analysis and decision-making across a wide range of disciplin

Are Video Models Emerging as Zero-Shot Learners and Reasoners in Medical Imaging?

arXiv:2510.10254v2 Announce Type: replace Abstract: Recent advances in large generative models have shown that simple autoregressive formulations, when scaled appropriately, can exhibit strong zero-sh

researcharxiv-cs-cv

ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding

arXiv:2604.22333v1 Announce Type: cross Abstract: Rapid situational awareness is critical in post-disaster response. While remote sensing damage assessment is evolving from pixel-level change detectio

CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution

model-releasesarxiv-cs-cv

arXiv:2604.22192v1 Announce Type: new Abstract: Chart-to-code generation demands strict visual precision and syntactic correctness from Vision-Language Models (VLMs). However, existing approaches are

Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivity

arXiv:2604.22018v1 Announce Type: cross Abstract: Several brain foundation models (FM) have recently been proposed to predict brain disorders by modelling dynamic functional connectivity (FC). While t

Local Ai

GenMatter: Perceiving Physical Objects with Generative Matter Models

arXiv:2604.22160v1 Announce Type: cross Abstract: Human visual perception offers valuable insights for understanding computational principles of motion-based scene interpretation. Humans robustly dete

local-aiarxiv-cs-ai

Lightweight Retrieval-Augmented Generation and Large Language Model-Based Modeling for Scalable Patient-Trial Matching

arXiv:2604.22061v1 Announce Type: cross Abstract: Patient-trial matching requires reasoning over long, heterogeneous electronic health records (EHRs) and complex eligibility criteria, posing significa

Manifold Learning for Personalized and Label-Free Detection of Cardiac Arrhythmias

arXiv:2506.16494v3 Announce Type: replace Abstract: Electrocardiograms (ECGs) provide non-invasive measurements of heart activity and are established tools for detecting cardiac arrhythmias. Although

applicationsarxiv-cs-lg

Presenting DiaData for Research on Type 1 Diabetes

arXiv:2508.09160v2 Announce Type: replace Abstract: Type 1 diabetes (T1D) is an autoimmune disorder that leads to the destruction of insulin-producing cells, resulting in insulin deficiency, as to why

researcharxiv-cs-lg

ReLeVAnT: Relevance Lexical Vectors for Accurate Legal Text Classification

arXiv:2604.22292v1 Announce Type: cross Abstract: The classification of legal documents from an unstructured data corpus has several crucial applications in downstream tasks. Documents relevant to cou

ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression

arXiv:2604.22180v1 Announce Type: cross Abstract: Large language model (LLM) based listwise reranking has emerged as the dominant paradigm for achieving state-of-the-art ranking effectiveness in infor

Local Ai

SNGR: Selective Non-Gaussian Refinement for Ambiguous SLAM Factor Graphs

arXiv:2604.22065v1 Announce Type: new Abstract: We present Selective Non-Gaussian Refinement (SNGR), a SLAM framework that augments iSAM2 with targeted nested sampling on windows where Gaussian approx

local-aiarxiv-cs-ro

Hardware

SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

arXiv:2604.22575v1 Announce Type: new Abstract: Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenec

hardwarearxiv-cs-lg

Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Dataset

arXiv:2604.22260v1 Announce Type: cross Abstract: Urban transportation systems face growing safety challenges that require scalable intelligence for emerging smart mobility infrastructures. While rece

Transferable Physical-World Adversarial Patches Against Pedestrian Detection Models

arXiv:2604.22552v1 Announce Type: new Abstract: Physical adversarial patch attacks critically threaten pedestrian detection, causing surveillance and autonomous driving systems to miss pedestrians and

safetyarxiv-cs-cv

UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions

arXiv:2604.22209v1 Announce Type: cross Abstract: Generative audio modeling has largely been fragmented into specialized tasks, text-to-speech (TTS), text-to-music (TTM), and text-to-audio (TTA), each

Addressing Image Authenticity When Cameras Use Generative AI

arXiv:2604.21879v1 Announce Type: cross Abstract: The ability of generative AI (GenAI) methods to photorealistically alter camera images has raised awareness about the authenticity of images shared on

Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts

arXiv:2604.21344v1 Announce Type: cross Abstract: Charts are widely used to present complex information. Deriving meaningful insights in real-world contexts often requires interpreting multiple relate

CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction

arXiv:2604.21573v1 Announce Type: new Abstract: Spatial transcriptomics (ST) enables spatially resolved gene profiling but remains expensive and low-throughput, limiting large-cohort studies and routi

safetyarxiv-cs-cv

Deep Interest Mining with Cross-Modal Alignment for SemanticID Generation in Generative Recommendation

arXiv:2604.20861v1 Announce Type: cross Abstract: Generative Recommendation (GR) has demonstrated remarkable performance in next-token prediction paradigms, which relies on Semantic IDs (SIDs) to comp

Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos

arXiv:2604.21814v1 Announce Type: cross Abstract: Capsule endoscopy (CE) enables non-invasive gastrointestinal screening, but current CE research remains largely limited to frame-level classification

Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models

arXiv:2507.04023v3 Announce Type: replace Abstract: Large language models (LLMs) achieve impressive performance on complex mathematical benchmarks yet sometimes fail on basic math reasoning while gene

Efficient Logic Gate Networks for Video Copy Detection

arXiv:2604.21694v1 Announce Type: cross Abstract: Video copy detection requires robust similarity estimation under diverse visual distortions while operating at very large scale. Although deep neural

FunduSegmenter: Leveraging the RETFound Foundation Model for Joint Optic Disc and Optic Cup Segmentation in Retinal Fundus Images

arXiv:2508.11354v3 Announce Type: replace-cross Abstract: Purpose: This study introduces the first adaptation of RETFound for joint optic disc (OD) and optic cup (OC) segmentation. RETFound is a well-

Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuning

arXiv:2509.21976v3 Announce Type: replace-cross Abstract: Referring expression understanding in remote sensing poses unique challenges, as it requires reasoning over complex object-context relationshi