Search: “pretraining”

Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation

arXiv:2604.27263v1 Announce Type: new Abstract: Subword tokenization is an essential part of modern large language models (LLMs), yet its specific contributions to training efficiency and model perfor

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Energy-Efficient Plant Monitoring via Knowledge Distillation

DGX agent

arXiv:2604.27178v1 Announce Type: new Abstract: Recent advances in large-scale visual representation learning have significantly improved performance in plant species and plant disease recognition tas

applicationsarxiv-cs-cv

1 May 2026

Leveraging Quantum-Based Architectures for Robust Diagnostics

arXiv:2511.12386v2 Announce Type: replace Abstract: Quantum machine learning has emerged as a promising approach for medical image analysis, particularly in settings where compact models and expressiv

Tutorials

Noise2Map: End-to-End Diffusion Model for Semantic Segmentation and Change Detection

arXiv:2604.27889v1 Announce Type: new Abstract: Semantic segmentation and change detection are two fundamental challenges in remote sensing, requiring models to capture either spatial semantics or tem

tutorialsarxiv-cs-cv

TypeBandit: Type-Level Context Allocation and Reweighting for Effective Attribute Completion in Heterogeneous Graph Neural Networks

arXiv:2604.27356v1 Announce Type: cross Abstract: Heterogeneous graphs are widely used to model multi-relational systems, but missing node attributes remain a major bottleneck for downstream learning.

VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking

model-releasesarxiv-cs-ai

arXiv:2601.08611v2 Announce Type: replace-cross Abstract: The growing scale of online misinformation urgently demands Automated Fact-Checking (AFC). Existing benchmarks for evaluating AFC systems, how

Agents

Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusion

arXiv:2604.25574v1 Announce Type: new Abstract: In autonomous driving, camera-radar fusion offers complementary sensing and low deployment cost. Existing methods perform fusion through input mixing, f

agentsarxiv-cs-cv

IAM: Identity-Aware Human Motion and Shape Joint Generation

arXiv:2604.25164v1 Announce Type: new Abstract: Recent advances in text-driven human motion generation enable models to synthesize realistic motion sequences from natural language descriptions. Howeve

Learning Structure, Energy, and Dynamics: A Survey of Artificial Intelligence for Protein Dynamics

arXiv:2604.25244v1 Announce Type: cross Abstract: Protein dynamics underlie many biological functions, yet remain difficult to characterize due to the high computational cost of molecular dynamics sim

Local Ai

MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment

arXiv:2603.15954v2 Announce Type: replace Abstract: Real-time AI experiences call for on-device large language models (OD-LLMs) optimized for efficient deployment on resource-constrained hardware. The

local-aiarxiv-cs-lg

Revisiting the Past: Data Unlearning with Model State History

arXiv:2506.20941v3 Announce Type: replace Abstract: Large language models are trained on massive corpora of web data, which may include private data, copyrighted material, factually inaccurate data, o

Towards Robust Deep Learning-based Rumex Obtusifolius Detection from Drone Images

arXiv:2604.25316v1 Announce Type: new Abstract: Domain adaptation (DA) addresses the challenge of transferring a machine learning model trained on a source domain to a target domain with a different d

Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation

arXiv:2511.21517v2 Announce Type: replace Abstract: Unlike text, speech conveys information about the speaker, such as gender, through acoustic cues like pitch. This gives rise to modality-specific bi

safetyarxiv-cs-cl

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

arXiv:2604.23165v1 Announce Type: new Abstract: Spiking Vision Transformers (S-ViTs) offer a promising framework for energy-efficient visual learning. However, existing designs remain limited by two f

Diffusion Model as a Generalist Segmentation Learner

arXiv:2604.24575v1 Announce Type: new Abstract: Diffusion models are primarily trained for image synthesis, yet their denoising trajectories encode rich, spatially aligned visual priors. In this paper

Domain-Adapted Fine-Tuning of ECG Foundation Models for Multi-Label Structural Heart Disease Screening

model-releasesarxiv-cs-lg

arXiv:2604.23385v1 Announce Type: new Abstract: Transthoracic echocardiography is the reference standard for confirming structural heart disease (SHD), but first-line screening is limited by cost, wor

Enhancing molecular dynamics with equivariant machine-learned densities

arXiv:2604.24563v1 Announce Type: cross Abstract: Machine-learning interatomic potentials (MLIPs) have enabled molecular dynamics at near ab initio accuracy, yet remain limited to energies and forces

Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval

arXiv:2604.24469v1 Announce Type: cross Abstract: Content-based image retrieval (CBIR) systems enable users to search images based on visual content instead of relying on metadata. The text domain has

In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions

arXiv:2604.22817v1 Announce Type: cross Abstract: Recent advances in speech-aware language models have coupled strong acoustic encoders with large language models, enabling systems that move beyond tr

safetyarxiv-cs-cl

Intervention-Aware Multiscale Representation Learning from Imaging Phenomics and Perturbation Transcriptomics

arXiv:2604.22832v1 Announce Type: cross Abstract: Microscopy-based phenotypic profiling is scalable for drug discovery but lacks the mechanistic depth of transcriptomics, which remains costly and scar

safetyarxiv-cs-ai

Large language model-enabled automated data extraction for concrete materials informatics

arXiv:2604.22938v1 Announce Type: cross Abstract: The promise of data-driven materials discovery remains constrained by the scarcity of large, high-quality, and accessible experimental datasets. Here,

Agents

Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation

arXiv:2604.23604v1 Announce Type: new Abstract: Understanding the surrounding environment is fundamental in autonomous driving and robotic perception. Distinguishing between known classes and previous

agentsarxiv-cs-cv

LoFi: Location-Aware Fine-Grained Representation Learning for Chest X-ray

arXiv:2603.19451v2 Announce Type: replace-cross Abstract: Fine-grained representation learning is crucial for retrieval and phrase grounding in chest X-rays, where clinically relevant findings are oft

Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling

model-releasesarxiv-cs-cl

arXiv:2604.24715v1 Announce Type: new Abstract: Hybrid sequence models that combine efficient Transformer components with linear sequence modeling blocks are a promising alternative to pure Transforme

Neural Grammatical Error Correction for Romanian

arXiv:2604.23627v1 Announce Type: new Abstract: Resources for Grammatical Error Correction (GEC) in non-English languages are scarce, while available spellcheckers in these languages are mostly limite

Applications

Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data

arXiv:2503.05587v3 Announce Type: replace-cross Abstract: Robustness has become a critical attribute for the deployment of RAG systems in real-world applications. Existing research focuses on robustne

applicationsarxiv-cs-ai

Quantum Kernel Advantage over Classical Collapse in Medical Foundation Model Embeddings

arXiv:2604.24597v1 Announce Type: cross Abstract: We provide evidence of quantum kernel advantage under noiseless simulation in binary insurance classification on MIMIC-CXR chest radiographs using qua

RACANet: Reliability-Aware Crowd Anchor Network for RGB-T Crowd Counting

model-releasesarxiv-cs-cv

arXiv:2604.24543v1 Announce Type: new Abstract: RGB-Thermal (T) crowd counting aims to integrate visible-spectrum and thermal infrared information to improve the robustness of crowd density estimation

RL Token: Bootstrapping Online RL with Vision-Language-Action Models

arXiv:2604.23073v1 Announce Type: new Abstract: Vision-language-action (VLA) models can learn to perform diverse manipulation skills 'out of the box,' but achieving the precision and speed that real-w

safetyarxiv-cs-lg

Seer: Language Instructed Video Prediction with Latent Diffusion Models

arXiv:2303.14897v4 Announce Type: replace Abstract: Imagining the future trajectory is the key for robots to make sound planning and successfully reach their goals. Therefore, text-conditioned video p

safetyarxiv-cs-cv

SGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation

arXiv:2604.22825v1 Announce Type: cross Abstract: Large segmentation foundation models such as the Segment Anything Model (SAM) have reshaped promptable segmentation in natural images, and recent effo

Shape: A Self-Supervised 3D Geometry Foundation Model for Industrial CAD Analysis

model-releasesarxiv-cs-cv

arXiv:2604.22826v1 Announce Type: new Abstract: Industrial CAD workflows require robust, generalizable 3D geometric representations supporting accuracy and explainability. We introduce Shape, a self-s

SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding

arXiv:2511.17411v2 Announce Type: replace-cross Abstract: Robotic Foundation Models (RFMs) hold great promise as generalist, end-to-end systems for robot control. Yet their ability to generalize acros

The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptation

model-releasesarxiv-cs-ai

arXiv:2604.23750v1 Announce Type: cross Abstract: Hypernetwork-based methods such as Doc-to-LoRA internalize a document into an LLM's weights in a single forward pass, but they fail systematically on

Tutorials

Distilling Vision Transformers for Distortion-Robust Representation Learning

arXiv:2604.22529v1 Announce Type: new Abstract: Self-supervised learning has achieved remarkable success in learning visual representations from clean data, yet remains challenging when clean observat

tutorialsarxiv-cs-cv

Estimating Tail Risks in Language Model Output Distributions

arXiv:2604.22167v1 Announce Type: cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale. As a result, the safety of these models is increa

safetyarxiv-cs-ai

Multi-Token Prediction via Self-Distillation

arXiv:2602.06019v2 Announce Type: replace Abstract: Existing techniques for accelerating language model inference, such as speculative decoding, require training auxiliary speculator models and buildi

Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection

model-releasesarxiv-cs-lg

arXiv:2604.22753v1 Announce Type: new Abstract: Scaling laws are used to plan multi-million-dollar training runs, but fitting those laws can itself cost millions. In modern large-scale workflows, asse

Context Unrolling in Omni Models

arXiv:2604.21921v1 Announce Type: new Abstract: We present Omni, a unified multimodal model natively trained on diverse modalities, including text, images, videos, 3D geometry, and hidden representati

Do Masked Autoencoders Improve Downhole Prediction? An Empirical Study on Real Well Drilling Data

arXiv:2604.20909v1 Announce Type: new Abstract: Downhole drilling telemetry presents a fundamental labeling asymmetry: surface sensor data are generated continuously at 1~Hz, while labeled downhole me

Encoder-Free Human Motion Understanding via Structured Motion Descriptions

arXiv:2604.21668v1 Announce Type: new Abstract: The world knowledge and reasoning capabilities of text-based large language models (LLMs) are advancing rapidly, yet current approaches to human motion

safetyarxiv-cs-cv

Listen and Chant Before You Read: The Ladder of Beauty in LM Pre-Training

arXiv:2604.21265v1 Announce Type: new Abstract: We show that pre-training a Transformer on music before language significantly accelerates language acquisition. Using piano performances (MAESTRO datas

Serialisation Strategy Matters: How FHIR Data Format Affects LLM Medication Reconciliation

model-releasesarxiv-cs-ai

arXiv:2604.21076v1 Announce Type: cross Abstract: Medication reconciliation at clinical handoffs is a high-stakes, error-prone process. Large language models are increasingly proposed to assist with t

Supervised Learning Has a Necessary Geometric Blind Spot: Theory, Consequences, and Minimal Repair

arXiv:2604.21395v1 Announce Type: cross Abstract: We prove that empirical risk minimisation (ERM) imposes a necessary geometric constraint on learned representations: any encoder that minimises superv

safetyarxiv-cs-ai

'This Wasn't Made for Me': Recentering User Experience and Emotional Impact in the Evaluation of ASR Bias

arXiv:2604.21148v1 Announce Type: new Abstract: Studies on bias in Automatic Speech Recognition (ASR) tend to focus on reporting error rates for speakers of underrepresented dialects, yet less researc

safetyarxiv-cs-cl

Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text

model-releasesarxiv-cs-cl

arXiv:2604.20051v1 Announce Type: new Abstract: Self-play has recently emerged as a promising paradigm to train Large Language Models (LLMs). In self-play, the target LLM creates the task input (e.g.,

23 Apr 2026

CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values

model-releasesarxiv-cs-cl

arXiv:2509.03740v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) like CLIP have shown impressive zero-shot and few-shot learning capabilities across diverse applications. Howeve

23 Apr 2026

Applications

Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs