Search: “pretraining”

Do Masked Autoencoders Improve Downhole Prediction? An Empirical Study on Real Well Drilling Data

arXiv:2604.20909v1 Announce Type: new Abstract: Downhole drilling telemetry presents a fundamental labeling asymmetry: surface sensor data are generated continuously at 1~Hz, while labeled downhole me

researcharxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Encoder-Free Human Motion Understanding via Structured Motion Descriptions

DGX agent

arXiv:2604.21668v1 Announce Type: new Abstract: The world knowledge and reasoning capabilities of text-based large language models (LLMs) are advancing rapidly, yet current approaches to human motion

safetyarxiv-cs-cv

24 Apr 2026

Listen and Chant Before You Read: The Ladder of Beauty in LM Pre-Training

arXiv:2604.21265v1 Announce Type: new Abstract: We show that pre-training a Transformer on music before language significantly accelerates language acquisition. Using piano performances (MAESTRO datas

Serialisation Strategy Matters: How FHIR Data Format Affects LLM Medication Reconciliation

model-releasesarxiv-cs-ai

arXiv:2604.21076v1 Announce Type: cross Abstract: Medication reconciliation at clinical handoffs is a high-stakes, error-prone process. Large language models are increasingly proposed to assist with t

Supervised Learning Has a Necessary Geometric Blind Spot: Theory, Consequences, and Minimal Repair

arXiv:2604.21395v1 Announce Type: cross Abstract: We prove that empirical risk minimisation (ERM) imposes a necessary geometric constraint on learned representations: any encoder that minimises superv

'This Wasn't Made for Me': Recentering User Experience and Emotional Impact in the Evaluation of ASR Bias

arXiv:2604.21148v1 Announce Type: new Abstract: Studies on bias in Automatic Speech Recognition (ASR) tend to focus on reporting error rates for speakers of underrepresented dialects, yet less researc

Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text

arXiv:2604.20051v1 Announce Type: new Abstract: Self-play has recently emerged as a promising paradigm to train Large Language Models (LLMs). In self-play, the target LLM creates the task input (e.g.,

CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values

arXiv:2509.03740v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) like CLIP have shown impressive zero-shot and few-shot learning capabilities across diverse applications. Howeve

Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs

arXiv:2604.19765v1 Announce Type: cross Abstract: Recent work identifies a sparse set of 'hallucination neurons' (H-neurons), less than 0.1% of feed-forward network neurons, that reliably predict when

applicationsarxiv-cs-ai

From Diffusion to Flow: Efficient Motion Generation in MotionGPT3

arXiv:2603.26747v2 Announce Type: replace Abstract: Recent text-driven motion generation methods span both discrete token-based approaches and continuous-latent formulations. MotionGPT3 exemplifies th

PASTA: A Patch-Agnostic Twofold-Stealthy Backdoor Attack on Vision Transformers

arXiv:2604.20047v1 Announce Type: new Abstract: Vision Transformers (ViTs) have achieved remarkable success across vision tasks, yet recent studies show they remain vulnerable to backdoor attacks. Exi

The Existential Theory of Research: Why Discovery Is Hard

arXiv:2604.19810v1 Announce Type: new Abstract: Can scientific discovery be made arbitrarily easy by choosing the right representation, collecting enough data, and deploying sufficiently powerful algo

Benign Overfitting in Adversarial Training for Vision Transformers

arXiv:2604.19724v1 Announce Type: cross Abstract: Despite the remarkable success of Vision Transformers (ViTs) across a wide range of vision tasks, recent studies have revealed that they remain vulner

applicationsarxiv-cs-ai

Bridging Foundation Models and ASTM Metallurgical Standards for Automated Grain Size Estimation from Microscopy Images

model-releasesarxiv-cs-cv

arXiv:2604.18957v1 Announce Type: new Abstract: Extracting standardized metallurgical metrics from microscopy images remains challenging due to complex grain morphology and the data demands of supervi

Co-Refine: AI-Powered Tool Supporting Qualitative Analysis

arXiv:2604.19309v1 Announce Type: cross Abstract: Qualitative coding relies on a researcher's application of codes to textual data. As coding proceeds across large datasets, interpretations of codes o

researcharxiv-cs-ai

Local Ai

COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition

arXiv:2503.07259v2 Announce Type: replace-cross Abstract: The goal of creating intelligent, human-centered wearable systems for continuous activity understanding faces a fundamental trade-off: Egocent

local-aiarxiv-cs-ai

Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval

arXiv:2604.19135v1 Announce Type: new Abstract: This paper presents the first exploration of text-to-image diffusion models for zero-shot sketch-based 3D shape retrieval (ZS-SBSR). Existing sketch-bas

safetyarxiv-cs-cv

LASER: Learning Active Sensing for Continuum Field Reconstruction

arXiv:2604.19355v1 Announce Type: cross Abstract: High-fidelity measurements of continuum physical fields are essential for scientific discovery and engineering design but remain challenging under spa

LiveVLN: Breaking the Stop-and-Go Loop in Vision-Language Navigation

model-releasesarxiv-cs-ro

arXiv:2604.19536v1 Announce Type: new Abstract: Recent navigation systems achieve strong benchmark results, yet real-world deployment often remains visibly stop-and-go. This bottleneck arises because

ReImagine: Rethinking Controllable High-Quality Human Video Generation via Image-First Synthesis

arXiv:2604.19720v1 Announce Type: new Abstract: Human video generation remains challenging due to the difficulty of jointly modeling human appearance, motion, and camera viewpoint under limited multi-

SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing

arXiv:2604.19587v1 Announce Type: new Abstract: Traditional photographic image editing typically requires users to possess sufficient aesthetic understanding to provide appropriate instructions for ad

Structure-Semantic Decoupled Modulation of Global Geospatial Embeddings for High-Resolution Remote Sensing Mapping

arXiv:2604.19591v1 Announce Type: new Abstract: Fine-grained high-resolution remote sensing mapping typically relies on localized visual features, which restricts cross-domain generalizability and oft

User Simulation in the Era of Generative AI: User Modeling, Synthetic Data Generation, and System Evaluation

arXiv:2501.04410v2 Announce Type: replace Abstract: User simulation is an emerging interdisciplinary topic with multiple critical applications in the era of Generative AI. It involves creating an inte

When Active Learning Falls Short: An Empirical Study on Chemical Reaction Extraction

arXiv:2604.19335v1 Announce Type: new Abstract: The rapid growth of chemical literature has generated vast amounts of unstructured data, where reaction information is particularly valuable for applica

researcharxiv-cs-lg

A Text-To-Text Alignment Algorithm for Better Evaluation of Modern Speech Recognition Systems

arXiv:2509.24478v2 Announce Type: replace Abstract: Modern neural networks have greatly improved performance across speech recognition benchmarks. However, gains are often driven by frequent words wit

Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning

arXiv:2604.16622v1 Announce Type: new Abstract: Backchannels (e.g., `yeah', `mhm', and `right') are short, non-interruptive feedback signals whose lexical form and prosody jointly convey pragmatic mea

AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusion

arXiv:2604.17818v1 Announce Type: new Abstract: Reconstructing 3D human motion and human-object interactions (HOI) from Internet videos is a fundamental step toward building large-scale datasets of hu

Arch: An AI-Native Hardware Description Language for Register-Transfer Clocked Hardware Design

arXiv:2604.05983v2 Announce Type: replace-cross Abstract: We present Arch (AI-native Register-transfer Clocked Hardware), a hardware description language for micro-architecture specification and AI-as

BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation

arXiv:2604.16514v1 Announce Type: new Abstract: Autoregressive vision-language models (VLMs) deliver strong multimodal capability, but their token-by-token decoding imposes a fundamental inference bot

C-GenReg: Training-Free 3D Point Cloud Registration by Multi-View-Consistent Geometry-to-Image Generation with Probabilistic Modalities Fusion

arXiv:2604.16680v1 Announce Type: new Abstract: We introduce C-GenReg, a training-free framework for 3D point cloud registration that leverages the complementary strengths of world-scale generative pr

safetyarxiv-cs-cv

Closing the Modality Reasoning Gap for Speech Large Language Models

arXiv:2601.05543v2 Announce Type: replace Abstract: Although Speech Large Language Models have achieved notable progress, a substantial modality reasoning gap remains: their reasoning performance on s

Data Compressibility Quantifies LLM Memorization

arXiv:2507.06056v4 Announce Type: replace Abstract: Large Language Models (LLMs) are known to memorize portions of their training data, sometimes even reproduce content verbatim when prompted appropri

Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings

arXiv:2505.15353v3 Announce Type: replace Abstract: Log-likelihood vectors define a common space for comparing language models as probability distributions, enabling unified comparisons across heterog

Foundation Model for Cardiac Time Series via Masked Latent Attention

arXiv:2603.26475v2 Announce Type: replace Abstract: Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent fou

safetyarxiv-cs-lg

MambaKick: Early Penalty Direction Prediction from HAR Embeddings

arXiv:2604.16588v1 Announce Type: new Abstract: Penalty kicks in soccer are decided under extreme time constraints, where goalkeepers benefit from anticipating shot direction from the kickers motion b

applicationsarxiv-cs-cv

Multi-Scale Reversible Chaos Game Representation: A Unified Framework for Sequence Classification

arXiv:2604.18477v1 Announce Type: new Abstract: Biological classification with interpretability remains a challenging task. For this, we introduce a novel encoding framework, Multi-Scale Reversible Ch

researcharxiv-cs-lg

Tutorials

OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction

arXiv:2604.16878v1 Announce Type: new Abstract: Early prediction of severe clinical deterioration and remaining length of stay can enable timely intervention and better resource allocation in high-acu

tutorialsarxiv-cs-lg

ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition

arXiv:2503.21248v3 Announce Type: replace Abstract: Large language models (LLMs) have shown potential in assisting scientific research, yet their ability to discover high-quality research hypotheses r

Synthetic Data Generation for Training Diversified Commonsense Reasoning Models

arXiv:2603.18361v2 Announce Type: replace Abstract: Conversational agents are required to respond to their users not only with high quality (i.e. commonsense bearing) responses, but also considering m

TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation

model-releasesarxiv-cs-cv

arXiv:2603.02972v2 Announce Type: replace Abstract: Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch: V

The Gait Signature of Frailty: Transfer Learning based Deep Gait Models for Scalable Frailty Assessment

arXiv:2603.24434v2 Announce Type: replace Abstract: Frailty is a condition in aging medicine characterized by diminished physiological reserve and increased vulnerability to stressors. However, frailt

The Role of Vocabularies in Learning Sparse Representations for Ranking

arXiv:2509.16621v2 Announce Type: replace-cross Abstract: Learned Sparse Retrieval (LSR) such as SPLADE has growing interest for effective semantic 1st stage matching while enjoying the efficiency of

applicationsarxiv-cs-cl

Transition-Matrix Regularization for Next Dialogue Act Prediction in Counselling Conversations

arXiv:2604.18539v1 Announce Type: new Abstract: This paper studies how empirical dialogue-flow statistics can be incorporated into Next Dialogue Act Prediction (NDAP). A KL regularization term is prop

Understanding the Prompt Sensitivity

arXiv:2604.18389v1 Announce Type: new Abstract: Prompt sensitivity, which refers to how strongly the output of a large language model (LLM) depends on the exact wording of its input prompt, raises con

Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic

arXiv:2510.17001v2 Announce Type: replace Abstract: Large language models (LLMs) often encode word-form variation (e.g., walk vs. walked) as linear directions in the embedding space. However, standard

Where Do Self-Supervised Speech Models Become Unfair?

arXiv:2604.18249v1 Announce Type: new Abstract: Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing wh

Comparison Study: Glacier Calving Front Delineation in Synthetic Aperture Radar Images With Deep Learning

arXiv:2501.05281v2 Announce Type: replace Abstract: Continuous monitoring of glacier calving fronts is essential for sea level rise projections. This study benchmarks Deep Learning systems for front d

20 Apr 2026

Olmo Hybrid: From Theory to Practice and Back

arXiv:2604.03444v3 Announce Type: replace-cross Abstract: Recent work has demonstrated the potential of non-transformer language models, especially linear recurrent neural networks (RNNs) and hybrid m