Search: “pretraining”

arXiv:2607.24516v1 Announce Type: cross Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely he

arXiv:2607.24112v1 Announce Type: new Abstract: We introduce State Transition Pretraining (STP) as a new scaling axis for GUI agents. During the STP stage, we continually pretrain a unified multimodal

arXiv:2607.24519v1 Announce Type: cross Abstract: Pretrained EEG foundation models are increasingly proposed for clinical decoding, but their transfer across populations and robustness to negative con

arXiv:2607.04081v1 Announce Type: new Abstract: In-context learning (ICL) has emerged as a central capability of pretrained language models, yet its theoretical analysis has focused primarily on causa

arXiv:2607.04988v1 Announce Type: new Abstract: Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through

arXiv:2602.06285v2 Announce Type: replace Abstract: Recent research in geospatial machine learning has demonstrated that models pretrained with self-supervised learning on Earth observation data can p

arXiv:2602.16249v2 Announce Type: replace Abstract: Self-supervised pretraining has transformed computer vision by enabling data-efficient fine-tuning, yet high-resolution pretraining typically requir

arXiv:2606.04010v1 Announce Type: cross Abstract: Brain foundation models (BFMs) are self-supervised Transformers pretrained on fMRI data. We posit that these models should capture each subject's cogn

arXiv:2605.17379v1 Announce Type: cross Abstract: Large language models pretrained on general-domain corpora often exhibit tokenization inefficiencies when applied to specialized domains. Although con

arXiv:2605.10759v1 Announce Type: cross Abstract: Diffusion and flow-matching models scale because pretraining is supervised regression: a clean sample is noised analytically, and a model regresses ag

arXiv:2604.27987v1 Announce Type: new Abstract: Fine-tuning pretrained models has become a standard approach to adapting pretrained knowledge to improve the accuracy on new sparse, imbalance datasets.

arXiv:2509.25424v5 Announce Type: replace-cross Abstract: Reinforcement learning fine-tuning (RLFT) is a dominant paradigm for improving pretrained policies for downstream tasks. These pretrained poli

arXiv:2604.22557v1 Announce Type: cross Abstract: The emergence of large-scale pretrained foundation models has transformed computer vision, enabling strong performance across diverse downstream tasks

arXiv:2604.22686v1 Announce Type: new Abstract: We present SS3D, a web-scale SfM-based self-supervision pretraining pipeline for feed-forward 3D estimation from monocular video. Our model jointly pred

arXiv:2604.11679v1 Announce Type: new Abstract: Clinical deployment of automated brain MRI analysis faces a fundamental challenge: clinical data is heterogeneous and noisy, and high-quality labels are

arXiv:2610.15677v1 Announce Type: new Abstract: Continual learning aims to adapt pretrained large language models (LLMs) to evolving data while preserving their pretrained capabilities. A central chal

arXiv:2610.16355v1 Announce Type: new Abstract: Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for wo

arXiv:2610.10524v1 Announce Type: new Abstract: Highly compressed video autoencoders offer an effective way to accelerate video diffusion models, as the Diffusion Transformer (DiT) operates on far few

arXiv:2610.08183v1 Announce Type: cross Abstract: Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performan

arXiv:2610.07074v1 Announce Type: cross Abstract: Estimating a decision threshold requires locating observations near an unknown boundary. We study how gradient-based pretraining learns this statistic

arXiv:2602.14512v3 Announce Type: replace Abstract: Autoregressive pretraining has been key to the scalability of large language models, yet medical generative foundation models remain predominantly b

arXiv:2610.02657v1 Announce Type: new Abstract: Converting a pretrained autoregressive (AR) model to a diffusion language model (dLLM) enables parallel generation without pretraining a new model. Publ

arXiv:2610.02260v1 Announce Type: new Abstract: Flow matching models excel at generative modeling, and many downstream applications require their samples to satisfy prescribed constraints, such as obs

arXiv:2609.39387v1 Announce Type: new Abstract: Neural operators increasingly benefit from pretraining on numerical simulations, yet adapting them for real-world prediction remains challenging. We int

arXiv:2609.35947v1 Announce Type: new Abstract: Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the

arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrain

arXiv:2609.35348v1 Announce Type: new Abstract: Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI, a pretrained data

arXiv:2601.04301v3 Announce Type: replace Abstract: As frontier AI systems are pretrained on web-scale data, test set contamination has become a critical concern for accurately assessing their capabil

arXiv:2609.35405v1 Announce Type: new Abstract: Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but th

arXiv:2609.32183v1 Announce Type: new Abstract: High-resolution plant imaging enables detailed characterization of plant morphology, but dense scientific analysis remains limited by costly pixel-level

arXiv:2511.22245v2 Announce Type: replace Abstract: Personalizing text-to-image diffusion models extends pretrained models to represent novel user-specific concepts from only a few reference images. H

arXiv:2609.30391v1 Announce Type: new Abstract: Existing in-context reinforcement learning methods mainly pretrain Transformers with supervised behavior-prediction objectives. This enables task infere

arXiv:2609.31394v1 Announce Type: cross Abstract: World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate

arXiv:2609.30316v1 Announce Type: cross Abstract: Language models used in financial backtests suffer from look-ahead bias, as a model trained on text published after the study period has already obser

arXiv:2609.29792v1 Announce Type: cross Abstract: We present TimeBraid, a series of unified time-series and language models that align pretrained language models and pretrained time-series foundation

arXiv:2609.29060v1 Announce Type: cross Abstract: Transformers achieve remarkable performance by jointly learning broad families of tasks during pretraining and adapting to unseen tasks from only a sh

arXiv:2609.22271v1 Announce Type: new Abstract: Multimodal stroke recurrence prediction requires effective integration of heterogeneous clinical and imaging data, yet modality imbalance often causes m

arXiv:2609.22146v1 Announce Type: cross Abstract: We study how post-training changes the weights of Large Language Models (LLMs) relative to their pretrained weights. Across 12 post-training chains wi

arXiv:2609.15037v1 Announce Type: cross Abstract: Low-rank gradient projection reduces the optimizer-state memory cost of large language model (LLM) pretraining, but the steps and wall-clock time need

arXiv:2609.14759v1 Announce Type: cross Abstract: Alignment applied after pretraining is shallow in a measurable way: a single direction in a model's residual stream can be edited out, and the model s

arXiv:2609.12316v1 Announce Type: new Abstract: Collecting high-quality robot data remains a fundamental challenge for training robot foundation models. Task and motion planning (TAMP) offers a scalab

arXiv:2609.10796v1 Announce Type: new Abstract: Pretrained diabetic retinopathy (DR) prediction models differ in their input fields, serialization formats, preprocessing requirements, and output seman

arXiv:2609.10715v1 Announce Type: new Abstract: We introduce NCP-ArchPreview, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP). Alongsid

arXiv:2609.06807v1 Announce Type: new Abstract: In this work, we comprehensively evaluate three popular feature-extraction paradigms in AI-based neuroimaging modeling: (1) computation of anatomical su

arXiv:2609.06912v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are commonly pretrained on large collections of procedurally generated synthetic tasks, yet it remains unclear how well

arXiv:2609.00746v1 Announce Type: new Abstract: Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that req