Search: “pretraining”

arXiv:2610.10324v1 Announce Type: new Abstract: Pretrained models for cell and nuclear instance segmentation differ substantially in architecture, pretraining data and objectives, parameter count, inf

arXiv:2610.08400v1 Announce Type: cross Abstract: Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to gen

arXiv:2609.40295v1 Announce Type: new Abstract: Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tok

arXiv:2609.39386v1 Announce Type: new Abstract: Pretrained time-series foundation models (TSFMs) are evaluated as forecasters of future values, yet for sparse series many decisions depend only on whic

arXiv:2609.27232v1 Announce Type: new Abstract: Scaling laws have guided large-model development in computer vision and natural language processing, but the relationships among data, model size, and c

arXiv:2609.24787v1 Announce Type: new Abstract: Forest inventories increasingly rely on artificial intelligence (AI) models to derive forest attributes from large-scale 3D point clouds. Current models

arXiv:2609.09338v1 Announce Type: new Abstract: Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distri

arXiv:2608.06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they r

arXiv:2608.01751v1 Announce Type: new Abstract: Geospatial foundation models (GeoFMs), pretrained on large-scale geospatial data such as Earth observation (EO), climate, and weather data, have shown p

arXiv:2606.09396v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement le

arXiv:2602.11137v2 Announce Type: replace-cross Abstract: Large language models are typically trained in two broad phases: pretraining to produce a base model, followed by further training to improve

arXiv:2605.28544v1 Announce Type: new Abstract: Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to vision-language models pretrained primaril

arXiv:2605.19133v1 Announce Type: cross Abstract: Self-supervised learning (SSL) is now a standard way to pretrain medical image models, but performance is still mostly judged by downstream accuracy.

arXiv:2610.06450v1 Announce Type: new Abstract: Foundation models (FMs) are increasingly being developed for general time series and physiological signals, yet their transferability to downstream phys

arXiv:2602.02180v2 Announce Type: replace Abstract: Linearizing pretrained large language models (LLMs) primarily relies on intra-layer hybrid attention mechanisms to alleviate the quadratic complexit

arXiv:2610.03432v1 Announce Type: cross Abstract: Online data selection has demonstrated substantial efficiency gains for LLM pretraining by training on the most valuable candidates within each batch.

arXiv:2610.00414v1 Announce Type: new Abstract: Foundation models pretrained on large-scale datasets demonstrate strong transferability to medical imaging tasks. However, understanding how their laten

arXiv:2609.36426v1 Announce Type: cross Abstract: A detector pretrained on a broad corpus is fine-tuned on a narrow domain, its in-domain accuracy improves, and it ships. We ask what happens meanwhile

arXiv:2609.22973v1 Announce Type: new Abstract: Adapting a pretrained Vision-Language-Action (VLA) model to a new robot, environment, or task requires demonstrations that are collected locally and oft

arXiv:2406.02465v2 Announce Type: replace-cross Abstract: Can pretrained models generalize to new datasets without any retraining? We deploy pretrained image models on datasets they were not trained f

arXiv:2609.03350v1 Announce Type: cross Abstract: Pretraining data for Armenian, a morphologically rich and low-resource language, is scarce, and no open Armenian LLM has been released with the data a

arXiv:2609.01041v1 Announce Type: cross Abstract: We introduce ViTAMINS, a method that integrates synthetic hard negatives into unsupervised vision transformer pretraining to improve representation qu

arXiv:2608.24093v1 Announce Type: new Abstract: Self-supervised representation learning for 4D point cloud videos is challenging because annotations are costly and reconstruction-based pretraining can

arXiv:2608.12989v1 Announce Type: new Abstract: Pretrained tabular foundation models have demonstrated strong predictive capability; however, their application to large-scale datasets remains constrai

arXiv:2608.12327v1 Announce Type: cross Abstract: Multilingual pretrained models nominally support Nepali, yet no controlled benchmark has compared them under a single fine-tuning protocol. We fine-tu

arXiv:2608.00586v1 Announce Type: new Abstract: Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pret

arXiv:2607.24516v1 Announce Type: cross Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely he

arXiv:2607.24112v1 Announce Type: new Abstract: We introduce State Transition Pretraining (STP) as a new scaling axis for GUI agents. During the STP stage, we continually pretrain a unified multimodal

arXiv:2607.24519v1 Announce Type: cross Abstract: Pretrained EEG foundation models are increasingly proposed for clinical decoding, but their transfer across populations and robustness to negative con

arXiv:2607.04081v1 Announce Type: new Abstract: In-context learning (ICL) has emerged as a central capability of pretrained language models, yet its theoretical analysis has focused primarily on causa

arXiv:2607.04988v1 Announce Type: new Abstract: Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through

arXiv:2602.06285v2 Announce Type: replace Abstract: Recent research in geospatial machine learning has demonstrated that models pretrained with self-supervised learning on Earth observation data can p

arXiv:2602.16249v2 Announce Type: replace Abstract: Self-supervised pretraining has transformed computer vision by enabling data-efficient fine-tuning, yet high-resolution pretraining typically requir

arXiv:2606.04010v1 Announce Type: cross Abstract: Brain foundation models (BFMs) are self-supervised Transformers pretrained on fMRI data. We posit that these models should capture each subject's cogn

arXiv:2605.17379v1 Announce Type: cross Abstract: Large language models pretrained on general-domain corpora often exhibit tokenization inefficiencies when applied to specialized domains. Although con

arXiv:2605.10759v1 Announce Type: cross Abstract: Diffusion and flow-matching models scale because pretraining is supervised regression: a clean sample is noised analytically, and a model regresses ag

arXiv:2604.27987v1 Announce Type: new Abstract: Fine-tuning pretrained models has become a standard approach to adapting pretrained knowledge to improve the accuracy on new sparse, imbalance datasets.

arXiv:2509.25424v5 Announce Type: replace-cross Abstract: Reinforcement learning fine-tuning (RLFT) is a dominant paradigm for improving pretrained policies for downstream tasks. These pretrained poli

arXiv:2604.22557v1 Announce Type: cross Abstract: The emergence of large-scale pretrained foundation models has transformed computer vision, enabling strong performance across diverse downstream tasks

arXiv:2604.22686v1 Announce Type: new Abstract: We present SS3D, a web-scale SfM-based self-supervision pretraining pipeline for feed-forward 3D estimation from monocular video. Our model jointly pred

arXiv:2604.11679v1 Announce Type: new Abstract: Clinical deployment of automated brain MRI analysis faces a fundamental challenge: clinical data is heterogeneous and noisy, and high-quality labels are

arXiv:2610.10524v1 Announce Type: new Abstract: Highly compressed video autoencoders offer an effective way to accelerate video diffusion models, as the Diffusion Transformer (DiT) operates on far few

arXiv:2610.08183v1 Announce Type: cross Abstract: Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performan

arXiv:2610.07074v1 Announce Type: cross Abstract: Estimating a decision threshold requires locating observations near an unknown boundary. We study how gradient-based pretraining learns this statistic

arXiv:2602.14512v3 Announce Type: replace Abstract: Autoregressive pretraining has been key to the scalability of large language models, yet medical generative foundation models remain predominantly b

arXiv:2610.02657v1 Announce Type: new Abstract: Converting a pretrained autoregressive (AR) model to a diffusion language model (dLLM) enables parallel generation without pretraining a new model. Publ