Search: “pretraining”

arXiv:2605.28060v1 Announce Type: new Abstract: Explaining the predictions of neural models in clinical NLP remains a significant challenge, especially for complex tasks involving long, unstructured m

arXiv:2605.26790v1 Announce Type: new Abstract: Low-thrust trajectory design relies heavily on repeated evaluations of fuel consumption and transfer feasibility, which require expensive optimal contro

arXiv:2507.16116v2 Announce Type: replace Abstract: The rapid advancement of video diffusion models has been hindered by fundamental limitations in temporal modeling, particularly the rigid synchroniz

arXiv:2509.15105v3 Announce Type: replace Abstract: Time series forecasting (TSF) is critical in domains like energy, finance, healthcare, and logistics, requiring models that generalize across divers

arXiv:2507.03122v2 Announce Type: replace-cross Abstract: This study investigates the feasibility and performance of federated learning (FL) for multi-label ICD code classification using clinical note

arXiv:2605.14893v1 Announce Type: cross Abstract: Contrastively pre-trained Vision-Language Models (VLMs) serve as powerful feature extractors. Yet, their shared latent spaces are prone to structural

arXiv:2403.18151v2 Announce Type: replace-cross Abstract: Objective: Cytology plays a crucial role in lung cancer diagnosis. Pulmonary cytology involves cell morphological characterization in the spec

arXiv:2604.17856v1 Announce Type: new Abstract: Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automatin

arXiv:2604.15756v1 Announce Type: new Abstract: Vision-language models (VLMs) such as CLIP exhibit strong Out-of-distribution (OOD) detection capabilities by aligning visual and textual representation

arXiv:2610.14871v1 Announce Type: new Abstract: By predicting future states, latent world models provide rich self-supervision for end-to-end driving, but often inherit dense features from pretrained

arXiv:2610.12832v1 Announce Type: new Abstract: Pretrained vision-language models (VLMs) already exhibit strong visual, spatial and reasoning capabilities relevant to robotic manipulation. The key cha

arXiv:2610.07997v1 Announce Type: new Abstract: Molecular encoders contrastively pretrained on paired molecule-morphology data, such as CLOOME and CellCLIP, have been proposed as cheap surrogates for

arXiv:2512.21004v2 Announce Type: replace Abstract: Recent advances in pretraining general foundation models have significantly improved performance across diverse downstream tasks. While autoregressi

arXiv:2609.13283v1 Announce Type: cross Abstract: We present a multimodal foundation model for lunar remote sensing, pretrained from scratch on SomBench, a geographically partitioned corpus of nearly

arXiv:2609.14970v1 Announce Type: new Abstract: Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing

arXiv:2606.21358v1 Announce Type: new Abstract: Supervised video pretraining is a common transfer learning practice for improving downstream action recognition performance. However, it requires large-

arXiv:2610.17914v1 Announce Type: new Abstract: Post-hoc linearization is an attractive way to accelerate pretrained Video Diffusion Transformers (Video DiTs), yet existing methods largely treat the q

arXiv:2610.10324v1 Announce Type: new Abstract: Pretrained models for cell and nuclear instance segmentation differ substantially in architecture, pretraining data and objectives, parameter count, inf

arXiv:2610.08400v1 Announce Type: cross Abstract: Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to gen

arXiv:2609.40295v1 Announce Type: new Abstract: Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tok

arXiv:2609.39386v1 Announce Type: new Abstract: Pretrained time-series foundation models (TSFMs) are evaluated as forecasters of future values, yet for sparse series many decisions depend only on whic

arXiv:2609.27232v1 Announce Type: new Abstract: Scaling laws have guided large-model development in computer vision and natural language processing, but the relationships among data, model size, and c

arXiv:2609.24787v1 Announce Type: new Abstract: Forest inventories increasingly rely on artificial intelligence (AI) models to derive forest attributes from large-scale 3D point clouds. Current models

arXiv:2609.09338v1 Announce Type: new Abstract: Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distri

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

arXiv:2608.06993v1 Announce Type: cross Abstract: Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they r

arXiv:2608.01751v1 Announce Type: new Abstract: Geospatial foundation models (GeoFMs), pretrained on large-scale geospatial data such as Earth observation (EO), climate, and weather data, have shown p

arXiv:2606.09396v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement le

arXiv:2602.11137v2 Announce Type: replace-cross Abstract: Large language models are typically trained in two broad phases: pretraining to produce a base model, followed by further training to improve

arXiv:2605.28544v1 Announce Type: new Abstract: Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to vision-language models pretrained primaril

arXiv:2605.19133v1 Announce Type: cross Abstract: Self-supervised learning (SSL) is now a standard way to pretrain medical image models, but performance is still mostly judged by downstream accuracy.

arXiv:2610.20494v1 Announce Type: new Abstract: Pretrained representations are increasingly used to share and reuse information from medical images without exposing the original scans, but it is uncle

arXiv:2610.06450v1 Announce Type: new Abstract: Foundation models (FMs) are increasingly being developed for general time series and physiological signals, yet their transferability to downstream phys

arXiv:2602.02180v2 Announce Type: replace Abstract: Linearizing pretrained large language models (LLMs) primarily relies on intra-layer hybrid attention mechanisms to alleviate the quadratic complexit

arXiv:2610.03432v1 Announce Type: cross Abstract: Online data selection has demonstrated substantial efficiency gains for LLM pretraining by training on the most valuable candidates within each batch.

arXiv:2610.00414v1 Announce Type: new Abstract: Foundation models pretrained on large-scale datasets demonstrate strong transferability to medical imaging tasks. However, understanding how their laten

arXiv:2609.36426v1 Announce Type: cross Abstract: A detector pretrained on a broad corpus is fine-tuned on a narrow domain, its in-domain accuracy improves, and it ships. We ask what happens meanwhile

arXiv:2609.22973v1 Announce Type: new Abstract: Adapting a pretrained Vision-Language-Action (VLA) model to a new robot, environment, or task requires demonstrations that are collected locally and oft

arXiv:2406.02465v2 Announce Type: replace-cross Abstract: Can pretrained models generalize to new datasets without any retraining? We deploy pretrained image models on datasets they were not trained f

arXiv:2609.03350v1 Announce Type: cross Abstract: Pretraining data for Armenian, a morphologically rich and low-resource language, is scarce, and no open Armenian LLM has been released with the data a

arXiv:2609.01041v1 Announce Type: cross Abstract: We introduce ViTAMINS, a method that integrates synthetic hard negatives into unsupervised vision transformer pretraining to improve representation qu

arXiv:2608.24093v1 Announce Type: new Abstract: Self-supervised representation learning for 4D point cloud videos is challenging because annotations are costly and reconstruction-based pretraining can

arXiv:2608.12989v1 Announce Type: new Abstract: Pretrained tabular foundation models have demonstrated strong predictive capability; however, their application to large-scale datasets remains constrai

arXiv:2608.12327v1 Announce Type: cross Abstract: Multilingual pretrained models nominally support Nepali, yet no controlled benchmark has compared them under a single fine-tuning protocol. We fine-tu

arXiv:2608.00586v1 Announce Type: new Abstract: Despite the widespread adoption of foundation models as feature extractors for medical imaging, relatively little is understood about how different pret

arXiv:2607.24516v1 Announce Type: cross Abstract: While data curation for Vision Language Models (VLMs) is increasingly active, public practice for constructing pretraining mixtures remains largely he