Search: “pretraining”

arXiv:2410.19842v2 Announce Type: replace-cross Abstract: Contrastive learning yields impressive results for self-supervision in computer vision. The approach relies on the creation of positive pairs,

arXiv:2605.23840v1 Announce Type: new Abstract: Mueller matrix imaging provides rich, physically meaningful contrast for biomedical tissue analysis, but supervised learning is hindered by scarce dense

arXiv:2510.25897v2 Announce Type: replace Abstract: The default paradigm of post-training text-to-image generators includes post-hoc selection of generated images, and subsequent training with one rew

arXiv:2605.18552v1 Announce Type: new Abstract: Classifying protein topology is essential for deciphering biological function, but progress is held back by the lack of large-scale benchmarks that avoi

arXiv:2511.08704v2 Announce Type: replace Abstract: This paper investigates the scaling properties of autoregressive next-pixel prediction, a simple, end-to-end yet under-explored framework for unifie

arXiv:2602.19710v2 Announce Type: replace Abstract: Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level percept

arXiv:2605.14747v1 Announce Type: cross Abstract: Recent advances in multimodal large language models have driven growing interest in graphical user interface (GUI) agents, yet their generalization re

arXiv:2605.12438v1 Announce Type: new Abstract: When adapting an encoder to a new domain, the standard approach is to continue training with Masked Language Modeling (MLM). We show that temporarily sw

arXiv:2605.11034v1 Announce Type: cross Abstract: We present MambaNetBurst, a compact tokenizer-free byte-level sequence classifier for network burst classification based on a Mamba-2 backbone. In con

arXiv:2605.11408v1 Announce Type: cross Abstract: Tabular data forms the backbone of high-stakes decision systems in finance, healthcare, and beyond. Yet industrial tabular datasets are inherently dif

arXiv:2510.25356v2 Announce Type: replace Abstract: In the U.S. judicial system, a widespread approach to legal interpretation entails assessing how a legal text would be understood by an `ordinary' s

arXiv:2605.12236v1 Announce Type: cross Abstract: Fine-tuning pre-trained robot policies with reinforcement learning (RL) often inherits the bottlenecks introduced by pre-training with behavioral clon

arXiv:2605.02968v1 Announce Type: new Abstract: We introduce a finite-size gradient-transport framework for real language-model training, based on five observables (D,z,eta,elta,v_{rel}) that separate

arXiv:2605.03968v1 Announce Type: new Abstract: Accurate school detection is essential for supporting education initiatives, including infrastructure planning and expanding internet connectivity to un

arXiv:2604.27039v1 Announce Type: new Abstract: Token serves as the fundamental unit of computation in modern autoregressive models, and generation length directly influences both inference cost and r

arXiv:2507.21545v3 Announce Type: replace Abstract: Robotic task planning in real-world environments requires reasoning over implicit constraints from language and vision. While LLMs and VLMs offer st

arXiv:2511.09282v3 Announce Type: replace-cross Abstract: Significant progress has been made in spoken question answering (SQA) in recent years. However, many existing methods, including large audio l

We've gone even farther: Nemotron 3 Super is 120B and pretrained on 25T tokens in NVFP4. Nemotron 3 Ultra is ~500B and also pretrained in NVFP4. Accelerated computing means we rethink every aspect of

arXiv:2512.14751v3 Announce Type: replace-cross Abstract: Finetuning pretrained large language models (LLMs) has become the standard paradigm for developing downstream applications. However, its secur

arXiv:2609.34756v1 Announce Type: cross Abstract: Adapting pretrained models to downstream tasks with limited data has become a central paradigm in modern deep learning. Yet, despite its widespread pr

arXiv:2504.10214v2 Announce Type: replace Abstract: Pretrained model-based incremental object detection (PTMIOD) leverages the rich detection priors of pretrained detectors to learn new categories inc

arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures

arXiv:2608.04389v1 Announce Type: new Abstract: Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neu

arXiv:2607.27203v1 Announce Type: new Abstract: Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) thi

arXiv:2509.10468v2 Announce Type: replace-cross Abstract: Recent advances in generative recommenders adopt a two-stage paradigm: items are first tokenized into semantic IDs using a pretrained tokenize

arXiv:2610.14558v1 Announce Type: new Abstract: Large pretrained models have achieved remarkable success across diverse domains, yet adapting them to multiple downstream tasks remains challenging: ful

arXiv:2609.16262v1 Announce Type: cross Abstract: Pretraining followed by fine-tuning introduces a compute-allocation problem: under a fixed training budget, compute spent improving the upstream objec

arXiv:2609.15344v1 Announce Type: new Abstract: We study the adaptation of pretrained language models to univariate time-series forecasting through a parameter-efficient transfer learning framework, w

arXiv:2609.10518v1 Announce Type: new Abstract: fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are com

arXiv:2610.14011v1 Announce Type: new Abstract: Modern experiments increasingly produce rich post-treatment objects---such as text, images, and video---that may provide early surrogate signals for out

arXiv:2609.32898v1 Announce Type: new Abstract: Pretrained tabular foundation models process every dataset at a fixed depth, with inference costs growing with dataset size. To address this, we present

arXiv:2609.30948v1 Announce Type: cross Abstract: The Job Shop Scheduling Problem (JSSP) is a fundamental combinatorial optimization problem in industrial optimization. This work introduces Pretrained

arXiv:2607.25157v1 Announce Type: new Abstract: Discrete masked diffusion language models support bidirectional generation and infilling, but adapting pretrained autoregressive (AR) transformers requi

arXiv:2606.30988v1 Announce Type: new Abstract: Robot manipulation often relies on sensory feedback beyond vision, particularly in contact-rich settings where force, tactile, or audio signals reveal i

arXiv:2605.30467v1 Announce Type: new Abstract: This study introduces a novel Arctic-focused remote sensing foundation model (RSFM) by combining diversity-aware regional-scale image curation with mask

arXiv:2605.16975v1 Announce Type: cross Abstract: Electrocardiogram (ECG) foundation models pretrained on typical diagnostic 10-second ECG segments, have demonstrated strong transferability across a r

arXiv:2605.07327v1 Announce Type: new Abstract: Sampling from pretrained diffusion and flow-matching models typically requires many forward passes to generate diverse and high-fidelity images. Existin

arXiv:2507.00994v4 Announce Type: replace Abstract: Learning high-quality text representations is fundamental to a wide range of NLP tasks. While encoder pretraining has traditionally relied on Masked

arXiv:2603.00110v2 Announce Type: replace Abstract: The scarcity of large-scale robotic data has motivated the repurposing of foundation models from other modalities for policy learning. In this work,

arXiv:2604.10970v1 Announce Type: new Abstract: Background: Task-specific microscopy datasets are often small, making it difficult to train deep learning models that learn robust features. While self-

arXiv:2604.08627v1 Announce Type: cross Abstract: Pretrained models have become standard in both vision and language, yet they typically do not provide reliable measures of confidence. Existing uncert

arXiv:2609.35470v1 Announce Type: cross Abstract: Applied researchers increasingly convert images into features with pretrained encoders, then use those features in a downstream prediction model. The

arXiv:2609.29592v1 Announce Type: cross Abstract: Adapting large pretrained vision models under limited data and frozen-backbone constraints remains a central challenge in transfer learning. While lig

arXiv:2603.02293v3 Announce Type: replace Abstract: Frozen pretrained features are often treated as a safe interface for downstream learning: only a small linear readout is trained, while the backbone

arXiv:2609.17868v1 Announce Type: new Abstract: Pathology vision-language models are commonly built by pretraining or fine-tuning large encoders on paired image-caption data. We asked whether a pathol

arXiv:2604.05809v2 Announce Type: replace-cross Abstract: This paper presents Text-Guided Backdoor (TGB), an adjustable backdoor attack against multimodal pretrained models that uses natural-word trig