Search: “pretraining”

arXiv:2602.01053v2 Announce Type: replace Abstract: Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only by lightwe

arXiv:2605.30348v1 Announce Type: cross Abstract: The pretraining data mixture of Large Language Models (LLMs) constitutes their 'digital DNA', shaping model behaviors, capabilities, and failure modes

arXiv:2605.28573v1 Announce Type: cross Abstract: The massive scaling of Large Language Models (LLMs) has made pretraining increasingly cost-prohibitive. While low-rank representation and orthonormal

arXiv:2605.27676v1 Announce Type: cross Abstract: Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent fact

arXiv:2605.27079v1 Announce Type: cross Abstract: Off-policy reinforcement learning of pretrained flow policies remains challenging due to the instability of optimization arising from the multi-step s

arXiv:2605.26026v1 Announce Type: cross Abstract: Light sheet fluorescence microscopy (LSM) enables high-resolution, three-dimensional (3D) imaging of biological specimens, providing rich volumetric d

arXiv:2605.22133v2 Announce Type: replace-cross Abstract: Recent advances in generative modeling show that pretrained representations can improve generation as conditioning features or alignment targe

arXiv:2605.19919v1 Announce Type: new Abstract: Pretrained imitation policies have become a strong foundation for robot manipulation, but they often require online improvement to overcome execution er

arXiv:2605.20082v1 Announce Type: cross Abstract: The rapid growth of autonomous driving datasets has enabled the scaling of powerful motion forecasting models. While large-scale pretraining provides

arXiv:2510.05213v2 Announce Type: replace-cross Abstract: Pretrained vision foundation models (VFMs) advance robotic learning via rich visual representations, yet individual VFMs typically excel only

arXiv:2605.12939v1 Announce Type: new Abstract: Recent diffusion- and flow-based VTON methods achieve strong results with pretrained generative models, but their reliance on multi-step sampling incurs

arXiv:2602.11618v4 Announce Type: replace Abstract: Chemical Language Models (CLMs) pre-trained on large scale molecular data are widely used for molecular property prediction. However, the common bel

arXiv:2510.10642v3 Announce Type: replace-cross Abstract: Building generalist robot policies that can handle diverse tasks in open-ended environments is a central challenge in robotics. To leverage kn

arXiv:2605.11369v1 Announce Type: new Abstract: Generating physically plausible dynamic motions of human-object interaction (HOI) remains challenging, mainly due to existing HOI datasets limited to st

arXiv:2605.09995v1 Announce Type: new Abstract: Post-training (via supervised fine-tuning) improves instruction-following, but often induces semantic mode collapse by biasing models toward low-entropy

arXiv:2605.10551v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) have achieved strong results in molecular property prediction, but polymers present distinct challenges: labeled datasets a

arXiv:2605.06859v1 Announce Type: cross Abstract: Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across differen

arXiv:2602.04556v2 Announce Type: replace Abstract: Weight tying is widely used in compact language models to reduce parameters by sharing the token table between the input embedding and the output pr

arXiv:2512.16504v3 Announce Type: replace Abstract: The self-supervised pretraining paradigm has achieved great success in learning 3D action representations for skeleton-based action recognition usin

arXiv:2605.00857v1 Announce Type: cross Abstract: Source-free domain adaptation (SFDA) provides a practical solution to cross-subject EEG decoding by adapting source-pretrained models to unlabeled tar

arXiv:2605.00809v1 Announce Type: new Abstract: In this paper, we present extbf{Gen}erative extbf{L}anguage-extbf{I}mage extbf{P}re-training (GenLIP), a minimalist generative pretraining framework for

arXiv:2604.27472v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models advance robotic control via strong visual-linguistic priors. However, existing VLAs predominantly frame pretraining

arXiv:2604.24763v1 Announce Type: new Abstract: Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creatin

arXiv:2604.19753v1 Announce Type: new Abstract: We propose a feature-free approach to algorithm selection that replaces hand-crafted instance features with pretrained text embeddings. Our method, Zero

arXiv:2601.18891v3 Announce Type: replace Abstract: Caribou across the Arctic has declined in recent decades, motivating scalable and accurate monitoring approaches to guide evidence-based conservatio

arXiv:2510.07248v3 Announce Type: replace Abstract: Small language models (SLMs) enable scalable tool-augmented multi-agent systems where multiple SLMs handle subtasks orchestrated by a powerful coord

arXiv:2509.18611v2 Announce Type: replace Abstract: Pretraining on large-scale collections of PDE-governed spatiotemporal trajectories has recently shown promise for building generalizable models of d

arXiv:2604.16443v1 Announce Type: cross Abstract: Data-driven models for building thermal dynamics are a scalable approach for enabling energy-efficient operation through fault detection & diagnosis o

arXiv:2604.13966v1 Announce Type: new Abstract: We study value adaptation in offline-to-online reinforcement learning under general function approximation. Starting from an imperfect offline pretraine

arXiv:2501.16154v4 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown impressive multilingual capabilities through pretraining on diverse corpora. Although these models sho

arXiv:2509.26404v2 Announce Type: replace-cross Abstract: Fingerprinting Large Language Models (LLMs)is essential for provenance verification and model attribution. Existing fingerprinting methods are

arXiv:2601.01041v3 Announce Type: replace Abstract: Deepfake detection remains highly challenging, particularly in cross-dataset scenarios and complex real-world settings. This challenge mainly arises

arXiv:2604.11590v1 Announce Type: new Abstract: Nowadays, pretrained models are increasingly used as general-purpose backbones and adapted at test-time to downstream environments where target data are

arXiv:2604.10451v1 Announce Type: new Abstract: Despite recent advancements in the field of medical image analysis with the use of pretrained foundation models, the issue of distribution shifts betwee

arXiv:2604.08168v1 Announce Type: new Abstract: Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due

arXiv:2610.21547v1 Announce Type: new Abstract: Federated learning (FL) is increasingly used to adapt a pretrained, centrally fine-tuned model to data that remain on user devices, beyond training a mo

arXiv:2610.22019v1 Announce Type: cross Abstract: Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been

arXiv:2610.21332v1 Announce Type: new Abstract: Pretrained latent spaces trade compression, reconstruction, and the learnability of the coordinates a downstream generator has to work in, which complic

arXiv:2610.21306v1 Announce Type: new Abstract: Vision-language-action (VLA) models transfer semantic knowledge acquired through vision-language pretraining to robot control, achieving strong performa

arXiv:2610.19882v1 Announce Type: new Abstract: Instruction-driven image editing has reached native 4K, yet how such editors are trained remains undisclosed, and pretraining one from scratch is beyond

arXiv:2610.14489v1 Announce Type: new Abstract: Reusable wearable PPG representations should capture local pulse morphology and broader physiological regularities. Existing pretraining objectives, how

arXiv:2610.20111v1 Announce Type: new Abstract: Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution ma

arXiv:2610.20608v1 Announce Type: new Abstract: Representation-alignment methods effectively accelerate the diffusion transformer training. REPA, for example, uses a pretrained vision encoder to super

arXiv:2610.19651v1 Announce Type: new Abstract: Hyperspectral imaging distinguishes RGB-similar objects, but a pretrained vision-language-action model needs a state its frozen decoder can use. SPECTRA

arXiv:2610.15028v1 Announce Type: cross Abstract: Recent bidirectional encoders have been modernized with innovations from causal large language models (LLMs). However, the pretraining recipe itself r

arXiv:2610.17573v1 Announce Type: new Abstract: Self-supervised pretraining has expanded the pool of 3D medical foundation models, but published evaluations use different datasets, endpoints and fine-