Search: “pretraining”

arXiv:2512.14067v2 Announce Type: replace-cross Abstract: Diffusion language models (dLMs) have emerged as a promising paradigm that enables parallel, non-autoregressive generation, but their learning

arXiv:2604.26772v1 Announce Type: new Abstract: Recent methods demonstrate that large-scale pretrained models, such as CLIP vision transformers, effectively detect AI-generated images (AIGIs) from uns

arXiv:2604.03260v2 Announce Type: replace-cross Abstract: Standard attention scales quadratically with sequence length. Efficient attention methods reduce this O(n^2) cost, but when retrofitted into p

arXiv:2410.06723v2 Announce Type: replace-cross Abstract: Pathology foundation models (PFMs) have emerged as powerful pretrained encoders for computational pathology, but their robustness under clinic

arXiv:2604.24679v1 Announce Type: new Abstract: Pathology foundation models (PFMs) have recently emerged as powerful pretrained encoders for computational pathology, enabling transfer learning across

arXiv:2604.23091v1 Announce Type: new Abstract: Scaling EEG foundation models requires pooling data across heterogeneous electrode montages, a prerequisite both for larger pretraining corpora and for

arXiv:2512.05359v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning has emerged as a promising paradigm in RGB-T tracking, enabling downstream task adaptation by freezing pretrained pa

arXiv:2604.22348v1 Announce Type: new Abstract: Clinical risk prediction using longitudinal medical data supports individualized care. Self-supervised foundation models have emerged as a promising app

arXiv:2604.21276v1 Announce Type: cross Abstract: As pretrained large language models replace task-specific decoders in speech recognition, a critical question arises: do their text-derived priors mak

arXiv:2505.11702v2 Announce Type: replace Abstract: This work develops a framework for post-training augmentation invariance, in which our goal is to add invariance properties to a pretrained network

arXiv:2604.21681v1 Announce Type: new Abstract: We present Sapiens2, a model family of high-resolution transformers for human-centric vision focused on generalization, versatility, and high-fidelity o

arXiv:2603.07474v2 Announce Type: replace-cross Abstract: What is the interplay between semantic representations learned by language models (LM) from surface form alone to those learned from more grou

arXiv:2604.20329v1 Announce Type: cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent

arXiv:2604.19728v1 Announce Type: cross Abstract: We present VLA Foundry, an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize

arXiv:2604.16879v1 Announce Type: new Abstract: With the rapid development of generative models and multimodal content editing technologies, the key challenge faced by synthetic image detection (SID)

arXiv:2604.17629v1 Announce Type: new Abstract: Pretrained biomedical vision-language models (VLMs) such as BioMedCLIP perform well on average but often degrade on challenging modalities where inter-c

arXiv:2604.18106v1 Announce Type: new Abstract: Adapting large language models (LLMs) to low-resource languages (LRLs) is constrained by the scarcity of task data and computational resources. Although

arXiv:2604.16780v1 Announce Type: new Abstract: This paper presents FairNVT, a lightweight debiasing framework for pretrained transformer-based encoders that improves both representation and predictio

arXiv:2604.17751v1 Announce Type: cross Abstract: Adapting foundation models under resource budgets relies heavily on Parameter-Efficient Fine-Tuning (PEFT), with LoRA being a standard modular solutio

arXiv:2510.02370v3 Announce Type: replace Abstract: Large language models leverage both parametric knowledge acquired during pretraining and in-context knowledge provided at inference time. Crucially,

arXiv:2604.17415v1 Announce Type: cross Abstract: Reward-based fine-tuning aims to steer a pretrained diffusion or flow-based generative model toward higher-reward samples while remaining close to the

arXiv:2603.24562v2 Announce Type: replace Abstract: While large-scale pretraining has revolutionized language modeling, its potential remains underexplored in healthcare with structured electronic hea

arXiv:2604.17782v1 Announce Type: new Abstract: Zero-shot EEG-to-image retrieval aims to decode perceived visual content from electroencephalography (EEG) by aligning neural responses with pretrained

arXiv:2510.09065v2 Announce Type: replace-cross Abstract: We introduce MMAudioSep, a generative model for video/text-queried sound separation that is founded on a pretrained video-to-audio model. By l

arXiv:2604.00313v2 Announce Type: replace Abstract: Automated species classification from underwater imagery is bottlenecked by the cost of expert annotation, and supervised models trained on one data

arXiv:2511.20645v2 Announce Type: replace Abstract: Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoe

arXiv:2604.12469v1 Announce Type: new Abstract: Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may

arXiv:2604.12223v1 Announce Type: cross Abstract: Pretrained language models (PLMs) like BERT provide strong semantic representations but are costly and opaque, while symbolic models such as the Tsetl

arXiv:2604.11868v1 Announce Type: new Abstract: While medical Vision-Language models (VLMs) achieve strong performance on tasks such as tumor or organ segmentation and diagnosis prediction, their opaq

arXiv:2603.00655v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance by aligning pretrained visual representations with the linguistic know

arXiv:2604.11135v1 Announce Type: cross Abstract: Pretrained video generation models provide strong priors for robot control, but existing unified world action models still struggle to decode reliable

arXiv:2604.11714v1 Announce Type: new Abstract: Pretrained detectors perform well on benchmarks but often suffer performance degradation in real-world deployments due to distribution gaps between trai

arXiv:2512.13255v3 Announce Type: replace Abstract: We introduce BezierFlow, a lightweight training approach for few-step generation with pretrained diffusion and flow models. BezierFlow achieves a 2-

arXiv:2512.24503v2 Announce Type: replace-cross Abstract: Data teams at frontier AI companies routinely train small proxy models to make critical decisions about pretraining data recipes for full-scal

arXiv:2604.10424v1 Announce Type: new Abstract: Foundation-style ECG encoders pretrained with self-supervised learning are increasingly reused across tasks, institutions, and deployment contexts, ofte

arXiv:2604.11803v1 Announce Type: new Abstract: Natural language processing (NLP) and speech technologies have made significant progress in recent years; however, they remain largely focused on standa

arXiv:2604.10334v1 Announce Type: new Abstract: Structured Illumination Microscopy (SIM) enables rapid, high-contrast optical sectioning of fresh tissue without staining or physical sectioning, making

arXiv:2411.10636v2 Announce Type: replace-cross Abstract: In this study, we investigate extrinsic gender bias in Bangla pretrained language models, a largely underexplored area in low-resource languag

arXiv:2603.04165v3 Announce Type: replace-cross Abstract: Large-scale 2D foundation models exhibit strong transferable representations, yet extending them to 3D volumetric data typically requires retr

arXiv:2604.04958v2 Announce Type: replace-cross Abstract: Recent work suggests that large-scale, multi-animal modeling can significantly improve neural recording analysis. However, for functional calc

arXiv:2604.04563v2 Announce Type: replace-cross Abstract: Recent advances in vision--language pretraining have enabled strong medical foundation models, yet most analyze radiographs in isolation, over

arXiv:2602.22251v3 Announce Type: replace-cross Abstract: General-purpose 3D chemical modeling encompasses molecules and materials, requiring both generative and predictive capabilities. However, most

arXiv:2606.21550v2 Announce Type: replace Abstract: Alignment from human feedback uses human judgments about model outputs to steer the behavior of language models after pretraining. When those judgme

arXiv:2610.20832v1 Announce Type: cross Abstract: Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather informatio

arXiv:2610.22300v1 Announce Type: cross Abstract: Not all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond thei

arXiv:2610.21833v1 Announce Type: new Abstract: Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and evaluate power flow orders of magnitude faster