Search: “pretraining”

arXiv:2510.02538v2 Announce Type: replace Abstract: We are interested in solving the problem of imitation learning with a limited amount of real-world expert data. Existing offline imitation methods o

arXiv:2609.22184v1 Announce Type: new Abstract: Drug-target relation prediction supports candidate screening, drug repositioning, and mechanism analysis. Existing models often use incomplete drug or p

arXiv:2609.23463v1 Announce Type: new Abstract: Federated learning lets multiple parties train a shared model without pooling their data, but a client with far less data than the others can end up poo

arXiv:2609.03824v1 Announce Type: new Abstract: 3D foundation models (3DFMs) excel at predicting camera poses and dense depth from multiple views of a scene, showcasing strong zero-shot generalization

arXiv:2608.30305v1 Announce Type: new Abstract: Image restoration models are typically trained with a fixed set of capabilities. When new restoration requirements emerge, existing solutions usually tr

arXiv:2608.27370v1 Announce Type: new Abstract: Language model pretraining has become almost synonymous with prohibitive cost, placing it out of reach for much of the academic and open-source communit

arXiv:2608.25410v1 Announce Type: cross Abstract: Token communications realize the semantic communication principle at the granularity of transformer tokens, providing a promising direction for client

arXiv:2608.22272v1 Announce Type: cross Abstract: Generative adversarial networks (GANs) can provide efficient image generation, while diffusion models offer high-quality image restoration but require

arXiv:2405.03728v3 Announce Type: replace-cross Abstract: Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or

arXiv:2608.08749v1 Announce Type: new Abstract: Long-horizon robot manipulation requires policies to track completed subtasks and critical interaction events. However, existing memory mechanisms heavi

arXiv:2608.00383v1 Announce Type: cross Abstract: Logical rule induction seeks interpretable rules that transfer across propositional schemas. This requires respecting symmetries: atom naming, example

arXiv:2607.28776v1 Announce Type: new Abstract: Generative machine learning is increasingly used for inorganic crystal structure generation. Most models and the corresponding evaluation approaches rel

arXiv:2607.07885v1 Announce Type: cross Abstract: Dynamic obstacle avoidance in unstructured outdoor environments remains a critical challenge for autonomous mobile robots, particularly when large-sca

arXiv:2607.07294v1 Announce Type: cross Abstract: Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot

arXiv:2603.21210v3 Announce Type: replace Abstract: Designing urban spaces that provide pedestrian wind comfort and safety requires time-resolved Computational Fluid Dynamics (CFD) simulations, but th

arXiv:2606.27371v1 Announce Type: new Abstract: State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple

arXiv:2606.25197v1 Announce Type: new Abstract: Learning effective policies for adaptive data acquisition remains challenging: posterior-based methods rely on surrogate models and posterior approximat

arXiv:2606.15678v2 Announce Type: replace Abstract: A feasibility and dynamics study of the Reservoir Attention Network (RAN), an architecture that injects a fixed, randomly-initialized reservoir into

arXiv:2606.09430v1 Announce Type: cross Abstract: Online task-free continual learning (TFCL) requires intelligent agents to sequentially accumulate knowledge from an unbounded, non-stationary data str

arXiv:2602.16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedba

arXiv:2603.26885v2 Announce Type: replace Abstract: Convolutional neural networks (CNNs) achieve state-of-the-art performance in medical image analysis yet remain opaque, limiting adoption in high-sta

arXiv:2605.30877v1 Announce Type: new Abstract: Large-scale Vision-Language-Action (VLA) pretraining is increasingly adopted as the foundation for robot policies, yet the evidence for pretrained VLAs

arXiv:2603.17945v2 Announce Type: replace Abstract: In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, n

arXiv:2605.24769v1 Announce Type: cross Abstract: Hyperspectral image restoration faces several challenges, including limited training data, strong sensor specificity, and high spectral dimensionality

arXiv:2605.10777v1 Announce Type: new Abstract: The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling the

arXiv:2604.10217v1 Announce Type: new Abstract: Cross-modal optical-SAR (Synthetic Aperture Radar) registration is a bottleneck for disaster-response via remote sensing, yet modern image matchers are

arXiv:2610.11694v1 Announce Type: cross Abstract: Existing reaction models primarily learn molecular transformations, whereas enzy- matic reactions depend jointly on molecular structure and catalytic

arXiv:2610.08433v1 Announce Type: new Abstract: Modern action recognition models rely on video transformers pretrained on massive collections of web-crawled videos, such as Kinetics-700. However, the

arXiv:2610.05711v1 Announce Type: cross Abstract: Strong image generation models are conditioned on class labels, aligned to frozen pretrained encoders, or built on separately trained autoencoders. Wh

arXiv:2610.01409v1 Announce Type: new Abstract: Reliable uncertainty estimation is essential for deploying object detectors when distribution/covariate shift and adversarial attacks may occur. Existin

arXiv:2609.32124v1 Announce Type: new Abstract: When adapting pre-trained models through fine-tuning, freezing weights alone might not preserve performance, as updates elsewhere can change the inputs

arXiv:2609.31199v1 Announce Type: cross Abstract: Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D

arXiv:2609.19865v1 Announce Type: new Abstract: Representation learning from medical code sequences in electronic health records and medical claims data has been successful in various clinical applica

arXiv:2609.20451v1 Announce Type: new Abstract: Numerical site-response predictions often deviate from observations, yet correcting these discrepancies is difficult because records are limited in both

arXiv:2609.20213v1 Announce Type: new Abstract: Dimensionality reduction is a well-known technique for improving space efficiency, typically applied uniformly across an entire dataset. This paper inve

arXiv:2609.16695v1 Announce Type: new Abstract: Large-scale pre-training has transformed representation learning in 2D vision, yet its transferability to 3D single object tracking (SOT) remains insuff

Hello! I've just uploaded a recent checkpoint of my model trained from scratch: https://huggingface.co/piotr-ai/polanka_3.7b_exp_wip_260901 It was pre-trained, mid-trained, and fine-tuned on a single

arXiv:2608.30844v1 Announce Type: cross Abstract: Interactive lesion segmentation in whole-body PET/CT requires a model to provide a strong initial prediction while also responding efficiently to spar

arXiv:2608.17432v1 Announce Type: new Abstract: Generative imitation learning policies excel at trajectory planning but lack closed-loop force regulation, while directly incorporating force modalities

The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardware and software plat

arXiv:2608.03629v1 Announce Type: new Abstract: A companion paper studies when activation patching and weight-space ablation agree, inside an idealized model where a conditional computation is carried

arXiv:2608.01306v1 Announce Type: new Abstract: Latents from vision foundation models (VFMs) are semantically rich and well suited for visual understanding. Recent representation autoencoder methods s

arXiv:2607.28236v1 Announce Type: cross Abstract: Pre-trained language models have significantly improved sentence representation learning, yet their embedding remain sensitive to semantic preserving

Visual generative models (e.g., diffusion models) typically operate in compressed latent spaces to balance training efficiency and sample quality. In parallel, there has been growing interest in lever

arXiv:2607.04919v1 Announce Type: new Abstract: Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We pro

arXiv:2607.01690v1 Announce Type: new Abstract: Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually believes the documents' core