Knowledge catalogue
arXiv:2605.27796v1 Announce Type: cross Abstract: Ultrasound is widely used in obstetric care due to its safety, accessibility, and real-time imaging. However, interpretation remains operator-dependen
arXiv:2605.28111v1 Announce Type: new Abstract: Predicting how a cell will change its transcriptional state under a developmental signal or a genetic perturbation is the computational core of in-silic
arXiv:2511.16870v3 Announce Type: replace Abstract: Enforcing alignment between the internal representations of diffusion or flow-based generative models and those of pretrained self-supervised encode
arXiv:2512.19097v3 Announce Type: replace-cross Abstract: Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is diffic
arXiv:2605.25459v1 Announce Type: cross Abstract: Language models are pretrained as passive predictors with no incentive to model the consequences of their own outputs. Post-training changes this: a m
arXiv:2602.13249v2 Announce Type: replace-cross Abstract: Small-molecule foundation models are typically pretrained on standalone molecular data, unlike vision and language models that often benefit f
arXiv:2605.23733v1 Announce Type: cross Abstract: Whole-body tracking (WBT) models have become a key foundation for humanoid robots, enabling them to imitate diverse motions with high fidelity. Traini
arXiv:2605.22869v1 Announce Type: new Abstract: Both full fine-tuning (Full FT) and parameter-efficient fine-tuning methods such as LoRA introduce weight updates without accounting for the spectral st
arXiv:2605.22222v1 Announce Type: new Abstract: Partial differential equation (PDE) foundation models are pretrained networks that forecast how physical fields like velocity and pressure evolve from a
arXiv:2602.04768v2 Announce Type: replace Abstract: Graph-structured data underpins many critical applications. While foundation models have transformed language and vision via large-scale pretraining
arXiv:2605.22132v1 Announce Type: new Abstract: Pretrained vision foundation models deliver strong performance across tasks with limited fine-tuning. However, their Vision Transformer (ViT) backbones
arXiv:2605.21654v1 Announce Type: cross Abstract: Reinforcement learning substantially improves pretrained language models, but it remains understudied why critic-free methods such as PPO and GRPO wor
arXiv:2605.20521v1 Announce Type: new Abstract: Fine-tuning adapts a pretrained machine learning model to a small, sensitive dataset, but this process risks memorizing individual new data points, maki
arXiv:2605.20316v1 Announce Type: new Abstract: Modern text-to-image diffusion models encode rich visual priors, but expose them only through one-way text-conditioned generation. Existing unified visi
arXiv:2605.21258v1 Announce Type: new Abstract: Current 3D-aware pretraining methods for embodied perception and manipulation are largely built on differentiable rendering frameworks, producing either
arXiv:2605.18911v1 Announce Type: cross Abstract: Wildfire prediction is important for early warning and resource allocation, yet existing Earth foundation models (Earth FMs) are pretrained for genera
arXiv:2605.20005v1 Announce Type: new Abstract: Fine-tuning large language models on new data improves task performance but degrades capabilities learned during pretraining, a phenomenon known as cata
arXiv:2605.19662v1 Announce Type: new Abstract: Tabular foundation models based on pretrained prior-data fitted networks~(PFNs) have shown strong generalization on diverse tabular tasks, but they are
arXiv:2602.02236v4 Announce Type: replace-cross Abstract: We study online fine-tuning of pretrained control policies for autonomous driving using Real-Time Recurrent Reinforcement Learning (RTRRL), a
arXiv:2603.22570v2 Announce Type: replace Abstract: Active computer vision promises efficient, biologically plausible perception through sequential, localized glimpses, but lacks scalable general-purp
arXiv:2605.18607v1 Announce Type: new Abstract: Progress in language model development is often driven by comparative decisions: which architecture to adopt, which pretraining corpus to use, or which
arXiv:2605.18324v1 Announce Type: cross Abstract: Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders. In this paper, we systematically investigate several design
arXiv:2605.16758v1 Announce Type: new Abstract: Large Language Models (LLMs) remain substantially less data-efficient than humans. Pre-pretraining (PPT) on synthetic languages has been proposed to clo
arXiv:2507.06384v2 Announce Type: replace-cross Abstract: Objective: Latent diffusion models (LDMs) could mitigate data scarcity challenges affecting machine learning development for medical image int
arXiv:2605.17340v1 Announce Type: new Abstract: Time series foundation models rely on large-scale pretraining over diverse datasets across domains, yet their heterogeneity in temporal patterns could h
arXiv:2605.18174v1 Announce Type: new Abstract: Muon has recently emerged as a strong alternative to AdamW for training neural networks, with encouraging large-scale pretraining results and growing ev
rsi is here. jesus https://x.com/nickevanjoseph/status/2056760504949842219?s=46 Excited to welcome Andrej to the Pretraining team! He'll be building a team focused on using Claude to accelerate pretra
arXiv:2605.17949v1 Announce Type: new Abstract: Remote sensing vision-language models commonly rely on pretrained visual encoders to convert images into semantic features before language-model reasoni
arXiv:2605.17017v1 Announce Type: cross Abstract: Behavior Foundation Models (BFMs) enable scalable imitation learning (IL) by pretraining task-agnostic representations that can be rapidly adapted to
arXiv:2602.23410v3 Announce Type: replace-cross Abstract: Brain foundation models have achieved remarkable advances across a wide range of neuroscience tasks. However, most existing models are limited
arXiv:2605.15735v1 Announce Type: cross Abstract: Vision--language--action (VLA) models are typically built by fine-tuning a pretrained vision--language model (VLM) on action data. However, we show th
arXiv:2605.13262v1 Announce Type: new Abstract: Modern SMILES-based chemical language models obtain strong MoleculeNet performance by treating SMILES as generic text and compensating with multi-millio
arXiv:2605.11405v1 Announce Type: new Abstract: Data curation has shifted the quality-compute frontier for language-model and contrastive image-text pretraining, but its role for vision-language model
arXiv:2605.11480v1 Announce Type: new Abstract: Reward fine-tuning has become a common approach for aligning pretrained diffusion and flow models with human preferences in text-to-image generation. Am
arXiv:2605.11871v1 Announce Type: new Abstract: Training-free camera control for pretrained flow-matching video generators is a partial-observation inverse problem: a depth-warped guidance video suppl
arXiv:2605.10973v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) improves in-domain performance but can degrade out-of-domain (OOD) generalization. Prior work suggests that this degradatio
arXiv:2605.10780v1 Announce Type: cross Abstract: Representation autoencoders that reuse frozen pretrained vision encoders as visual tokenizers have achieved strong reconstruction and generation quali
arXiv:2605.10903v1 Announce Type: new Abstract: This paper proposes a novel approach to address the challenge that pretrained VLA models often fail to effectively improve performance and reduce adapta
arXiv:2605.10642v1 Announce Type: new Abstract: Pretrained diffusion models provide powerful learned priors, but in scientific sampling the target distribution often depends on physical context that i
i like that there are models called bert and ernie, but in all seriousness, this update looks impressive ERNIE 5.1 is here 🚀 ERNIE 5.1 significantly reduces pretraining cost while compressing total pa
arXiv:2605.09993v1 Announce Type: new Abstract: Graph foundation models (GFMs), pretrained on massive graph data, have transformed graph machine learning by supporting general-purpose reasoning across
arXiv:2605.10616v1 Announce Type: cross Abstract: Tabular Foundation Models have recently established the state of the art in supervised tabular learning, by leveraging pretraining to learn generaliza
arXiv:2605.09485v1 Announce Type: new Abstract: Latent representations learned by neural networks often exhibit semantic structure, where concept similarity is reflected by geometric proximity in embe
arXiv:2605.08738v1 Announce Type: cross Abstract: Structured pruning and knowledge distillation (KD) are typical techniques for compressing large language models, but it remains unclear how they shoul
arXiv:2604.03928v2 Announce Type: replace-cross Abstract: Frozen pretrained image representations are widely used for transfer learning: a backbone is kept fixed, feature vectors are extracted, and a
arXiv:2605.09424v1 Announce Type: new Abstract: Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modality, r
arXiv:2605.07308v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have significantly advanced the capabilities of robotic agents in executing diverse tasks; however, they still face
arXiv:2605.07333v1 Announce Type: new Abstract: In-context reinforcement learning (ICRL) studies agents that, after pretraining, adapt to new tasks by conditioning on additional context without parame
arXiv:2605.07554v1 Announce Type: cross Abstract: Protein language models are trained primarily with masked language modeling (MLM), which predicts amino-acid identities at masked positions. We ask wh
arXiv:2605.07693v1 Announce Type: new Abstract: Geometric representation-conditioned molecule generation provides an effective paradigm that decouples molecule representation modeling from structure g
arXiv:2410.21438v3 Announce Type: replace Abstract: By pretraining on trillions of tokens, an LLM gains the capability of text generation. However, to enhance its utility and reduce potential harm, SF
arXiv:2605.04651v1 Announce Type: cross Abstract: Adapting pretrained models typically involves a trade-off between the high training costs of backpropagation and the heavy inference overhead of memor
arXiv:2605.02829v1 Announce Type: new Abstract: Adapting large pretrained models to diverse tasks is now routine, yet the two dominant strategies of parameter-efficient fine-tuning (PEFT) and low-rank
arXiv:2605.03297v1 Announce Type: cross Abstract: ASR systems based on self-supervised acoustic pretraining and CTC fine-tuning achieve strong performance on native speech but remain sensitive to acce
arXiv:2605.01256v1 Announce Type: new Abstract: A promising paradigm for adapting instruction-tuned language models is to learn task-specific updates on a pretrained base model and subsequently merge
arXiv:2605.02364v1 Announce Type: new Abstract: Upweighting high-quality data in LLM pretraining often improves performance, but in datalimited regimes, especially under overtraining, stronger upweigh
arXiv:2602.03668v2 Announce Type: replace-cross Abstract: Latent actions learned from diverse human videos serve as pseudo-labels for vision-language-action (VLA) pretraining, but provide effective su
arXiv:2605.02094v1 Announce Type: new Abstract: Subtle hand differences make sign language recognition challenging, yet many existing methods rely on encoders pretrained on generic action datasets tha
arXiv:2605.02258v1 Announce Type: new Abstract: Vision Foundation Models (VFMs) pretrained on large-scale RGB data have demonstrated remarkable representation quality, yet their applicability to multi
arXiv:2605.00225v1 Announce Type: cross Abstract: We show that pretrained acoustic embeddings classify elephant vocalisations at a level approaching that of end-to-end supervised neural networks, with