Search: “pretraining”

Hybrid ANN-SNN Pipeline with Local Plasticity

arXiv:2606.20151v2 Announce Type: replace-cross Abstract: This work proposes a hybrid ANN-SNN pipeline that effectively leverages the rich embeddings of pretrained artificial neural networks (ANNs) to

20 Aug 2026

Task Specialization Fine-Tuning for Contextual Reinforcement Learning

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.17180v1 Announce Type: cross Abstract: Contextual Reinforcement Learning (CRL) seeks to generalize classical RL by maximizing task coverage across a context space of related tasks. While pr

safetyarxiv-cs-ai

19 Aug 2026

Policy Iteration with Human Feedback: Bringing Post-Training RL to In-context Learning

arXiv:2608.16831v1 Announce Type: new Abstract: Generative pretraining established reusable task representations; later work on language-based task conditioning and in-context learning showed that a f

18 Aug 2026

Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision

arXiv:2505.12532v3 Announce Type: replace-cross Abstract: Efficiently adapting large pretrained models is critical under tight compute and memory budgets. While Parameter-Efficient Fine-Tuning (PEFT)

14 Aug 2026

MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment

arXiv:2608.11167v1 Announce Type: cross Abstract: Existing Multimodal Large Language Models (MLLMs) predominantly rely on image-text pairs for modality alignment pretraining, mapping global image repr

local-aiarxiv-cs-cl

12 Aug 2026

Applications

Retrieval-Augmented Vision Foundation Models for Robust Leukemia Cell Classification across Multiple Microscopy Datasets

arXiv:2608.10657v1 Announce Type: cross Abstract: Leukemia cell image classification is challenged by real-world domain shifts from acquisition, staining, illumination, and site protocols, causing sin

applicationsarxiv-cs-cv

12 Aug 2026

Hardware

AraSSM: A bidirectional state-space encoder for Arabic masked language modeling

arXiv:2608.08256v1 Announce Type: new Abstract: Pretrained Transformer encoders such as AraBERT, MARBERT, and CAMeLBERT have become the standard backbone for Arabic natural language understanding, but

hardwarearxiv-cs-cl

11 Aug 2026

Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature

arXiv:2608.05100v1 Announce Type: new Abstract: Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the imag

6 Aug 2026

MultiPathFormer: Towards a Foundation Model for Multipath Wireless Propagation

arXiv:2608.05076v1 Announce Type: cross Abstract: Recent advances in machine learning have enabled training of wireless foundation models, which aim to support tasks such as channel estimation, beam p

local-aiarxiv-cs-ai

6 Aug 2026

Tutorials

Sparse Weight Decomposition for Efficient Circuit Extraction

arXiv:2608.03913v1 Announce Type: cross Abstract: Dense pretrained transformers do not naturally expose interpretable units for circuit extraction. Existing approaches obtain such units by learning au

tutorialsarxiv-cs-cl

5 Aug 2026

RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding

arXiv:2608.00147v1 Announce Type: new Abstract: Vision-language pretraining learns rich medical image representations from radiology reports, but previous model variants commonly operate within a sing

4 Aug 2026

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

arXiv:2607.29213v1 Announce Type: cross Abstract: Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhan

safetyarxiv-cs-lg

3 Aug 2026

Models for minimalist RAG: B1ade 335M Embedding and 1B Parameter Small Language Models

model-releasesarxiv-cs-cl

arXiv:2607.27506v1 Announce Type: new Abstract: Language and embedding models used in RAG systems are conventionally assumed to require large-scale pretraining and explicit grounding supervision. We p

31 Jul 2026

Applications

Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers

arXiv:2512.17351v2 Announce Type: replace Abstract: Understanding architectural differences in language models is challenging, especially at academic-scale pretraining (e.g., 1.3B parameters, 100B tok

applicationsarxiv-cs-cl

29 Jul 2026

Knowledge-Guided Cross-Modal Fusion for Adult-to-Pediatric ECG Transfer via Label-Conditioned Contrastive Alignment

arXiv:2607.15928v2 Announce Type: replace Abstract: Adult and pediatric electrocardiogram (ECG) interpretation relies on age-sensitive criteria, and models pretrained mainly on adult ECGs often transf

safetyarxiv-cs-lg

28 Jul 2026

Post-Training in Time Series Foundation Models: A Unifying Framework

arXiv:2607.20002v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for

23 Jul 2026

Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

arXiv:2607.13429v1 Announce Type: cross Abstract: Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-langua

16 Jul 2026

In-Context Reinforcement Learning under Non-Stationarity: A Survey

arXiv:2607.11906v1 Announce Type: new Abstract: The development of decision-pretrained transformers, algorithm distillation, long-context meta-RL, and retrieval-augmented agents has renewed interest i

15 Jul 2026

Co-LMLM: Continuous-Query Limited Memory Language Models

arXiv:2607.07707v1 Announce Type: cross Abstract: Limited memory language models (LMLMs) externalize factual knowledge during pretraining to a knowledge base (KB), rather than memorizing it in their w

9 Jul 2026

RL Post-Training Builds Compositional Reasoning Strategies

arXiv:2607.07646v1 Announce Type: new Abstract: Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategie

9 Jul 2026

From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models

arXiv:2607.06553v1 Announce Type: new Abstract: Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and

researcharxiv-cs-cv

8 Jul 2026

CORA: Generalizable coronary artery disease assessment and risk stratification from coronary CT angiography using pathology-centric representation learning

arXiv:2603.24847v2 Announce Type: replace Abstract: Coronary artery disease, a leading cause of cardiovascular mortality worldwide, can be assessed non-invasively by coronary computed tomography angio

local-aiarxiv-cs-cv

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

model-releasesarxiv-cs-ro

arXiv:2607.02865v1 Announce Type: new Abstract: Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leadi

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

arXiv:2607.04733v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain

local-aiarxiv-cs-cl

Hardware

On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization

arXiv:2607.02834v1 Announce Type: new Abstract: Molecular optimization often starts from a pretrained generative model that captures a broad prior over valid molecular structures. At test time, howeve

hardwarearxiv-cs-lg

The Three Regimes of Offline-to-Online Reinforcement Learning

arXiv:2510.01460v4 Announce Type: replace-cross Abstract: Offline-to-online reinforcement learning (RL) has emerged as a practical paradigm that leverages offline datasets for pretraining and online i

safetyarxiv-cs-ai

LUMA: Benchmarking Segmentation via a Lightweight Universal Mask Adapter

arXiv:2607.00687v1 Announce Type: cross Abstract: Comparing transformer backbones for image segmentation is confounded: each is paired with a different decoder, recipe, and pretraining, so reported di

2 Jul 2026

Adapting Generalist Robot Policies with Semantic Reinforcement Learning

arXiv:2606.31958v1 Announce Type: new Abstract: Generalist robot policies learn a diverse repertoire of behaviors from large-scale pretraining. In principle, this makes them excellent priors for downs

safetyarxiv-cs-ro

1 Jul 2026

Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization

arXiv:2606.27663v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models leverage large-scale vision-language pretraining for flexible robot manipulation, yet at test time they remain britt

local-aiarxiv-cs-ro

29 Jun 2026

Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learning

model-releasesarxiv-cs-cv

arXiv:2606.24570v1 Announce Type: new Abstract: Vision-language contrastive pretraining has become the dominant recipe for 3D medical foundation models, leveraging the large volumes of paired scans an

24 Jun 2026

World Pilot: Steering Vision-Language-Action Models with World-Action Priors

model-releasesarxiv-cs-ro

arXiv:2606.12403v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining and perform competently across in-distribution manipulation

11 Jun 2026

Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models

arXiv:2606.09861v1 Announce Type: cross Abstract: While Next-Token Prediction (NTP) has unified LLM pretraining, its adaptation to unbounded, continuous time series (TS) remains open. To bridge the ga

10 Jun 2026

CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs

arXiv:2606.08420v1 Announce Type: new Abstract: Vision-language models (VLMs) pretrained on large-scale image-text pairs demonstrate strong image-level understanding, but are primarily optimized for g

Do Video Foundation Models Understand Intuitive Physics? A Layerwise Probing Analysis

arXiv:2606.09646v1 Announce Type: cross Abstract: We study whether pretrained video foundation models encode intuitive-physics information in their frozen representations, and how this information var

Foundation Inference Models for Ordinary Differential Equations

arXiv:2602.08733v2 Announce Type: replace Abstract: Ordinary differential equations (ODEs) are central to scientific modelling, but inferring their vector fields from noisy trajectories remains challe

researcharxiv-cs-lg

Muon Learns More Robust and Transferable Features than Adam

arXiv:2606.09658v1 Announce Type: cross Abstract: Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency

Applications

Self-Supervised Dynamical System Representations for Physiological Time-Series

arXiv:2512.00239v2 Announce Type: replace Abstract: The effectiveness of self-supervised learning (SSL) for physiological time series depends on the ability of a pretraining objective to preserve info

applicationsarxiv-cs-lg

Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models

arXiv:2601.09719v3 Announce Type: replace-cross Abstract: Pre-Layer Normalization (Pre-LN) is the de facto choice for large language models (LLMs) and is crucial for stable pretraining and effective t

4 Jun 2026

FACT: A Simple and Efficient Framework for Active Finetuning

model-releasesarxiv-cs-cv

arXiv:2606.02079v1 Announce Type: new Abstract: The main goal of active finetuning is to improve a pretrained model's performance on a specific task or domain by finetuning it with carefully selected

2 Jun 2026

Agents

LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents

arXiv:2602.01053v2 Announce Type: replace Abstract: Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only by lightwe

agentsarxiv-cs-lg

2 Jun 2026

LLMSurgeon: Diagnosing Data Mixture of Large Language Models

arXiv:2605.30348v1 Announce Type: cross Abstract: The pretraining data mixture of Large Language Models (LLMs) constitutes their 'digital DNA', shaping model behaviors, capabilities, and failure modes

29 May 2026

Efficient Pre-Training of LLMs through Truncated SVD Layers

arXiv:2605.28573v1 Announce Type: cross Abstract: The massive scaling of Large Language Models (LLMs) has made pretraining increasingly cost-prohibitive. While low-rank representation and orthonormal

28 May 2026

Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning

arXiv:2605.27676v1 Announce Type: cross Abstract: Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent fact

safetyarxiv-cs-lg

28 May 2026

Trust Region Q Adjoint Matching

arXiv:2605.27079v1 Announce Type: cross Abstract: Off-policy reinforcement learning of pretrained flow policies remains challenging due to the instability of optimization arising from the multi-step s

27 May 2026

A Multimodal 3D Foundation Model for Light Sheet Fluorescence Microscopy Enables Few-Shot Segmentation, Classification, and Deblurring

arXiv:2605.26026v1 Announce Type: cross Abstract: Light sheet fluorescence microscopy (LSM) enables high-resolution, three-dimensional (3D) imaging of biological specimens, providing rich volumetric d

safetyarxiv-cs-ai

26 May 2026

Atom-level Protein Representation Learning Improves Protein Structure Prediction

arXiv:2605.22133v2 Announce Type: replace-cross Abstract: Recent advances in generative modeling show that pretrained representations can improve generation as conditioning features or alignment targe

25 May 2026

Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning

arXiv:2605.19919v1 Announce Type: new Abstract: Pretrained imitation policies have become a strong foundation for robot manipulation, but they often require online improvement to overcome execution er

safetyarxiv-cs-ro

20 May 2026

Agents

VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving