Search: “pretraining”

Understanding the Weight Averaging Mechanism in LLM Training for Post-Training Quantization

arXiv:2610.05329v1 Announce Type: new Abstract: Large language models (LLMs) are typically pretrained in high precision but increasingly deployed with low-precision post-training quantization (PTQ). R

6 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters

DGX agent

arXiv:2610.02799v1 Announce Type: new Abstract: Fisheye cameras give mobile robots a single-sensor, low-cost view of their surroundings, yet the COCO-pretrained detectors that practitioners routinely

model-releasesarxiv-cs-cv

5 Oct 2026

Artifact Annotations Partially Substitute for Per-User Calibration: SAFE-EDA and a Normalization-Controlled Evaluation of Wrist-EDA Affect Recognition

arXiv:2610.01692v1 Announce Type: new Abstract: Wrist electrodermal activity (EDA) differs in amplitude from one person to the next, so affect-recognition models normalize their input before classific

Error-Corrected Inference-Time Scaling for Imperfect Diffusion Models

arXiv:2610.01933v1 Announce Type: cross Abstract: Inference-time scaling adapts pretrained diffusion models to new sampling tasks without additional training. Existing methods rely primarily on Monte

In-context Learning of Single-index Targets: Comparing Kernel and Feature Learners

arXiv:2610.01712v1 Announce Type: new Abstract: In-context learning (ICL) enables a pretrained model to infer a task from demonstrations without updating its parameters. While much of the existing the

VANDAM: Viewing a nucleotide sequence with DNA molecular priors

arXiv:2610.00411v1 Announce Type: new Abstract: Contemporary Genomic Foundation Models (GFMs) rely on a DNA-as-a-string paradigm that employs masked token prediction objectives for pretraining. Howeve

I Have a Stream: Making Self-Supervised Learning Work on Continuous Video

arXiv:2609.40333v1 Announce Type: new Abstract: Self-supervised learning draws inspiration from infant visual development, yet standard training pipelines bear little resemblance to it: images are ind

1 Oct 2026

NodeGround: A Node Classification Benchmark in the Graph Foundation Model Era

arXiv:2609.39673v1 Announce Type: new Abstract: Can a pretrained graph model replace training and tuning a separate predictor for each dataset? Answering this requires evaluating prediction quality al

1 Oct 2026

QuantCode Model: Specializing Language Models for Executable Algorithmic Trading Code

model-releasesarxiv-cs-cl

arXiv:2609.39420v1 Announce Type: new Abstract: Large language models are strong general-purpose code generators, but executable algorithmic trading remains a demanding specialization target: a model

1 Oct 2026

A Light Bilevel Refinement Aligns Self-Supervised Representations for Stronger Task-Specific Learning

model-releasesarxiv-cs-cv

arXiv:2609.33424v1 Announce Type: cross Abstract: Self-supervised pretraining learns representations that are broadly transferable across downstream tasks, yet direct fine-tuning can be suboptimal due

Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations

arXiv:2609.32743v1 Announce Type: new Abstract: Electroencephalography (EEG) foundation models (FMs) promise transferable neural representations, yet their advantages over strong supervised baselines

DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policies

arXiv:2609.32453v1 Announce Type: new Abstract: Robotic manipulation is inherently history-dependent, yet most pretrained robotic policies condition on only the current observation or a short temporal

safetyarxiv-cs-ro

From Static to Dynamic: On-Policy Distillation from Image to Video Diffusion Models

arXiv:2609.34371v1 Announce Type: new Abstract: On-policy distillation (OPD) specializes pretrained video diffusion models through teacher supervision along the student's own generation trajectory. Al

Handwritten Text Recognition Lives in the High-Pixel Variance Subspace

model-releasesarxiv-cs-cv

arXiv:2609.35473v1 Announce Type: new Abstract: In self-supervised pretraining for Handwritten Text Recognition (HTR), pixel reconstruction methods outperform contrastive methods, unlike in natural-im

Reuse or Relearn? A Spectral View of Earth Observation Foundation Models

arXiv:2609.32756v1 Announce Type: new Abstract: Foundation models are rarely used as generic, frozen feature extractors; instead, they are fine-tuned for the target downstream application. This practi

Spatial Grafting: Grounding 3D Features for Flow-Matching Robot Policies

model-releasesarxiv-cs-cv

arXiv:2609.35249v1 Announce Type: cross Abstract: Pretrained robot manipulation policies such as vision-language-action models (VLAs) or world-action models (WAMs) leave interaction-relevant metric ge

GridSFM: A Foundation Model for Solving AC Optimal Power Flow

arXiv:2609.30173v1 Announce Type: cross Abstract: We introduce GridSFM, a framework that combines a pretrained foundation model across grid topologies with physics-informed fine-tuning for solving AC

25 Sept 2026

Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matching

arXiv:2609.29405v1 Announce Type: cross Abstract: We adapt Reinforce Adjoint Matching (RAM), a reward-based post-training method, to generative speech enhancement (SE). Starting from a pretrained SE m

safetyarxiv-cs-lg

25 Sept 2026

Never Too Late for Force: Accelerating VLA Post-Training with Reactive Force Injection

arXiv:2607.14236v2 Announce Type: replace-cross Abstract: Pretrained vision-language-action (VLA) policies provide strong language-conditioned manipulation knowledge, but they remain largely vision-dr

safetyarxiv-cs-ai

24 Sept 2026

FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

model-releasesarxiv-cs-cl

arXiv:2609.25298v1 Announce Type: new Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely

23 Sept 2026

Hardware

TabICLv2: A better, faster, scalable, and open tabular foundation model

arXiv:2602.11139v2 Announce Type: replace Abstract: Tabular foundation models, such as TabPFNv2 and TabICL, have recently dethroned gradient-boosted trees at the top of predictive benchmarks, demonstr

hardwarearxiv-cs-lg

17 Sept 2026

Enemray: Toward Capable Language Models for Hassaniya

arXiv:2609.14829v1 Announce Type: cross Abstract: We introduce Enemray, a Hassaniya-centric language model that enables general-purpose interaction in Hassaniya. Enemray is trained around a stability-

15 Sept 2026

When does a scaling result justify a different allocation? A critical review of resource-allocation evidence for AI systems

arXiv:2609.14500v1 Announce Type: new Abstract: AI scaling studies increasingly evaluate systems that combine a pretrained model with retrieval, search, verification, tools, and interaction. Yet a hig

15 Sept 2026

CausalArena: Benchmarking Causal Discovery in the Foundation Model Era

arXiv:2609.11897v1 Announce Type: new Abstract: Causal discovery aims to uncover causal structures from data and is fundamental to scientific reasoning and intervention-based decision making. Its eval

11 Sept 2026

Full-Page Optical Music Recognition of Handwritten Monophonic Scores

arXiv:2609.05662v1 Announce Type: new Abstract: Full-page end-to-end Optical Music Recognition seeks to transcribe entire music pages directly into symbolic notation, avoiding the limitations of tradi

9 Sept 2026

MePo++: Unifying Representation Refinement and Reconciliation for General Continual Learning

arXiv:2609.05075v1 Announce Type: new Abstract: General continual learning (GCL) aims to learn from evolving data streams without task identities, explicit boundaries, or repeated access to previous d

safetyarxiv-cs-ai

7 Sept 2026

Object Concepts Emerge from Motion

arXiv:2609.04348v1 Announce Type: new Abstract: Object-centric visual representations are important for physical-world perception, but existing visual pretraining methods often capture semantic catego

7 Sept 2026

MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration

arXiv:2602.01734v2 Announce Type: replace Abstract: Training instability remains a critical challenge in large language model (LLM) pretraining, often manifesting as sudden gradient explosions that wa

4 Sept 2026

NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference

arXiv:2609.01657v1 Announce Type: cross Abstract: Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision enco

3 Sept 2026

Spatially Aware World Action Model via Geometric Latent Diffusion

arXiv:2609.02531v1 Announce Type: new Abstract: World Action Models (WAMs) leverage the capabilities of large-scale pretrained video diffusion models to jointly predict future observations and actions

3 Sept 2026

ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulation

model-releasesarxiv-cs-ro

arXiv:2609.02546v1 Announce Type: new Abstract: Zero-shot generalization to unseen embodiments is important for generalizable vision-language-action (VLA) models as robot hardware evolves and task-spe

3 Sept 2026

Learning Task-Specific Antibody Representations via Function-Aware Masking

arXiv:2609.00518v1 Announce Type: new Abstract: Antibody-specific language models pretrained via masked language modeling (MLM) learn representations that are critical for downstream sequence design a

2 Sept 2026

Probing Factual Knowledge Transfer with Training Data Interventions

arXiv:2609.01341v1 Announce Type: cross Abstract: Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned direc

researcharxiv-cs-ai

2 Sept 2026

Arkios: An Open Bilingual English-Nepali Language Model Trained From Scratch, with a Devanagari-Aware Tokenizer

arXiv:2608.30092v1 Announce Type: cross Abstract: We present Arkios, a 1.04B-parameter dense transformer pretrained from scratch on 150B tokens of bilingual English-Nepali text, using a custom single-

1 Sept 2026

Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization

arXiv:2608.26288v1 Announce Type: new Abstract: Muon has emerged as a strong optimizer for the matrix-valued parameters in large language model pretraining, approximately orthogonalizing its momentum

28 Aug 2026

Learning New Facts with QLoRA: An Acquisition-Retention Frontier

model-releasesarxiv-cs-cl

arXiv:2608.25677v1 Announce Type: new Abstract: Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that

27 Aug 2026

Tutorials

Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models

arXiv:2608.25693v1 Announce Type: new Abstract: Acquiring pixel-level annotations for medical image segmentation is a severe bottleneck. Traditional U-Net architectures, while effective, learn local t

tutorialsarxiv-cs-cv

27 Aug 2026

B-MIM: Biased Masked Image Modeling for Generalizable Segmentation of Fine-Grained Anatomical Structures

arXiv:2608.24364v1 Announce Type: new Abstract: Self-supervised pretraining enables transferable representations for medical imaging, yet most CT encoders remain biased toward coarse semantic understa

26 Aug 2026

Extending the Horizon of Early Diagnosis: Lung Cancer Prediction with Vision Transformers

arXiv:2608.21571v1 Announce Type: new Abstract: Lung cancer remains a leading cause of cancer-related mortality worldwide, and early diagnosis is critical for improving survival. However, early-stage

25 Aug 2026

Continuous Adversarial MeanFlow Transfer

arXiv:2608.19540v1 Announce Type: cross Abstract: Training fast generators on new domains with limited data remains challenging for two reasons. First, adapting a pretrained diffusion or flow model to

21 Aug 2026

Hybrid ANN-SNN Pipeline with Local Plasticity

arXiv:2606.20151v2 Announce Type: replace-cross Abstract: This work proposes a hybrid ANN-SNN pipeline that effectively leverages the rich embeddings of pretrained artificial neural networks (ANNs) to

20 Aug 2026

Task Specialization Fine-Tuning for Contextual Reinforcement Learning

arXiv:2608.17180v1 Announce Type: cross Abstract: Contextual Reinforcement Learning (CRL) seeks to generalize classical RL by maximizing task coverage across a context space of related tasks. While pr

safetyarxiv-cs-ai

19 Aug 2026

Policy Iteration with Human Feedback: Bringing Post-Training RL to In-context Learning

arXiv:2608.16831v1 Announce Type: new Abstract: Generative pretraining established reusable task representations; later work on language-based task conditioning and in-context learning showed that a f

18 Aug 2026

Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision

arXiv:2505.12532v3 Announce Type: replace-cross Abstract: Efficiently adapting large pretrained models is critical under tight compute and memory budgets. While Parameter-Efficient Fine-Tuning (PEFT)

14 Aug 2026

Local Ai

MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment

arXiv:2608.11167v1 Announce Type: cross Abstract: Existing Multimodal Large Language Models (MLLMs) predominantly rely on image-text pairs for modality alignment pretraining, mapping global image repr

local-aiarxiv-cs-cl

12 Aug 2026

Applications

Retrieval-Augmented Vision Foundation Models for Robust Leukemia Cell Classification across Multiple Microscopy Datasets

arXiv:2608.10657v1 Announce Type: cross Abstract: Leukemia cell image classification is challenged by real-world domain shifts from acquisition, staining, illumination, and site protocols, causing sin

applicationsarxiv-cs-cv

12 Aug 2026

Hardware

AraSSM: A bidirectional state-space encoder for Arabic masked language modeling

arXiv:2608.08256v1 Announce Type: new Abstract: Pretrained Transformer encoders such as AraBERT, MARBERT, and CAMeLBERT have become the standard backbone for Arabic natural language understanding, but

hardwarearxiv-cs-cl

11 Aug 2026

Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature

arXiv:2608.05100v1 Announce Type: new Abstract: Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the imag