Search: “pretraining”

Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flows

arXiv:2603.21210v3 Announce Type: replace Abstract: Designing urban spaces that provide pedestrian wind comfort and safety requires time-resolved Computational Fluid Dynamics (CFD) simulations, but th

1 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance

DGX agent

arXiv:2606.27371v1 Announce Type: new Abstract: State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple

safetyarxiv-cs-cv

26 Jun 2026

Efficient Adaptive Data Acquisition via Pretrained Belief Representations

arXiv:2606.25197v1 Announce Type: new Abstract: Learning effective policies for adaptive data acquisition remains challenging: posterior-based methods rely on surrogate models and posterior approximat

safetyarxiv-cs-lg

25 Jun 2026

Hardware

The Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection

arXiv:2606.15678v2 Announce Type: replace Abstract: A feasibility and dynamics study of the Reservoir Attention Network (RAN), an architecture that injects a fixed, randomly-initialized reservoir into

hardwarearxiv-cs-lg

23 Jun 2026

LargeMonitor: Monitoring Online Task-Free Continual Learning via Large Pretrained Models

arXiv:2606.09430v1 Announce Type: cross Abstract: Online task-free continual learning (TFCL) requires intelligent agents to sequentially accumulate knowledge from an unbounded, non-stationary data str

9 Jun 2026

Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models

arXiv:2602.16061v2 Announce Type: replace-cross Abstract: Estimating population quantities such as mean outcomes from user feedback is fundamental to platform evaluation and social science, yet feedba

9 Jun 2026

TTE-CAM: Self-Explainable Class Activation Maps for Pretrained Black-Box CNNs

arXiv:2603.26885v2 Announce Type: replace Abstract: Convolutional neural networks (CNNs) achieve state-of-the-art performance in medical image analysis yet remain opaque, limiting adoption in high-sta

1 Jun 2026

Wall-OSS-0.5 Technical Report

arXiv:2605.30877v1 Announce Type: new Abstract: Large-scale Vision-Language-Action (VLA) pretraining is increasingly adopted as the foundation for robot policies, yet the evidence for pretrained VLAs

safetyarxiv-cs-ro

1 Jun 2026

ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws

arXiv:2603.17945v2 Announce Type: replace Abstract: In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, n

29 May 2026

Leveraging pretrained RGB denoisers for hyperspectral image restoration

arXiv:2605.24769v1 Announce Type: cross Abstract: Hyperspectral image restoration faces several challenges, including limited training data, strong sensor specificity, and high spectral dimensionality

researcharxiv-cs-ai

26 May 2026

Locking Pretrained Weights via Deep Low-Rank Residual Distillation

arXiv:2605.10777v1 Announce Type: new Abstract: The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling the

12 May 2026

Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?

arXiv:2604.10217v1 Announce Type: new Abstract: Cross-modal optical-SAR (Synthetic Aperture Radar) registration is a bottleneck for disaster-response via remote sensing, yet modern image matchers are

14 Apr 2026

Elucidating the Space of Enzymatic Reaction: A Unified Benchmark and Pretrained Model

arXiv:2610.11694v1 Announce Type: cross Abstract: Existing reaction models primarily learn molecular transformations, whereas enzy- matic reactions depend jointly on molecular structure and catalytic

9 Oct 2026

HuC-VideoMAE: Human-Centric Video Masked Autoencoding from synthetic data

arXiv:2610.08433v1 Announce Type: new Abstract: Modern action recognition models rely on video transformers pretrained on massive collections of web-crawled videos, such as Kinetics-700. However, the

7 Oct 2026

Tutorials

From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model

arXiv:2610.05711v1 Announce Type: cross Abstract: Strong image generation models are conditioned on class labels, aligned to frozen pretrained encoders, or built on separately trained autoencoders. Wh

tutorialsarxiv-cs-cv

6 Oct 2026

Local Ai

Localisation-Aware Uncertainty for Pretrained Object Detection

arXiv:2610.01409v1 Announce Type: new Abstract: Reliable uncertainty estimation is essential for deploying object detectors when distribution/covariate shift and adversarial attacks may occur. Existin

local-aiarxiv-cs-cv

2 Oct 2026

Local Ai

What Should We Freeze? Guarded Freezing: Connectivity Shapes the Fine-Tuning of Pretrained Models

arXiv:2609.32124v1 Announce Type: new Abstract: When adapting pre-trained models through fine-tuning, freezing weights alone might not preserve performance, as updates elsewhere can change the inputs

local-aiarxiv-cs-lg

29 Sept 2026

Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning

arXiv:2609.31199v1 Announce Type: cross Abstract: Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D

28 Sept 2026

Pretrained Medical Representations for the Practical Screening of Drug Repositioning Candidates

arXiv:2609.19865v1 Announce Type: new Abstract: Representation learning from medical code sequences in electronic health records and medical claims data has been successful in various clinical applica

applicationsarxiv-cs-lg

18 Sept 2026

Seismic Site Response Prediction from Sparse Observations Using Finite-Element-Pretrained Latent Dynamics

arXiv:2609.20451v1 Announce Type: new Abstract: Numerical site-response predictions often deviate from observations, yet correcting these discrepancies is difficult because records are limited in both

18 Sept 2026

Subdomain-aware representation compression for pretrained image embeddings

arXiv:2609.20213v1 Announce Type: new Abstract: Dimensionality reduction is a well-known technique for improving space efficiency, typically applied uniformly across an entire dataset. This paper inve

18 Sept 2026

MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking

arXiv:2609.16695v1 Announce Type: new Abstract: Large-scale pre-training has transformed representation learning in 2D vision, yet its transferability to 3D single object tracking (SOT) remains insuff

16 Sept 2026

Tutorials

Pretrained, Curriculum-Tuned, and Ensembled: A Tracer-Aware Interactive Segmentation Pipeline for AutoPET V

arXiv:2608.30844v1 Announce Type: cross Abstract: Interactive lesion segmentation in whole-body PET/CT requires a model to provide a strong initial prediction while also responding efficiently to spar

tutorialsarxiv-cs-ai

1 Sept 2026

UniReflex: Plug-and-Play Force Control for Pretrained Generative Policies via Fast-Slow Reflex

arXiv:2608.17432v1 Announce Type: new Abstract: Generative imitation learning policies excel at trajectory planning but lack closed-loop force regulation, while directly incorporating force modalities

applicationsarxiv-cs-ro

19 Aug 2026

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

arXiv:2608.03629v1 Announce Type: new Abstract: A companion paper studies when activation patching and weight-space ablation agree, inside an idealized model where a conditional computation is carried

researcharxiv-cs-ai

5 Aug 2026

SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents

arXiv:2608.01306v1 Announce Type: new Abstract: Latents from vision foundation models (VFMs) are semantically rich and well suited for visual understanding. Recent representation autoencoder methods s

safetyarxiv-cs-cv

4 Aug 2026

Tutorials

CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising

arXiv:2607.28236v1 Announce Type: cross Abstract: Pre-trained language models have significantly improved sentence representation learning, yet their embedding remain sensitive to semantic preserving

tutorialsarxiv-cs-cl

31 Jul 2026

When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

arXiv:2607.04919v1 Announce Type: new Abstract: Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We pro

7 Jul 2026

Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing

arXiv:2607.01690v1 Announce Type: new Abstract: Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually believes the documents' core

safetyarxiv-cs-ai

3 Jul 2026

Tumor-aware augmentation with task-guided attention analysis improves rectal cancer segmentation from magnetic resonance images

model-releasesarxiv-cs-cv

arXiv:2605.05522v2 Announce Type: replace-cross Abstract: Although self-supervised pretraining is expected to learn broadly transferable representations, its effectiveness across imaging modalities su

30 Jun 2026

A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models

arXiv:2502.08660v4 Announce Type: replace Abstract: Semantic role labeling (SRL) is a central natural language processing task for understanding predicate-argument structures within texts and enabling

26 Jun 2026

Nemotron-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context

model-releasesarxiv-cs-cl

arXiv:2606.26493v1 Announce Type: new Abstract: Diffusion language models offer a promising alternative to autoregressive models due to their potential for parallel and iterative generation. However,

26 Jun 2026

TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems

arXiv:2606.25147v1 Announce Type: cross Abstract: User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints inherent to fixed-

25 Jun 2026

UPLOTS: A Unified Pretrained Language Model for Constrained Time-series Generation

arXiv:2606.10466v1 Announce Type: cross Abstract: In time-series generation, existing approaches typically handcraft ortrain a separate model for each dataset, which hinders their scalability and fail

applicationsarxiv-cs-ai

10 Jun 2026

Pretrained, Frozen, Still Leaking: Auditing Cross-Encoder Attribute Transfer in EEG Foundation Models

arXiv:2606.09189v1 Announce Type: cross Abstract: EEG foundation-model releases are usually audited one endpoint at a time: raw-reconstruction, membership inference, identity linkage, or DP-SGD on the

9 Jun 2026

Challenges in Explaining Pretrained Clinical Text Classifiers

arXiv:2605.28060v1 Announce Type: new Abstract: Explaining the predictions of neural models in clinical NLP remains a significant challenge, especially for complex tasks involving long, unstructured m

28 May 2026

Pretrained Approximators for Low-Thrust Trajectory Cost and Reachability

arXiv:2605.26790v1 Announce Type: new Abstract: Low-thrust trajectory design relies heavily on repeated evaluations of fuel consumption and transfer feasibility, which require expensive optimal contro

27 May 2026

Pusa V1.0: Unlocking Temporal Control in Pretrained Video Diffusion Models via Vectorized Timestep Adaptation

arXiv:2507.16116v2 Announce Type: replace Abstract: The rapid advancement of video diffusion models has been hindered by fundamental limitations in temporal modeling, particularly the rigid synchroniz

27 May 2026

Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting

arXiv:2509.15105v3 Announce Type: replace Abstract: Time series forecasting (TSF) is critical in domains like energy, finance, healthcare, and logistics, requiring models that generalize across divers

applicationsarxiv-cs-lg

25 May 2026

Federated Learning for ICD Classification with Lightweight Models and Pretrained Embeddings

model-releasesarxiv-cs-cl

arXiv:2507.03122v2 Announce Type: replace-cross Abstract: This study investigates the feasibility and performance of federated learning (FL) for multi-label ICD code classification using clinical note

20 May 2026

Your CLIP has 164 dimensions of noise: Exploring the embeddings covariance eigenspectrum of contrastively pretrained vision-language transformers

arXiv:2605.14893v1 Announce Type: cross Abstract: Contrastively pre-trained Vision-Language Models (VLMs) serve as powerful feature extractors. Yet, their shared latent spaces are prone to structural

researcharxiv-cs-ai

15 May 2026

Automated Description Generation of Cytologic Findings for Lung Cytological Images Using a Pretrained Vision Model and Dual Text Decoders: Preliminary Study

arXiv:2403.18151v2 Announce Type: replace-cross Abstract: Objective: Cytology plays a crucial role in lung cancer diagnosis. Pulmonary cytology involves cell morphological characterization in the spec

23 Apr 2026

PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation

arXiv:2604.17856v1 Announce Type: new Abstract: Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automatin

applicationsarxiv-cs-cv

21 Apr 2026

TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models

arXiv:2604.15756v1 Announce Type: new Abstract: Vision-language models (VLMs) such as CLIP exhibit strong Out-of-distribution (OOD) detection capabilities by aligning visual and textual representation

20 Apr 2026

VLMs are Native Action Models: Unleashing the Manipulation Capabilities of VLMs with Explicit Action-Space Grounding

arXiv:2610.12832v1 Announce Type: new Abstract: Pretrained vision-language models (VLMs) already exhibit strong visual, spatial and reasoning capabilities relevant to robotic manipulation. The key cha

safetyarxiv-cs-ro

12 Oct 2026

Can phenotypic activity be predicted without experimental readouts?

arXiv:2610.07997v1 Announce Type: new Abstract: Molecular encoders contrastively pretrained on paired molecule-morphology data, such as CLOOME and CellCLIP, have been proposed as cheap surrogates for

7 Oct 2026

Local Ai

Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations

arXiv:2512.21004v2 Announce Type: replace Abstract: Recent advances in pretraining general foundation models have significantly improved performance across diverse downstream tasks. While autoregressi

local-aiarxiv-cs-cv

28 Sept 2026

Multimodal-Multiresolution Foundation Model for Lunar Remote Sensing

arXiv:2609.13283v1 Announce Type: cross Abstract: We present a multimodal foundation model for lunar remote sensing, pretrained from scratch on SomBench, a geographically partitioned corpus of nearly