Search: “pretraining”

Noise is All You Need: Solving Linear Inverse Problems by Noise Combination Sampling with Diffusion Models

arXiv:2510.23633v2 Announce Type: replace-cross Abstract: Pretrained diffusion models have demonstrated strong capabilities in zero-shot inverse problem solving by incorporating observation informatio

researcharxiv-cs-cv

5 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

DGX agent

arXiv:2605.00973v1 Announce Type: new Abstract: Biosignals acquired from different locations on the body often provide temporally ordered views of the same underlying physiological process. However, m

researcharxiv-cs-lg

5 May 2026

Toward a foundational thermal model for residential buildings

arXiv:2605.01364v1 Announce Type: new Abstract: The building energy community lacks a foundational thermal model, i.e., a single pretrained model capable of generalizing across diverse buildings, clim

5 May 2026

Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptation

arXiv:2510.19897v2 Announce Type: replace Abstract: We investigate how agents built on pretrained large language models (LLMs) can learn target classification functions from labeled examples without p

RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

arXiv:2602.18196v3 Announce Type: replace Abstract: Structured dilated attention has an appealing inference-time efficiency knob: it reduces the FLOPs of attention and the KV cache size by a factor of

SIMON: Saliency-aware Integrative Multi-view Object-centric Neural Decoding

arXiv:2605.00401v1 Announce Type: new Abstract: Recent EEG-to-image retrieval methods leverage pretrained vision encoders and foveation-inspired priors, but typically assume a fixed, center-focused vi

TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning

arXiv:2605.00015v1 Announce Type: cross Abstract: Time Series Foundation Models (TSFMs) advance generalization and data efficiency in time series forecasting by unified large-scale pretraining. But TS

applicationsarxiv-cs-cv

Understanding Cognitive States from Head & Hand Motion Data

arXiv:2509.24255v2 Announce Type: replace-cross Abstract: As virtual reality (VR) becomes widespread, head and hand motion data captured by consumer systems has become substantially more common. Howev

HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Data

model-releasesarxiv-cs-ai

arXiv:2604.04306v2 Announce Type: replace-cross Abstract: The increasing frequency and severity of climate related disasters have intensified the need for real time monitoring, early warning, and info

1 May 2026

Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Prediction

model-releasesarxiv-cs-lg

arXiv:2604.26498v1 Announce Type: new Abstract: The rapid growth of molecular foundation models and general-purpose large language models has encouraged a scale-centric view of artificial intelligence

30 Apr 2026

Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models

model-releasesarxiv-cs-ai

arXiv:2601.03423v3 Announce Type: replace-cross Abstract: Adapting language models to the clinical domain through continued pretraining and instruction tuning requires costly retraining for each new m

30 Apr 2026

Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising

arXiv:2604.26694v1 Announce Type: cross Abstract: We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruc

researcharxiv-cs-ai

30 Apr 2026

A Systematic Post-Train Framework for Video Generation

arXiv:2604.25427v1 Announce Type: new Abstract: While large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content, a signif

29 Apr 2026

PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection

arXiv:2604.25599v1 Announce Type: cross Abstract: Code understanding models increasingly rely on pretrained language models (PLMs) and graph neural networks (GNNs), which capture complementary semanti

29 Apr 2026

Agents

Accelerating Reinforcement Learning for Wind Farm Control via Expert Demonstrations

arXiv:2604.22794v1 Announce Type: cross Abstract: Reinforcement learning (RL) offers a promising approach for adaptive wind farm flow control, yet its practical deployment is hindered by slow training

agentsarxiv-cs-lg

Bridging Restoration and Generation Manifolds in One-Step Diffusion for Real-World Super-Resolution

arXiv:2604.24136v1 Announce Type: new Abstract: Pretrained diffusion models have revolutionized real-world image super-resolution (Real-ISR) but suffer from computational bottlenecks due to iterative

applicationsarxiv-cs-cv

Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach

model-releasesarxiv-cs-lg

arXiv:2601.18399v2 Announce Type: replace Abstract: Separating liquid-liquid dispersions in gravity settlers is critical in chemical, pharmaceutical, and recycling processes. The dense-packed zone hei

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers

arXiv:2604.23452v1 Announce Type: new Abstract: Vision Transformers trained only on image classification routinely transfer to tasks that demand spatial understanding, yet they receive no spatial supe

researcharxiv-cs-cv

Learning an Image Editing Model without Image Editing Pairs

arXiv:2510.14978v2 Announce Type: replace Abstract: Recent image editing models have achieved impressive results while following natural language editing instructions, but they rely on supervised fine

researcharxiv-cs-cv

LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation

arXiv:2604.00829v3 Announce Type: replace-cross Abstract: Adapting pretrained language models (LMs) into vision-language models (VLMs) can degrade their native linguistic capability due to representat

safetyarxiv-cs-cl

Quasi-Equivariant Metanetworks

model-releasesarxiv-cs-lg

arXiv:2604.23720v1 Announce Type: new Abstract: Metanetworks are neural architectures designed to operate directly on pretrained weights to perform downstream tasks. However, the parameter space serve

Tutorials

SRL-CLIP: Efficient CLIP Video Adaptation via Structured Semantic Role Labels

arXiv:2401.07669v2 Announce Type: replace Abstract: Adapting CLIP for videos has gained popularity due to its semantic and rich representation. While CLIP is a good starting point, it typically underg

tutorialsarxiv-cs-cv

The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry

arXiv:2604.22778v1 Announce Type: cross Abstract: We present the first systematic study of weight matrix singular value spectra during transformer pretraining, tracking full SVD decompositions of ever

researcharxiv-cs-ai

Selective Contrastive Learning For Gloss Free Sign Language Translation

arXiv:2604.22374v1 Announce Type: new Abstract: Sign language translation (SLT) converts continuous sign videos into spoken-language text, yet it remains challenging due to the intrinsic modality mism

safetyarxiv-cs-cl

27 Apr 2026

Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging

model-releasesarxiv-cs-cv

arXiv:2510.05971v3 Announce Type: replace Abstract: The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing sel

27 Apr 2026

Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection

arXiv:2601.02438v3 Announce Type: replace-cross Abstract: Software vulnerability detection can be formulated as a binary classification problem that determines whether a given code snippet contains se

researcharxiv-cs-ai

Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training

arXiv:2604.21741v1 Announce Type: new Abstract: Post-training is essential for turning pretrained generalist robot policies into reliable task-specific controllers, but existing human-in-the-loop pipe

safetyarxiv-cs-ro

Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection

arXiv:2604.21728v1 Announce Type: new Abstract: Pretrained vision-language models such as CLIP exhibit strong zero-shot generalization but remain sensitive to distribution shifts. Test-time adaptation

Tumor-anchored deep feature random forests for out-of-distribution detection in lung cancer segmentation

arXiv:2512.08216v3 Announce Type: replace-cross Abstract: Accurate segmentation of lung tumors from 3D computed tomography (CT) scans is essential for automated treatment planning and response assessm

JoyAI-RA 0.1: A Foundation Model for Robotic Autonomy

arXiv:2604.20100v1 Announce Type: new Abstract: Robotic autonomy in open-world environments is fundamentally limited by insufficient data diversity and poor cross-embodiment generalization. Existing r

applicationsarxiv-cs-ro

23 Apr 2026

Membership Inference for Contrastive Pre-training Models with Text-only PII Queries

arXiv:2603.14222v2 Announce Type: replace-cross Abstract: Contrastive pretraining models such as CLIP and CLAP, serve as the ubiquitous perceptual backbones for modern multimodal large models, yet the

safetyarxiv-cs-ai

23 Apr 2026

NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

arXiv:2602.15353v2 Announce Type: replace-cross Abstract: Large pretrained language models and neural reasoning systems have advanced many natural language tasks, yet they remain challenged by knowled

safetyarxiv-cs-ai

23 Apr 2026

ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning

model-releasesarxiv-cs-ai

arXiv:2604.19254v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) reduces the training cost of full-parameter fine-tuning for large language models (LLMs) by training only a sma

22 Apr 2026

StochasTok: Improving Fine-Grained Subword Understanding in LLMs

arXiv:2506.01687v3 Announce Type: replace Abstract: Subword-level understanding is integral to numerous tasks, including understanding multi-digit numbers, spelling mistakes, abbreviations, rhyming, a

researcharxiv-cs-cl

22 Apr 2026

Decomposing the Depth Profile of Fine-Tuning

arXiv:2604.17177v1 Announce Type: new Abstract: Fine-tuning adapts pretrained networks to new objectives. Whether the resulting depth profile of representational change reflects an intrinsic property

Domain-oriented RAG Assessment (DoRA): Synthetic Benchmarking for RAG-based Question Answering on Defense Documents

arXiv:2604.17943v1 Announce Type: new Abstract: Open-domain RAG benchmarks over public corpora can overestimate deployment performance due to pretraining overlap and weak attribution requirements. We

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

arXiv:2604.18109v1 Announce Type: new Abstract: This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover t

Generative Semantic Communication via Alternating Dual-Domain Posterior Sampling

arXiv:2604.16796v1 Announce Type: new Abstract: Generative semantic communication (SemCom) harnesses pretrained generative priors to improve the perceptual quality of wireless image transmission. Exis

Lizard: An Efficient Linearization Framework for Large Language Models

arXiv:2507.09025v4 Announce Type: replace Abstract: We propose Lizard, a linearization framework that transforms pretrained Transformer-based Large Language Models (LLMs) into subquadratic architectur

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

arXiv:2604.18360v1 Announce Type: cross Abstract: Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, the

applicationsarxiv-cs-cl

Agents

OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

arXiv:2604.17915v1 Announce Type: new Abstract: Vision-Language Models(VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured out

agentsarxiv-cs-cv

REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning

arXiv:2604.17257v1 Announce Type: new Abstract: Recent text embedding models are often adapted to specialized domains via contrastive pre-finetuning (PFT) on a naive collection of scattered, heterogen

safetyarxiv-cs-cl

Hardware

Scalable and Adaptive Parallel Training of Graph Transformer on Large Graphs

arXiv:2604.16715v1 Announce Type: cross Abstract: Graph foundation models have demonstrated remarkable adaptability across diverse downstream tasks through large-scale pretraining on graphs. However,

hardwarearxiv-cs-lg

SHRUG-FM: Reliability-Aware Foundation Models for Earth Observation

model-releasesarxiv-cs-cv

arXiv:2511.10370v2 Announce Type: replace Abstract: Geospatial foundation models (GFMs) for Earth observation often fail to perform reliably in environments underrepresented during pretraining. We int

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton

arXiv:2510.09378v2 Announce Type: replace Abstract: Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This rais

HyCal: A Training-Free Prototype Calibration Method for Cross-Discipline Few-Shot Class-Incremental Learning

model-releasesarxiv-cs-cv

arXiv:2604.15678v1 Announce Type: new Abstract: Pretrained Vision-Language Models (VLMs) like CLIP show promise in continual learning, but existing Few-Shot Class-Incremental Learning (FSCIL) methods

20 Apr 2026

Tutorials

On Optimal Hyperparameters for Differentially Private Deep Transfer Learning

arXiv:2510.20616v2 Announce Type: replace Abstract: Differentially private (DP) transfer learning, i.e., fine-tuning a pretrained model on private data, is the current state-of-the-art approach for tr

tutorialsarxiv-cs-lg

20 Apr 2026

TriagerX: Dual Transformers for Bug Triaging Tasks with Content and Interaction Based Rankings