Search: “pretraining”

BoHA: Blockwise Hadamard Product Adaptation for Parameter-Efficient Fine-Tuning

arXiv:2509.21637v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning (PEFT) of large language models trains a small task-specific parameter set while keeping the pretrained model frozen

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Cognitive Agent Compilation for Explicit Problem Solver Modeling

DGX agent

arXiv:2605.07040v1 Announce Type: cross Abstract: Large language models (LLMs) are widely used for tutoring, feedback generation, and content creation, but their broad pretraining makes them hard to c

safetyarxiv-cs-ai

11 May 2026

Continually Evolving Skill Knowledge in Vision Language Action Model

arXiv:2511.18085v4 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models show promising knowledge accumulation ability from pretraining, yet continual learning in VLA remains chal

Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment

arXiv:2605.06885v1 Announce Type: cross Abstract: Diffusion language models (DLMs) have recently demonstrated capabilities that complement standard autoregressive (AR) models, particularly in non-sequ

safetyarxiv-cs-ai

ModelLens: Finding the Best for Your Task from Myriads of Models

arXiv:2605.07075v1 Announce Type: new Abstract: The open-source model ecosystem now contains hundreds of thousands of pretrained models, yet picking the best model for a new dataset is increasingly in

Tutorials

Tables Guide Vision: Learning to See the Heart through Tabular Data

arXiv:2503.14998v3 Announce Type: replace Abstract: Contrastive learning methods in computer vision typically rely on augmented views of the same image or multimodal pretraining strategies that align

tutorialsarxiv-cs-cv

TopoPrune: Robust Data Pruning via Unified Latent Space Topology

arXiv:2602.02739v2 Announce Type: replace-cross Abstract: Geometric data pruning methods, while practical for leveraging pretrained models, are fundamentally unstable. Their reliance on extrinsic geom

TRAJGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations

arXiv:2605.06990v1 Announce Type: new Abstract: Multimodal self-supervised learning (MSSL) has emerged as a key paradigm for pretraining geospatial foundation models. However, existing geospatial MSSL

Understanding Robustness of Model Editing in Code LLMs

arXiv:2511.03182v2 Announce Type: replace-cross Abstract: Large language models (LLMs) for code are increasingly used in software development, but they remain static after pretraining while APIs and s

The Top AI Papers of the Week (May 4 - 10) - Conductor - HeavySkill - Horizon Generalization - 1,000 Synthetic Computers - Self-Improving Pr…

The Top AI Papers of the Week (May 4 - 10) - Conductor - HeavySkill - Horizon Generalization - 1,000 Synthetic Computers - Self-Improving Pretraining - Coordination as Architecture - Connect Four Alph

researchdair-ai--x

10 May 2026

EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics

arXiv:2605.03871v1 Announce Type: new Abstract: Language models encode substantial evaluative knowledge from pretraining, yet current post-training methods rely on external supervision (human annotati

7 May 2026

Implicit Representations of Grammaticality in Language Models

arXiv:2605.05197v1 Announce Type: new Abstract: Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitte

researcharxiv-cs-cl

7 May 2026

Agents

Technical highlights: CogViT Vision Encoder - Built with dual-teacher distillation: SigLIP2 for semantics, DINOv3 for texture. A two-stage r…

Technical highlights: CogViT Vision Encoder - Built with dual-teacher distillation: SigLIP2 for semantics, DINOv3 for texture. A two-stage recipe, masked modeling, then contrastive pretraining, with Q

agentszhipu-ai--x

7 May 2026

Hardware

Learning Zero-Shot Subject-Driven Video Generation Using 1% Compute

arXiv:2504.17816v3 Announce Type: replace Abstract: Subject-driven video generation (SDV-Gen) aims to produce videos of a specific subject by adapting a pretrained video model, enabling personalized a

hardwarearxiv-cs-cv

Memory-Efficient Continual Learning with CLIP Models

arXiv:2605.03866v1 Announce Type: new Abstract: Contrastive Language-Image Pretraining (CLIP) models excel at understanding image-text relationships but struggle with adapting to new data without forg

Mix3R: Mixing Feed-forward Reconstruction and Generative 3D Priors for Joint Multi-view Aligned 3D Reconstruction and Pose Estimation

arXiv:2605.03359v1 Announce Type: new Abstract: Recent trends in sparse-view 3D reconstruction have taken two different paths: feed-forward reconstruction that predicts pixel-aligned point maps withou

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning

model-releasesarxiv-cs-cl

arXiv:2605.03229v1 Announce Type: new Abstract: Adapting a pretrained language model to a new task often hurts the general capabilities it already had, a problem known as catastrophic forgetting. Spar

LatentDiff: Scaling Semantic Dataset Comparison to Millions of Images

model-releasesarxiv-cs-cv

arXiv:2605.00899v1 Announce Type: new Abstract: We present LatentDiff, a scalable framework for semantic dataset comparison that operates directly in the latent space of pretrained vision encoders. By

Noise is All You Need: Solving Linear Inverse Problems by Noise Combination Sampling with Diffusion Models

arXiv:2510.23633v2 Announce Type: replace-cross Abstract: Pretrained diffusion models have demonstrated strong capabilities in zero-shot inverse problem solving by incorporating observation informatio

researcharxiv-cs-cv

Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

arXiv:2605.00973v1 Announce Type: new Abstract: Biosignals acquired from different locations on the body often provide temporally ordered views of the same underlying physiological process. However, m

Toward a foundational thermal model for residential buildings

arXiv:2605.01364v1 Announce Type: new Abstract: The building energy community lacks a foundational thermal model, i.e., a single pretrained model capable of generalizing across diverse buildings, clim

Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptation

model-releasesarxiv-cs-cl

arXiv:2510.19897v2 Announce Type: replace Abstract: We investigate how agents built on pretrained large language models (LLMs) can learn target classification functions from labeled examples without p

RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

arXiv:2602.18196v3 Announce Type: replace Abstract: Structured dilated attention has an appealing inference-time efficiency knob: it reduces the FLOPs of attention and the KV cache size by a factor of

SIMON: Saliency-aware Integrative Multi-view Object-centric Neural Decoding

arXiv:2605.00401v1 Announce Type: new Abstract: Recent EEG-to-image retrieval methods leverage pretrained vision encoders and foveation-inspired priors, but typically assume a fixed, center-focused vi

Applications

TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning

arXiv:2605.00015v1 Announce Type: cross Abstract: Time Series Foundation Models (TSFMs) advance generalization and data efficiency in time series forecasting by unified large-scale pretraining. But TS

applicationsarxiv-cs-cv

Understanding Cognitive States from Head & Hand Motion Data

arXiv:2509.24255v2 Announce Type: replace-cross Abstract: As virtual reality (VR) becomes widespread, head and hand motion data captured by consumer systems has become substantially more common. Howev

HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Data

arXiv:2604.04306v2 Announce Type: replace-cross Abstract: The increasing frequency and severity of climate related disasters have intensified the need for real time monitoring, early warning, and info

1 May 2026

Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Prediction

arXiv:2604.26498v1 Announce Type: new Abstract: The rapid growth of molecular foundation models and general-purpose large language models has encouraged a scale-centric view of artificial intelligence

30 Apr 2026

Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models

arXiv:2601.03423v3 Announce Type: replace-cross Abstract: Adapting language models to the clinical domain through continued pretraining and instruction tuning requires costly retraining for each new m

30 Apr 2026

Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising

arXiv:2604.26694v1 Announce Type: cross Abstract: We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruc

30 Apr 2026

A Systematic Post-Train Framework for Video Generation

arXiv:2604.25427v1 Announce Type: new Abstract: While large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content, a signif

29 Apr 2026

PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection

arXiv:2604.25599v1 Announce Type: cross Abstract: Code understanding models increasingly rely on pretrained language models (PLMs) and graph neural networks (GNNs), which capture complementary semanti

29 Apr 2026

Agents

Accelerating Reinforcement Learning for Wind Farm Control via Expert Demonstrations

arXiv:2604.22794v1 Announce Type: cross Abstract: Reinforcement learning (RL) offers a promising approach for adaptive wind farm flow control, yet its practical deployment is hindered by slow training

agentsarxiv-cs-lg

Applications

Bridging Restoration and Generation Manifolds in One-Step Diffusion for Real-World Super-Resolution

arXiv:2604.24136v1 Announce Type: new Abstract: Pretrained diffusion models have revolutionized real-world image super-resolution (Real-ISR) but suffer from computational bottlenecks due to iterative

applicationsarxiv-cs-cv

Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach

arXiv:2601.18399v2 Announce Type: replace Abstract: Separating liquid-liquid dispersions in gravity settlers is critical in chemical, pharmaceutical, and recycling processes. The dense-packed zone hei

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers

arXiv:2604.23452v1 Announce Type: new Abstract: Vision Transformers trained only on image classification routinely transfer to tasks that demand spatial understanding, yet they receive no spatial supe

researcharxiv-cs-cv

Learning an Image Editing Model without Image Editing Pairs

arXiv:2510.14978v2 Announce Type: replace Abstract: Recent image editing models have achieved impressive results while following natural language editing instructions, but they rely on supervised fine

researcharxiv-cs-cv

LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation

arXiv:2604.00829v3 Announce Type: replace-cross Abstract: Adapting pretrained language models (LMs) into vision-language models (VLMs) can degrade their native linguistic capability due to representat

safetyarxiv-cs-cl

Quasi-Equivariant Metanetworks

arXiv:2604.23720v1 Announce Type: new Abstract: Metanetworks are neural architectures designed to operate directly on pretrained weights to perform downstream tasks. However, the parameter space serve

Tutorials

SRL-CLIP: Efficient CLIP Video Adaptation via Structured Semantic Role Labels

arXiv:2401.07669v2 Announce Type: replace Abstract: Adapting CLIP for videos has gained popularity due to its semantic and rich representation. While CLIP is a good starting point, it typically underg

tutorialsarxiv-cs-cv

The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry

arXiv:2604.22778v1 Announce Type: cross Abstract: We present the first systematic study of weight matrix singular value spectra during transformer pretraining, tracking full SVD decompositions of ever

Selective Contrastive Learning For Gloss Free Sign Language Translation

arXiv:2604.22374v1 Announce Type: new Abstract: Sign language translation (SLT) converts continuous sign videos into spoken-language text, yet it remains challenging due to the intrinsic modality mism

safetyarxiv-cs-cl

27 Apr 2026

Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging

model-releasesarxiv-cs-cv

arXiv:2510.05971v3 Announce Type: replace Abstract: The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing sel

27 Apr 2026

Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection

arXiv:2601.02438v3 Announce Type: replace-cross Abstract: Software vulnerability detection can be formulated as a binary classification problem that determines whether a given code snippet contains se

Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training

arXiv:2604.21741v1 Announce Type: new Abstract: Post-training is essential for turning pretrained generalist robot policies into reliable task-specific controllers, but existing human-in-the-loop pipe

safetyarxiv-cs-ro

Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection

arXiv:2604.21728v1 Announce Type: new Abstract: Pretrained vision-language models such as CLIP exhibit strong zero-shot generalization but remain sensitive to distribution shifts. Test-time adaptation

Tumor-anchored deep feature random forests for out-of-distribution detection in lung cancer segmentation

arXiv:2512.08216v3 Announce Type: replace-cross Abstract: Accurate segmentation of lung tumors from 3D computed tomography (CT) scans is essential for automated treatment planning and response assessm

Applications

JoyAI-RA 0.1: A Foundation Model for Robotic Autonomy