Search: “pretraining”

Steerable Visual Representations

arXiv:2604.02327v2 Announce Type: replace-cross Abstract: Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tas

The Platonic Defense: Backdoor Defense for Self-Supervised Encoders in the Era of Large Scale Pre-training

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2606.29451v1 Announce Type: new Abstract: Self-supervised learning (SSL) pretrained models have become a dominant paradigm for visual representation learning, but they are vulnerable to backdoor

researcharxiv-cs-cv

30 Jun 2026

Towards Evaluating Data Priors for Tabular Foundation Models

arXiv:2606.29241v1 Announce Type: new Abstract: Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However,

researcharxiv-cs-lg

TrafficAlign: Aligning Large Language Models for Traffic Scenario Generation

arXiv:2606.29097v1 Announce Type: new Abstract: Recent research has investigated the use of large language models (LLMs) to generate traffic scenarios for autonomous driving. However, pretrained LLMs

Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D

arXiv:2606.29964v1 Announce Type: new Abstract: Score distillation turns a pretrained 2D diffusion model into a 3D generator, but the per-step gradient is estimated from a single randomly chosen view:

Agents

Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution

arXiv:2606.20014v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has achieved strong performance in sequential decision-making, yet scaling to complex multi-agent environments rem

agentsarxiv-cs-ai

Large Language Model Teaches Visual Students: Cross-Modality Transfer of Fine-Grained Conceptual Knowledge

arXiv:2606.27527v1 Announce Type: cross Abstract: Large Language Models (LLMs) possess broad conceptual knowledge acquired through large-scale text pretraining, yet their potential to supervise models

StableMotion: One-Step Motion Estimation with Diffusion Prior

arXiv:2505.06668v2 Announce Type: replace Abstract: We present StableMotion, a novel framework that leverages geometric and content priors from pretrained large-scale image diffusion models for motion

researcharxiv-cs-cv

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

model-releasesarxiv-cs-lg

arXiv:2606.28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models. When applied sequentially across

Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean

model-releasesarxiv-cs-cl

arXiv:2606.26618v1 Announce Type: new Abstract: Large pretrained text-to-speech (TTS) models sound almost human for well-resourced languages, but much worse for languages that are rare in their traini

Data-Free Reservoir Features for Efficient Long-Horizon Cold-Start Continual Learning

arXiv:2606.27095v1 Announce Type: cross Abstract: Cold-start exemplar-free class-incremental learning requires learning a growing set of classes without replay, external pretraining, or a large initia

safetyarxiv-cs-ai

Generative Retrieval via Diffusion Transformer with Metric-Ordered Sequence Training and Hybrid-Policy Preference Optimization

arXiv:2606.26899v1 Announce Type: new Abstract: Embedding-based retrieval ranks items by their similarity to a query in a shared vector space and usually aims to return the highest-scoring items. In m

safetyarxiv-cs-ai

VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation

arXiv:2606.26534v1 Announce Type: cross Abstract: Recently, zero-shot text-to-speech (TTS) has enabled high-fidelity and expressive speech synthesis, but it often fails to imitate unseen speaking styl

Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns

arXiv:2606.25010v1 Announce Type: cross Abstract: Neural scaling laws for transformer language models predict smooth improvements in pretraining loss with increasing parameters, but downstream capabil

researcharxiv-cs-cl

Naturalness Predicts but Does Not Cause Transferability in Image Encodings of Real-World Streams

arXiv:2606.25844v1 Announce Type: new Abstract: A common practice converts a one-dimensional signal into an image so that a vision backbone pretrained on natural photographs can be reused for recognit

applicationsarxiv-cs-cv

TACO: Towards Task-Consistent Open-Vocabulary Adaptation in Video Recognition

arXiv:2606.25478v1 Announce Type: new Abstract: Adapting CLIP for open-vocabulary video recognition necessitates a delicate balance between newly acquired video knowledge and the pretrained generaliza

Velocity Prediction in Automatic Guitar Transcription

arXiv:2606.24912v1 Announce Type: cross Abstract: Automatic Music Transcription (AMT) models have achieved a high level of success in polyphonic transcription of various instruments. Velocity, typical

DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model

arXiv:2606.24051v1 Announce Type: new Abstract: Vision-Language-Action driving models convert a pretrained Vision-Language Model into a driving policy, allowing them to use world knowledge and follow

Ensemble Learning for Large Language Models in Text and Code Generation: A Survey

arXiv:2503.13505v3 Announce Type: replace-cross Abstract: Generative Pretrained Transformers (GPTs) are foundational Large Language Models (LLMs) for text generation. However, individual LLMs often pr

applicationsarxiv-cs-ai

MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systems

arXiv:2504.17070v3 Announce Type: replace-cross Abstract: Robots need task planning methods to achieve goals that require more than one action. Recently, large pretrained models have demonstrated impr

Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs

model-releasesarxiv-cs-ai

arXiv:2606.23938v1 Announce Type: new Abstract: Driving VLA models incorporating Chain-of-Thought (CoT) reasoning are attractive because they leverage pretrained VLM representations and expose interme

PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

arXiv:2606.24102v1 Announce Type: new Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot direc

researcharxiv-cs-cl

TuringViT: Making SOTA Vision Transformers Accessible to All

arXiv:2606.24253v1 Announce Type: new Abstract: Modern VLMs and VLA systems commonly adopt off-the-shelf ViTs such as SigLIP2 as visual encoders, but diverse downstream requirements in latency, tempor

applicationsarxiv-cs-cv

What Do Flow-Based Inverse Solvers Approximate? A Posterior-Transport View

arXiv:2606.24516v1 Announce Type: new Abstract: A growing family of training-free solvers -- FlowDPS, FLOWER, PnP-Flow and their diffusion ancestors (DPS, DAPS) -- repurpose a pretrained flow-matching

Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

arXiv:2606.24178v1 Announce Type: cross Abstract: Pretrained vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object clas

AEF-Econ: Toward Plug-and-Play Socioeconomic Foundation Embeddings from AlphaEarth for Urban Remote Sensing

arXiv:2606.20697v1 Announce Type: new Abstract: AlphaEarth Foundations (AEF) unify global remote sensing foundation embeddings through multimodal self-supervised learning, but their pretraining focuse

Evo-RAD: Navigating Rare Retinal Disease Diagnosis via Self-Evolving Agentic Retrieval

arXiv:2606.22955v1 Announce Type: new Abstract: Large-scale pretrained foundation models have revolutionized general medical screening, but often falter on rare diseases because such conditions are un

Flatness Preserves Instruction Following in Vision-Language-Action Models

arXiv:2606.23641v1 Announce Type: new Abstract: Vision-language-action (VLA) models have the potential for open-world generalization by leveraging pretrained vision-language representations, yet downs

applicationsarxiv-cs-ro

FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation

arXiv:2606.20867v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models enable general-purpose robotic control via large-scale multimodal pretraining, yet their effectiveness under few-sho

GraphPFN: A Prior-Data Fitted Graph Foundation Model

arXiv:2509.21489v3 Announce Type: replace Abstract: Graph foundation models face several fundamental challenges including transferability across diverse domains and data scarcity, which calls into que

applicationsarxiv-cs-lg

MOOZY: A Patient-First Foundation Model for Computational Pathology

arXiv:2603.27048v3 Announce Type: replace Abstract: Computational pathology needs whole-slide image (WSI) foundation models that transfer across diverse clinical tasks, yet current approaches remain l

PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models

arXiv:2606.22958v1 Announce Type: cross Abstract: Inference-time alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance, atte

Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian

model-releasesarxiv-cs-lg

arXiv:2606.21876v1 Announce Type: new Abstract: The Categorical Jacobian (CJ) of Zhang et al. (2024) reads protein contacts from a language model by perturbing every residue with every alternative ami

Tutorials

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

arXiv:2606.22938v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains

tutorialsarxiv-cs-lg

Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers

arXiv:2606.22383v1 Announce Type: new Abstract: Parameter-efficient fine-tuning (PEFT) has become a practical solution for adapting large pretrained vision transformers (ViTs) to downstream tasks whil

Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning

model-releasesarxiv-cs-lg

arXiv:2606.14970v2 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse dow

TSAuditor: A time-series auditing framework [P]

researchr-machinelearning

TSAuditor is a contamination auditing framework for time series foundation models that uses probe adaptation dynamics to detect when evaluation datasets may have been exposed during pretraining, valid

20 Jun 2026

Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs

arXiv:2602.00945v2 Announce Type: replace-cross Abstract: LLMs are multilingual by training, yet their lingua franca is often English, reflecting English language dominance in pretraining. Other langu

11 Jun 2026

Tac-DINO: Learning Vision-Tactile Features with Patch Alignment

arXiv:2606.12069v1 Announce Type: new Abstract: Touch is the primary medium through which humans interact with the environment. Currently, tactile learning mainly focuses on image-level pretraining or

11 Jun 2026

Are We Evaluating Knowledge or Phrasing? Mitigating MCQA Sensitivity with ParaEval

arXiv:2606.10657v1 Announce Type: new Abstract: Multiple-choice (MCQA) benchmarks are the standard for evaluating pretrained large language models, but their reliance on log-likelihood scoring makes t

researcharxiv-cs-cl

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development 'Any topic related to buil…

model-releasesyann-lecun--x

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development 'Any topic related to building pretraining pipelines, distributed training infrastruct

Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning

model-releasesarxiv-cs-ai

arXiv:2606.10196v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) aims to adapt pretrained models with a small trainable parameter subset, however, most existing methods choose

IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder

arXiv:2606.11096v1 Announce Type: new Abstract: Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing s

Reasoning over Semantic IDs Enhances Generative Recommendation

arXiv:2603.23183v2 Announce Type: replace-cross Abstract: Recent advances in generative recommendation have leveraged pretrained LLMs by formulating sequential recommendation as autoregressive generat

safetyarxiv-cs-ai

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

model-releasesarxiv-cs-ai

arXiv:2606.10929v1 Announce Type: cross Abstract: Task vectors, LoRA, activation steering, and random search around pretrained weights all suggest that learned behaviour can be controlled by linear di

Speaker Group Encoding in Self-supervised Speech Recognition Models

arXiv:2606.10654v1 Announce Type: new Abstract: We investigate what self-supervised speech recognition models (S3Ms) learn about speaker groups (SGs). We examine several states of S3Ms: pretrained, fi

safetyarxiv-cs-cl

STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model

arXiv:2606.10653v1 Announce Type: new Abstract: Although pretrained text-to-image (T2I) generation models can produce high-quality images, they often fail to faithfully reflect the semantic intent of

BCG-FM: A Foundation Model for Ambient Cardiac Health Sensing

arXiv:2606.07692v1 Announce Type: cross Abstract: Foundation models for wearable biosignals have matched or exceeded supervised specialists across a range of clinical tasks, yet all rely on modalities