Search: “pretraining”

Generative Retrieval via Diffusion Transformer with Metric-Ordered Sequence Training and Hybrid-Policy Preference Optimization

arXiv:2606.26899v1 Announce Type: new Abstract: Embedding-based retrieval ranks items by their similarity to a query in a shared vector space and usually aims to return the highest-scoring items. In m

safetyarxiv-cs-ai

26 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation

DGX agent

arXiv:2606.26534v1 Announce Type: cross Abstract: Recently, zero-shot text-to-speech (TTS) has enabled high-fidelity and expressive speech synthesis, but it often fails to imitate unseen speaking styl

researcharxiv-cs-ai

26 Jun 2026

Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns

arXiv:2606.25010v1 Announce Type: cross Abstract: Neural scaling laws for transformer language models predict smooth improvements in pretraining loss with increasing parameters, but downstream capabil

researcharxiv-cs-cl

Naturalness Predicts but Does Not Cause Transferability in Image Encodings of Real-World Streams

arXiv:2606.25844v1 Announce Type: new Abstract: A common practice converts a one-dimensional signal into an image so that a vision backbone pretrained on natural photographs can be reused for recognit

applicationsarxiv-cs-cv

TACO: Towards Task-Consistent Open-Vocabulary Adaptation in Video Recognition

arXiv:2606.25478v1 Announce Type: new Abstract: Adapting CLIP for open-vocabulary video recognition necessitates a delicate balance between newly acquired video knowledge and the pretrained generaliza

Velocity Prediction in Automatic Guitar Transcription

arXiv:2606.24912v1 Announce Type: cross Abstract: Automatic Music Transcription (AMT) models have achieved a high level of success in polyphonic transcription of various instruments. Velocity, typical

DriveStack-VLA: Render-Teacher Alignment for BEV-Based DeepStack Vision-Language-Action Model

arXiv:2606.24051v1 Announce Type: new Abstract: Vision-Language-Action driving models convert a pretrained Vision-Language Model into a driving policy, allowing them to use world knowledge and follow

Ensemble Learning for Large Language Models in Text and Code Generation: A Survey

arXiv:2503.13505v3 Announce Type: replace-cross Abstract: Generative Pretrained Transformers (GPTs) are foundational Large Language Models (LLMs) for text generation. However, individual LLMs often pr

applicationsarxiv-cs-ai

MuTRAP: Multi-trigger Trojans Attacking Robot Task Planning Systems

arXiv:2504.17070v3 Announce Type: replace-cross Abstract: Robots need task planning methods to achieve goals that require more than one action. Recently, large pretrained models have demonstrated impr

Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs

arXiv:2606.23938v1 Announce Type: new Abstract: Driving VLA models incorporating Chain-of-Thought (CoT) reasoning are attractive because they leverage pretrained VLM representations and expose interme

PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Models

arXiv:2606.24102v1 Announce Type: new Abstract: Most electronic health record (EHR) foundation models encode clinical events as discrete event tokens from a fixed vocabulary and therefore cannot direc

researcharxiv-cs-cl

TuringViT: Making SOTA Vision Transformers Accessible to All

arXiv:2606.24253v1 Announce Type: new Abstract: Modern VLMs and VLA systems commonly adopt off-the-shelf ViTs such as SigLIP2 as visual encoders, but diverse downstream requirements in latency, tempor

applicationsarxiv-cs-cv

What Do Flow-Based Inverse Solvers Approximate? A Posterior-Transport View

arXiv:2606.24516v1 Announce Type: new Abstract: A growing family of training-free solvers -- FlowDPS, FLOWER, PnP-Flow and their diffusion ancestors (DPS, DAPS) -- repurpose a pretrained flow-matching

Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

arXiv:2606.24178v1 Announce Type: cross Abstract: Pretrained vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object clas

AEF-Econ: Toward Plug-and-Play Socioeconomic Foundation Embeddings from AlphaEarth for Urban Remote Sensing

arXiv:2606.20697v1 Announce Type: new Abstract: AlphaEarth Foundations (AEF) unify global remote sensing foundation embeddings through multimodal self-supervised learning, but their pretraining focuse

Evo-RAD: Navigating Rare Retinal Disease Diagnosis via Self-Evolving Agentic Retrieval

arXiv:2606.22955v1 Announce Type: new Abstract: Large-scale pretrained foundation models have revolutionized general medical screening, but often falter on rare diseases because such conditions are un

Flatness Preserves Instruction Following in Vision-Language-Action Models

arXiv:2606.23641v1 Announce Type: new Abstract: Vision-language-action (VLA) models have the potential for open-world generalization by leveraging pretrained vision-language representations, yet downs

applicationsarxiv-cs-ro

FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation

arXiv:2606.20867v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models enable general-purpose robotic control via large-scale multimodal pretraining, yet their effectiveness under few-sho

GraphPFN: A Prior-Data Fitted Graph Foundation Model

arXiv:2509.21489v3 Announce Type: replace Abstract: Graph foundation models face several fundamental challenges including transferability across diverse domains and data scarcity, which calls into que

applicationsarxiv-cs-lg

MOOZY: A Patient-First Foundation Model for Computational Pathology

arXiv:2603.27048v3 Announce Type: replace Abstract: Computational pathology needs whole-slide image (WSI) foundation models that transfer across diverse clinical tasks, yet current approaches remain l

PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models

arXiv:2606.22958v1 Announce Type: cross Abstract: Inference-time alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance, atte

Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian

arXiv:2606.21876v1 Announce Type: new Abstract: The Categorical Jacobian (CJ) of Zhang et al. (2024) reads protein contacts from a language model by perturbing every residue with every alternative ami

Tutorials

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

arXiv:2606.22938v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains

tutorialsarxiv-cs-lg

Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers

arXiv:2606.22383v1 Announce Type: new Abstract: Parameter-efficient fine-tuning (PEFT) has become a practical solution for adapting large pretrained vision transformers (ViTs) to downstream tasks whil

Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning

arXiv:2606.14970v2 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse dow

Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs

arXiv:2602.00945v2 Announce Type: replace-cross Abstract: LLMs are multilingual by training, yet their lingua franca is often English, reflecting English language dominance in pretraining. Other langu

11 Jun 2026

Tac-DINO: Learning Vision-Tactile Features with Patch Alignment

arXiv:2606.12069v1 Announce Type: new Abstract: Touch is the primary medium through which humans interact with the environment. Currently, tactile learning mainly focuses on image-level pretraining or

11 Jun 2026

Are We Evaluating Knowledge or Phrasing? Mitigating MCQA Sensitivity with ParaEval

arXiv:2606.10657v1 Announce Type: new Abstract: Multiple-choice (MCQA) benchmarks are the standard for evaluating pretrained large language models, but their reliance on log-likelihood scoring makes t

researcharxiv-cs-cl

Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning

arXiv:2606.10196v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) aims to adapt pretrained models with a small trainable parameter subset, however, most existing methods choose

IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder

arXiv:2606.11096v1 Announce Type: new Abstract: Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing s

Reasoning over Semantic IDs Enhances Generative Recommendation

arXiv:2603.23183v2 Announce Type: replace-cross Abstract: Recent advances in generative recommendation have leveraged pretrained LLMs by formulating sequential recommendation as autoregressive generat

safetyarxiv-cs-ai

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

arXiv:2606.10929v1 Announce Type: cross Abstract: Task vectors, LoRA, activation steering, and random search around pretrained weights all suggest that learned behaviour can be controlled by linear di

Speaker Group Encoding in Self-supervised Speech Recognition Models

arXiv:2606.10654v1 Announce Type: new Abstract: We investigate what self-supervised speech recognition models (S3Ms) learn about speaker groups (SGs). We examine several states of S3Ms: pretrained, fi

safetyarxiv-cs-cl

STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model

arXiv:2606.10653v1 Announce Type: new Abstract: Although pretrained text-to-image (T2I) generation models can produce high-quality images, they often fail to faithfully reflect the semantic intent of

BCG-FM: A Foundation Model for Ambient Cardiac Health Sensing

arXiv:2606.07692v1 Announce Type: cross Abstract: Foundation models for wearable biosignals have matched or exceeded supervised specialists across a range of clinical tasks, yet all rely on modalities

Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery

arXiv:2606.09672v1 Announce Type: new Abstract: Ask a pretrained biomedical language model whether 'cortisol 28 ug/dL' and 'stock-market volatility' are related, and it returns a cosine similarity of

Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space

arXiv:2603.29824v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning methods such as LoRA enable efficient adaptation of large pretrained models, but often lag behind full fine-tuning i

GNSS-FM: A Self-Supervised Foundation Model for Daily GNSS Displacement Time Series

arXiv:2606.07725v1 Announce Type: cross Abstract: Displacement time series from Global Navigation Satellite Systems (GNSS) are essential for a wide range of applications, including monitoring tectonic

In-Context Learning of Temporal Point Processes with Foundation Inference Models

arXiv:2509.24762v3 Announce Type: replace Abstract: Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamic

INFUSER: Influence-Guided Self-Evolution Improves Reasoning

arXiv:2606.09052v1 Announce Type: cross Abstract: Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet e

MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training

arXiv:2606.08788v1 Announce Type: new Abstract: Representation alignment with pretrained vision models has recently shown strong potential for accelerating diffusion transformer training. By aligning

Local Ai

MosaicIMU: Composing Carrier Experts for Generalizable Neural Inertial Odometry

arXiv:2606.09355v1 Announce Type: new Abstract: Robust inertial odometry is essential for various carriers when external sensing is unreliable. Learning-based methods reduce integration drift by captu

local-aiarxiv-cs-ro

Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation

arXiv:2606.08679v1 Announce Type: cross Abstract: Pretrained models are often evaluated on multi-task leaderboards to measure their applicability in diverse contexts. However, current methods for aggr

Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning

arXiv:2606.09303v1 Announce Type: new Abstract: The rapid development of pretrained foundation models has enabled more general image segmentation. Multimodal large language models (MLLMs) have been wi

Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

arXiv:2506.06891v3 Announce Type: replace Abstract: We study the corruption-robustness of in-context reinforcement learning (ICRL), focusing on the Decision-Pretrained Transformer (DPT, Lee et al., 20

TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech

arXiv:2606.09019v1 Announce Type: cross Abstract: Codec-based autoregressive (AR) speech language models have achieved strong text-to-speech (TTS) quality by modeling speech as sequences of discrete a

UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models

arXiv:2602.18020v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models leverage pretrained Vision-Language Models (VLMs) as backbones to map images and instructions to actions, demons

applicationsarxiv-cs-cv

A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Design

arXiv:2510.16023v2 Announce Type: replace Abstract: Linear polymers, macromolecules formed from monomers covalently bonded into continuous chains, underpin countless technologies and are indispensable