Search: “pretraining”

Learning-State-Aware Dynamic Generative Data Augmentation on Small-Scale Datasets

arXiv:2608.18907v1 Announce Type: cross Abstract: Small-scale image classification is often limited by the scarcity of training data. Generative data augmentation (GDA) based on pretrained generative

tutorialsarxiv-cs-ai

20 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers

DGX agent

arXiv:2608.18399v1 Announce Type: new Abstract: Vision transformers (ViTs) trained to copy a pretrained teacher's attention maps recover most of fine-tuning's in-distribution accuracy yet fall measura

researcharxiv-cs-cv

20 Aug 2026

Looking Beyond the Scale: Do Surgical Skill Models Learn Transferable Representations Across Assessment Rubrics?

arXiv:2608.17519v1 Announce Type: new Abstract: Vision-based surgical skill assessment has shown strong in-domain results, yet a fundamental question remains unasked: do these models learn transferabl

tutorialsarxiv-cs-cv

19 Aug 2026

MoNe: Modular Neural Memory for Efficient Long Context Inference

arXiv:2608.17616v1 Announce Type: new Abstract: We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retrain

19 Aug 2026

VLCP: Vision Language Control Policy Closed-Loop Code Replanning for Robot Manipulation

arXiv:2608.16978v1 Announce Type: new Abstract: Turning a frontier vision-language model into a robot policy usually means fine-tuning it to emit an action representation it never saw in pretraining,

safetyarxiv-cs-ro

19 Aug 2026

6G Native AI and Channel Foundation Models

arXiv:2608.14591v1 Announce Type: cross Abstract: The integration of artificial intelligence (AI) and wireless communications is widely regarded as a core objective of sixth-generation (6G) systems. H

researcharxiv-cs-lg

A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classification

model-releasesarxiv-cs-lg

arXiv:2608.14824v1 Announce Type: cross Abstract: We present a parameter-free episodic evaluation of nearest-centroid classification for elephant vocalisations on fixed pretrained acoustic embeddings,

CSMoE: An Efficient Remote Sensing Foundation Model with Soft Mixture-of-Experts

arXiv:2509.14104v2 Announce Type: replace Abstract: Self-supervised learning (SSL) through masked autoencoders (MAEs) has recently attracted great attention for remote sensing (RS) foundation model (F

researcharxiv-cs-cv

Do Visual Grounding Decoders Need Feed-Forward Networks? A Controlled Study over Frozen Vision-Language Features

arXiv:2608.15061v1 Announce Type: new Abstract: Do feed-forward networks (FFNs) in visual grounding decoders add essential computation once a pretrained vision-language model has already encoded image

PhaseLoRA: Control-Regime-Conditioned Low-Rank Adaptation for Continuous-Action Vision-Language-Action Policies

arXiv:2608.15285v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) is a natural way to adapt pretrained vision-language-action (VLA) policies, but most adapter designs apply temp

Quantum Large Language Models via Tensor Network Disentanglers

arXiv:2410.17397v2 Announce Type: replace-cross Abstract: We introduce a framework for seamlessly integrating quantum computing into pretrained large language models (LLMs). The key idea is to constru

Qwen-Video-Edit: Instruction-Based Video Editing by Repurposing an Image Editing Model

arXiv:2608.14790v1 Announce Type: new Abstract: Instruction-based video editing is commonly built on video-pretrained generative backbones: a video diffusion transformer is adapted, at considerable co

Scale-Consistent Posterior Dynamics for Diffusion Inverse Problems

arXiv:2608.15144v1 Announce Type: cross Abstract: Posterior sampling with a pretrained diffusion prior is governed by a conditional score whose intermediate likelihood component is generally intractab

STAIR: Semantic-Temporal Automaton for Interpretable Reasoning in Temporal Question Answering

arXiv:2608.16224v1 Announce Type: cross Abstract: By leveraging large-scale pretraining, LLMs can interpret diverse temporal expressions and question formulations without task-specific training. Howev

TAHB: A Comprehensive Benchmark for Text-Attributed Hypergraph Learning

arXiv:2608.15055v1 Announce Type: new Abstract: Hypergraphs effectively model higher-order groupwise relationships beyond pairwise interactions, while pretrained language models (PLMs) and large langu

Applications

Take it Personally: The Limits of General SSL Representations for Real-Life PPG Emotion Detection

arXiv:2608.14675v1 Announce Type: cross Abstract: While Self-Supervised Learning (SSL) effectively extracts general representations from noisy, unconstrained physiological signals such as photoplethys

applicationsarxiv-cs-ai

TransfHAR: Self-Supervised Wrist Representations for On-Demand Activity Recognition

arXiv:2608.15861v1 Announce Type: new Abstract: Fine-grained wrist activity recognition can support applications such as procedural step guidance and context-aware assistance, yet acquiring labeled da

researcharxiv-cs-lg

CytoBERT: A Foundation Model for Cytometry Data

arXiv:2608.14414v1 Announce Type: new Abstract: Cytometry measures the complex characteristics of single cells (e.g., counts and protein expression of immune cells) and is widely used across immunolog

tutorialsarxiv-cs-lg

Model-agnostic Retrieval-Augmented Extended Forecasting for time series

model-releasesarxiv-cs-lg

arXiv:2608.14054v1 Announce Type: new Abstract: Time series forecasting with pretrained foundation models has demonstrated strong zero-shot capabilities. However, achieving optimal performance on time

Rethinking Auxiliary Modalities in Multimodal Zero-shot Anomaly Detection: From Semantic Fusion to Conditional Modulation

arXiv:2608.13973v1 Announce Type: new Abstract: Recent foundation model-based methods have endowed RGB images with strong zero-shot anomaly detection (ZSAD) through vision-language pretraining. Howeve

researcharxiv-cs-cv

Local Ai

The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

arXiv:2608.14229v1 Announce Type: new Abstract: Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform g

local-aiarxiv-cs-cl

PixSDS: Why Latent SDS Makes Noisy Pixels

arXiv:2608.12997v1 Announce Type: new Abstract: Score Distillation Sampling (SDS) enables text-to-3D generation by optimizing rendered images with a pretrained diffusion prior, but latent SDS often pr

researcharxiv-cs-cv

14 Aug 2026

SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation

arXiv:2606.16454v2 Announce Type: replace-cross Abstract: Low-Rank Adaptation (LoRA) enables efficient adaptation of large pretrained models to downstream tasks by parameterizing weight updates with l

14 Aug 2026

SpatialVAM:Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policy

arXiv:2604.03181v2 Announce Type: replace-cross Abstract: Robotic manipulation requires understanding both the 3D spatial structure of the environment and its temporal evolution, yet most existing pol

safetyarxiv-cs-cv

14 Aug 2026

VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation

arXiv:2603.17450v2 Announce Type: replace-cross Abstract: Sequential Recommendation (SR) in multimodal settings typically relies on small frozen pretrained encoders, which limits semantic capacity and

13 Aug 2026

Can Computational Reducibility Lead to Transferable Models for Graph Combinatorial Optimization?

arXiv:2603.02462v2 Announce Type: replace-cross Abstract: A key challenge in developing unified neural solvers for combinatorial optimization (CO) is the efficient generalization of models from a give

Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?

model-releasesarxiv-cs-cl

arXiv:2608.10690v1 Announce Type: new Abstract: Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus

Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts

arXiv:2608.10605v1 Announce Type: cross Abstract: In large-scale pretraining, the algorithm, architecture, and systems decisions are conventionally made in disconnected stages. A scaling law stage sel

Frozen Brain-MRI Foundation Models Are Site Fingerprints

arXiv:2608.10295v1 Announce Type: cross Abstract: Frozen foundation-model (FM) embeddings are increasingly used as off-the-shelf brain-MRI representations, on the assumption that they capture anatomy.

Putting Registers to Work: Task Registers for Token Pruning in Vision Transformers

arXiv:2608.10989v1 Announce Type: cross Abstract: Token-pruning policies are usually designed for a single recognition pipeline, but pretrained Vision Transformers are reused across tasks with differe

Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

arXiv:2608.11204v1 Announce Type: cross Abstract: Learning reliable surgical manipulation policies is bottlenecked by the scarcity of action-labeled demonstrations: teleoperated surgical robot (e.g.,

Local Ai

Temporal Straightening for Latent Planning

arXiv:2603.12231v3 Announce Type: replace Abstract: Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual fe

local-aiarxiv-cs-lg

AdaDINO: Pair-Aware In-Backbone Adaptation of Frozen DINO for Efficient Remote Sensing Change Detection

arXiv:2608.07982v1 Announce Type: new Abstract: Vision foundation models (VFMs) such as DINO are pretrained for single-image representation, whereas remote sensing change detection requires reasoning

LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding

arXiv:2608.07596v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models transform representations from pretrained vision-language models (VLMs) into robot actions, yet the interface that

RAG-Audio: Retrieval-Augmented Generation for Faithful Brain-to-Audio Reconstruction

arXiv:2608.09331v1 Announce Type: cross Abstract: Brain-to-audio reconstruction is limited by prior domination: when a pretrained generator is conditioned on a weak neural signal, it produces realisti

SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

arXiv:2608.09006v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Tra

tutorialsarxiv-cs-ai

SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection

arXiv:2603.22213v2 Announce Type: replace-cross Abstract: While large language models (LLMs) are pretrained on massive amounts of data, their knowledge coverage remains incomplete in specialized, data

Test-Time Prototype Adaptation for Open-Vocabulary Semantic Segmentation

arXiv:2608.08290v1 Announce Type: new Abstract: Open-vocabulary semantic segmentation (OVSS) repurposes a pretrained CLIP encoder for dense prediction without additional labeled supervision. Existing

Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation

arXiv:2608.09052v1 Announce Type: cross Abstract: Semi-supervised adaptation of vision foundation models (VFMs) commonly freezes the pretrained backbone and updates lightweight modules such as LoRA. H

UniDFKD: A Unified Semantic Prior Framework for Architecture-Agnostic Data-Free Knowledge Distillation

arXiv:2608.09287v1 Announce Type: cross Abstract: Data-Free Knowledge Distillation (DFKD) transfers knowledge from a pretrained teacher model to a compact student model by synthesizing semantically in

tutorialsarxiv-cs-ai

Hardware

verdi: retrieval is not transfer for continual world model optimization

arXiv:2608.09537v1 Announce Type: new Abstract: Foundation world models have made remarkable progress in planning, simulation, and embodied intelligence. However, optimizing a pretrained world model t

hardwarearxiv-cs-ai

CrystalGRPO: Target-Aligned and Coverage-Preserving Reinforcement Learning for Flow-Based Crystal Structure Prediction

arXiv:2608.06582v1 Announce Type: new Abstract: Flow-based generative models can efficiently produce candidate structures for crystal structure prediction (CSP), but their pretrained objectives do not

safetyarxiv-cs-lg

Do 3D Medical Foundation Models See Through MRI Artifacts? A Controlled Study of Representation Robustness

arXiv:2608.06613v1 Announce Type: cross Abstract: Self-supervised 3D medical foundation models are increasingly used as general-purpose feature extractors, yet their sensitivity to MRI artifacts remai

Same physical state, different collective dynamics: state encodings select synchronization outcomes in language-model agents

arXiv:2608.06968v1 Announce Type: cross Abstract: Language-model agents act on state encodings of their environment, yet these are treated as interchangeable interfaces. Using pretrained language mode

WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN

arXiv:2608.07267v1 Announce Type: new Abstract: Recent vision-language navigation (VLN) systems increasingly adapt pretrained vision-language models (VLMs) into vision-language-action (VLA) policies t

Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination

arXiv:2608.07341v1 Announce Type: cross Abstract: Test data from public benchmarks inevitably leaks into pretraining corpora, inflating evaluation scores once memorized. extbf{Contamination mitigation

Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation

arXiv:2608.05999v1 Announce Type: new Abstract: Vision-language-action (VLA) models have demonstrated remarkable capabilities in robotic manipulation by leveraging pretrained vision-language models. H

researcharxiv-cs-ro

7 Aug 2026

Simulator-Grounded Large Language Models for Industrial Causal Reasoning: Tool-Use, Structured Injection, and Plant-Portable Retrieval for Wastewater Treatment Decision Support

arXiv:2608.05151v1 Announce Type: cross Abstract: Wastewater operators need answers grounded in how their plant's variables interact and how fast effects propagate, not in generic pretraining text, wh