Search: “pretraining”

Reverse-Engineering Model Editing on Language Models

arXiv:2602.10134v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive informatio

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark

DGX agent

arXiv:2605.18232v1 Announce Type: cross Abstract: Somali is a Cushitic language of the Horn of Africa with ~25 million speakers, yet no documented dedicated Somali pretraining corpus with a companion

model-releasesarxiv-cs-ai

19 May 2026

TabH2O: A Unified Foundation Model for Tabular Prediction

model-releasesarxiv-cs-lg

arXiv:2605.18383v1 Announce Type: new Abstract: We present TabH2O, a foundation model for tabular data that performs classification and regression in a single forward pass via in-context learning. Tab

Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning

model-releasesarxiv-cs-ro

arXiv:2602.10503v2 Announce Type: replace Abstract: Pretrained on large-scale and diverse datasets, VLA models demonstrate strong generalization and adaptability as general-purpose robotic policies. H

Training-Free Occluded Text Rendering via Glyph Priors and Attention-Guided Semantic Blending

arXiv:2605.16810v1 Announce Type: new Abstract: We present a training-free framework for occluded text rendering with a pretrained FLUX.1-dev backbone. The task requires a model to render recognizable

local-aiarxiv-cs-cv

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time

arXiv:2605.15220v1 Announce Type: cross Abstract: Data mixing decides how to combine different sources or types of data and is a consequential problem throughout language model training. In pretrainin

18 May 2026

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

arXiv:2605.15687v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Ex

18 May 2026

Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation

arXiv:2603.00574v2 Announce Type: replace-cross Abstract: Adapting pretrained multi-modal models to evolving test-time distributions, known as multi-modal test-time adaptation, presents a significant

Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2

arXiv:2605.13904v1 Announce Type: cross Abstract: Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks, and are typically evalu

GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation

arXiv:2605.14406v1 Announce Type: cross Abstract: Large-scale pretraining on Earth observation imagery has yielded powerful representations of the natural and built environment. However, most existing

local-aiarxiv-cs-cv

MeMo: Memory as a Model

arXiv:2605.15156v1 Announce Type: cross Abstract: Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Ma

applicationsarxiv-cs-ai

Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

arXiv:2605.14063v1 Announce Type: new Abstract: Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a frozen source che

ROAD: Adaptive Data Mixing for Offline-to-Online Reinforcement Learning via Bi-Level Optimization

arXiv:2605.14497v1 Announce Type: cross Abstract: Offline-to-online reinforcement learning harnesses the stability of offline pretraining and the flexibility of online fine-tuning. A key challenge lie

BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning

arXiv:2605.13382v1 Announce Type: new Abstract: While autoregressive (AR) Vision-Language-Action (VLA) models have demonstrated formidable reasoning capabilities in robotic tasks, their sequential dec

local-aiarxiv-cs-ro

Early Data Exposure Improves Robustness to Subsequent Fine-Tuning

arXiv:2605.12705v1 Announce Type: new Abstract: How can we train models whose post-trained capabilities survive subsequent fine-tuning? Rather than focusing on downstream interventions to mitigate for

EvObj: Learning Evolving Object-centric Representations for 3D Instance Segmentation without Scene Supervision

arXiv:2605.13152v1 Announce Type: cross Abstract: We introduce EvObj for unsupervised 3D instance segmentation that bridges the geometric domain gap between synthetic pretraining data and real-world p

applicationsarxiv-cs-ai

Margin-calibrated Classifier Guidance for Property-driven Synthesis Planning

arXiv:2605.13101v1 Announce Type: cross Abstract: Synthesis planning seeks an efficient sequence of chemical reactions that produce a target molecule. Typically, a pretrained single-step (autoregressi

RotVLA: Rotational Latent Action for Vision-Language-Action Model

arXiv:2605.13403v1 Announce Type: cross Abstract: Latent Action Models (LAMs) have emerged as an effective paradigm for handling heterogeneous datasets during Vision-Language-Action (VLA) model pretra

applicationsarxiv-cs-cv

Survival In-Context: Amortized Bayesian Survival Analysis via Prior-Fitted Networks

arXiv:2603.29475v2 Announce Type: replace Abstract: Survival analysis is crucial for many medical applications, but remains challenging for modern machine learning due to limited data, censoring, and

applicationsarxiv-cs-lg

Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization

arXiv:2605.12756v1 Announce Type: cross Abstract: Large language models (LLMs) are pretrained by minimizing the cross-entropy loss for next-token prediction. In this paper, we study whether this optim

Compositional Neural Operators for Multi-Dimensional Fluid Dynamics

arXiv:2605.11691v1 Announce Type: new Abstract: Partial differential equations (PDEs) govern diverse physical phenomena, yet high-fidelity numerical solutions are computationally expensive and Machine

Discrete Flow Matching for Offline-to-Online Reinforcement Learning

arXiv:2605.12379v1 Announce Type: new Abstract: Many reinforcement learning (RL) tasks have discrete action spaces, but most generative policy methods based on diffusion and flow matching are designed

safetyarxiv-cs-lg

Efficient and Adaptive Human Activity Recognition via LLM Backbones

model-releasesarxiv-cs-lg

arXiv:2605.12019v1 Announce Type: new Abstract: Human Activity Recognition (HAR) is a core task in pervasive computing systems, where models must operate under strict computational constraints while r

Estimating Subgraph Importance with Structural Prior Domain Knowledge

arXiv:2605.12009v1 Announce Type: new Abstract: We propose a subgraph importance estimation method for pretrained Graph Neural Networks (GNNs) on graph-level tasks, formulated as a linear Group Lasso

applicationsarxiv-cs-lg

GaitProtector: Impersonation-Driven Gait De-Identification via Training-Free Diffusion Latent Optimization

arXiv:2605.12431v1 Announce Type: new Abstract: Conventional gait de-identification methods often encounter an inherent trade-off: they either provide insufficient identity suppression or introduce sp

researcharxiv-cs-cv

In-Context Multi-Objective Optimization

arXiv:2512.11114v2 Announce Type: replace Abstract: Balancing competing objectives is omnipresent across disciplines, from drug design to autonomous systems. Multi-objective Bayesian optimization is a

safetyarxiv-cs-lg

Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data

model-releasesarxiv-cs-cv

arXiv:2605.11881v1 Announce Type: new Abstract: The high-dimensional features extracted from large-scale unlabeled data via various pretrained models with diverse architectures are referred to as hete

LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection

model-releasesarxiv-cs-lg

arXiv:2605.11872v1 Announce Type: new Abstract: Orthogonal parameter-efficient fine-tuning (PEFT) adapts pretrained weights through structure-preserving multiplicative transformations, but existing me

Agents

LychSim: A Controllable and Interactive Simulation Framework for Vision Research

arXiv:2605.12449v1 Announce Type: new Abstract: While self-supervised pretraining has reduced vision systems' reliance on synthetic data, simulation remains an indispensable tool for closed-loop optim

agentsarxiv-cs-cv

READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling

model-releasesarxiv-cs-cl

arXiv:2312.06950v3 Announce Type: replace-cross Abstract: Fully fine-tuning pretrained large-scale transformer models has become a popular paradigm for video-language modeling tasks, such as temporal

ReasonEdit: Editing Vision-Language Models using Human Reasoning

arXiv:2602.02408v4 Announce Type: replace Abstract: Model editing aims to correct errors in large, pretrained models without altering unrelated behaviors. While some recent works have edited vision-la

researcharxiv-cs-cv

Reviving In-domain Fine-tuning Methods for Source-Free Cross-domain Few-shot Learning

model-releasesarxiv-cs-cv

arXiv:2605.11659v1 Announce Type: new Abstract: Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot

CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization

arXiv:2602.05902v2 Announce Type: replace-cross Abstract: Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats withou

CORTEG: Foundation Models Enable Cross-Modality Representation Transfer from Scalp to Intracranial Brain Recordings

arXiv:2605.10337v1 Announce Type: new Abstract: Intracranial electrocorticography (ECoG) offers high-signal-to-noise access to cortical activity for brain-computer interfaces, yet limited per-patient

FactoryNet: A Large-Scale Dataset toward Industrial Time-Series Foundation Models

arXiv:2605.09081v1 Announce Type: cross Abstract: We introduce the first universal pretraining corpus for industrial time-series data: FactoryNet. 51M datapoints across 23k end-to-end task executions

fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery

arXiv:2605.09438v1 Announce Type: new Abstract: Many features in pretrained Transformers span multiple layers: they emerge through stages of inference, persist in the residual stream, or are built joi

local-aiarxiv-cs-lg

Hyperspherical Autoencoder for High-Fidelity Image Reconstruction and Generation

arXiv:2601.22904v2 Announce Type: replace-cross Abstract: Recent studies have explored using pretrained Vision Foundation Models (VFMs) such as DINO for generative autoencoders, showing strong generat

JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusion

arXiv:2601.22143v2 Announce Type: replace-cross Abstract: Audio-Visual Foundation Models, which are pretrained to jointly generate sound and visual content, have recently shown an unprecedented abilit

applicationsarxiv-cs-cv

Learngene Search Across Multiple Datasets for Building Variable-Sized Models

arXiv:2605.08209v1 Announce Type: new Abstract: Deep learning methods are widely used under diverse resource constraints, resulting in models of varying sizes, such as the Vision Transformer (ViT) ser

MC-RFM: Geometry-Aware Few-Shot Adaptation via Mixed-Curvature Riemannian Flow Matching

arXiv:2605.08557v1 Announce Type: cross Abstract: Parameter-efficient adaptation of pretrained vision models is commonly performed through linear probes, prompts, low-rank updates, or lightweight resi

Overconfident and Blind to Details: Fixing Prompt Insensitivity with Abductive Preference Learning

model-releasesarxiv-cs-cl

arXiv:2510.09887v2 Announce Type: replace Abstract: Vision and language models frequently ignore semantically critical input edits, defaulting to pretraining priors. For example, models will confident

PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models

arXiv:2605.08800v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) may memorize sensitive cross-modal information during pretraining. However, existing MLLM unlearning benchmar

Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World

arXiv:2605.09189v1 Announce Type: new Abstract: The scaling laws guiding modern model training were calibrated for a single regime: data-rich, single-epoch pretraining. The dominant such scaling law f

Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputs

arXiv:2605.10385v1 Announce Type: cross Abstract: Guided-diffusion black-box optimization (BO) has shown strong empirical performance on structured design problems such as molecules and crystals, but

S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain

model-releasesarxiv-cs-cv

arXiv:2605.08589v1 Announce Type: new Abstract: Parameter Efficient Fine-Tuning (PEFT) is a key technique for adapting a large pretrained model to downstream tasks by fine-tuning only a small number o

StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception

arXiv:2605.09989v1 Announce Type: cross Abstract: Recent advances in robot imitation learning have yielded powerful visuomotor policies capable of manipulating a wide variety of objects directly from

safetyarxiv-cs-cv

XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies

arXiv:2605.10734v1 Announce Type: new Abstract: For reinforcement learning in the real world online exploration is expensive A common practice in robotic reinforcement learning is to incorporate addit

safetyarxiv-cs-lg

Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning