Search: “pretraining”

Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modeling

arXiv:2605.27286v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) are transforming the forecasting paradigm through large-scale cross-domain pretraining. However, most existing T

27 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Tutorials

Model Unlearning Objectives Vary for Distinct Language Functions

DGX agent

arXiv:2605.26454v1 Announce Type: new Abstract: Large language models (LLMs) learn undesirable properties during pretraining, including dangerous knowledge and toxic text generation. Just as post-trai

tutorialsarxiv-cs-cl

27 May 2026

All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting

arXiv:2602.17234v2 Announce Type: replace Abstract: Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledg

Hardware

Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs

arXiv:2605.24684v1 Announce Type: cross Abstract: Multimodal Attributed Graph Learning (MAGL) integrates intrinsic node attributes with structural topology via graph aggregation. However, as pretraine

hardwarearxiv-cs-ai

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models

arXiv:2605.25765v1 Announce Type: cross Abstract: Concept unlearning aims to erase a target concept from a pretrained text-to-image diffusion model without retraining. Closed-form methods are attracti

Context-CoT: Enhancing Context Learning via High-Quality Reasoning Synthesis

arXiv:2605.25354v1 Announce Type: new Abstract: While LLMs excel at reasoning over prompts using static pretrained knowledge, they struggle significantly with context learning-the ability to dynamical

Tutorials

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models

arXiv:2605.25477v1 Announce Type: cross Abstract: The ability to efficiently and reliably learn new tasks has been a foundational challenge in robotics. Vision-Language-Action (VLA) models have demons

tutorialsarxiv-cs-ai

Tutorials

Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting

arXiv:2605.24911v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have recently achieved strong zero-shot forecasting performance through large-scale pretraining and retrieval-au

tutorialsarxiv-cs-ai

Tutorials

Uncovering the Latent Potential of Deep Intermediate Representations

arXiv:2605.23033v1 Announce Type: cross Abstract: Foundational Models pretrained on huge amount of data learn representations that evolve across depth, forming a hierarchy of embeddings with distinct

tutorialsarxiv-cs-ai

25 May 2026

Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation

arXiv:2602.02214v3 Announce Type: replace Abstract: To achieve real-time interactive video generation, current methods distill pretrained bidirectional video diffusion models into few-step autoregress

researcharxiv-cs-cv

Hierarchical Variational Policies for Reward-Guided Diffusion

arXiv:2605.21661v1 Announce Type: cross Abstract: Adapting pretrained diffusion models to downstream objectives such as inverse problems often requires expensive test-time guidance or optimization. We

MAP4TS: A Multi-Aspect Prompting Framework for Time-Series Forecasting with Large Language Models

arXiv:2510.23090v2 Announce Type: replace Abstract: Recent advances have investigated the use of pretrained large language models (LLMs) for time-series forecasting by aligning numerical inputs with L

Supervised Classification Heads as Semantic Prototypes: Unlocking Vision-Language Alignment via Weight Recycling

arXiv:2605.22484v1 Announce Type: new Abstract: Vision-Language Models (VLMs) excel at tasks like zero-shot classification and cross-modal retrieval by mapping images and text to a shared space, but t

Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding

arXiv:2605.20268v1 Announce Type: cross Abstract: Real-world time series come with text: metadata, descriptions, news, reports. Yet time series foundation models process numerical sequences in isolati

GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation

arXiv:2605.20752v1 Announce Type: new Abstract: Vision-language-action (VLA) policies have advanced language-conditioned robotic manipulation by transferring semantic priors from pretrained vision-lan

applicationsarxiv-cs-ro

Latent Space Guided Scenario Sampling for Multimodal Segmentation Under Missing Modalities

arXiv:2605.20372v1 Announce Type: new Abstract: Multimodal semantic segmentation benefits remote sensing analysis by combining complementary information from different sensor modalities. In real-world

applicationsarxiv-cs-cv

LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws

arXiv:2502.12120v3 Announce Type: replace-cross Abstract: Scaling laws guide the development of large language models (LLMs) by offering estimates for the optimal balance of model size, tokens, and co

Multimodal Optimal Transport for Training-free Temporal Segmentation in Surgical Robotics

model-releasesarxiv-cs-cv

arXiv:2602.24138v2 Announce Type: replace Abstract: Automated recognition of surgical phases and steps is a fundamental capability for intraoperative decision support, workflow automation, and skill a

ROAR-3D: Routing Arbitrary Views for High-Fidelity 3D Generation

arXiv:2605.21121v1 Announce Type: new Abstract: Single-image-to-3D generative models can now produce high-quality geometry, yet conditioning on a single view inevitably introduces ambiguity about unse

researcharxiv-cs-cv

Variance Reduction for Expectations with Diffusion Teachers

arXiv:2605.21489v1 Announce Type: cross Abstract: Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution.

researcharxiv-cs-cv

A Bitter Lesson for Data Filtering

arXiv:2605.19407v1 Announce Type: cross Abstract: We investigate data filtering for large model pretraining via new scaling studies that target the high compute, data-scarce regime. In spite of an app

Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls

model-releasesarxiv-cs-cv

arXiv:2605.19728v1 Announce Type: new Abstract: Foundation video models produce visually impressive results, but their use in embodied AI remains limited because they are primarily trained on natural

Block-Based Double Decoders

arXiv:2605.18807v1 Announce Type: cross Abstract: Encoder-decoder models offer substantial inference-time savings over decoder-only models, but their pretraining objectives suffer from sparse supervis

Efficient and Minimax Optimal In-context Nonparametric Regression with Transformers

arXiv:2601.15014v2 Announce Type: replace-cross Abstract: We study in-context learning for nonparametric regression with alpha-Holder smooth regression functions, for some alpha>0. We prove that, with

Nonlinearity as Rank: Generative Low-Rank Adapter with Radial Basis Functions

arXiv:2602.05709v2 Announce Type: replace Abstract: Low-rank adaptation (LoRA) approximates the update of a pretrained weight matrix using the product of two low-rank matrices. However, standard LoRA

TrajTok: Adaptive Spatial Tokenization for Trajectory Representation Learning

arXiv:2605.20134v1 Announce Type: new Abstract: Learning generalizable trajectory representations from raw GPS traces remains difficult because the data is continuous, noisy, and irregularly sampled.

Where Does Authorship Signal Emerge in Encoder-Based Language Models?

arXiv:2605.19908v1 Announce Type: new Abstract: Authorship attribution models fine-tuned with the same pretrained encoder, data, and loss can differ four-fold in performance depending only on their sc

researcharxiv-cs-cl

ADMEDTAGGER: an annotation framework for distillation of expert knowledge for the Polish medical language

arXiv:2601.09722v2 Announce Type: replace-cross Abstract: In this work, we present an annotation framework that demonstrates how a multilingual LLM pretrained on a large corpus can be used as a teache

local-aiarxiv-cs-ai

Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent

model-releasesarxiv-cs-cv

arXiv:2603.21787v2 Announce Type: replace Abstract: Event cameras are attractive for industrial robotics because they provide high temporal resolution, high dynamic range, and reduced motion blur. How

MixSD: Mixed Contextual Self-Distillation for Knowledge Injection

arXiv:2605.16865v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models, but it often degrades pretrained capabilities such as reasonin

MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation

arXiv:2605.17743v1 Announce Type: new Abstract: Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams while retaining past competence, yet texture

RAVE: Re-Allocating Visual Attention in Large Multimodal Models

arXiv:2605.18359v1 Announce Type: new Abstract: Large multimodal models (LMMs) inherit the self-attention mechanism of pretrained language backbones, yet standard attention can exhibit suboptimal allo

Reverse-Engineering Model Editing on Language Models

arXiv:2602.10134v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive informatio

SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark

arXiv:2605.18232v1 Announce Type: cross Abstract: Somali is a Cushitic language of the Horn of Africa with ~25 million speakers, yet no documented dedicated Somali pretraining corpus with a companion

TabH2O: A Unified Foundation Model for Tabular Prediction

model-releasesarxiv-cs-lg

arXiv:2605.18383v1 Announce Type: new Abstract: We present TabH2O, a foundation model for tabular data that performs classification and regression in a single forward pass via in-context learning. Tab

Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning

model-releasesarxiv-cs-ro

arXiv:2602.10503v2 Announce Type: replace Abstract: Pretrained on large-scale and diverse datasets, VLA models demonstrate strong generalization and adaptability as general-purpose robotic policies. H

Training-Free Occluded Text Rendering via Glyph Priors and Attention-Guided Semantic Blending

arXiv:2605.16810v1 Announce Type: new Abstract: We present a training-free framework for occluded text rendering with a pretrained FLUX.1-dev backbone. The task requires a model to render recognizable

local-aiarxiv-cs-cv

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time

arXiv:2605.15220v1 Announce Type: cross Abstract: Data mixing decides how to combine different sources or types of data and is a consequential problem throughout language model training. In pretrainin

safetyarxiv-cs-ai

18 May 2026

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

arXiv:2605.15687v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Ex

safetyarxiv-cs-ai

18 May 2026

Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation

arXiv:2603.00574v2 Announce Type: replace-cross Abstract: Adapting pretrained multi-modal models to evolving test-time distributions, known as multi-modal test-time adaptation, presents a significant

Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2

arXiv:2605.13904v1 Announce Type: cross Abstract: Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks, and are typically evalu

GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation

arXiv:2605.14406v1 Announce Type: cross Abstract: Large-scale pretraining on Earth observation imagery has yielded powerful representations of the natural and built environment. However, most existing

local-aiarxiv-cs-cv

MeMo: Memory as a Model

arXiv:2605.15156v1 Announce Type: cross Abstract: Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Ma

applicationsarxiv-cs-ai

Reliability-Gated Source Anchoring for Continual Test-Time Adaptation

arXiv:2605.14063v1 Announce Type: new Abstract: Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a frozen source che

ROAD: Adaptive Data Mixing for Offline-to-Online Reinforcement Learning via Bi-Level Optimization

arXiv:2605.14497v1 Announce Type: cross Abstract: Offline-to-online reinforcement learning harnesses the stability of offline pretraining and the flexibility of online fine-tuning. A key challenge lie

safetyarxiv-cs-ai

BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning

arXiv:2605.13382v1 Announce Type: new Abstract: While autoregressive (AR) Vision-Language-Action (VLA) models have demonstrated formidable reasoning capabilities in robotic tasks, their sequential dec

local-aiarxiv-cs-ro

14 May 2026

Early Data Exposure Improves Robustness to Subsequent Fine-Tuning

arXiv:2605.12705v1 Announce Type: new Abstract: How can we train models whose post-trained capabilities survive subsequent fine-tuning? Rather than focusing on downstream interventions to mitigate for

14 May 2026

EvObj: Learning Evolving Object-centric Representations for 3D Instance Segmentation without Scene Supervision