Search: “pretraining”

Achieving Tokenizer Flexibility in Language Models through Heuristic Adaptation and Supertoken Learning

arXiv:2505.09738v2 Announce Type: replace-cross Abstract: Pretrained language models (LLMs) are often constrained by their fixed tokenization schemes, leading to inefficiencies and performance limitat

researcharxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

BeatGraph: Self-Supervised Heartbeat Graphs for Infant ECG Representations from the Home Environment

DGX agent

arXiv:2609.31546v1 Announce Type: new Abstract: Electrocardiogram (ECG) foundation models typically tokenize the signal into fixed-length patches that ignore cardiac structure, so a patch may split a

model-releasesarxiv-cs-lg

28 Sept 2026

Can Linguistic Reasoning Vectors Enhance Multimodal Reasoning Ability?

arXiv:2609.31140v1 Announce Type: new Abstract: Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However,

Applications

EXAONE Demand 1.0: A Time Series Foundation Model for Demand Forecasting

arXiv:2609.30880v1 Announce Type: new Abstract: Time series foundation models (TSFMs) are pretrained on series from diverse domains, where demand series make up only a small fraction. Demand data has

applicationsarxiv-cs-ai

FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning

arXiv:2609.31204v1 Announce Type: cross Abstract: Recent fMRI foundation models differ substantially in the spatial scale at which they represent brain activity. ROI- and connectivity-based models are

Local Ai

Region-Level Black-Box Defense Against Stealthy Embedding-Space Backdoors in CLIP

arXiv:2609.31558v1 Announce Type: new Abstract: Contrastive Language--Image Pretraining (CLIP) has emerged as a dominant vision backbone due to its strong transferability and zero-shot capabilities. H

local-aiarxiv-cs-cv

Towards Understanding Momentum Acceleration in River-Valley Loss Landscape

arXiv:2609.30957v1 Announce Type: new Abstract: The empirical success of pretraining large language models has inspired a deeper investigation into the underlying loss landscapes and the optimization

ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs

arXiv:2609.31448v1 Announce Type: cross Abstract: Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answer

A Manifold-Aware Topic Modeling Approach via Rank-Based Prototypes

arXiv:2609.29630v1 Announce Type: cross Abstract: Recent topic models leverage pretrained embeddings, but neural architectures produce latent representations without grounding in specific texts, and c

researcharxiv-cs-ai

An Analysis of Streaming Deep Reinforcement Learning for Adaptive Continual Learning in Robotics

arXiv:2609.28807v1 Announce Type: new Abstract: Over the course of a lifetime, robots may encounter novel scenarios unaccounted for in its original training that result in performance degradation. One

Applications

DeltaWAM: Delta World Action Models for Bimanual Manipulation

arXiv:2609.28811v1 Announce Type: new Abstract: World-action models (WAMs) transfer visual and motion priors from pretrained video generators to robot control by jointly modeling visual dynamics and a

applicationsarxiv-cs-cv

Tutorials

Generative Atmospheric Super-Resolution from Heterogeneous In Situ Observations through Composable Interfaces

arXiv:2609.29027v1 Announce Type: new Abstract: Atmospheric observations are sparse, heterogeneous, and unevenly distributed, whereas many generative atmospheric models learn distributions over regula

tutorialsarxiv-cs-lg

Spectral-Guided Diffusion: Accelerating Inference via Static Spectral Layer Scheduling

arXiv:2609.29505v1 Announce Type: new Abstract: Diffusion inference repeatedly evaluates the same large network. We ask whether pretrained weights alone can identify residual branches that need not be

Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy

model-releasesarxiv-cs-lg

arXiv:2609.28838v1 Announce Type: cross Abstract: Online reinforcement learning fine-tuning of pretrained flow-matching vision-language-action (VLA) policies promises robots that keep learning after d

EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields

arXiv:2605.06192v2 Announce Type: replace-cross Abstract: Pretrained video diffusion models provide powerful spatiotemporal generative priors, making them a natural foundation for robotic world models

GaussPDE: Graph-Based Partial Differential Equation-Driven Rendering for 3D Gaussian Splatting

arXiv:2609.27264v1 Announce Type: new Abstract: We present GaussPDE, a framework that injects physically structured partial differential equation (PDE) dynamics into pretrained 3D Gaussian scenes with

I-SplineFlow: Learning Monotone Spline Stochastic Interpolant Schedulers for Few-Step Generation

model-releasesarxiv-cs-lg

arXiv:2609.27963v1 Announce Type: new Abstract: Few-step generation with pretrained diffusion and flow models can be accelerated by lightweight training that optimizes the sampling trajectory rather t

On the Diffusibility of High-Dimensional Latents

arXiv:2609.28473v1 Announce Type: new Abstract: Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf e

Uncheatable Eval: Dynamic Compression-Based Evaluation of Language Models

arXiv:2609.27510v1 Announce Type: cross Abstract: Modern large language models are pretrained on massive datasets, making it difficult to prevent benchmark data from entering their training sets and u

ChainDoRA: Tensor-Train Factorized Weight-Decomposed Low-Rank Adaptation for Parameter-Efficient LLM Fine-Tuning

model-releasesarxiv-cs-cl

arXiv:2609.25058v1 Announce Type: new Abstract: Parameter-efficient fine-tuning (PEFT) adapts large language models (LLMs) to downstream tasks while updating only a small fraction of their pretrained

Foundation model embeddings capture pre-diagnostic changes on screening mammograms

arXiv:2609.26605v1 Announce Type: cross Abstract: Foundation model embeddings of screening mammograms may encode pre-diagnostic tissue change without task-specific adaptation. We tested whether embedd

NAWE: Digital Watermarking with Neural-Assisted Watermark Extraction

arXiv:2609.25972v1 Announce Type: new Abstract: NAWE (Neural-Assisted Watermark Extraction) combines an explicit signal-processing watermarking construction with a pretrained neural host predictor. A

Applications

StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Models

arXiv:2609.26071v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models inherit strong semantic priors from large-scale vision-language pretraining, yet remain limited in robotic manipulat

applicationsarxiv-cs-ro

Active Visual Sampling with a Connectome-Constrained Fly Model for One-Shot Hatch Recognition in Architectural Drawings

arXiv:2609.24565v1 Announce Type: new Abstract: Architectural drawings encode material classes through repeated hatch patterns. We test whether a connectome-constrained fly visual network, pretrained

An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond

arXiv:2609.24170v1 Announce Type: new Abstract: Embodied AI systems are often organized into System 1 and System 2. System 1 is typically a pretrained policy that generates actions at high frequency,

Bridge3D: Enabling Vision-Language-Action Models to See and Act in 3D

model-releasesarxiv-cs-ro

arXiv:2609.24525v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have demonstrated remarkable generalization in robotic manipulation via large-scale multimodal pretraining. However,

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

arXiv:2609.24976v1 Announce Type: cross Abstract: Dexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple pred

DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinement

arXiv:2609.24868v1 Announce Type: new Abstract: World Action Models (WAMs) jointly generate robot actions and predict future world states, transferring priors from video pretraining to robot control.

researcharxiv-cs-ro

Evaluating the Generalization of Neuroimaging Foundation Models on African Brain MRI

arXiv:2609.23983v1 Announce Type: new Abstract: Neuroimaging foundation models pretrained on large, predominantly western cohorts are increasingly proposed as general-purpose backbones for brain MRI a

Grounded Action Model: 3D Grounding as a Foundation for Robotics

arXiv:2609.23863v1 Announce Type: new Abstract: Manipulation policies must know which objects matter and where they are, yet the pretrained backbones that current robot foundation models build on, fro

Local Ai

ME-Brain-1.0: Memory, Cognition and Action for Evolving Embodied Intelligence

arXiv:2609.24271v1 Announce Type: new Abstract: Current embodied systems largely rely on pretrained capabilities that remain fixed after deployment, limiting their ability to learn from physical inter

local-aiarxiv-cs-ro

Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging

arXiv:2609.24517v1 Announce Type: cross Abstract: Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional join

Hardware

Streaming Video Editing with Easy Adaptation

arXiv:2609.24788v1 Announce Type: new Abstract: In this paper, we propose SVEET, a framework that requires merely training on a pretrained bidirectional video diffusion model but supports high-quality

hardwarearxiv-cs-cv

SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroboration

arXiv:2609.24303v1 Announce Type: cross Abstract: Post-training often improves task performance but can degrade confidence calibration, leaving post-trained language models (PoLMs) more overconfident

safetyarxiv-cs-cl

Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention

arXiv:2609.22005v1 Announce Type: cross Abstract: Gating the value pathway of attention reportedly improves language model pretraining, and prior studies disagree on why. We argue and provide experime

researcharxiv-cs-cl

Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAs

arXiv:2609.21022v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by combining semantic knowledge from pretrained vision-lang

researcharxiv-cs-ro

Tutorials

FOCAL-VLA: Subtask-Guided Geometry Distillation and Implicit World Modeling for Vision-Language-Action Models

arXiv:2609.21228v1 Announce Type: cross Abstract: Vision-language-action (VLA) models built on pretrained vision-language models have demonstrated strong performance across diverse robotic manipulatio

tutorialsarxiv-cs-ai

Joint Remaining Useful Life Prediction and Capacity Estimation of Lithium-Ion Batteries Using Partial-Charging Data

arXiv:2609.21932v1 Announce Type: new Abstract: Joint remaining useful life (RUL) prediction and capacity estimation require representations of both gradual degradation and recent battery behavior. Th

Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers

arXiv:2609.21126v1 Announce Type: new Abstract: We propose a decoupled, layerwise method for structurally sparsifying the fully connected layers of pretrained neural networks. Rather than penalizing a

NaViRrator: Robot Navigation from Human-Readable Maps through a Learned Visual Route

arXiv:2609.21316v1 Announce Type: new Abstract: Human-readable maps provide an intuitive interface for specifying robot destinations, but connecting their schematic geometry to egocentric observations

Safe Real-Time Policy Steering via Noise-Space Trajectory Optimization for One-Step Generative Policies

arXiv:2609.21220v1 Announce Type: new Abstract: Generative robot policies can represent diverse, multimodal behaviors, but adapting pretrained policies to deployment-time constraints such as collision

Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models

model-releasesarxiv-cs-ro

arXiv:2609.21740v1 Announce Type: new Abstract: Latent world models enable planning by predicting the effects of actions in a learned representation space, but their predictions can become unreliable

A Free Lunch? Adapting PP-OCRv6 for Historical Text Recognition

arXiv:2609.20064v1 Announce Type: new Abstract: Despite impressive reported scores, large vision-language models have seen limited practical uptake in historical automatic text recognition because of

AnyviewMeter: Adapting Robotic Reward Models with Camera Geometry and Multi-View Attention

arXiv:2609.20106v1 Announce Type: new Abstract: Robotic reward models evaluate task execution from visual observations, but their predictions can change with camera viewpoint and occlusion even when t

dQwen3.5: Hybrid-Attention Diffusion Language Models

arXiv:2609.20751v1 Announce Type: new Abstract: Adapting a pretrained autoregressive (AR) model is a cost-efficient route to a diffusion language model (DLM). While nearly all such adaptations start f

researcharxiv-cs-cl

Improving Cross-embodiment Transfer in Latent Action Models with Action-Similarity Supervision

arXiv:2609.19846v1 Announce Type: cross Abstract: As generalist robot policies gain vision and language from web-scale pretraining, demonstrations remain costly to collect and tied to the robot that r

safetyarxiv-cs-ai

JEPA-WAM: Connecting Generated Visual Instructions to World Action Models through JEPA Latent Representations

arXiv:2609.20277v1 Announce Type: new Abstract: World Action Models (WAMs) have demonstrated strong robotic manipulation capabilities by augmenting pretrained video generative models with action exper

Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance

arXiv:2602.01346v2 Announce Type: replace Abstract: While open sourced Vision-Language Models (VLMs) have proliferated, selecting the optimal pretrained model for a specific downstream task remains ch

safetyarxiv-cs-ai