Search: “pretraining”

On-Demand Attention: Language Models Know When to Recall

arXiv:2609.20734v1 Announce Type: new Abstract: Reasoning and agentic workloads increasingly demand efficient long-context inference. Yet full-attention decoding reads the growing history at every ste

18 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models

DGX agent

arXiv:2509.19276v2 Announce Type: replace-cross Abstract: Solving ill-posed inverse problems requires powerful and flexible priors. We propose leveraging pretrained latent diffusion models for this ta

researcharxiv-cs-lg

17 Sept 2026

EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation

arXiv:2603.05757v2 Announce Type: replace Abstract: Video generative models (VGMs) pretrained on large-scale internet data can produce temporally coherent rollout videos that capture rich object dynam

safetyarxiv-cs-ro

17 Sept 2026

Tutorials

Attention is All You Need Until You Need Retention

arXiv:2501.09166v2 Announce Type: replace-cross Abstract: Pretrained Transformers keep what they learned in their weights and lose what they observe once a session ends. The first version of this pape

tutorialsarxiv-cs-ai

Beyond In-Distribution Metrics: A Systematic Out-of-Distribution Evaluation of Congenital Heart Disease Segmentation

arXiv:2609.17068v1 Announce Type: cross Abstract: Congenital heart disease (CHD) diagnosis and surgical planning often require patient-specific 3D anatomical models, but manual segmentation is labor-i

GrowMTP: Can RL Grow Its Own Draft Head?

arXiv:2609.16648v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training drives the frontier capabilities of large language models, with its wall-clock dominated by autoregressive r

safetyarxiv-cs-cl

InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy

arXiv:2609.17218v1 Announce Type: new Abstract: Label-free clustering of frozen pretrained visual embeddings offers a scalable route to biodiversity monitoring, but image-only fine-grained taxonomy ex

researcharxiv-cs-cv

Not Another Text Benchmark: Putting the 'Visual' Back in Visual Question Answering for Large Video Models

arXiv:2609.17112v1 Announce Type: new Abstract: Large video models have exhibited impressive performance on a wide range of visual question answering tasks, owing to the rise of powerful, pretrained t

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

arXiv:2609.17435v1 Announce Type: new Abstract: We submit MeTRON-FR, a 125M GPT-2 pretrained on 92.47M words of French, to the BabyLM 2026 Strict track. It scores 85.97 +/- 0.17% on QFrBLiMP (a native

Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data

arXiv:2609.16532v1 Announce Type: new Abstract: Continued pretraining (CPT) with data augmentation such as paraphrasing can store inside a large language model (LLM) the knowledge of a small source co

Hardware

Communication-Efficient LLM Adaptation over Decentralized GPU Meshes

arXiv:2609.14339v1 Announce Type: new Abstract: Decentralized training enables large-model training over low-end GPUs and internet-grade connections, but communication along both data-parallel and pip

hardwarearxiv-cs-lg

Depth and Scale in the Sub-150M Regime: JugnuLM-53M vs JugnuLM-110M

arXiv:2609.14715v1 Announce Type: new Abstract: We scale our conventional sub-150M pretraining recipe from 53.5M to 109.7M parameters, holding the method fixed (Qwen3-style decoder with grouped-query

Do Tabular Foundation Models Still Need Feature Engineering?

arXiv:2609.13202v1 Announce Type: new Abstract: Feature engineering has long been a cornerstone of tabular machine learning. Tabular foundation models (TFMs) are pretrained on a wide range of tabular

Freeze, Share, Shrink: Rethinking the Action Backbone in Diffusion Policies

arXiv:2511.12101v3 Announce Type: replace-cross Abstract: Many recent Vision-Language-Action models employ diffusion or flow-matching backbones with hundreds of millions of parameters for action gener

Applications

GeomVLA: Unifying Scene, Motion, and Action in 3D

arXiv:2609.13812v1 Announce Type: new Abstract: We present GeomVLA, a Vision-Language-Action (VLA) model that unifies perception, latent scene motion prediction, and action generation within a shared

applicationsarxiv-cs-ro

LPA-CWM: A Learned Physical Adjudicator for Motion Reasoning with Counterfactual World Models

arXiv:2609.14073v1 Announce Type: cross Abstract: Counterfactual world models (CWM) extract motion from pretrained video predictors by comparing factual and intervened predictions. However, responses

Multimodal Foundation Models Adaptation based on Domain-Aware Relaxed Orthogonal Subspace for Remote Sensing

arXiv:2609.13654v1 Announce Type: new Abstract: Pretrained foundation models (FMs) have achieved remarkable success in computer vision, yet their high fine-tuning cost limits practical deployment. Par

Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs

arXiv:2609.15322v1 Announce Type: cross Abstract: Pretrained driving vision-language models (VLMs) integrate visual, route, language, and driving context into rich driving priors, yet their representa

Agents

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

arXiv:2609.15364v1 Announce Type: new Abstract: Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce

agentsarxiv-cs-ai

Steering Generative Robot Policies with Lexicographic Preferences

arXiv:2609.15014v1 Announce Type: cross Abstract: Pretrained generative robot policies can produce effective behaviors across diverse environments, but deployment can lead to requirements and preferen

TotalSynth: Robust Whole-Body Synthetic CT from MRI and CBCT

arXiv:2609.13838v1 Announce Type: new Abstract: Purpose: To develop and evaluate TotalSynth, a reusable pretrained framework for whole-body synthetic CT (sCT) generation from MRI and cone-beam CT (CBC

Visible Touch: Rendering Contact for Visuomotor Policies

model-releasesarxiv-cs-ro

arXiv:2609.14156v1 Announce Type: new Abstract: Integrating contact information into visuomotor policies remains an open problem. Touch is essential to robust manipulation, yet most modern policies, i

Local Ai

What Makes an Efficient VLA? Navigating Action-Head Design, Scaling, and Latency

arXiv:2609.13984v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models combine a pretrained vision encoder, a language backbone, and an action head, but their relative contribution has

local-aiarxiv-cs-cv

GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models

arXiv:2609.12731v1 Announce Type: new Abstract: Text-to-image (T2I) diffusion models inevitably internalize sensitive or non-compliant concepts from large-scale pretraining data, necessitating post-ho

Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation

arXiv:2609.12993v1 Announce Type: cross Abstract: We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WSL

researcharxiv-cs-cl

Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature

arXiv:2609.12123v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) is an effective approach for adapting large pretrained models by learning low-rank weight updates. In practice, the LoRA rank

Agents

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

arXiv:2609.13141v1 Announce Type: new Abstract: Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by selecting a small set of context un

agentsarxiv-cs-cl

Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks

arXiv:2501.04234v2 Announce Type: replace-cross Abstract: Modern artificial intelligence is supported by machine learning models (e.g., foundation models) that are pretrained on a massive data corpus

Very Exciting: Zero-Shot Model Predictive Control of Buildings via Excitation-Based Generalized Transfer Learning Models

arXiv:2609.12853v1 Announce Type: cross Abstract: The widespread adoption of data-driven, energy-efficient model predictive control (MPC) in buildings remains hindered by substantial effort to collect

researcharxiv-cs-lg

Beyond Noise Steering: Dual-Latent Space Reinforcement Learning for Generative Robot Policy

arXiv:2609.11270v1 Announce Type: new Abstract: Pretrained generative robot policies learn expressive action priors from demonstrations. However, existing reinforcement learning methods only steer the

safetyarxiv-cs-ro

11 Sept 2026

IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies

arXiv:2609.10915v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies leverage pretrained vision-language backbones to achieve strong cross-task generalization. A leading design coup

11 Sept 2026

Why Does Post-Training Quantization Work?

arXiv:2609.11716v1 Announce Type: cross Abstract: Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces

researcharxiv-cs-cl

11 Sept 2026

Learning transferable human physiology from two million hours of sleep with SleepFM-2

arXiv:2609.06849v1 Announce Type: new Abstract: Sleep provides a nightly window into health by capturing coordinated activity across the brain, heart, muscles and respiratory system. We introduce Slee

Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training

arXiv:2609.08115v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training i

What Makes Adversarial Examples Transfer Across Deepfake Detectors?

arXiv:2609.10002v1 Announce Type: new Abstract: Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and tr

researcharxiv-cs-cv

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

arXiv:2609.10355v1 Announce Type: cross Abstract: Video understanding has rapidly evolved toward video large language models (VideoLLMs): systems that couple video representations with pretrained larg

Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance

arXiv:2609.05730v1 Announce Type: new Abstract: Contrastive Language-Image Pretraining (CLIP) is a building block of many machine learning applications. Scaling laws have guided resource allocation fo

FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures

arXiv:2601.08026v5 Announce Type: replace Abstract: Scientific compound figures combine multiple labeled panels into a single image, and downstream pretraining and retrieval require panel-aligned visu

safetyarxiv-cs-cv

Tutorials

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

arXiv:2609.06079v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies leverage pretrained vision-language models (VLMs) to guide action generation for robot control. VLMs provide hie

tutorialsarxiv-cs-lg

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

arXiv:2609.09123v1 Announce Type: new Abstract: Autoregressive (AR) video diffusion models have shown great potential in real-time video generation. Recent methods distill pretrained bidirectional vid

researcharxiv-cs-cv

PatchFormer: A Patch-Based Time Series Foundation Model with Hierarchical Masked Reconstruction and Cross-Domain Transfer Learning for Zero-Shot Multi-Horizon Forecasting

arXiv:2601.20845v2 Announce Type: replace Abstract: Time series forecasting is a fundamental problem with applications in climate, energy, healthcare, and finance. Many existing approaches require dom

Reading Decoder Trajectories: Training-Free Counterfactual Query-Trajectory Reliability for Small-Object Detection

arXiv:2609.06581v1 Announce Type: new Abstract: Small-object detection remains challenging because limited pixels cause information loss and suppress the scale knowledge encoded in pretrained detector

EXAONE Forecast for Finance

arXiv:2609.04239v1 Announce Type: new Abstract: This technical report presents EXAONE Forecast for Finance (EXAONE Finance), a financial time series (TS) foundation model (TSFM) tailored to financial

7 Sept 2026

VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models

arXiv:2609.04355v1 Announce Type: cross Abstract: Pretrained vision-language-action (VLA) models enable broad manipulation but remain unreliable in tasks demanding precision and repeatability. Applyin

safetyarxiv-cs-ai

7 Sept 2026

WEECFP-SuRGE: Wide Embedded Extended Connectivity Fingerprint with Substructure Rotary Graph-distance Encoding

arXiv:2609.04672v1 Announce Type: new Abstract: We introduce WEECFP, a parameter-free 1024-dimensional continuous molecular fingerprint that scatters each Morgan substructure across roughly thirty-two

7 Sept 2026

Learnable composition for neural operators

arXiv:2609.03069v1 Announce Type: new Abstract: Neural operators are fast, differentiable surrogates for physical simulation, but their accuracy often degrades when domain geometry, size, or operating

researcharxiv-cs-lg

4 Sept 2026

Hardware

Solving the Needle-in-a-Haystack Problem in Mammography Vision-Language Model with Differentiable Subset Sampling

arXiv:2609.03085v1 Announce Type: new Abstract: There is growing interest in adopting CLIP-style vision--language model (VLM) pretraining for mammography. However, models that directly employ the stan

hardwarearxiv-cs-cv

4 Sept 2026

Xiaomi-TabLDM: A Tabular Foundation Model Technical Report

arXiv:2609.03880v1 Announce Type: new Abstract: We introduce Xiaomi-TabLDM, a tabular large data foundation model for classification and regression via in-context learning, which delivers superior pre