Search: “pretraining”

A foundation model for energy and radiation systems built on heterogeneous scientific interfaces

arXiv:2609.38067v1 Announce Type: new Abstract: Scientific foundation models are commonly evaluated after heterogeneous physical problems have already been translated into a compatible gridded, tokeni

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation

DGX agent

arXiv:2609.37591v1 Announce Type: cross Abstract: Test-time adaptation for vision-language navigation (TTA-VLN) enables pretrained policies to adapt online to unseen environments using only test-time

safetyarxiv-cs-ai

30 Sept 2026

Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging

arXiv:2609.37564v1 Announce Type: new Abstract: Merging pretrained models has emerged as an effective approach for consolidating diverse capabilities into a single unified model. However, prevailing m

researcharxiv-cs-cl

TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Tasks

arXiv:2609.36968v1 Announce Type: new Abstract: Unsupervised tabular anomaly detection (TAD) aims to identify anomalous rows in tabular data using normal training samples. While conventional methods r

applicationsarxiv-cs-lg

The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

arXiv:2609.36118v1 Announce Type: new Abstract: Vision-language-action (VLA) policies connect a pretrained vision-language backbone to an action head through a latent interface, but which backbone lay

safetyarxiv-cs-ai

Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

arXiv:2609.37924v1 Announce Type: cross Abstract: Recent work on anchored diffusion language models improves denoising by shaping an intermediate latent space with supervised important-token targets.

researcharxiv-cs-cl

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

arXiv:2609.36585v1 Announce Type: new Abstract: Pretrained transformers use little of their depth to follow references in context. Thirteen base models reliably follow only 1.4-3.6 lines, and extra pr

researcharxiv-cs-ai

You Only Reprogram Once: Rethinking Prolonged Training for Visual Reprogramming

arXiv:2609.36661v1 Announce Type: new Abstract: Visual reprogramming is a parameter-efficient method for adapting pretrained models, yet its training can remain computationally expensive: even with a

Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training

arXiv:2609.34272v1 Announce Type: new Abstract: BF16 is now standard in large-scale pretraining, including in fused attention kernels such as FlashAttention, and these kernels are widely trusted. When

Can Protein-Derived Knowledge Improve Pathology Foundation Models?

arXiv:2609.33178v1 Announce Type: new Abstract: Molecularly guided pathology foundation models (PFMs) exploit transcriptomic or proteomic information to enrich whole-slide image (WSI) representations,

ConvCue: Complementary Visual Inductive Biases for Vision-Language Models

arXiv:2609.34196v1 Announce Type: new Abstract: Modern vision-language models (VLMs) achieve strong performance across a broad range of multimodal tasks, yet still struggle with visual questions that

Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse

arXiv:2609.33707v1 Announce Type: cross Abstract: Vision-language-action (VLA) models adapt pretrained vision-language models (VLMs) for closed-loop robot control, transferring their perceptual and se

DreamingGoose: Staged Distillation from Autoregressive Transformers to Bidirectional Recurrent Diffusion Language Models

arXiv:2609.34253v1 Announce Type: new Abstract: Pretrained autoregressive Transformers represent a large sunk investment in compute. Existing conversion methods reuse that investment by changing eithe

researcharxiv-cs-lg

E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding

arXiv:2609.33530v1 Announce Type: cross Abstract: Natural Language Understanding (NLU) plays a crucial role in various applications, yet its performance suffers from weaknesses in handling the complex

applicationsarxiv-cs-lg

From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers

arXiv:2609.34866v1 Announce Type: new Abstract: Most post-training quantization pipelines fit each weight matrix to its pretrained counterpart, one matrix at a time. Whether that proxy tracks what an

Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing

arXiv:2609.32928v1 Announce Type: new Abstract: Millimeter-wave (mmWave) radar and RGB-D cameras can record cardiac and respiratory waveforms continuously and without contact, but labeled recordings r

researcharxiv-cs-lg

Prompt-Anchored Residual Adaptation for Biomedical Vision-Language Models

arXiv:2609.33701v1 Announce Type: new Abstract: Pretrained biomedical vision-language models achieve strong zero-shot performance in biomedical image classification. However, downstream biomedical cla

Residual Denoising Enables Sample-Efficient Multi-Agent Coordination on Demand

arXiv:2609.32129v1 Announce Type: new Abstract: Pretrained robot policies offer strong manipulation skills but are typically limited to single-agent settings, where a robot acts in isolation. In this

RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG

arXiv:2609.31698v1 Announce Type: new Abstract: Most existing MEG and EEG (M/EEG) visual decoding methods align brain signals with a single global embedding extracted from a pretrained visual encoder,

safetyarxiv-cs-cv

RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning

arXiv:2603.09160v2 Announce Type: replace Abstract: Dense image captioning is critical for vision-language pretraining and for text-to-image generation, but scaling expert-quality annotations is prohi

safetyarxiv-cs-cv

WAM-OPD: Sharpening World Action Models via On-Policy Distillation

arXiv:2609.34250v1 Announce Type: new Abstract: Pretrained world action models (WAMs) provide generalist capabilities across diverse robotic manipulation tasks, yet improving target-task performance t

WirelessMathBench-XL: An Auditable Benchmark for Wireless Mathematical Reasoning

arXiv:2509.23219v2 Announce Type: replace Abstract: Technical-domain benchmarks constructed from arXiv papers can overlap the same public text used in LLM pretraining. Auditing this risk at training-c

BAT-CLIP: Trimodal Alignment of Brain, Audio and Text

model-releasesarxiv-cs-ai

arXiv:2609.31180v1 Announce Type: cross Abstract: Decoding and interpreting naturalistic speech from the brain increasingly relies on alignment to pretrained speech and language representation spaces.

Tutorials

Combining General and Domain-Specific Pretext Tasks for Brain MR Image Segmentation

arXiv:2609.30708v1 Announce Type: cross Abstract: A key challenge in medical image analysis is the scarcity of large annotated datasets for specific populations and diseases. As deep learning models r

tutorialsarxiv-cs-ai

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

arXiv:2609.31620v1 Announce Type: new Abstract: Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual re

Neural Parameter Estimation of RC Thermal Building Models for Model Predictive Control

arXiv:2604.05904v2 Announce Type: replace-cross Abstract: Gray-box RC models are widely used to enable energy-efficient model predictive control (MPC) in buildings. However, estimating RC parameters r

Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra

arXiv:2609.31206v1 Announce Type: new Abstract: Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be

Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs

arXiv:2609.29382v1 Announce Type: cross Abstract: Flow-matching Vision-Language-Action (VLA) models have emerged as a potential solution for generalist robot control, designed by combining a pretraine

safetyarxiv-cs-lg

25 Sept 2026

Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning

arXiv:2609.29822v1 Announce Type: new Abstract: Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are dif

25 Sept 2026

Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

arXiv:2609.28339v1 Announce Type: new Abstract: Pretrained generative Diffusion Transformers (DiTs) capture rich pixel-level visual and language-conditioned structure through large-scale image and vid

researcharxiv-cs-ro

Latent evolving World Action Model

arXiv:2609.27455v1 Announce Type: new Abstract: World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). In

safetyarxiv-cs-cv

Agents

TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuning

arXiv:2609.28314v1 Announce Type: new Abstract: Human teleoperators spend substantial time demonstrating behaviors that robots can already perform autonomously, limiting the scalability of data collec

agentsarxiv-cs-ro

VGM-VS: Rethinking Visual Geometry Model for High-Precision Visual Servoing

arXiv:2609.28312v1 Announce Type: cross Abstract: We present VGM-VS, a visual servoing method built on a pretrained feed-forward visual geometry model. Given the current view and a reference image cap

applicationsarxiv-cs-cv

TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection

arXiv:2609.25775v1 Announce Type: new Abstract: Recent advances in generative video models have enabled the synthesis of visually realistic content, posing significant challenges to synthetic video de

23 Sept 2026

Training a Language Model End-to-End in Rust: An Experience Report

model-releasesarxiv-cs-cl

arXiv:2609.25008v1 Announce Type: new Abstract: I pretrained a language model end-to-end in Rust - alone, with no team, no PyTorch, and no Python in the training path - for $164 in rented GPU time. I

23 Sept 2026

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

model-releasesarxiv-cs-cl

arXiv:2609.25081v1 Announce Type: new Abstract: We describe ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly li

23 Sept 2026

Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery

arXiv:2609.23924v1 Announce Type: new Abstract: Pretrained EEG foundation models are increasingly proposed as general-purpose encoders for brain-computer interfaces, yet recent benchmarks disagree abo

researcharxiv-cs-lg

SPeaR: Test-Time Adaptation with Steering Primitives for Realigning Representations

arXiv:2609.24111v1 Announce Type: cross Abstract: Test-time adaptation (TTA) addresses distribution shift using only unlabeled test data. Existing methods typically adapt pretrained models by updating

TRACE: Tractable Routing Autoencoder for Clinical ECG

arXiv:2609.23460v1 Announce Type: new Abstract: Deep learning has advanced automated electrocardiogram (ECG) diagnosis, but the field's most accurate models, foundation models pretrained on millions o

Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea

arXiv:2609.24064v1 Announce Type: new Abstract: New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer tha

When Wider Views Fail: Stress-Testing Feed-Forward 3D Reconstruction

arXiv:2609.24839v1 Announce Type: new Abstract: Feed-forward 3D reconstruction models enable efficient geometry estimation from sparse images, but their pretrained nature can make them vulnerable to d

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

model-releasesarxiv-cs-ai

arXiv:2603.26556v3 Announce Type: replace-cross Abstract: Converting a pretrained Transformer into a more efficient hybrid model through distillation offers a promising approach to reducing inference

18 Sept 2026

Reinforcement Learning for Real-Time Vision-Language-Action Policies

arXiv:2609.18207v1 Announce Type: cross Abstract: Reinforcement learning fine-tuning on top of large, pretrained Vision-Language-Action (VLA) models offers promise for highly reliable robot deployment

17 Sept 2026

WholeBodyWAM: Learning Whole-Body World Action Models with Scalable Motion Priors

arXiv:2609.18197v1 Announce Type: new Abstract: Humanoid whole-body manipulation requires coordinated whole-body dynamics, yet large-scale trajectories from a target robot are expensive to collect and

applicationsarxiv-cs-ro

17 Sept 2026

A Controlled Reevaluation of Coreference Resolution Models

arXiv:2404.00727v3 Announce Type: replace Abstract: All state-of-the-art coreference resolution (CR) models involve finetuning a pretrained language model. Whether the superior performance of one CR m

researcharxiv-cs-cl

Backward SDEs-based Diffusion for Physics-Constrained Generation

arXiv:2609.15702v1 Announce Type: new Abstract: Pretrained score-based diffusion models provide strong unconditional priors, yet enforcing measurement or physics consistency in inverse problems is oft

applicationsarxiv-cs-lg

WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

arXiv:2506.24092v3 Announce Type: replace Abstract: Adapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations.

When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control

model-releasesarxiv-cs-ai

arXiv:2609.15781v1 Announce Type: cross Abstract: Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to