Search: “pretraining”

Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training

arXiv:2609.34272v1 Announce Type: new Abstract: BF16 is now standard in large-scale pretraining, including in fused attention kernels such as FlashAttention, and these kernels are widely trusted. When

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Can Protein-Derived Knowledge Improve Pathology Foundation Models?

DGX agent

arXiv:2609.33178v1 Announce Type: new Abstract: Molecularly guided pathology foundation models (PFMs) exploit transcriptomic or proteomic information to enrich whole-slide image (WSI) representations,

researcharxiv-cs-cv

29 Sept 2026

ConvCue: Complementary Visual Inductive Biases for Vision-Language Models

model-releasesarxiv-cs-cv

arXiv:2609.34196v1 Announce Type: new Abstract: Modern vision-language models (VLMs) achieve strong performance across a broad range of multimodal tasks, yet still struggle with visual questions that

Does Adversarial Training Improve Generalization in Multi-View VLAs? Revealing and Mitigating View Collapse

arXiv:2609.33707v1 Announce Type: cross Abstract: Vision-language-action (VLA) models adapt pretrained vision-language models (VLMs) for closed-loop robot control, transferring their perceptual and se

DreamingGoose: Staged Distillation from Autoregressive Transformers to Bidirectional Recurrent Diffusion Language Models

arXiv:2609.34253v1 Announce Type: new Abstract: Pretrained autoregressive Transformers represent a large sunk investment in compute. Existing conversion methods reuse that investment by changing eithe

researcharxiv-cs-lg

E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding

arXiv:2609.33530v1 Announce Type: cross Abstract: Natural Language Understanding (NLU) plays a crucial role in various applications, yet its performance suffers from weaknesses in handling the complex

applicationsarxiv-cs-lg

From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers

arXiv:2609.34866v1 Announce Type: new Abstract: Most post-training quantization pipelines fit each weight matrix to its pretrained counterpart, one matrix at a time. Whether that proxy tracks what an

Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing

arXiv:2609.32928v1 Announce Type: new Abstract: Millimeter-wave (mmWave) radar and RGB-D cameras can record cardiac and respiratory waveforms continuously and without contact, but labeled recordings r

researcharxiv-cs-lg

Prompt-Anchored Residual Adaptation for Biomedical Vision-Language Models

arXiv:2609.33701v1 Announce Type: new Abstract: Pretrained biomedical vision-language models achieve strong zero-shot performance in biomedical image classification. However, downstream biomedical cla

Residual Denoising Enables Sample-Efficient Multi-Agent Coordination on Demand

arXiv:2609.32129v1 Announce Type: new Abstract: Pretrained robot policies offer strong manipulation skills but are typically limited to single-agent settings, where a robot acts in isolation. In this

RPA: Residual Patch-Token Adapter for Image Retrieval from EEG and MEG

arXiv:2609.31698v1 Announce Type: new Abstract: Most existing MEG and EEG (M/EEG) visual decoding methods align brain signals with a single global embedding extracted from a pretrained visual encoder,

RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning

arXiv:2603.09160v2 Announce Type: replace Abstract: Dense image captioning is critical for vision-language pretraining and for text-to-image generation, but scaling expert-quality annotations is prohi

WAM-OPD: Sharpening World Action Models via On-Policy Distillation

arXiv:2609.34250v1 Announce Type: new Abstract: Pretrained world action models (WAMs) provide generalist capabilities across diverse robotic manipulation tasks, yet improving target-task performance t

WirelessMathBench-XL: An Auditable Benchmark for Wireless Mathematical Reasoning

arXiv:2509.23219v2 Announce Type: replace Abstract: Technical-domain benchmarks constructed from arXiv papers can overlap the same public text used in LLM pretraining. Auditing this risk at training-c

BAT-CLIP: Trimodal Alignment of Brain, Audio and Text

model-releasesarxiv-cs-ai

arXiv:2609.31180v1 Announce Type: cross Abstract: Decoding and interpreting naturalistic speech from the brain increasingly relies on alignment to pretrained speech and language representation spaces.

Tutorials

Combining General and Domain-Specific Pretext Tasks for Brain MR Image Segmentation

arXiv:2609.30708v1 Announce Type: cross Abstract: A key challenge in medical image analysis is the scarcity of large annotated datasets for specific populations and diseases. As deep learning models r

tutorialsarxiv-cs-ai

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

arXiv:2609.31620v1 Announce Type: new Abstract: Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual re

Neural Parameter Estimation of RC Thermal Building Models for Model Predictive Control

arXiv:2604.05904v2 Announce Type: replace-cross Abstract: Gray-box RC models are widely used to enable energy-efficient model predictive control (MPC) in buildings. However, estimating RC parameters r

Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra

arXiv:2609.31206v1 Announce Type: new Abstract: Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be

Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs

arXiv:2609.29382v1 Announce Type: cross Abstract: Flow-matching Vision-Language-Action (VLA) models have emerged as a potential solution for generalist robot control, designed by combining a pretraine

safetyarxiv-cs-lg

25 Sept 2026

Qwengram-0.8B: I transferred Qwen3.8 Flash-Next’s n-gram memory into Qwen3.5-0.8B — 5.05% lower validation perplexity

model-releasesr-localllama

I’ve been experimenting with whether Qwen3.8-Flash-Next’s pretrained PLE n-gram memory can improve a much smaller Qwen3.5-0.8B model. I trained the 0.8B setup with limited resources, mostly using free

25 Sept 2026

Self-Supervised Anchoring of Fingertip Sensing to Proprioception and Proactive Actions for Robot Imitation Learning

arXiv:2609.29822v1 Announce Type: new Abstract: Robotic imitation learning often relies on external cameras, yet local interaction cues such as object proximity, contact onset, and grasp state are dif

25 Sept 2026

Beyond Future Prediction: Denoising as Generative Adaptation for Robot Control

arXiv:2609.28339v1 Announce Type: new Abstract: Pretrained generative Diffusion Transformers (DiTs) capture rich pixel-level visual and language-conditioned structure through large-scale image and vid

researcharxiv-cs-ro

Latent evolving World Action Model

arXiv:2609.27455v1 Announce Type: new Abstract: World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). In

Agents

TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuning

arXiv:2609.28314v1 Announce Type: new Abstract: Human teleoperators spend substantial time demonstrating behaviors that robots can already perform autonomously, limiting the scalability of data collec

agentsarxiv-cs-ro

VGM-VS: Rethinking Visual Geometry Model for High-Precision Visual Servoing

arXiv:2609.28312v1 Announce Type: cross Abstract: We present VGM-VS, a visual servoing method built on a pretrained feed-forward visual geometry model. Given the current view and a reference image cap

applicationsarxiv-cs-cv

TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection

arXiv:2609.25775v1 Announce Type: new Abstract: Recent advances in generative video models have enabled the synthesis of visually realistic content, posing significant challenges to synthetic video de

23 Sept 2026

Training a Language Model End-to-End in Rust: An Experience Report

model-releasesarxiv-cs-cl

arXiv:2609.25008v1 Announce Type: new Abstract: I pretrained a language model end-to-end in Rust - alone, with no team, no PyTorch, and no Python in the training path - for $164 in rented GPU time. I

23 Sept 2026

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

model-releasesarxiv-cs-cl

arXiv:2609.25081v1 Announce Type: new Abstract: We describe ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly li

23 Sept 2026

Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery

arXiv:2609.23924v1 Announce Type: new Abstract: Pretrained EEG foundation models are increasingly proposed as general-purpose encoders for brain-computer interfaces, yet recent benchmarks disagree abo

researcharxiv-cs-lg

SPeaR: Test-Time Adaptation with Steering Primitives for Realigning Representations

arXiv:2609.24111v1 Announce Type: cross Abstract: Test-time adaptation (TTA) addresses distribution shift using only unlabeled test data. Existing methods typically adapt pretrained models by updating

TRACE: Tractable Routing Autoencoder for Clinical ECG

arXiv:2609.23460v1 Announce Type: new Abstract: Deep learning has advanced automated electrocardiogram (ECG) diagnosis, but the field's most accurate models, foundation models pretrained on millions o

Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea

model-releasesarxiv-cs-cv

arXiv:2609.24064v1 Announce Type: new Abstract: New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer tha

When Wider Views Fail: Stress-Testing Feed-Forward 3D Reconstruction

arXiv:2609.24839v1 Announce Type: new Abstract: Feed-forward 3D reconstruction models enable efficient geometry estimation from sparse images, but their pretrained nature can make them vulnerable to d

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

model-releasesarxiv-cs-ai

arXiv:2603.26556v3 Announce Type: replace-cross Abstract: Converting a pretrained Transformer into a more efficient hybrid model through distillation offers a promising approach to reducing inference

18 Sept 2026

Reinforcement Learning for Real-Time Vision-Language-Action Policies

arXiv:2609.18207v1 Announce Type: cross Abstract: Reinforcement learning fine-tuning on top of large, pretrained Vision-Language-Action (VLA) models offers promise for highly reliable robot deployment

17 Sept 2026

WholeBodyWAM: Learning Whole-Body World Action Models with Scalable Motion Priors

arXiv:2609.18197v1 Announce Type: new Abstract: Humanoid whole-body manipulation requires coordinated whole-body dynamics, yet large-scale trajectories from a target robot are expensive to collect and

applicationsarxiv-cs-ro

17 Sept 2026

A Controlled Reevaluation of Coreference Resolution Models

arXiv:2404.00727v3 Announce Type: replace Abstract: All state-of-the-art coreference resolution (CR) models involve finetuning a pretrained language model. Whether the superior performance of one CR m

researcharxiv-cs-cl

Backward SDEs-based Diffusion for Physics-Constrained Generation

arXiv:2609.15702v1 Announce Type: new Abstract: Pretrained score-based diffusion models provide strong unconditional priors, yet enforcing measurement or physics consistency in inverse problems is oft

applicationsarxiv-cs-lg

WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

model-releasesarxiv-cs-cv

arXiv:2506.24092v3 Announce Type: replace Abstract: Adapting large pretrained vision models to medical image classification is often limited by memory, computation, and task-specific specializations.

When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control

model-releasesarxiv-cs-ai

arXiv:2609.15781v1 Announce Type: cross Abstract: Pretrained world models, learned simulators that encode an observation into a latent state and predict how it evolves under actions, are beginning to

iBrain: A Unified Foundation Model Reading the Brain from Surface to Spikes

arXiv:2609.06960v1 Announce Type: new Abstract: Invasive neural recordings provide high-fidelity measurements of brain activity, with signals such as intracranial EEG (iEEG) and intracortical spiking

safetyarxiv-cs-ai

10 Sept 2026

Tutorials

Learning Counterfactual World Models for Embodied Reasoning under Partial Observability

arXiv:2609.05834v1 Announce Type: new Abstract: World models promise a general route to embodied intelligence: learn predictive dynamics once, then reason, plan, and act with them. Increasingly, the r

tutorialsarxiv-cs-ai

10 Sept 2026

RoboDrop: Curating VLA Post-Training Data via Local Gradient Compatibility

arXiv:2609.10021v1 Announce Type: new Abstract: Vision--language--action (VLA) models acquire broad generalization through large-scale pretraining, yet adapting them to a new task and robot embodiment

10 Sept 2026

3DWay: Generalizing Robot Manipulation via 3D Consistent Waypoints

model-releasesarxiv-cs-ro

arXiv:2609.08224v1 Announce Type: new Abstract: Intermediate representations are key to bridging the modality gap between generalizable manipulation policies and large-scale pretrained vision-language

Anthropic researcher’s resignation sparks broad AI safety discussion

An Anthropic PBC researcher has resigned over concerns that artificial intelligence labs are “gambling with our lives.” Jacob Coxon was part of the company’s AI pretraining team until today. He announ

safetysiliconangle

Agents

Precision at Scale: Domain-Specific Datasets On-Demand

arXiv:2407.03463v2 Announce Type: replace Abstract: In the realm of self-supervised learning (SSL), conventional wisdom has gravitated towards the utility of massive, general domain datasets for pretr

agentsarxiv-cs-cv

Scaling 3D Generative Priors to Large-Scale Scene Meshes from Multi-View Images

arXiv:2609.06385v1 Announce Type: new Abstract: Pretrained 3D generative models produce detailed geometry and appearance but are primarily designed for object-centric generation within a limited spati