Search: “pretraining”

BCG-FM: A Foundation Model for Ambient Cardiac Health Sensing

arXiv:2606.07692v1 Announce Type: cross Abstract: Foundation models for wearable biosignals have matched or exceeded supervised specialists across a range of clinical tasks, yet all rely on modalities

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery

DGX agent

arXiv:2606.09672v1 Announce Type: new Abstract: Ask a pretrained biomedical language model whether 'cortisol 28 ug/dL' and 'stock-market volatility' are related, and it returns a cosine similarity of

model-releasesarxiv-cs-ai

9 Jun 2026

Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space

model-releasesarxiv-cs-lg

arXiv:2603.29824v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning methods such as LoRA enable efficient adaptation of large pretrained models, but often lag behind full fine-tuning i

GNSS-FM: A Self-Supervised Foundation Model for Daily GNSS Displacement Time Series

arXiv:2606.07725v1 Announce Type: cross Abstract: Displacement time series from Global Navigation Satellite Systems (GNSS) are essential for a wide range of applications, including monitoring tectonic

In-Context Learning of Temporal Point Processes with Foundation Inference Models

model-releasesarxiv-cs-lg

arXiv:2509.24762v3 Announce Type: replace Abstract: Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamic

INFUSER: Influence-Guided Self-Evolution Improves Reasoning

arXiv:2606.09052v1 Announce Type: cross Abstract: Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet e

MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training

arXiv:2606.08788v1 Announce Type: new Abstract: Representation alignment with pretrained vision models has recently shown strong potential for accelerating diffusion transformer training. By aligning

Local Ai

MosaicIMU: Composing Carrier Experts for Generalizable Neural Inertial Odometry

arXiv:2606.09355v1 Announce Type: new Abstract: Robust inertial odometry is essential for various carriers when external sensing is unreliable. Learning-based methods reduce integration drift by captu

local-aiarxiv-cs-ro

Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation

arXiv:2606.08679v1 Announce Type: cross Abstract: Pretrained models are often evaluated on multi-task leaderboards to measure their applicability in diverse contexts. However, current methods for aggr

Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning

model-releasesarxiv-cs-cv

arXiv:2606.09303v1 Announce Type: new Abstract: The rapid development of pretrained foundation models has enabled more general image segmentation. Multimodal large language models (MLLMs) have been wi

Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

arXiv:2506.06891v3 Announce Type: replace Abstract: We study the corruption-robustness of in-context reinforcement learning (ICRL), focusing on the Decision-Pretrained Transformer (DPT, Lee et al., 20

TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech

arXiv:2606.09019v1 Announce Type: cross Abstract: Codec-based autoregressive (AR) speech language models have achieved strong text-to-speech (TTS) quality by modeling speech as sequences of discrete a

UAOR: Uncertainty-aware Observation Reinjection for Vision-Language-Action Models

arXiv:2602.18020v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models leverage pretrained Vision-Language Models (VLMs) as backbones to map images and instructions to actions, demons

applicationsarxiv-cs-cv

A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Design

arXiv:2510.16023v2 Announce Type: replace Abstract: Linear polymers, macromolecules formed from monomers covalently bonded into continuous chains, underpin countless technologies and are indispensable

MoDA: Modulation Adapter for Fine-Grained Visual Grounding in Instructional MLLMs

arXiv:2506.01850v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success in instruction-following tasks by integrating pretrained visual enco

One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models

arXiv:2602.00541v2 Announce Type: replace Abstract: Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical me

Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning

arXiv:2606.06835v1 Announce Type: new Abstract: The performance gap across languages in LLMs is well documented, and closing it natively requires pretraining or fine-tuning on corpora that, for most l

safetyarxiv-cs-cl

Local Ai

AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding

arXiv:2606.06155v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models leverage the rich world knowledge of pretrained vision-language models (VLMs) to enable instruction-following robo

local-aiarxiv-cs-cv

Flow-based Policy Adaptation without Policy Updates

arXiv:2606.06461v1 Announce Type: new Abstract: Leveraging prior knowledge from pretrained policies, foundation models, or human operators offers an efficient alternative to learning robot skills from

safetyarxiv-cs-ro

Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learning

arXiv:2606.05506v1 Announce Type: new Abstract: We propose a sensor-guided adaptive contrastive learning framework for visual representation learning in PointGoal navigation. During training, privileg

Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning

arXiv:2601.21700v3 Announce Type: replace Abstract: Large Language Models (LLMs) increasingly support culturally sensitive decision making, yet often exhibit misalignment due to skewed pretraining dat

safetyarxiv-cs-cl

Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction

arXiv:2602.23214v2 Announce Type: replace Abstract: Plug-and-Play diffusion prior (PnPDP) frameworks have emerged as a powerful paradigm for solving imaging inverse problems by treating pretrained gen

4 Jun 2026

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization

arXiv:2505.11166v3 Announce Type: replace-cross Abstract: Despite advances in pretraining with extended context sizes, large language models (LLMs) still face challenges in effectively utilizing real-

safetyarxiv-cs-ai

4 Jun 2026

An Asymptotic Theory of Chain-of-Thought in In-Context Learning

model-releasesarxiv-cs-lg

arXiv:2606.03217v1 Announce Type: cross Abstract: Chain-of-thought (CoT) reasoning has become a widely used mechanism for eliciting multi-step reasoning in large language models by generating intermed

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

model-releasesarxiv-cs-ai

arXiv:2606.03657v1 Announce Type: new Abstract: Large language models for code generation often need to use APIs that are absent from their pretraining data. This requires more than recalling a functi

How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models

arXiv:2601.22841v2 Announce Type: replace Abstract: Large-scale foundation models (FMs) in remote sensing (RS) (denoted as RS FMs) are developed following paradigms established in computer vision (CV)

researcharxiv-cs-cv

Investigating Adversarial Robustness of Multi-modal Large Language Models

model-releasesarxiv-cs-cv

arXiv:2606.03713v1 Announce Type: new Abstract: Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder

Typhoon: Towards an Effective Task-Specific Masking Strategy for Pre-trained Language Models

arXiv:2303.15619v2 Announce Type: replace-cross Abstract: The choice of which tokens to mask is a central, under-examined design decision in masked language modeling (MLM). Standard pretraining masks

A Foundation Model for Wearable Movement Data in Mental Health Research

arXiv:2411.15240v5 Announce Type: replace-cross Abstract: Wearable movement data is collected by nearly all commercially available smartwatches and is a valuable resource for mental health research, r

applicationsarxiv-cs-ai

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

arXiv:2606.02194v1 Announce Type: new Abstract: Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robot

safetyarxiv-cs-lg

Echo: A Joint-Embedding Predictive Architecture for Speaker Diarization and Speech Recognition in a Shared Latent Space

model-releasesarxiv-cs-ai

arXiv:2606.01909v1 Announce Type: cross Abstract: We present Echo, a proof-of-concept audio system built around a single 25 M-parameter ViT encoder. The encoder is pretrained with a JEPA objective and

Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models

model-releasesarxiv-cs-ai

arXiv:2504.03635v4 Announce Type: replace Abstract: Reasoning is a core capability of language models (LMs), yet it remains unclear how much model capacity is necessary to support reasoning during pre

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606.00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models. However, most exi

applicationsarxiv-cs-ai

Information-Theoretic Lower Bounds for Bit-Constrained Stochastic Optimization via a Reduction to Compressed Gaussian Mean Estimation

arXiv:2606.00703v1 Announce Type: cross Abstract: Low-precision pretraining (FP8, MXFP4, NVFP4) is now standard for frontier language models, yet the literature is almost entirely achievability -- alg

KDH-CAD: Knowledge-data hybrid CAD learning under data scarcity

arXiv:2606.01702v1 Announce Type: cross Abstract: Deep learning in computer-aided design (CAD) remains fundamentally constrained by the data scarcity challenge: authentic CAD data is difficult to coll

applicationsarxiv-cs-lg

Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning

arXiv:2602.08689v2 Announce Type: replace Abstract: Diffusion models generate samples through an iterative denoising process guided by a pretrained neural network. Once the denoiser is fixed, the samp

safetyarxiv-cs-lg

On the Collapse of Generative Paths: A Criterion and Correction for Diffusion Steering

arXiv:2512.10339v2 Announce Type: replace Abstract: Inference-time steering adapts pretrained diffusion and flow models to new tasks without retraining, often utilizing ratio-of-densities construction

Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models

model-releasesarxiv-cs-cl

arXiv:2606.00284v1 Announce Type: new Abstract: While continual pretraining~(CPT) is a practical way to extend large language models to new languages, naive finetuning on targeted data erodes existing

RoboSemanticBench: Diagnosing Semantic Grounding in Action Prediction for VLA Models

model-releasesarxiv-cs-ro

arXiv:2606.02277v1 Announce Type: new Abstract: Vision-language-action (VLA) models are built on the premise that semantic understanding from pretrained language or vision-language backbones should gu

Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression

arXiv:2606.00512v1 Announce Type: new Abstract: In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce

TempRet: Temporal Enhancement and Two-Stage Reranking for CVPR 2026 EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge

model-releasesarxiv-cs-cv

arXiv:2605.24470v2 Announce Type: replace Abstract: Video-text retrieval has witnessed remarkable progress driven by large-scale vision-language pretraining, yet most existing approaches inherit an im

Why Are DMD Students Lazy? Understanding the Copying Behavior in Few-Step Distillation

arXiv:2606.02237v1 Announce Type: new Abstract: Distribution Matching Distillation (DMD) compresses pretrained diffusion models into efficient few-step generators by aligning their noised distribution

Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS

arXiv:2605.30748v1 Announce Type: cross Abstract: We present Chatterbox-Flash, a zero-shot text-to-speech model obtained by fine-tuning a pretrained autoregressive TTS decoder into a block-diffusion d

NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models

arXiv:2605.30393v1 Announce Type: cross Abstract: Public numeric benchmarks appear in pretraining, so an evaluation that conditions on a date may be measuring memorized recall rather than out-of-sampl

applicationsarxiv-cs-ai

Parallel Tempering Initial Sampling in Inference-Time Reward Alignment

arXiv:2605.30991v1 Announce Type: cross Abstract: Inference-time reward alignment steers pretrained diffusion and flow-based generative models to satisfy user-specified rewards without retraining. Rec

which is NOT new; see this quote from 5 years ago. the fact that all this is still true says a lot.

which is NOT new; see this quote from 5 years ago. the fact that all this is still true says a lot. This was right five years ago, and still is: “Large scale pretrained models are certainly likely to

safetygary-marcus--x

Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems

arXiv:2506.00175v5 Announce Type: replace-cross Abstract: Modern AI systems are typically developed through multiple stages-pretraining, fine-tuning rounds, and subsequent adaptation or alignment, whe

safetyarxiv-cs-ai

AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching