Search: “pretraining”

SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization

model-releasesarxiv-cs-lg

arXiv:2604.07663v2 Announce Type: replace Abstract: The AdamW optimizer, while standard for LLM pretraining, is a critical memory bottleneck, consuming optimizer states equivalent to twice the model's

17 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Enhancing Mixture-of-Experts Specialization via Cluster-Aware Upcycling

DGX agent

arXiv:2604.13508v1 Announce Type: new Abstract: Sparse Upcycling provides an efficient way to initialize a Mixture-of-Experts (MoE) model from pretrained dense weights instead of training from scratch

researcharxiv-cs-cv

16 Apr 2026

Frozen Forecasting: A Unified Evaluation

arXiv:2507.13942v2 Announce Type: replace Abstract: Forecasting future events is a fundamental capability for general-purpose systems that plan or act across different levels of abstraction. Yet, eval

16 Apr 2026

(How) Learning Rates Regulate Catastrophic Overtraining

arXiv:2604.13627v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a hel

researcharxiv-cs-cl

16 Apr 2026

RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real World

model-releasesarxiv-cs-cl

arXiv:2604.05096v2 Announce Type: replace Abstract: Large language models (LLMs) acquire most of their knowledge during pretraining, which ties them to a fixed snapshot of the world and makes adaptati

16 Apr 2026

IMU: Influence-guided Machine Unlearning

arXiv:2508.01620v3 Announce Type: replace-cross Abstract: Machine Unlearning (MU) aims to selectively erase the influence of specific data points from pretrained models. However, most existing MU meth

Models Know Their Shortcuts: Deployment-Time Shortcut Mitigation

arXiv:2604.12277v1 Announce Type: new Abstract: Pretrained language models often rely on superficial features that appear predictive during training yet fail to generalize at test time, a phenomenon k

safetyarxiv-cs-lg

RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair

arXiv:2604.12820v1 Announce Type: new Abstract: Large language models (LLMs) inherently absorb harmful knowledge, misinformation, and personal data during pretraining on large-scale web corpora, with

local-aiarxiv-cs-ai

Applications

Robotic Manipulation is Vision-to-Geometry Mapping (f(v) rightarrow G): Vision-Geometry Backbones over Language and Video Models

arXiv:2604.12908v1 Announce Type: new Abstract: At its core, robotic manipulation is a problem of vision-to-geometry mapping (f(v) rightarrow G). Physical actions are fundamentally defined by geometri

applicationsarxiv-cs-ro

Tutorials

SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learning

arXiv:2602.08590v4 Announce Type: replace Abstract: Vision-language pretrained models offer strong transferable representations, yet adapting them in privacy-sensitive multi-party settings is challeng

tutorialsarxiv-cs-lg

Tutorials

Siamese Foundation Models for Crystal Structure Prediction

arXiv:2503.10471v2 Announce Type: replace-cross Abstract: Predicting crystal structures from chemical compositions is a fundamental challenge in materials discovery, complicated by complex 3D geometri

tutorialsarxiv-cs-ai

Task Alignment: A simple and effective proxy for model merging in computer vision

arXiv:2604.12935v1 Announce Type: new Abstract: Efficiently merging several models fine-tuned for different tasks, but stemming from the same pretrained base model, is of great practical interest. Des

safetyarxiv-cs-cv

Agents

BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving

arXiv:2604.10856v1 Announce Type: cross Abstract: Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in c

agentsarxiv-cs-ai

DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation

arXiv:2604.10882v1 Announce Type: cross Abstract: Graph Neural Network pretraining is pivotal for leveraging unlabeled graph data. However, generalizing across heterogeneous domains remains a major ch

Agents

EDFNet: Early Fusion of Edge and Depth for Thin-Obstacle Segmentation in UAV Navigation

arXiv:2604.09694v1 Announce Type: new Abstract: Autonomous Unmanned Aerial Vehicles (UAVs) must reliably detect thin obstacles such as wires, poles, and branches to navigate safely in real-world envir

agentsarxiv-cs-cv

FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer

arXiv:2604.10023v1 Announce Type: cross Abstract: With the growing availability of open-sourced adapters trained on the same diffusion backbone for diverse scenes and objects, combining these pretrain

Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

arXiv:2604.10103v1 Announce Type: new Abstract: Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window att

local-aiarxiv-cs-cv

Masked Contrastive Pre-Training Improves Music Audio Key Detection

arXiv:2604.10021v1 Announce Type: cross Abstract: Self-supervised music foundation models underperform on key detection, which requires pitch-sensitive representations. In this work, we present the fi

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers

arXiv:2604.11508v1 Announce Type: cross Abstract: Fine-tuning pretrained image classifiers is standard practice, yet which individual samples are forgotten during this process, and whether forgetting

Training-Free Cross-Lingual Dysarthria Severity Assessment via Phonological Subspace Analysis in Self-Supervised Speech Representations

arXiv:2604.10123v1 Announce Type: new Abstract: Dysarthric speech severity assessment typically requires trained clinicians or supervised models built from labelled pathological speech, limiting scala

safetyarxiv-cs-cl

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting

model-releasesarxiv-cs-ai

arXiv:2604.10544v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have recently achieved remarkable success in universal forecasting by leveraging large-scale pretraining on dive

Another BRIXEL in the Wall: Towards Cheaper Dense Features

arXiv:2511.05168v2 Announce Type: replace Abstract: Vision foundation models achieve strong performance on both global and locally dense downstream tasks. Pretrained on large images, the recent DINOv3

local-aiarxiv-cs-cv

13 Apr 2026

Efficient Unlearning through Maximizing Relearning Convergence Delay

arXiv:2604.09391v1 Announce Type: cross Abstract: Machine unlearning poses challenges in removing mislabeled, contaminated, or problematic data from a pretrained model. Current unlearning approaches a

13 Apr 2026

On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment

arXiv:2604.08579v1 Announce Type: cross Abstract: We study cross-modal alignment between independently pretrained vision (DINOv2) and language (all-MiniLM-L6-v2) encoders using the functional map fram

13 Apr 2026

Applications

Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise

arXiv:2604.06468v1 Announce Type: new Abstract: Most methods for learning with noisy labels require privileged knowledge such as noise transition matrices, clean subsets or pretrained feature extracto

applicationsarxiv-cs-lg

Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models

arXiv:2604.08456v1 Announce Type: cross Abstract: Despite rapid progress, pretrained vision-language models still struggle when answers depend on tiny visual details or on combining clues spread acros

researcharxiv-cs-cl

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning

arXiv:2604.07941v1 Announce Type: new Abstract: Post-training has become central to turning pretrained large language models (LLMs) into aligned and deployable systems. Recent progress spans supervise

safetyarxiv-cs-cl

RewardFlow: Generate Images by Optimizing What You Reward

arXiv:2604.08536v1 Announce Type: new Abstract: We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward La

safetyarxiv-cs-cv

SeMoBridge: Semantic Modality Bridge for Efficient Few-Shot Adaptation of CLIP

arXiv:2509.26036v3 Announce Type: replace Abstract: While Contrastive Language-Image Pretraining (CLIP) excels at zero-shot tasks by aligning image and text embeddings, its performance in few-shot cla

safetyarxiv-cs-cv

Toward a universal foundation model for graph-structured data

model-releasesarxiv-cs-ai

arXiv:2604.06391v1 Announce Type: cross Abstract: Graphs are a central representation in biomedical research, capturing molecular interaction networks, gene regulatory circuits, cell--cell communicati

Visual prompting reimagined: The power of the Activation Prompts

model-releasesarxiv-cs-lg

arXiv:2604.06440v1 Announce Type: cross Abstract: Visual prompting (VP) has emerged as a popular method to repurpose pretrained vision models for adaptation to downstream tasks. Unlike conventional mo

AdaLoop: Thinking with the Vision Encoder

arXiv:2610.21323v1 Announce Type: new Abstract: Most vision-language models (VLMs) treat the vision encoder as a static feature extractor: it encodes an image once, independently of the question, and

Cross-Prompt Inference Mitigates Rare-Class Suppression in Imbalanced VLM Finetuning

arXiv:2610.21470v1 Announce Type: new Abstract: When a vision-language model is finetuned on class-imbalanced data, rare-class recall can fall well below the model's own zero-shot performance. In this

DK-VLM: Geometry as Position for Spatial Reasoning in Vision-Language Models

arXiv:2610.22356v1 Announce Type: new Abstract: Vision-language models still struggle to understand 3D spatial relations across views and update spatial judgments as the observer moves. A common appro

In-Context Learning Before Specialisation: Theory of Learning in a Solvable Topic Model

arXiv:2610.21901v1 Announce Type: cross Abstract: Sequence models can generalise in fundamentally distinct ways: extrapolating structures seen in training, or using their prompt to adapt to novel task

Applications

In-Context Preference-Conditioned Pareto Set Learning from Limited Offline Data

arXiv:2610.21736v1 Announce Type: new Abstract: Multi-objective optimization involves diverse preferences over conflicting objectives. Rather than returning a finite set of trade-off solutions, prefer

applicationsarxiv-cs-ai

L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

arXiv:2610.21356v1 Announce Type: new Abstract: Healthcare institutions routinely accumulate high-dimensional computed tomography (CT) volumes, but only a fraction can be annotated by experts. We pres

local-aiarxiv-cs-cv

MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data

arXiv:2610.20965v1 Announce Type: new Abstract: Large multimodal models have achieved remarkable progress by scaling model capacity and training on massive paired multimodal data. However, this paradi

Noise-Space Reward Alignment for Few-Step Generative Models

arXiv:2610.21125v1 Announce Type: new Abstract: Few-step generative models produce high-quality samples in one or a few evaluations. However, steering them toward desired properties remains difficult,

safetyarxiv-cs-lg

Quantile-Guided Diffusion for Probabilistic Time-Series Forecasting

arXiv:2610.20739v1 Announce Type: new Abstract: Quantile forecasts summarize uncertainty at each future time step, while diffusion forecasters represent it through sampled future trajectories. We stud

Where, How, and When: Query-Guided Grounding for Video Understanding

arXiv:2610.21422v1 Announce Type: cross Abstract: Understanding video content requires reasoning at multiple levels of granularity, from identifying objects to interpreting their interactions and loca

Back-Translation Still Matters: Leveraging Monolingual Data for Low-Resource Machine Translation with LLMs

arXiv:2610.18818v2 Announce Type: cross Abstract: Pre-trained Large Language Model (LLMs) are effective on many tasks in zero-shot and few-shot settings. For machine translation of low-resource langua

Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

model-releasesarxiv-cs-cl

arXiv:2311.04902v3 Announce Type: replace Abstract: Large Language Models (LLMs) with billions of parameters are prime targets for network pruning, which removes some model weights without hurting per

Can Learning to Answer Spatial Questions Improve Novel-View Generation?

arXiv:2610.20610v1 Announce Type: new Abstract: Generating novel views from limited observations requires preserving both the appearance and spatial structure of a scene. Although recent approaches ex

CktHGVAE: A Symmetric Design/State Heterogeneous Graph Variational Autoencoder for Transistor-level Analog/Mixed-signal Circuit Design and Analysis

arXiv:2610.19508v1 Announce Type: new Abstract: We propose CktHGVAE, a heterogeneous graph variational autoencoder for transistor-level analog/mixedsignal CMOS circuits that takes not only design vari

Data Density Scaling Laws for Image Self-Distillation

arXiv:2610.19013v1 Announce Type: new Abstract: Current scaling law formulations suggest that increasing the number of unique samples in a dataset improves model performance, while repeated exposure t

Tutorials

Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction

arXiv:2610.19212v1 Announce Type: new Abstract: World models of visual information predict future image frames conditioned on given action sequences. This predicted scene evolution is driven by determ

tutorialsarxiv-cs-cv

Depth-progressive 3D seismic velocity model building via 2D generative diffusion models

model-releasesarxiv-cs-ai

arXiv:2610.14812v1 Announce Type: cross Abstract: Three-dimensional (3D) velocity model building (VMB) remains computationally challenging with conventional approaches and native 3D neural networks, l