Search: “pretraining”

Membership Inference for Contrastive Pre-training Models with Text-only PII Queries

arXiv:2603.14222v2 Announce Type: replace-cross Abstract: Contrastive pretraining models such as CLIP and CLAP, serve as the ubiquitous perceptual backbones for modern multimodal large models, yet the

safetyarxiv-cs-ai

23 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

DGX agent

arXiv:2602.15353v2 Announce Type: replace-cross Abstract: Large pretrained language models and neural reasoning systems have advanced many natural language tasks, yet they remain challenged by knowled

safetyarxiv-cs-ai

23 Apr 2026

Chroma1, V41, V48, Radiance, delivering a look similar to Midjourney.

local-air-stablediffusion

Chroma1 is an 8.9B parameter text-to-image foundational model , with different versions including V48 (512x raw pretrained checkpoint) and variants like Chroma1-HD (mixed resolution finetune) . Chroma

22 Apr 2026

ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning

model-releasesarxiv-cs-ai

arXiv:2604.19254v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) reduces the training cost of full-parameter fine-tuning for large language models (LLMs) by training only a sma

22 Apr 2026

StochasTok: Improving Fine-Grained Subword Understanding in LLMs

arXiv:2506.01687v3 Announce Type: replace Abstract: Subword-level understanding is integral to numerous tasks, including understanding multi-digit numbers, spelling mistakes, abbreviations, rhyming, a

researcharxiv-cs-cl

22 Apr 2026

Decomposing the Depth Profile of Fine-Tuning

arXiv:2604.17177v1 Announce Type: new Abstract: Fine-tuning adapts pretrained networks to new objectives. Whether the resulting depth profile of representational change reflects an intrinsic property

researcharxiv-cs-lg

Domain-oriented RAG Assessment (DoRA): Synthetic Benchmarking for RAG-based Question Answering on Defense Documents

arXiv:2604.17943v1 Announce Type: new Abstract: Open-domain RAG benchmarks over public corpora can overestimate deployment performance due to pretraining overlap and weak attribution requirements. We

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

arXiv:2604.18109v1 Announce Type: new Abstract: This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover t

Generative Semantic Communication via Alternating Dual-Domain Posterior Sampling

arXiv:2604.16796v1 Announce Type: new Abstract: Generative semantic communication (SemCom) harnesses pretrained generative priors to improve the perceptual quality of wireless image transmission. Exis

safetyarxiv-cs-cv

Lizard: An Efficient Linearization Framework for Large Language Models

arXiv:2507.09025v4 Announce Type: replace Abstract: We propose Lizard, a linearization framework that transforms pretrained Transformer-based Large Language Models (LLMs) into subquadratic architectur

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

arXiv:2604.18360v1 Announce Type: cross Abstract: Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, the

applicationsarxiv-cs-cl

Agents

OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

arXiv:2604.17915v1 Announce Type: new Abstract: Vision-Language Models(VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured out

agentsarxiv-cs-cv

REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning

arXiv:2604.17257v1 Announce Type: new Abstract: Recent text embedding models are often adapted to specialized domains via contrastive pre-finetuning (PFT) on a naive collection of scattered, heterogen

safetyarxiv-cs-cl

Hardware

Scalable and Adaptive Parallel Training of Graph Transformer on Large Graphs

arXiv:2604.16715v1 Announce Type: cross Abstract: Graph foundation models have demonstrated remarkable adaptability across diverse downstream tasks through large-scale pretraining on graphs. However,

hardwarearxiv-cs-lg

SHRUG-FM: Reliability-Aware Foundation Models for Earth Observation

model-releasesarxiv-cs-cv

arXiv:2511.10370v2 Announce Type: replace Abstract: Geospatial foundation models (GFMs) for Earth observation often fail to perform reliably in environments underrepresented during pretraining. We int

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton

arXiv:2510.09378v2 Announce Type: replace Abstract: Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This rais

researcharxiv-cs-lg

HyCal: A Training-Free Prototype Calibration Method for Cross-Discipline Few-Shot Class-Incremental Learning

model-releasesarxiv-cs-cv

arXiv:2604.15678v1 Announce Type: new Abstract: Pretrained Vision-Language Models (VLMs) like CLIP show promise in continual learning, but existing Few-Shot Class-Incremental Learning (FSCIL) methods

20 Apr 2026

Tutorials

On Optimal Hyperparameters for Differentially Private Deep Transfer Learning

arXiv:2510.20616v2 Announce Type: replace Abstract: Differentially private (DP) transfer learning, i.e., fine-tuning a pretrained model on private data, is the current state-of-the-art approach for tr

tutorialsarxiv-cs-lg

20 Apr 2026

TriagerX: Dual Transformers for Bug Triaging Tasks with Content and Interaction Based Rankings

arXiv:2508.16860v2 Announce Type: replace-cross Abstract: Pretrained Language Models or PLMs are transformer-based architectures that can be used in bug triaging tasks. PLMs can better capture token s

applicationsarxiv-cs-ai

20 Apr 2026

SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization

model-releasesarxiv-cs-lg

arXiv:2604.07663v2 Announce Type: replace Abstract: The AdamW optimizer, while standard for LLM pretraining, is a critical memory bottleneck, consuming optimizer states equivalent to twice the model's

17 Apr 2026

Enhancing Mixture-of-Experts Specialization via Cluster-Aware Upcycling

arXiv:2604.13508v1 Announce Type: new Abstract: Sparse Upcycling provides an efficient way to initialize a Mixture-of-Experts (MoE) model from pretrained dense weights instead of training from scratch

Frozen Forecasting: A Unified Evaluation

arXiv:2507.13942v2 Announce Type: replace Abstract: Forecasting future events is a fundamental capability for general-purpose systems that plan or act across different levels of abstraction. Yet, eval

(How) Learning Rates Regulate Catastrophic Overtraining

arXiv:2604.13627v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a hel

researcharxiv-cs-cl

RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real World

arXiv:2604.05096v2 Announce Type: replace Abstract: Large language models (LLMs) acquire most of their knowledge during pretraining, which ties them to a fixed snapshot of the world and makes adaptati

IMU: Influence-guided Machine Unlearning

arXiv:2508.01620v3 Announce Type: replace-cross Abstract: Machine Unlearning (MU) aims to selectively erase the influence of specific data points from pretrained models. However, most existing MU meth

Models Know Their Shortcuts: Deployment-Time Shortcut Mitigation

arXiv:2604.12277v1 Announce Type: new Abstract: Pretrained language models often rely on superficial features that appear predictive during training yet fail to generalize at test time, a phenomenon k

safetyarxiv-cs-lg

RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair

arXiv:2604.12820v1 Announce Type: new Abstract: Large language models (LLMs) inherently absorb harmful knowledge, misinformation, and personal data during pretraining on large-scale web corpora, with

local-aiarxiv-cs-ai

Robotic Manipulation is Vision-to-Geometry Mapping (f(v) rightarrow G): Vision-Geometry Backbones over Language and Video Models

arXiv:2604.12908v1 Announce Type: new Abstract: At its core, robotic manipulation is a problem of vision-to-geometry mapping (f(v) rightarrow G). Physical actions are fundamentally defined by geometri

applicationsarxiv-cs-ro

Tutorials

SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learning

arXiv:2602.08590v4 Announce Type: replace Abstract: Vision-language pretrained models offer strong transferable representations, yet adapting them in privacy-sensitive multi-party settings is challeng

tutorialsarxiv-cs-lg

Tutorials

Siamese Foundation Models for Crystal Structure Prediction

arXiv:2503.10471v2 Announce Type: replace-cross Abstract: Predicting crystal structures from chemical compositions is a fundamental challenge in materials discovery, complicated by complex 3D geometri

tutorialsarxiv-cs-ai

Task Alignment: A simple and effective proxy for model merging in computer vision

arXiv:2604.12935v1 Announce Type: new Abstract: Efficiently merging several models fine-tuned for different tasks, but stemming from the same pretrained base model, is of great practical interest. Des

safetyarxiv-cs-cv

Agents

BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving

arXiv:2604.10856v1 Announce Type: cross Abstract: Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in c

agentsarxiv-cs-ai

DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation

arXiv:2604.10882v1 Announce Type: cross Abstract: Graph Neural Network pretraining is pivotal for leveraging unlabeled graph data. However, generalizing across heterogeneous domains remains a major ch

researcharxiv-cs-ai

Agents

EDFNet: Early Fusion of Edge and Depth for Thin-Obstacle Segmentation in UAV Navigation

arXiv:2604.09694v1 Announce Type: new Abstract: Autonomous Unmanned Aerial Vehicles (UAVs) must reliably detect thin obstacles such as wires, poles, and branches to navigate safely in real-world envir

agentsarxiv-cs-cv

FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer

arXiv:2604.10023v1 Announce Type: cross Abstract: With the growing availability of open-sourced adapters trained on the same diffusion backbone for diverse scenes and objects, combining these pretrain

safetyarxiv-cs-ai

Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

arXiv:2604.10103v1 Announce Type: new Abstract: Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window att

local-aiarxiv-cs-cv

Masked Contrastive Pre-Training Improves Music Audio Key Detection

arXiv:2604.10021v1 Announce Type: cross Abstract: Self-supervised music foundation models underperform on key detection, which requires pitch-sensitive representations. In this work, we present the fi

researcharxiv-cs-lg

Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers

arXiv:2604.11508v1 Announce Type: cross Abstract: Fine-tuning pretrained image classifiers is standard practice, yet which individual samples are forgotten during this process, and whether forgetting

researcharxiv-cs-ai

Training-Free Cross-Lingual Dysarthria Severity Assessment via Phonological Subspace Analysis in Self-Supervised Speech Representations

arXiv:2604.10123v1 Announce Type: new Abstract: Dysarthric speech severity assessment typically requires trained clinicians or supervised models built from labelled pathological speech, limiting scala

safetyarxiv-cs-cl

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting

model-releasesarxiv-cs-ai

arXiv:2604.10544v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have recently achieved remarkable success in universal forecasting by leveraging large-scale pretraining on dive

Another BRIXEL in the Wall: Towards Cheaper Dense Features

arXiv:2511.05168v2 Announce Type: replace Abstract: Vision foundation models achieve strong performance on both global and locally dense downstream tasks. Pretrained on large images, the recent DINOv3

local-aiarxiv-cs-cv

13 Apr 2026

Efficient Unlearning through Maximizing Relearning Convergence Delay

arXiv:2604.09391v1 Announce Type: cross Abstract: Machine unlearning poses challenges in removing mislabeled, contaminated, or problematic data from a pretrained model. Current unlearning approaches a

13 Apr 2026

On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment

arXiv:2604.08579v1 Announce Type: cross Abstract: We study cross-modal alignment between independently pretrained vision (DINOv2) and language (all-MiniLM-L6-v2) encoders using the functional map fram

safetyarxiv-cs-ai

13 Apr 2026

Concepts

All Tools

Auto-generated index of all tools mentioned across the wiki.

conceptstoolsindex

11 Apr 2026

Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise

arXiv:2604.06468v1 Announce Type: new Abstract: Most methods for learning with noisy labels require privileged knowledge such as noise transition matrices, clean subsets or pretrained feature extracto

applicationsarxiv-cs-lg

Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models

arXiv:2604.08456v1 Announce Type: cross Abstract: Despite rapid progress, pretrained vision-language models still struggle when answers depend on tiny visual details or on combining clues spread acros

researcharxiv-cs-cl

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning

arXiv:2604.07941v1 Announce Type: new Abstract: Post-training has become central to turning pretrained large language models (LLMs) into aligned and deployable systems. Recent progress spans supervise

safetyarxiv-cs-cl

RewardFlow: Generate Images by Optimizing What You Reward

arXiv:2604.08536v1 Announce Type: new Abstract: We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward La

safetyarxiv-cs-cv