Search: “pretraining”

Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation

arXiv:2607.21582v1 Announce Type: cross Abstract: Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferrin

safetyarxiv-cs-cv

24 Jul 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

SoccerSynth Field: enhancing field detection with synthetic data from virtual soccer simulator

DGX agent

arXiv:2503.13969v2 Announce Type: replace Abstract: Field detection in team sports is an essential task in sports video analysis. However, collecting large-scale and diverse real-world datasets for tr

applicationsarxiv-cs-cv

24 Jul 2026

GATE-3D: Geometry-Aware Test-time Adaptive Reranking for Open-Set 3D Shape Retrieval

arXiv:2607.19111v1 Announce Type: new Abstract: Large pretrained vision models have substantially improved appearance-based 3D shape retrieval, but they still confuse shapes that look similar while di

23 Jul 2026

The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability

arXiv:2607.20301v1 Announce Type: cross Abstract: Fine-tuning has been widely used to adapt large language models (LLMs) for domain-specific tasks. Parameter efficient fine-tuning (PEFT) methods such

23 Jul 2026

Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule

model-releasesarxiv-cs-lg

arXiv:2607.19383v1 Announce Type: cross Abstract: Pretrained generative foundation models cast forecasting as conditional generation from a learned predictive distribution and forecast unseen series z

23 Jul 2026

AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization

arXiv:2607.13805v1 Announce Type: new Abstract: Contrastive Language-Image Pretraining learns a shared representation space through large-scale contrastive learning. However, existing methods that enf

safetyarxiv-cs-cv

16 Jul 2026

Tutorials

The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model

arXiv:2607.13660v1 Announce Type: new Abstract: Contrastive Language-Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity, reflecting an intrinsic hyp

tutorialsarxiv-cs-lg

16 Jul 2026

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

arXiv:2607.12463v1 Announce Type: new Abstract: Coding agents must integrate external tool returns into ongoing reasoning - a capability that standard left-to-right pretraining on code exposes only in

safetyarxiv-cs-ai

15 Jul 2026

Rough Path Signature-Guided Geometry Augmentation for Few-Shot Industrial Surface Defect Detection

arXiv:2607.12245v1 Announce Type: new Abstract: Few-shot industrial defect detection remains difficult for standard supervised detectors, which achieve poor performance on boundary-dominated industria

15 Jul 2026

Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps

arXiv:2607.12360v1 Announce Type: new Abstract: The cooldown phase of a warmup-stable-decay (WSD) learning-rate schedule, now a default in large-model pretraining, lowers the final training loss in so

researcharxiv-cs-lg

15 Jul 2026

Are Current Continual Learning Methods Truly Agnostic? Introducing OPRE, a Step Toward Agnostic Continual Learning

arXiv:2511.08226v2 Announce Type: replace-cross Abstract: In order to achieve Continual Learning (CL), the problem of catastrophic forgetting, one that has plagued neural networks since their inceptio

Applications

DreamCharacter-1: From 3D Generative Foundation Models to Product-Ready Character Generation

arXiv:2607.07817v1 Announce Type: cross Abstract: We present DreamCharacter-1, a lightweight post-adaptation framework that calibrates pretrained 3D foundation models toward high-fidelity, production-

applicationsarxiv-cs-ai

LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity

arXiv:2607.08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language

Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

arXiv:2202.13142v3 Announce Type: replace Abstract: A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degra

Tutorials

Can We Really Learn One Representation to Optimize All Rewards?

arXiv:2602.11399v2 Announce Type: replace-cross Abstract: As unsupervised pretraining becomes increasingly ubiquitous in reinforcement learning, a more thorough theoretical understanding of these meth

tutorialsarxiv-cs-ai

9 Jul 2026

Stage-Aware Adaptation and Distribution Calibration for Subject-Driven Personalized Text-to-Image Generation

arXiv:2607.07173v1 Announce Type: new Abstract: Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images whi

9 Jul 2026

STST-JEPA: Shallow-Target Spatio-Temporal Joint Embedding Prediction Architecture For EEG Self-Supervised Learning

arXiv:2607.06629v1 Announce Type: new Abstract: Brain age -- the age inferred from a physiological recording -- is an emerging biomarker whose deviation from chronological age tracks neurological and

researcharxiv-cs-lg

9 Jul 2026

Local Ai

AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring

arXiv:2607.05859v1 Announce Type: new Abstract: Vision-Language Models (VLMs) are promising for construction-site monitoring, and recent construction-tailored VLMs have primarily adapted pretrained VL

local-aiarxiv-cs-cv

Local Ai

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning

arXiv:2409.06067v3 Announce Type: replace Abstract: Previous studies on federated learning (FL) often encounter performance degradation due to data heterogeneity among different clients. In light of t

local-aiarxiv-cs-ai

RoboTALES: Learning Reasoning-Guided Robot Policies via Task-Aligned Simulated Futures

arXiv:2607.06018v1 Announce Type: new Abstract: Pretrained video generative models are promising backbones for visuomotor control, but their imagined futures often drift from task intent and are not r

safetyarxiv-cs-ro

VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

arXiv:2510.13808v2 Announce Type: replace Abstract: Large Vision Language Models (VLMs) excel at general visual reasoning but experience significant performance degradation when deployed in novel doma

X-FEMR: A Token-level Explainable Approach for Electronic Health Records Foundation Models using Transformer-based Models

arXiv:2607.06163v1 Announce Type: cross Abstract: Foundation Models for Electronic Health Records (FEMRs) are pretrained on large-scale structured patient data, enabling them to convert longitudinal p

safetyarxiv-cs-ai

Agents

EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

arXiv:2607.05155v1 Announce Type: new Abstract: Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deploy

agentsarxiv-cs-cl

Modular Foundation Models for Time-Series Perception in Digital Twins

model-releasesarxiv-cs-lg

arXiv:2607.03585v1 Announce Type: new Abstract: Engineering Digital Twins and Prognostics and Health Management (PHM) systems rely on robust perception modules to extract actionable information from h

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

arXiv:2607.03644v1 Announce Type: cross Abstract: Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, rada

Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling

arXiv:2607.04780v1 Announce Type: cross Abstract: Sequential Monte Carlo (SMC) methods are a natural tool for post-hoc conditioning of pretrained generative models, but in many applications the mutati

safetyarxiv-cs-lg

PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesis

arXiv:2607.04880v1 Announce Type: new Abstract: Recent advances in large-scale pretrained vision-language-action models have improved robot policy learning, but directly deploying such policies in use

safetyarxiv-cs-ro

Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens

arXiv:2507.02964v2 Announce Type: replace-cross Abstract: The increasing scale of AI workloads demands High-Performance Computing (HPC) infrastructure and training methodologies that are both scalable

3 Jul 2026

Towards Robustness against Typographic Attack with Training-free Concept Localization

arXiv:2607.02494v1 Announce Type: cross Abstract: Models trained via Contrastive Language-Image Pretraining (CLIP) serve as the foundational vision encoders for most modern Large Vision Language Model

3 Jul 2026

AdaBoosting Text Prompts for Vision-Language Models

arXiv:2607.00684v1 Announce Type: new Abstract: The classification accuracy of pretrained Vision-Language Models (VLMs) relies on the quality of the text prompts. Handcrafted templates and Large Langu

researcharxiv-cs-lg

Calibrated Test-Time Guidance for Bayesian Inference

arXiv:2602.22428v2 Announce Type: replace-cross Abstract: Test-time guidance is a widely used mechanism for steering pretrained diffusion models toward outcomes specified by a reward function. Existin

researcharxiv-cs-ai

From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning

arXiv:2603.10263v2 Announce Type: replace-cross Abstract: We introduce Distribution Contractive Reinforcement Learning (DICE-RL), a framework that uses reinforcement learning (RL) as a 'distribution c

safetyarxiv-cs-lg

LV-ROVER: Multi-Stream Tesseract Voting for Maltese Paragraph OCR

arXiv:2607.00250v1 Announce Type: new Abstract: Maltese has decent text corpora and pretrained language models, but, like many languages outside the handful with large OCR benchmarks, only a single kn

Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifold

arXiv:2607.00647v1 Announce Type: new Abstract: Training-free guidance (TFG) steers a pretrained diffusion model toward a desired attribute at inference. To be effective, this guidance must be applied

FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning

arXiv:2511.17979v2 Announce Type: replace Abstract: Diffusion models have achieved remarkable success in generative modeling, yet how to effectively adapt large pretrained models to new tasks remains

Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning

arXiv:2606.30686v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) systems, built on pretrained vision-language models (VLMs), have shown rapidly improving performance on robot manipulatio

PRISM: Latent Composition Consistency for Single-Image Reflection Removal

arXiv:2606.31513v1 Announce Type: new Abstract: Single-image reflection removal (SIRR) seeks to recover the transmission layer from a mixture corrupted by reflections -- a severely ill-posed problem.

Warp RL: Reshaping Base Policy Distributions for Dynamics Adaptation

arXiv:2606.31043v1 Announce Type: new Abstract: Residual reinforcement learning adapts a pretrained robot policy by learning an additive correction to its actions. While effective when adaptation amou

safetyarxiv-cs-lg

Benchmarking Geospatial Foundation Models for Agriculture Applications

arXiv:2606.29664v1 Announce Type: new Abstract: Geospatial foundation models pretrained on satellite imagery promise broad generalization across remote sensing tasks and regions, but their geographic

Hardware

CLEAR-MoE: Shared-Basis Expert Extraction from Frozen Vision Transformers via Calibration-Driven Layer Selection

arXiv:2606.28516v1 Announce Type: new Abstract: We present CLEAR-MoE, a four-phase post-training pipeline that converts a frozen pretrained Vision Transformer (ViT) into a sparse Mixture-of-Experts (M

hardwarearxiv-cs-cv

DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks

arXiv:2606.30140v1 Announce Type: cross Abstract: Recent breakthroughs in foundation models and Large Language Models (LLMs) have introduced new opportunities for studying and decoding genomic sequenc

Flow Matching in Feature Space for Stochastic World Modeling

arXiv:2606.29059v1 Announce Type: cross Abstract: World modeling requires forecasting uncertain futures while preserving information useful for downstream perception. Existing visual world models ofte

From Failure Taxonomy to Intervention: A Diagnostic Methodology for Industry-Scale AVLM in Video and Live-Streaming Platform Moderation

model-releasesarxiv-cs-lg

arXiv:2606.30059v1 Announce Type: new Abstract: Industry-scale video and live-streaming moderation imposes requirements that are difficult to satisfy with generic pretrained public models or external

From Fog Chamber to Aircraft Window: Pixel-Registered Imaging and Synthetic Fine-Tuning Enable Cross-Domain Defogging

arXiv:2606.29093v1 Announce Type: new Abstract: A deep defogging pipeline pretrained on controlled laboratory fog and fine-tuned with domain-randomized synthetic fog applied to clear outdoor scenes ge

Improving Large-Scale Weakly Supervised ASR by Filtering and Selection

arXiv:2606.28728v1 Announce Type: cross Abstract: Leveraging large-scale weakly supervised datasets is crucial to train robust end-to-end automatic speech recognition (ASR) models. However, such datas

researcharxiv-cs-cl

KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models

arXiv:2606.30258v1 Announce Type: cross Abstract: Tabular foundation models have advanced deep learning for tabular data by delivering strong default performance across many small and medium tasks. Ye

Momentum Guidance: Plug-and-Play Guidance for Flow Models

arXiv:2602.20360v2 Announce Type: replace-cross Abstract: Flow-based generative methods offer a simple and effective framework for high-fidelity generation, yet pretrained flow models are rarely used

RePer-360: Releasing Perspective Priors for 360^irc Depth Estimation via Self-Modulation

arXiv:2603.05999v2 Announce Type: replace Abstract: Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360^irc images due to the substan

safetyarxiv-cs-cv