Search: “pretraining”

arXiv:2607.02466v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models are fundamentally bottlenecked by the scarcity of expert demonstrations -- triplets of observations, instructions,

arXiv:2602.23135v2 Announce Type: replace-cross Abstract: Edge classification on directed dynamic graphs requires modeling interactions between source and destination nodes exhibiting asymmetrical beh

arXiv:2606.15129v2 Announce Type: replace-cross Abstract: Color fundus photography (CFP) is the mainstay of large-scale retinal screening, but its diagnostic capacity is limited by the lack of depth-r

arXiv:2606.19781v2 Announce Type: replace-cross Abstract: Neural scaling laws describe how model performance improves as a power law in compute, model size, and dataset size. While well-established fo

arXiv:2602.17901v3 Announce Type: replace-cross Abstract: Self-supervised learning (SSL) and diffusion models have respectively advanced representation learning and generative modeling for high-dimens

arXiv:2606.26428v1 Announce Type: cross Abstract: Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. Th

arXiv:2606.06892v1 Announce Type: new Abstract: Scalable data attribution methods typically assign isolated utility scores to individual training examples. This prevalent additive assumption fundament

arXiv:2606.05515v1 Announce Type: new Abstract: Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the na

arXiv:2512.24698v2 Announce Type: replace Abstract: Generating dynamic motions for legged robots remains a challenging problem. While reinforcement learning has achieved notable success in various leg

arXiv:2606.04876v1 Announce Type: new Abstract: Tabular foundation models, such as TabPFN, achieve strong performance on tabular datasets with numerical and categorical data, but do not natively handl

arXiv:2606.02971v1 Announce Type: new Abstract: Extracting reporting obligations from EU legislation is critical for assessing and reducing regulatory reporting burden. However, distinguishing reporti

arXiv:2506.14842v2 Announce Type: replace-cross Abstract: Building image classification models remains cumbersome in data-scarce domains, where collecting large labeled datasets is impractical. In-con

arXiv:2605.23857v1 Announce Type: cross Abstract: Knowledge distillation generally assumes a strong-to-weak relationship where stronger teachers yield better students. In this work, we examine this as

arXiv:2605.19359v1 Announce Type: new Abstract: Deep learning methods have demonstrated promising results in predicting BI-RADS scores from mammography images. However, the interpretation of these ima

arXiv:2605.06663v2 Announce Type: replace Abstract: Large language models are typically deployed as monolithic systems, requiring the full model even when applications need only a narrow subset of cap

arXiv:2605.08241v1 Announce Type: cross Abstract: Self-supervised learning (SSL) has transformed representation learning for large models, yet remains unexplored for microcontroller (MCU)-class models

arXiv:2510.18516v3 Announce Type: replace-cross Abstract: Neural recordings exhibit a distinctive form of heterogeneity rooted in differences in cell types, intrinsic circuit dynamics, and stochastic

arXiv:2512.17817v3 Announce Type: replace Abstract: While 3DGS has emerged as a high-fidelity scene representation, encoding rich, general-purpose features directly from its primitives remains under-e

arXiv:2605.03722v1 Announce Type: new Abstract: We propose Evolutionary Dynamic Loss (EDL), a framework that learns a transferable classification loss in the probability space using unlimited syntheti

arXiv:2605.00933v1 Announce Type: new Abstract: Continuous Glucose Monitoring (CGM) can detect early metabolic subphenotypes (insulin resistance, IR; eta-cell dysfunction), but population-scale deploy

arXiv:2604.27715v1 Announce Type: new Abstract: Test-time prompt tuning (TPT) has emerged as a promising technique for enhancing the adaptability of vision-language models by optimizing textual prompt

arXiv:2604.24044v1 Announce Type: new Abstract: Accurate 3D object detection is critical for autonomous driving, necessitating reliable, cost-effective sensors capable of operating in adverse weather

arXiv:2604.22854v1 Announce Type: cross Abstract: Transformer architectures, including nnFormer,have demonstrated promising results in volumetric medical image segmentation by being able to capture lo

arXiv:2604.17633v1 Announce Type: new Abstract: Large language models exhibit impressive cross-lingual capabilities. However, prior work analyzes this phenomenon through isolated factors and at sparse

arXiv:2604.16391v1 Announce Type: cross Abstract: Vision-language-action (VLA) models have shown great potential in building generalist robots, but still face a dilemma-misalignment of 2D image foreca

arXiv:2604.16422v1 Announce Type: new Abstract: The injection of domain-specific knowledge is crucial for adapting language models (LMs) to specialized fields such as biomedicine. While most current a

arXiv:2604.10609v1 Announce Type: new Abstract: Instance segmentation enables the analysis of spatial and temporal properties of cells in microscopy images by identifying the pixels belonging to each

arXiv:2610.09372v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) layers replace the feed-forward block of a Transformer with E expert networks, and each token is routed to k of these experts.

arXiv:2610.07652v1 Announce Type: cross Abstract: The ability to interact with articulated objects is essential for embodied intelligent systems, but collecting large-scale real-world demonstrations f

arXiv:2610.04867v1 Announce Type: new Abstract: Greedy block-wise local learning splits a network into gradient-isolated blocks trained by local auxiliary losses, deleting the backward pass between bl

arXiv:2610.03265v1 Announce Type: cross Abstract: Large-scale pre-training has improved the generalization of neural operators across diverse PDEs. However, existing PDE foundation models still strugg

arXiv:2610.01165v1 Announce Type: new Abstract: Layer interventions are widely used to probe the internal organization of language models, yet most analyses examine a single training checkpoint even t

arXiv:2609.33419v1 Announce Type: new Abstract: Comparisons in video self-supervised learning often evaluate complete training recipes rather than isolating the method itself: architecture, objective,

arXiv:2609.31461v1 Announce Type: new Abstract: Background: Large volumes of unlabeled knee MRI scans are available across repositories but remain insufficiently leveraged. We developed KneePreM, a kn

arXiv:2609.26034v1 Announce Type: new Abstract: Water treatment research is expanding rapidly, but much of the knowledge acquired from this research remains scattered across unstructured literature. T

arXiv:2609.02881v1 Announce Type: new Abstract: We introduce the Graph Machine (GM), an architecture that maintains an O(n)-sized state and accesses it through sparse, dynamic routing. Unlike methods

arXiv:2608.30392v1 Announce Type: new Abstract: Global food security and sustainable climate action increasingly rely on robust, scalable agricultural monitoring. Earth observation foundation models h

arXiv:2603.02041v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are predominantly trained on English-centric data, resulting in uneven performance for smaller languages. We stud

arXiv:2608.17521v1 Announce Type: new Abstract: We introduce BrainNorm, a normative foundation model, trained and tested on ~66,000 T1-weighted structural MRI (T1w sMRI) scans. By leveraging language-

arXiv:2608.12419v1 Announce Type: new Abstract: Large language models (LLMs) have achieved remarkable breakthroughs across various applications. However, their architectures remain inefficient in pret

arXiv:2608.13277v1 Announce Type: cross Abstract: We ask whether language-model pre-training can be decomposed into smaller, independently trainable jobs that can later be recomposed into a coherent l

arXiv:2508.05149v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-resource languages, reaching state-of-the-art perf

arXiv:2608.02252v1 Announce Type: new Abstract: Recent vision-language models for chest X-ray understanding are largely built on image-report alignment and therefore rely heavily on MIMIC-CXR as the d

arXiv:2504.19683v4 Announce Type: replace Abstract: Fine-grained robotic manipulation often fails when inaccurate initial grasps propagate errors and necessitate complex pose correction. We propose Gr

arXiv:2607.20486v1 Announce Type: new Abstract: Designing optimizers for modern deep learning remains a challenging scientific problem, requiring the joint consideration of optimization geometry, stat

arXiv:2607.20548v1 Announce Type: cross Abstract: Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges hav