Search: “pretraining”

arXiv:2605.13581v1 Announce Type: new Abstract: Hyperspectral image (HSI) restoration is crucial for reliable analysis, as real HSIs suffer from degradations like noise, blur, and resolution loss. How

Interesting position paper on agentic AI as a foreseeable pathway to AGI. (bookmark it) There has been strong debate on whether a larger single model get us there or a multi-agent system. The authors

arXiv:2605.12725v1 Announce Type: new Abstract: Recent video anomaly detection research has expanded rapidly with an emphasis on general models of normality intended to work across many different scen

arXiv:2605.13013v1 Announce Type: new Abstract: Diffusion world models have recently become competitive for online model-based reinforcement learning, but current approaches expose a tension: pixel di

arXiv:2605.12556v1 Announce Type: new Abstract: Low-light image enhancement is challenging due to complex degradations, including amplified noise, artifacts, and color distortion. While Retinex-based

arXiv:2605.13711v1 Announce Type: new Abstract: Multimodal irregular time series (MITS) consist of asynchronous and irregularly sampled observations from heterogeneous numerical and textual channels.

arXiv:2507.10797v2 Announce Type: replace Abstract: This paper introduces the framework of multi-armed sampling, which serves as the sampling counterpart to the optimization problem of multi-armed ban

arXiv:2605.13190v1 Announce Type: cross Abstract: Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrad

arXiv:2605.12991v1 Announce Type: cross Abstract: LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagreement at rates we term yield, a vulnerability widel

arXiv:2601.21033v2 Announce Type: replace Abstract: Diffusion models cannot enforce hard constraints, yet applications in the physical sciences demand exact satisfaction of conservation laws, boundary

arXiv:2605.12855v1 Announce Type: new Abstract: Clinical trial studies indicate benefit of watch-and-wait (WW) surveillance for patients with rectal cancer showing a complete or near clinical response

arXiv:2605.12567v1 Announce Type: cross Abstract: The inherent electronic and speckle noise complicates clinical interpretation of ultrasound images. Conventional denoising methods rely on explicit no

arXiv:2411.15913v4 Announce Type: replace-cross Abstract: Music style transfer blends source structure with reference style to enable personalized music creation. However, existing zero-shot methods o

arXiv:2603.20527v3 Announce Type: replace Abstract: Preconditioned adaptive methods have gained significant attention for training deep neural networks, as they capture rich curvature information of t

arXiv:2605.13672v1 Announce Type: new Abstract: Few-shot classification (FSC) is widely used for learning from limited labeled data, yet most evaluations implicitly assume that target concepts are ind

arXiv:2605.11033v1 Announce Type: cross Abstract: TokaMind is a multi-modal transformer (MMT) foundation model pre-trained on tokamak plasma diagnostics data from MAST, where it was shown to outperfor

arXiv:2511.17001v2 Announce Type: replace Abstract: Cross-embodiment robot learning requires a unified action representation with consistent semantics across robot platforms. Existing representations

arXiv:2605.12904v1 Announce Type: new Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new tabul

arXiv:2605.11311v1 Announce Type: cross Abstract: Diffusion models typically generate image batches from independent Gaussian initial noises. We argue that this independence assumption is only one cho

arXiv:2605.11666v1 Announce Type: new Abstract: The reasoning frontier of Large Language Models (LLMs) has advanced significantly through modern post-training paradigms (e.g., Reinforcement Learning f

arXiv:2605.11898v1 Announce Type: new Abstract: Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive

arXiv:2505.11356v4 Announce Type: replace Abstract: Graph Contrastive Learning (GCL) relies on semantically consistent graph augmentations, but common local perturbations provide limited control over

arXiv:2605.12167v1 Announce Type: cross Abstract: Video generation models offer a promising imagination mechanism for robot manipulation by predicting long-horizon future observations, but effectively

arXiv:2605.11130v1 Announce Type: new Abstract: Critical events in multivariate time series, from turbine failures to cardiac arrhythmias, demand accurate prediction, yet labeled data is scarce becaus

arXiv:2605.11208v1 Announce Type: new Abstract: Automated, clinician-grade assessment reports for surgical procedures could reduce documentation burden and provide objective feedback, yet remain chall

arXiv:2605.11195v1 Announce Type: new Abstract: Large language models (LLMs) trained on web-scale corpora can memorize sensitive training data, posing significant privacy risks. Differential privacy (

Interesting paper. What I like about this is that it is a relatively low-commitment attention modification. I.e., one can use it during most of training, switch back to vanilla attention near the end,

arXiv:2603.17510v2 Announce Type: replace Abstract: Robots operating in human-shared environments must not only achieve task-level navigation objectives such as safety and efficiency, but also adapt t

arXiv:2605.12471v1 Announce Type: cross Abstract: We introduce KV-Fold, a simple, training-free long-context inference protocol that treats the key-value (KV) cache as the accumulator in a left fold o

arXiv:2605.11115v1 Announce Type: new Abstract: High Dynamic Range (HDR) generation remains challenging for generative models, which are largely limited to low dynamic range outputs. Recent diffusionb

arXiv:2503.09051v2 Announce Type: replace Abstract: We propose a novel model-level GNN explanation framework that shifts the explanation target from class-wise rule extraction to rule-based logit reco

arXiv:2605.11762v1 Announce Type: new Abstract: Learning robust navigation policies remains a core challenge in robotics. Offline imitation learning suffers from distribution shift and compounding err

arXiv:2605.12492v1 Announce Type: new Abstract: We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additi

arXiv:2605.11506v1 Announce Type: new Abstract: Score-based diffusion models achieve state-of-the-art performance for inverse problems, but their practical deployment is hindered by long inference tim

arXiv:2605.11151v1 Announce Type: cross Abstract: Offline-to-online reinforcement learning (RL) improves sample efficiency by leveraging pre-collected datasets prior to online interaction. A key chall

arXiv:2605.11502v1 Announce Type: new Abstract: Accurately and consistently indexing biomedical literature by publication type and study design is essential for supporting evidence synthesis and knowl

arXiv:2605.11840v1 Announce Type: new Abstract: Radar-camera depth estimation must turn an ultra-sparse, all-weather, metric radar signal into a dense per-pixel depth map. Existing methods -- concaten

arXiv:2605.11854v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive language models, offering stronger global awareness

arXiv:2605.11547v1 Announce Type: new Abstract: Flow matching models generate samples by numerically integrating a learned velocity field, with each integration step requiring a neural network evaluat

arXiv:2605.11689v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures have become standard in large language models, yet many of their core design choices - expert count, granularit

arXiv:2605.12466v1 Announce Type: cross Abstract: Looped Transformers offer a promising alternative to purely feed-forward computation by iteratively refining latent representations, improving languag

arXiv:2511.07767v2 Announce Type: replace Abstract: Schedule-Free SGD, proposed in [Defazio et al., 2024], achieves optimal convergence rates without requiring the training horizon in advance, by repl

arXiv:2605.11167v1 Announce Type: new Abstract: Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretra

TST has two moving parts that can be ablated independently: the input-side averaging and the output-side multi-hot cross-entropy. Both, on their own, outperform the matched-FLOPs baseline. Their effec

arXiv:2605.12088v1 Announce Type: new Abstract: Multi-reference image generation aims to synthesize images from textual instructions while faithfully preserving subject identities from multiple refere

arXiv:2605.11864v1 Announce Type: cross Abstract: Listwise reranking is a key yet computationally expensive component in vision-centric retrieval and multimodal retrieval-augmented generation (M-RAG)