Search: “pretraining”

Measuring and Decomposing Mode Separation via the Canonical Diffusion

arXiv:2605.08777v1 Announce Type: cross Abstract: Mode separation, namely how sharply a distribution fragments into barrier-separated clusters, is a fundamental geometric property of densities, diffic

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Mela: Test-Time Memory Consolidation based on Transformation Hypothesis

DGX agent

arXiv:2605.10537v1 Announce Type: new Abstract: Memory consolidation, the process by which transient experiences are transformed into stable, structured representations, is a foundational organizing p

researcharxiv-cs-cl

12 May 2026

ML-CLIPSim: Multi-Layer CLIP Similarity for Machine-Oriented Image Quality

arXiv:2605.09479v1 Announce Type: cross Abstract: We study full-reference image quality assessment from a machine-centric perspective, where images are evaluated by how well they preserve information

MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration

arXiv:2603.28254v2 Announce Type: replace Abstract: Orthogonalized-update optimizers such as Muon improve training of matrix-valued parameters, but existing extensions typically either rescale updates

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers

arXiv:2605.09176v1 Announce Type: cross Abstract: Training large language models requires optimization algorithms that are not only statistically effective, but also computationally and memory efficie

Optimizer-Induced Mode Connectivity: From AdamW to Muon

arXiv:2605.09991v1 Announce Type: new Abstract: Mode connectivity has been widely studied, yet the role of the optimizer remains underexplored. We revisit it through optimizer-induced implicit regular

OracleTSC: Oracle-Informed Reward Hurdle and Uncertainty Regularization for Traffic Signal Control

arXiv:2605.08516v1 Announce Type: new Abstract: Transparent decision-making is essential for traffic signal control (TSC) systems to earn public trust. However, traditional reinforcement learning-base

Hardware

PermuQuant: Lowering Per-Group Quantization Error by Reordering Channels for Diffusion Models

arXiv:2605.09503v1 Announce Type: new Abstract: Large-scale visual generative models have achieved remarkable performance. However, their high computational and memory costs make deployment challengin

hardwarearxiv-cs-cv

Phoenix-VL 1.5 Medium Technical Report

arXiv:2605.10391v1 Announce Type: cross Abstract: We introduce Phoenix-VL 1.5 Medium, a 123B-parameter natively multimodal and multilingual foundation model, adapted to regional languages and the Sing

ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulation

arXiv:2605.08774v1 Announce Type: cross Abstract: Long-horizon robotic manipulation requires dense feedback that reflects how a task advances through its procedural stages, not merely whether the fina

Radar-Guided Polynomial Fitting for Metric Depth Estimation

arXiv:2503.17182v4 Announce Type: replace Abstract: We propose POLAR, a novel radar-guided depth estimation method that introduces polynomial fitting to efficiently transform scaleless depth predictio

Reducing Annotation Burden for Femoral Cartilage Segmentation in Knee MRI via Cross-Sequence Transfer Learning

arXiv:2605.09067v1 Announce Type: new Abstract: Purpose: To develop and evaluate cross-sequence transfer learning for automatic femoral cartilage segmentation, testing bidirectional transfer between d

Reinforcement learning for inverse structural design and rapid laser cutting of kirigami prototypes

arXiv:2605.08098v1 Announce Type: new Abstract: Kirigami is an increasingly useful fabrication method to produce shape-programmable metamaterial structures. However, inverse design remains difficult b

RelBench v2: A Large-Scale Benchmark and Repository for Relational Data

arXiv:2602.12606v2 Announce Type: replace Abstract: Relational deep learning (RDL) has emerged as a powerful paradigm for learning directly on relational databases by modeling entities and their relat

Restoration-Aligned Generative Flow Models for Blind Motion Deblurring

arXiv:2605.08854v1 Announce Type: new Abstract: Generative flow models offer powerful priors learned from large-scale natural images, but directly adapting them to restoration tasks such as motion deb

Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models

arXiv:2602.11824v2 Announce Type: replace Abstract: Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visu

Applications

SABER: A Scalable Action-Based Embodied Dataset for Real-World VLA Adaptation

arXiv:2605.09613v1 Announce Type: cross Abstract: Robotic deployment in real-world environments depends on rich, domain-specific action data as much as on strong model architecture. General-purpose ro

applicationsarxiv-cs-cv

SDG-MoE: Signed Debate Graph Mixture-of-Experts

arXiv:2605.08322v1 Announce Type: cross Abstract: Sparse MoE models achieve a good balance between capacity and compute by routing each token to a small subset of experts. However, in most MoE archite

Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention

arXiv:2605.08453v1 Announce Type: cross Abstract: This paper studies the role of sinks and diagonal patterns as attention switch and anti-oversmoothing mechanisms. We analyze geometric conditions unde

SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMs

arXiv:2605.09449v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) have made remarkable progress in visual understanding and language-based reasoning, yet they lack a pers

Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse

arXiv:2601.11042v2 Announce Type: replace-cross Abstract: Sequential knowledge editing in large language models often causes catastrophic collapse of the model's general abilities, especially for para

SpectraLLM: Uncovering the Ability of LLMs for Molecular Structure Elucidation from Multi-Spectral Data

arXiv:2508.08441v3 Announce Type: replace-cross Abstract: Automated molecular structure elucidation remains challenging, as existing approaches often depend on pre-compiled databases or restrict thems

Structural Alignment Improves Graph Test-Time Adaptation

arXiv:2502.18334v5 Announce Type: replace Abstract: Graph-based learning excels at capturing interaction patterns in diverse domains like recommendation, fraud detection, and particle physics. However

The Echo Amplifies the Knowledge: Somatic Marker Analogues in Language Models via Emotion Vector Re-Injection

arXiv:2605.08611v1 Announce Type: new Abstract: Current language model memory systems store what happened but not how it felt. This distinction -- between semantic memory (knowing about a past event)

Time-Warping Recurrent Neural Networks for Transfer Learning

arXiv:2604.02474v2 Announce Type: replace Abstract: Dynamical systems describe how a physical system evolves over time. Physical processes can evolve faster or slower in different environmental condit

Tutorials

Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm

arXiv:2605.10640v1 Announce Type: cross Abstract: Continual Pre-Training (CPT) is essential for enabling Language Models (LMs) to integrate new knowledge without erasing old. While classical CPT techn

tutorialsarxiv-cs-ai

TrajTok: Learning Trajectory Tokens enables better Video Understanding

arXiv:2602.22779v2 Announce Type: replace Abstract: Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens. This severely limits video ef

safetyarxiv-cs-cv

Transfer Learning for Dead Fuel Moisture Prediction Using Time-Warping Recurrent Neural Networks

arXiv:2605.08379v1 Announce Type: cross Abstract: This paper proposes a time-warping transfer learning method, a technique for temporally rescaling the learned dynamics of a recurrent neural network (

Unified Noise Steering for Efficient Human-Guided VLA Adaptation

arXiv:2605.10821v1 Announce Type: new Abstract: Diffusion-based vision-language-action (VLA) models have emerged as strong priors for robotic manipulation, yet adapting them to real-world distribution

safetyarxiv-cs-ro

Agents

VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving

arXiv:2605.08830v1 Announce Type: cross Abstract: End-to-end autonomous driving requires models to understand traffic scenes, infer driving intent, and generate executable motion plans. Recent vision-

agentsarxiv-cs-ai

VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2605.10485v1 Announce Type: new Abstract: Precise spatial reasoning is fundamental to robotic manipulation, yet the visual backbones of current vision-language-action (VLA) models are predominan

Why is prompting hard? Understanding prompts on binary sequence predictors

arXiv:2502.10760v2 Announce Type: replace Abstract: Frontier models can be prompted or conditioned to do many tasks, but finding good prompts is not always easy, nor is understanding some performant p

researcharxiv-cs-cl

WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records

arXiv:2605.09765v1 Announce Type: cross Abstract: Representation learning in electronic health records (EHR) has largely followed paradigms inherited from natural language processing, relying on seque

Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation

arXiv:2605.09981v1 Announce Type: cross Abstract: Multimodal models that jointly reason over protein sequences, structures, and function annotations within a unified representation hold immense potent

YEZE at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization via Heterogeneous Ensembling

arXiv:2605.06231v2 Announce Type: replace Abstract: This paper presents our system for SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization, which identifies p

researcharxiv-cs-cl

Anisotropic Modality Align

arXiv:2605.07825v1 Announce Type: cross Abstract: Training multimodal large language models has long been limited by the scarcity of high-quality paired multimodal data. Recent studies show that the s

safetyarxiv-cs-cv

Arrow: A Foundation Model for Causal Discovery

arXiv:2605.07204v1 Announce Type: new Abstract: We introduce Arrow, a foundation model for zero-shot causal discovery on observational tabular data. Arrow factorizes a directed acyclic graph into an u

Conservative Flows: A New Paradigm of Generative Models

arXiv:2605.06905v1 Announce Type: new Abstract: Modern generative modeling is dominated by transport from a noise prior to data. We propose an alternative paradigm in which generation is performed by

Consistency Regularised Gradient Flows for Inverse Problems

arXiv:2605.07907v1 Announce Type: cross Abstract: Vision-Language Latent Diffusion Models (LDMs) (Rombach et al., 2022) provide powerful generative priors for inverse problems. However, existing LDM-b

Convergence and Emergence of In-Context Reinforcement Learning with Chain of Thought

arXiv:2605.07123v1 Announce Type: new Abstract: In-context reinforcement learning (ICRL) refers to the ability of RL agents to adapt to new tasks at inference time without parameter updates by conditi

Coupling Models for One-Step Discrete Generation

arXiv:2605.07193v1 Announce Type: new Abstract: Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large l

Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentation

model-releasesarxiv-cs-cv

arXiv:2508.20909v2 Announce Type: replace Abstract: Foundation models pre-trained on large-scale natural image datasets offer a powerful paradigm for medical image segmentation. However, effectively t

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

arXiv:2605.07483v1 Announce Type: cross Abstract: Successful deep neural networks discover salient features of data. We show when and why they fail to learn out-of-distribution (OOD)-relevant represen

Agents

Dynamic Mode Decomposition along Depth in Vision Transformers

arXiv:2605.07556v1 Announce Type: new Abstract: Recent work has shown that contiguous vision transformer (ViT) blocks (a) can be replaced by a linear map and (b) organize into recurrent phases of comp

agentsarxiv-cs-cv

EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing

arXiv:2605.07455v1 Announce Type: new Abstract: Visual-prompt-guided edit transfer aims to learn image transformations directly from example pairs, offering more precise and controllable editing than

safetyarxiv-cs-cv

Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer

arXiv:2605.07407v1 Announce Type: new Abstract: Health foundation models (FMs) learn useful representations from wearable sensors, but interpreting what they encode and transferring that knowledge acr

Applications

Flexible Routing via Uncertainty Decomposition

arXiv:2605.07805v1 Announce Type: new Abstract: A key strategy for balancing performance and cost in modern machine learning systems is to dynamically route queries to either a low-cost model or a mor

applicationsarxiv-cs-lg

Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs

arXiv:2605.07961v1 Announce Type: new Abstract: Federated fine-tuning (FFT) has emerged as a privacy-preserving paradigm for collaboratively adapting large language models (LLMs). Built upon federated