Search: “arxiv-cs-lg”

Understanding Generalization Requires Universal Induction

arXiv:2609.34458v1 Announce Type: cross Abstract: Classical statistical theory is insufficient to explain the successes of general-purpose AI models, because it depends on handcrafted inductive biases

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Understanding Quantization of Optimizer States in LLM Pre-training: Dynamics of State Staleness and Effectiveness of State Resets

DGX agent

arXiv:2603.16731v2 Announce Type: replace Abstract: Quantizing optimizer states is becoming an important ingredient of memory-efficient large-scale pre-training, but the resulting optimizer dynamics r

researcharxiv-cs-lg

29 Sept 2026

Understanding the Subspace Stabilization of the Hessian and Gradient Covariance Matrix

arXiv:2609.31983v1 Announce Type: new Abstract: The phenomenon of the top subspace stabilization of the Hessian matrix is an surprising and critical aspect in study of the second-order information of

Understanding the Synergy between SFT, RLVR, and OPD in LLM Post-Training

arXiv:2609.31900v1 Announce Type: new Abstract: Modern LLM post-training composes supervised fine-tuning (SFT), reinforcement learning with verifiable rewards (RLVR), and on-policy distillation (OPD)

UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models

arXiv:2609.32831v1 Announce Type: new Abstract: Unified multimodal models combine understanding, generation, and editing within a single network, offering a promising foundation for versatile multimod

Uniform Race: Parameter-Free Approximate Rejection Sampling

arXiv:2609.34639v1 Announce Type: new Abstract: We study approximate sampling: given N independent samples from a proposal distribution mu, the goal is to select one whose distribution is close to a t

Unifying Distributional Training for One-Step Visual Generation

arXiv:2609.35763v1 Announce Type: new Abstract: Distributional training provides collective supervision for one-step visual generation by matching real and generated features in frozen representation

Tutorials

UniPool: Learning Expert-to-Layer Ownership from Brief Global Access

arXiv:2605.06665v2 Announce Type: replace Abstract: Most Mixture-of-Experts (MoE) transformers preassign each expert to one layer for the whole of training. We study whether a brief global-access phas

tutorialsarxiv-cs-lg

Universal Approximation of Measure-to-Measure Operators by Pushforwards

arXiv:2609.35483v1 Announce Type: new Abstract: Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using

Universal Dynamic Portfolios

arXiv:2609.34643v1 Announce Type: new Abstract: Cover's Universal Portfolio (Cover, 1991) matches the performance of the best constant rebalanced portfolio in hindsight. We generalize this framework t

Universality and Generalization of Causal Transformers Across Context Lengths

arXiv:2609.35055v1 Announce Type: new Abstract: Long contexts are central to modern transformer systems, but most expressivity results choose a different network for each fixed sequence length. We stu

Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation

arXiv:2602.05548v4 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR), particularly GRPO, has become the standard for eliciting LLM reasoning. However, its efficien

UOPD: Uncertainty-Aware Intervention for On-Policy Distillation of Multi-Turn Agents

arXiv:2609.34036v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own rollouts using dense supervision from a teacher. In multi-turn environments, a mistake at a cri

Using Machine Learning to Investigate Predictors of Fasting Blood Glucose: Insights into Circadian Timing and Age Interactions

arXiv:2609.32386v1 Announce Type: new Abstract: Impaired glucose regulation is a major contributor to metabolic dysfunction and type 2 diabetes. This study developed an interpretable machine-learning

Vanilla Policy Optimization Is Both Optimal and Differentially Private for Stochastic Contextual Bandits

arXiv:2609.33888v1 Announce Type: new Abstract: Can vanilla policy optimization explore enough to achieve near-optimal regret in stochastic contextual bandits? We show that standard exponential policy

VC Dimension and Expressivity of Real-Valued Transformers

arXiv:2609.31999v1 Announce Type: new Abstract: Whereas previous results on abilities and limitations of transformers have restricted the definition of transformers in various ways, here we study soft

Verifying Neural Networks with Reinforcement Learning

arXiv:2609.34553v1 Announce Type: new Abstract: Formal verification can play a key role in ensuring the reliability of Deep Neural Networks (DNNs) deployed in safety-critical systems. Modern DNN verif

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

arXiv:2609.35028v1 Announce Type: new Abstract: Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosyste

Vision--Language Signals in Constrained RL: Safety Gains Without Anticipation

arXiv:2609.34041v1 Announce Type: new Abstract: Safe reinforcement learning seeks policies that maximise task performance while satisfying safety constraints. In driving benchmarks, however, collision

Agents

VLM-Guided Experience Replay

arXiv:2602.01915v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) have enabled powerful semantic and multimodal reasoning capabiliti

agentsarxiv-cs-lg

Wasserstein Convergence of ODE-Based Samplers in Decentralized Diffusion Model via Velocity Field Decomposition

arXiv:2606.15835v2 Announce Type: replace Abstract: Diffusion models have achieved impressive empirical success in generative tasks, and their convergence theory is now relatively well understood. Mot

Watch the Model Think: On-Policy Extraction of Activation Steering Vectors

arXiv:2602.14143v2 Announce Type: replace Abstract: When a model solves a problem on one attempt and fails it on the next, what separates the two is rarely the final answer token; it is the trajectory

Wavelet-Based Parity Detection Revisited: Representation Dependence, Generalization, and Mechanistic Analysis

arXiv:2511.00071v2 Announce Type: replace Abstract: Parity is exactly determined by the least significant bit (LSB), so machine learning is unnecessary for solving the task. This work instead uses par

Weighted Spline-Expanded Networks with Distributional Balancing for Continuous Treatment Effects

arXiv:2609.33740v1 Announce Type: cross Abstract: Estimating causal effects with continuous treatments in observational studies is challenging due to confounding, model misspecification, and high-dime

Tutorials

Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Data

arXiv:2609.35322v1 Announce Type: new Abstract: Score-based generative models generate new samples by integrating a time-dependent drift that carries Gaussian noise onto the target distribution. In pr

tutorialsarxiv-cs-lg

What Can a Leaderboard Certify? Compositional Controllability for Fair Evaluation and Training of Biomedical Literature-Review Agents

arXiv:2609.32318v1 Announce Type: new Abstract: Leaderboards rank long-horizon agents by their final outputs. Yet a higher score alone does not establish whether two systems are comparable or which st

What Do Latent Predictive Vehicle Representations Retain? Measuring State, Geometry, and Local Response

arXiv:2609.32512v1 Announce Type: new Abstract: Models of vehicle dynamics learned from logged states and commands complement physics-based models, and latent world models, which predict in a learned

What Does a ProcGen Generalization Gap Measure? Action Rules, Convergence, and the Missing Random Floor

arXiv:2609.32532v1 Announce Type: new Abstract: A generalization gap in reinforcement learning (return on training levels minus return on held-out levels) is usually reported without a reference point

What Does a Stream Model Buy You in Flow Matching?

arXiv:2609.34123v1 Announce Type: new Abstract: Stream-level flow matching replaces the linear interpolant of conditional flow matching (CFM) by a Gaussian-process (GP) stream connecting each source--

What does FFN compression change downstream? Same-state causal restoration in diffusion language models

arXiv:2609.31685v1 Announce Type: new Abstract: Diffusion language models (DLMs) enable flexible, parallel generation, but their iterative denoising remains computationally expensive, motivating incre

What Does Model Growth Really Add? Functional Capacity Beyond Parameter Count

arXiv:2607.14571v2 Announce Type: replace Abstract: Model growth adds parameters to a trained checkpoint and continues training, aiming to increase capacity while reusing the existing model instead of

What Does the Rank Buy? A Spectral and Distributional Analysis of Low-Rank Adaptation

arXiv:2609.32002v1 Announce Type: cross Abstract: The rank r in LoRA is widely treated as a capacity control: a smaller rank is assumed to yield a simpler model that generalizes better. We show that,

What masking geometry works best for EEG foundation models?

arXiv:2609.33487v1 Announce Type: new Abstract: EEG foundation models hold promise for scalable brain-signal decoding across clinical and cognitive neuroscience applications, yet their pre-training pi

What Must a World Model Distinguish for Planning?

arXiv:2609.33030v1 Announce Type: new Abstract: World models simulate the consequences of action candidates, but good planning need not preserve every physical distinction required for accurate predic

What Next-Event Accuracy Cannot See: Closed-Loop Evaluation of Emergency Department Trajectory Simulators

arXiv:2609.31635v1 Announce Type: new Abstract: Clinical trajectory models are usually evaluated by next-event accuracy on observed histories. Simulation is different: models must condition on their o

What Should Data Teach? Moving Bottlenecks Across Circuit, Store, and Use

arXiv:2609.32991v1 Announce Type: new Abstract: What should data teach a language model at a particular point in training? A circuit view reveals three distinct bottlenecks: forming a computation, mak

What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment

arXiv:2609.32485v1 Announce Type: new Abstract: Federated low-rank adaptation (LoRA) requires identifying an update structure that is shared across heterogeneous clients. Prior work reports strong sim

What Should the Reflector See? An Empirical Study of Evidence in Reflective Prompt Optimization

arXiv:2609.32452v1 Announce Type: cross Abstract: Reflective prompt optimization revises instructions using examples of a model's behavior, but which evidence the reflector should receive remains uncl

What Should We Freeze? Guarded Freezing: Connectivity Shapes the Fine-Tuning of Pretrained Models

arXiv:2609.32124v1 Announce Type: new Abstract: When adapting pre-trained models through fine-tuning, freezing weights alone might not preserve performance, as updates elsewhere can change the inputs

Agents

What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discovery

arXiv:2609.31760v1 Announce Type: cross Abstract: Can a robot improve itself the way coding agents now improve software? We built an agentic system to find out. It watches a robot fail, works out whic

agentsarxiv-cs-lg

What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection

arXiv:2609.33375v1 Announce Type: cross Abstract: The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly tho

What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanation

arXiv:2609.32670v1 Announce Type: cross Abstract: When a language model explains an answer it has already given, does it reuse the computation that produced the answer or reconstruct a story from the

When Better Gets Worse: Improvement Fidelity for Self-Improving Agents in Adaptive Worlds

arXiv:2609.32677v1 Announce Type: cross Abstract: Self-improving agents increasingly rely on proxy verifiers to choose policy updates, yet deployment can change the world in which those updates are ev

Tutorials

When Can Attention Heads Be Statically Defined?

arXiv:2609.34650v1 Announce Type: new Abstract: Some attention heads learn similar patterns across inputs. Reusing these patterns could reduce training cost by avoiding repeated query-key score comput

tutorialsarxiv-cs-lg

When Can Old Evaluations Certify a New Model? Label-Efficient Release Decisions under Evaluator Drift

arXiv:2609.32267v1 Announce Type: new Abstract: Releasing a model update requires certifying that its current-population risk stays below a threshold. Trusted labels are expensive, while a cheap evalu

When Can We Trust the Sparse Lens? A Certification Framework for SAE Faithfulness

arXiv:2606.18383v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are increasingly used to study internal representations in language models (LMs), but it remains unclear when an SAE-base

When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety

arXiv:2609.34771v1 Announce Type: cross Abstract: Reliable AI safeguards require both control mechanisms that reduce unsafe behavior and monitoring mechanisms that detect safety risks during model int

When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learning

arXiv:2609.33220v1 Announce Type: new Abstract: Outcome-based reinforcement learning can produce models with similar task performance but very different ways of communicating about their mistakes. We