Search: “arxiv-cs-lg”

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers

arXiv:2603.17771v2 Announce Type: replace Abstract: Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing explanations have largely focuse

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Automated Formal Proofs of Combinatorial Identities via Wilf-Zeilberger Guidance and LLMs

DGX agent

arXiv:2605.04472v1 Announce Type: new Abstract: Automating formal proofs of combinatorial identities is challenging for LLM-based provers, as long-horizon proof planning is required and unconstrained

model-releasesarxiv-cs-lg

7 May 2026

Average Attention Transformers and Arithmetic Circuits

arXiv:2605.04683v1 Announce Type: cross Abstract: We analyse the computational power of transformer encoders as sequence-to-sequence functions on vectors. We show that average hard attention can be us

AxMoE: Characterizing the Impact of Approximate Multipliers on Mixture-of-Experts DNN Architectures

arXiv:2605.04754v1 Announce Type: new Abstract: Deep neural network (DNN) inference at the edge demands simultaneous improvements in accuracy, computational efficiency, and energy consumption. Approxi

Back to Blackwell: Closing the Loop on Intransitivity in Multi-Objective Preference Fine-Tuning

arXiv:2602.19041v2 Announce Type: replace Abstract: A recurring challenge in preference fine-tuning (PFT) is handling extit{intransitive} (i.e., cyclic) preferences. Intransitive preferences often ste

Bayesian Parameter Shift Rule in Variational Quantum Eigensolvers

arXiv:2502.02625v2 Announce Type: replace Abstract: Parameter shift rules (PSRs) are key techniques for efficient gradient estimation in variational quantum eigensolvers (VQEs). In this paper, we prop

Benchmarking LLMs on the Massive Sound Embedding Benchmark (MSEB)

arXiv:2605.04556v1 Announce Type: cross Abstract: The Massive Sound Embedding Benchmark (MSEB) has emerged as a standard for evaluating the functional breadth of audio models. While initial baselines

Beyond Rigid Geometries: The Spline-Pullback Metric for Universal Diffeomorphic SPD Representation Learning

arXiv:2605.04406v1 Announce Type: new Abstract: The integration of Symmetric Positive Definite (SPD) matrices into deep learning has historically relied on fixed algebraic Riemannian metrics. Analogou

Beyond Variance: Prompt-Efficient RLVR via Rare-Event Amplification and Bidirectional Pairing

arXiv:2602.03452v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) is effective for training large language models on deterministic outcome reasoning tasks. Prio

Bilinear Mamba-Koopman Neural MPC for Varying Dynamics

arXiv:2605.04793v1 Announce Type: new Abstract: Koopman-based neural MPC models generate time-varying dynamics from historical data, but preserve convexity by enforcing that the system operator is ind

BOOOM: Loss-Function-Agnostic Black-Box Optimization over Orthonormal Manifolds for Machine Learning and Statistical Inference

arXiv:2605.04087v1 Announce Type: cross Abstract: Optimization over the Stiefel manifold St(p,d), the set of p imes d column-orthonormal matrices, is fundamental in statistics, machine learning, and s

Applications

Breaking the Quality-Privacy Tradeoff in Tabular Data Generation via In-Context Learning

arXiv:2605.04911v1 Announce Type: new Abstract: Tabular data synthesis aims to generate high-quality data while preserving privacy. However, we find that existing tabular generative models exhibit a c

applicationsarxiv-cs-lg

Bridging Input Feature Spaces Towards Graph Foundation Models

arXiv:2605.04834v1 Announce Type: new Abstract: Unlike vision and language domains, graph learning lacks a shared input space, as input features differ across graph datasets not only in semantics, but

Hardware

Budget-aware Auto Optimizer Configurator

arXiv:2605.04711v1 Announce Type: cross Abstract: Optimizer states occupy massive GPU memory in large-scale model training. However, gradients in different network blocks exhibit distinct behaviors, s

hardwarearxiv-cs-lg

Building informative materials datasets beyond targeted objectives

arXiv:2605.05104v1 Announce Type: cross Abstract: Materials science data collection can be expensive, making the reuse and long-term utility of datasets critical important for future discovery campaig

Calibrating Tabular Anomaly Detection via Optimal Transport

arXiv:2602.06810v2 Announce Type: replace Abstract: Tabular anomaly detection (TAD) remains challenging due to the heterogeneity of tabular data: features lack natural relationships, vary widely in di

Capabilities of Auto-encoders and Principal Component Analysis of the Reduction of Microstructural Images; Application on the Acceleration of Phase-Field Simulations

arXiv:2605.04229v1 Announce Type: new Abstract: In this work, a data-driven framework based on Phase-Field simulations data is proposed to highlight the capabilities of neural networks to ensure accur

Capacity-Aware Mixture Law Enables Efficient LLM Data Optimization

arXiv:2603.08022v2 Announce Type: replace Abstract: A data mixture refers to how different data sources are combined to train large language models, and selecting an effective mixture is crucial for o

Causal discovery under mean independence and linearity

arXiv:2605.04381v1 Announce Type: cross Abstract: Causal discovery methods such as LiNGAM identify causal structure from observational data by assuming mutually independent disturbances. This assumpti

Centrality-Based Pruning for Efficient Echo State Networks

arXiv:2603.20684v2 Announce Type: replace Abstract: Echo State Networks (ESNs) are a reservoir computing framework widely used for nonlinear time-series prediction. However, despite their effectivenes

Applications

Climate-based Pre-screening of Self-sustaining Regreening Opportunities in Drylands: A Case Study for Saudi Arabia

arXiv:2605.04206v1 Announce Type: new Abstract: Large-scale restoration in drylands is widely promoted to address land degradation and biodiversity loss, yet many efforts rely on long-term irrigation,

applicationsarxiv-cs-lg

Cognitive Twins: Investigating Personalized Thinking Model Building and Its Performance Enhancement with Human-in-the-Loop

arXiv:2605.04761v1 Announce Type: new Abstract: This paper presents the Personalized Thinking Model (PTM), a hierarchical and interpretable learner representation designed for AI supported education.

Combining Abstract Argumentation and Machine Learning for Efficiently Analyzing Low-Level Process Event Streams

arXiv:2505.05880v2 Announce Type: replace-cross Abstract: Monitoring and analyzing process traces is a critical task for modern companies and organizations. In scenarios where there is a gap between t

Local Ai

Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models

arXiv:2605.04830v1 Announce Type: new Abstract: Diffusion models undergo a phase transition in a critical time window during generation dynamics, with two complementary diagnoses of criticality. The s

local-aiarxiv-cs-lg

Conditional Flow-VAE for Safety-Critical Traffic Scenario Generation

arXiv:2605.04366v1 Announce Type: cross Abstract: Safety-critical scenarios are essential for the development of autonomous vehicles (AVs) but are rare in real-world driving data. While simulation off

Conditional outlier detection for clinical alerting

arXiv:2605.05124v1 Announce Type: new Abstract: We develop and evaluate a data-driven approach for detecting unusual (anomalous) patient-management actions using past patient cases stored in an electr

Confronting Label Indeterminacy in Automated Bail Decisions

arXiv:2605.04073v1 Announce Type: new Abstract: Bail decisions present a fundamental challenge for data-driven decision support systems. When bail is denied, the counterfactual outcome of whether the

Constrained Extreme Gradient Boosting for Adapting Reduced-Order Models

arXiv:2605.04130v1 Announce Type: new Abstract: High-fidelity simulations, such as computational fluid dynamics and finite element analysis, are essential for modeling complex engineering systems but

Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing

arXiv:2605.04185v1 Announce Type: new Abstract: When deploying reinforcement learning policies to physical robots, actuator rate constraints -- hard limits on how fast each joint can move per control

Agents

ContextPilot: Fast Long-Context Inference via Context Reuse

arXiv:2511.03475v4 Announce Type: replace Abstract: AI applications increasingly depend on long-context inference, where LLMs consume substantial context to support stronger reasoning. Common examples

agentsarxiv-cs-lg

Contextual Memory-Enhanced Source Coding for Low-SNR Communications

arXiv:2605.04400v1 Announce Type: cross Abstract: While Separate Source-Channel Coding (SSCC) retains the practical benefits of modular system design, its effectiveness in noisy text transmission is f

Counter-Dyna: Data-Efficient RL-Based HVAC Control using Counterfactual Building Models

arXiv:2605.04555v1 Announce Type: new Abstract: Model-based reinforcement learning (MBRL) offers a promising approach for data-efficient energy management in buildings, combining the strengths of pred

Local Ai

Counterfactual identifiability beyond global monotonicity: non-monotone triangular structural causal models

arXiv:2605.04413v1 Announce Type: new Abstract: Structural causal models provide a unified semantics for interventions and counterfactuals, but most identifiability results rely on restrictive assumpt

local-aiarxiv-cs-lg

CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies

arXiv:2605.04470v1 Announce Type: new Abstract: Open-loop imitation learning has advanced modern autonomous driving policy architectures, but closed-loop deployment remains vulnerable to policy-induce

Critical Windows of Complexity Control: When Transformers Decide to Reason or Memorize

arXiv:2605.04396v1 Announce Type: new Abstract: Recent work has shown that Transformers' compositional generalization is governed by complexity control, initialization scale and weight decay, which st

Cross-Model Consistency of Feature Importance in Electrospinning: Separating Robust from Model-Dependent Features

arXiv:2605.04905v1 Announce Type: new Abstract: Electrospinning is a highly sensitive fabrication process in which small variations in operating parameters can significantly influence fiber morphology

Hardware

CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels

arXiv:2605.05023v1 Announce Type: new Abstract: Efficient CUDA implementations of attention mechanisms are critical to modern deep learning systems, yet supporting diverse and evolving attention varia

hardwarearxiv-cs-lg

Data-dependent Exploration for Online Reinforcement Learning from Human Feedback

arXiv:2605.04477v1 Announce Type: new Abstract: Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for aligning large language models (LLMs) by continuously c

Dataset-Driven Channel Masks in Transformers for Multivariate Time Series

arXiv:2410.23222v3 Announce Type: replace Abstract: Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale

Hardware

Deep Wave Network for Modeling Multi-Scale Physical Dynamics

arXiv:2605.04198v1 Announce Type: new Abstract: Performance of deep learning models is strongly governed by architectural capacity, with width and depth as primary controls. However, in physical-scien

hardwarearxiv-cs-lg

DeFed-GMM-DaDiL: A Decentralized Federated Framework for Domain Adaptation

arXiv:2605.04324v1 Announce Type: new Abstract: Decentralized multi-source domain adaptation seeks to transfer knowledge from multiple heterogeneous and related source domains to an unlabeled target d

Applications

Delving into Non-Exchangeability for Conformal Prediction in Graph-Structured Multivariate Time Series

arXiv:2605.04957v1 Announce Type: new Abstract: Point forecasting for graph-structured multivariate time series is a fundamental problem, but rigorous uncertainty quantification for such predictions i

applicationsarxiv-cs-lg

Demystifying Manifold Constraints in LLM Pre-training

arXiv:2605.04418v1 Announce Type: new Abstract: The empirical success of large language model (LLM) pre-training relies heavily on heuristic stabilization techniques, such as explicit normalization la

Denoising Particle Filters: Learning State Estimation with Single-Step Objectives

arXiv:2602.19651v2 Announce Type: replace-cross Abstract: Learning-based methods commonly treat state estimation in robotics as a sequence modeling problem. While this paradigm can be effective at max

Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone

arXiv:2605.04454v1 Announce Type: cross Abstract: Alignment evaluation in machine learning has largely become evaluation of models. Influential benchmarks score model outputs under fixed inputs, such

Agents

Designing a double deep reinforcement learning selection tool for resilient demand prediction

arXiv:2605.04068v1 Announce Type: new Abstract: The use of artificial intelligence in supply chain forecasting has attracted many scientific studies for several decades. However, the process of select

agentsarxiv-cs-lg

Differentiable Chemistry in PINNs for Solving Parameterized and Stiff Reaction Systems

arXiv:2605.04708v1 Announce Type: new Abstract: From neural ODEs to continuous-time machine learning, differentiable solvers allow physics, optimization, and simulation to become trainable components

Discovering New Theorems via LLMs with In-Context Proof Learning in Lean

arXiv:2509.14274v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated significant promise in formal theorem proving. In this study, we investigate the ability of LLMs to d