Search: “inference”

Compute Where it Counts: Self Optimizing Language Models

arXiv:2605.10875v1 Announce Type: cross Abstract: Efficient LLM inference research has largely focused on reducing the cost of each decoding step (e.g., using quantization, pruning, or sparse attentio

safetyarxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Deterministic Differentiable Structured Pruning for Large Language Models

DGX agent

arXiv:2603.08065v2 Announce Type: replace-cross Abstract: Structured pruning reduces LLM inference cost by removing low-importance architectural components. This can be viewed as learning a multiplica

researcharxiv-cs-cl

12 May 2026

Enhancing Consistency Models for Multi-Agent Trajectory Prediction

arXiv:2605.08572v1 Announce Type: new Abstract: Diffusion models for multi-agent trajectory prediction are limited by iterative denoising, which causes inference latency that hinders their use in time

agentsarxiv-cs-cv

Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality

arXiv:2605.10206v1 Announce Type: cross Abstract: Distributional causal inference requires estimating not only average treatment effects but also interventional outcome distributions, including quanti

safetyarxiv-cs-lg

Local Ai

FLARE: One-Shot PE-Level Fault Localization in Systolic Arrays via Algebraic Test Vectors

arXiv:2605.08594v1 Announce Type: cross Abstract: Systolic arrays are the dominant compute fabric for neural network inference. Prior work has addressed column-level fault detection efficiently with u

local-aiarxiv-cs-lg

Guided Streaming Stochastic Interpolant Policy

arXiv:2605.10051v1 Announce Type: cross Abstract: Inference-time guidance is essential for steering generative robot policies toward dynamic objectives without retraining, yet existing methods are lar

safetyarxiv-cs-ai

NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning

arXiv:2605.08221v1 Announce Type: cross Abstract: This paper presents NoisyCoconut, a novel inference-time method that enhances large language model (LLM) reliability by manipulating internal represen

Hardware

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes

arXiv:2605.08913v1 Announce Type: cross Abstract: Autoregressive inference is typically assumed to scale predictably with decoding length, and key-value (KV) caching is widely regarded as a universall

hardwarearxiv-cs-cl

Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner

arXiv:2506.01301v2 Announce Type: replace Abstract: Theory-of-Mind (ToM) enables humans to infer mental states-such as beliefs, desires, and intentions-forming the foundation of social cognition. Howe

PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding

arXiv:2605.08632v1 Announce Type: cross Abstract: Speculative decoding accelerates Large Language Models (LLMs) inference by using a lightweight draft model to propose candidate tokens that are verifi

PRIM: Meta-Learned Bayesian Root Cause Analysis

arXiv:2605.08786v1 Announce Type: new Abstract: Root cause analysis (RCA) in complex systems is challenging due to error propagation across multiple variables, the need for structural causal knowledge

RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache

arXiv:2605.08317v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong performance across diverse tasks, but their inference with long input contexts is bottlenecked by memor

Recursive Language Models

arXiv:2512.24601v3 Announce Type: replace Abstract: We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive

RubricRefine: Improving Tool-Use Agent Reliability with Training-Free Pre-Execution Refinement

arXiv:2605.09730v1 Announce Type: new Abstract: Iterative self-refinement is a popular inference-time reliability technique, but its effectiveness in code-mode tool use depends heavily on the structur

SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation

model-releasesarxiv-cs-cv

arXiv:2605.10187v1 Announce Type: new Abstract: Scientific reasoning is a key aspect of human intelligence, requiring the integration of multimodal inputs, domain expertise, and multi-step inference a

TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems

model-releasesarxiv-cs-cl

arXiv:2605.09539v1 Announce Type: new Abstract: Multi-agent systems (MAS) have emerged as a promising paradigm for solving complex tasks. Recent work has explored self-evolving MAS that automatically

Test-Time Speculation

model-releasesarxiv-cs-cl

arXiv:2605.09329v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by using a fast draft model to generate tokens and a more accurate target model to verify them. Its perfo

Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning

arXiv:2605.08741v1 Announce Type: new Abstract: Inference-time harnesses substantially improve large language models on complex reasoning tasks. However, the intrinsic capabilities of the underlying m

safetyarxiv-cs-cl

When Independent Sampling Outperforms Agentic Reasoning

arXiv:2605.08478v1 Announce Type: new Abstract: We study how to allocate inference-time compute for competitive programming under fixed budgets. Evaluating 216 Codeforces problems across Divisions 1-3

agentsarxiv-cs-lg

WindINR: Latent-State INR for Fast Local Wind Query and Correction in Complex Terrain

arXiv:2605.09511v1 Announce Type: new Abstract: Many downstream decisions in complex terrain require fast wind estimates at a small number of user-specified locations and heights for a given forecast

AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers

arXiv:2602.13357v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) achieve state-of-the-art performance in high-fidelity image and video generation but suffer from expensive infer

EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing

arXiv:2605.07455v1 Announce Type: new Abstract: Visual-prompt-guided edit transfer aims to learn image transformations directly from example pairs, offering more precise and controllable editing than

safetyarxiv-cs-cv

Local Ai

HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learning

arXiv:2605.07211v1 Announce Type: cross Abstract: Mobile devices face diverse resource constraints and non-IID data class distributions, requiring fast on-device inference for local in-distribution (I

local-aiarxiv-cs-ai

Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation

arXiv:2605.06892v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) have achieved state-of-the-art video generation quality, but they incur immense computational cost because standard infere

researcharxiv-cs-cv

Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression

arXiv:2502.01941v3 Announce Type: replace-cross Abstract: While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrie

Setting-Matched and Semantics-Scaled Benchmarking of One-Step Generative Models Against Multistep Diffusion and Flow Models

model-releasesarxiv-cs-cv

arXiv:2603.14186v4 Announce Type: replace Abstract: State-of-the-art text-to-image models produce high-quality images, but inference remains expensive as generation requires several sequential ODE or

Hardware

SOCKET: SOft Collision Kernel EsTimator for Sparse Attention

arXiv:2602.06283v2 Announce Type: replace Abstract: Exploiting sparsity during long-context inference is key to scaling large language models, as attention dominates the cost of autoregressive decodin

hardwarearxiv-cs-lg

SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting

arXiv:2605.07243v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting a tree of candidate continuations and verifying it in one target forward. Existing drafters f

researcharxiv-cs-cl

Switchcraft: AI Model Router for Agentic Tool Calling

arXiv:2605.07112v1 Announce Type: new Abstract: Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Mo

agentsarxiv-cs-ai

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers

arXiv:2605.07772v1 Announce Type: new Abstract: Transformers perform inference by iteratively transforming token representations across layers. This layerwise computation has been studied empirically,

WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems

arXiv:2605.07068v1 Announce Type: cross Abstract: The LLM Wiki pattern, to compile and provide domain knowledge into a persistent artifact and serve it to LLMs via KV cache inference, promises context

A Fast Model Counting Algorithm for Two-Variable Logic with Counting and Modulo Counting Quantifiers

arXiv:2605.03391v1 Announce Type: cross Abstract: Weighted first-order model counting (WFOMC) is a central task in lifted probabilistic inference: It asks for the weighted sum of all models of a first

Local Ai

Exact Dual Geometry of SOC-ICNN Value Functions

arXiv:2605.04722v1 Announce Type: new Abstract: Input Convex Neural Networks (ICNNs) are commonly used in a two-stage manner: one first trains a convex network and then minimizes it over its input in

local-aiarxiv-cs-lg

FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation

arXiv:2605.04651v1 Announce Type: cross Abstract: Adapting pretrained models typically involves a trade-off between the high training costs of backpropagation and the heavy inference overhead of memor

researcharxiv-cs-cl

GeoDecider: A Coarse-to-Fine Agentic Workflow for Explainable Lithology Classification

arXiv:2605.03383v1 Announce Type: new Abstract: Lithology classification aims to infer subsurface rock types from well-logging signals, supporting downstream applications like reservoir characterizati

agentsarxiv-cs-ai

Applications

Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy

arXiv:2605.05072v1 Announce Type: new Abstract: 3D occupancy prediction aims to infer dense, voxel-wise scene semantics from sensor observations, where the 2D-to-3D view transformation serves as a cru

applicationsarxiv-cs-cv

Tree-Conditioned Edit Flows for Ancestral Sequence Reconstruction

arXiv:2605.04119v1 Announce Type: cross Abstract: Ancestral sequence reconstruction (ASR) aims to infer extinct protein sequences at internal nodes of a phylogenetic tree. Classical ASR methods are ty

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving

arXiv:2605.00831v1 Announce Type: cross Abstract: The rise of million-token, agent-based applications has placed unprecedented demands on large language model (LLM) inference services. The long-runnin

agentsarxiv-cs-ai

Hardware

MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier

arXiv:2603.03756v3 Announce Type: replace-cross Abstract: While large language models (LLMs) show promise in scientific discovery, existing research focuses on inference or feedback-driven training, l

hardwarearxiv-cs-cl

On the Privacy of LLMs: An Ablation Study

arXiv:2605.02255v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in interactive and retrieval-augmented settings, raising significant privacy concerns. While at

Position: LLM Serving Needs Mathematical Optimization and Algorithmic Foundations, Not Just Heuristics

arXiv:2605.01280v1 Announce Type: cross Abstract: This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic founda

S2O: Early Stopping for Sparse Attention via Online Permutation

arXiv:2602.22575v2 Announce Type: replace Abstract: Attention scales quadratically with sequence length, fundamentally limiting long-context inference. Existing block-granularity sparsification can re

Speculative Speculative Decoding

arXiv:2603.03251v3 Announce Type: replace Abstract: Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by using a

researcharxiv-cs-lg

Adaptive GoGI-Skip: Coupling Goal-Gradient Importance with Dynamic Uncertainty for Efficient Reasoning

arXiv:2505.08392v3 Announce Type: replace Abstract: Chain-of-Thought (CoT) prompting trades inference speed for reasoning accuracy. Existing compressors force a compromise as static gradient technique

safetyarxiv-cs-cl

Applications

Arbitrarily Conditioned Hierarchical Flows for Spatiotemporal Events

arXiv:2605.01226v1 Announce Type: new Abstract: Events in spatiotemporal systems are ubiquitous, yet modeling their complex distributions remains challenging. Existing point process models often rely

applicationsarxiv-cs-lg

ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection

arXiv:2502.06774v4 Announce Type: replace Abstract: Ensuring neural networks adhere to domain-specific constraints is crucial for addressing safety and trustworthiness while also enhancing inference a

TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs

model-releasesarxiv-cs-cl

arXiv:2510.15545v4 Announce Type: replace Abstract: Accelerating the inference of large language models (LLMs) has been a critical challenge in generative AI. Speculative decoding (SD) substantially i

Applications

Beyond Continuity: Simulation-free Reconstruction of Discrete Branching Dynamics from Single-cell Snapshots