Search: “inference”

Flow-Controlled Scheduling for LLM Inference with Provable Stability Guarantees

arXiv:2604.11001v1 Announce Type: new Abstract: Large language models (LLMs) have been widely adopted due to their great performance across a wide range of applications. ChatGPT and Gemini now serve h

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

Hardware Utilization and Inference Performance of Edge Object Detection Under Fault Injection

DGX agent

arXiv:2604.09631v1 Announce Type: cross Abstract: As deep learning models are deployed on resource constrained edge platforms in autonomous driving systems, reli able knowledge of hardware behavior un

hardwarearxiv-cs-ai

14 Apr 2026

Membership Inference Attacks Expose Participation Privacy in ECG Foundation Encoders

arXiv:2604.10424v1 Announce Type: new Abstract: Foundation-style ECG encoders pretrained with self-supervised learning are increasingly reused across tasks, institutions, and deployment contexts, ofte

Powerful Training-Free Membership Inference Against Autoregressive Language Models

arXiv:2601.12104v2 Announce Type: replace-cross Abstract: Fine-tuned language models pose significant privacy risks, as they may memorize and expose sensitive information from their training data. Mem

Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference

arXiv:2604.11496v1 Announce Type: cross Abstract: Dual-encoder Vision-Language Models (VLMs) such as CLIP are often characterized as bag-of-words systems due to their poor performance on compositional

safetyarxiv-cs-cl

A Closer Look at the Application of Causal Inference in Graph Representation Learning

arXiv:2604.08890v1 Announce Type: cross Abstract: Modeling causal relationships in graph representation learning remains a fundamental challenge. Existing approaches often draw on theories and methods

13 Apr 2026

TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference

model-releasesarxiv-cs-cv

arXiv:2604.09220v1 Announce Type: new Abstract: Implicit neural video representations encode entire video sequences within the parameters of a neural network and enable constant time frame reconstruct

13 Apr 2026

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures

arXiv:2604.09048v1 Announce Type: cross Abstract: While the large energy consumption of Large Language Models (LLMs) is recognized by the community, system operators lack guidance for energy-efficient

13 Apr 2026

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference

model-releasesarxiv-cs-cl

arXiv:2604.08133v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) has become a dominant architecture for scaling large language models due to their sparse activation mechanism. However, the s

Tutorials

Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement

arXiv:2507.08390v4 Announce Type: replace Abstract: Discrete diffusion models have recently emerged as strong alternatives to autoregressive language models, matching their performance through large-s

tutorialsarxiv-cs-lg

MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization

arXiv:2604.06798v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) based large language models (LLMs) offer strong performance but suffer from high memory and computation costs. Weight binariz

local-aiarxiv-cs-ai

ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference

arXiv:2508.16703v4 Announce Type: replace-cross Abstract: On-device running Large Language Models (LLMs) is nowadays a critical enabler towards preserving user privacy. We observe that the attention o

local-aiarxiv-cs-ai

Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation

arXiv:2604.06831v1 Announce Type: cross Abstract: Current LLM-based services typically require users to submit raw text regardless of its sensitivity. While intuitive, such practice introduces substan

safetyarxiv-cs-ai

Acceleration and its Limits in Gaussian Variational Inference

arXiv:2610.21020v1 Announce Type: cross Abstract: We study whether the classical momentum-based acceleration that improves the condition-number dependence of gradient descent from kappa to sqrt{kappa}

DPA-MoE: A Runtime Precision Control Framework for MoE Inference

arXiv:2610.21378v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed pr

researcharxiv-cs-cl

Lethe: Link Inference Attacks For Evaluation of Edge Unlearning Methods

arXiv:2610.21869v1 Announce Type: new Abstract: Graph Unlearning (GU) aims to remove the influence of specific training data from a Graph Neural Network without retraining anew. While most methods add

SALTO: Solver-Anchored Lifted Transfer Operators for Nonlinear Event Inference

arXiv:2610.21049v1 Announce Type: new Abstract: Learned dynamical models can accurately predict trajectories while producing unreliable sensitivities when differentiated or transposed, limiting their

local-aiarxiv-cs-lg

Cross-Lingual TTS for Indian Languages: A Benchmark and Inference-Time Duration Estimation Strategies

arXiv:2610.14826v1 Announce Type: cross Abstract: Recent alignment-free Text-to-Speech (TTS) models achieve high synthesis quality by removing explicit alignment modules and simplifying training. Howe

Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware

arXiv:2610.14389v1 Announce Type: new Abstract: Benchmark scores tell us what an LLM can do, but not how much energy is required to deliver that capability. We directly measure this trade-off for 86 o

Tutorials

ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition

arXiv:2601.03822v2 Announce Type: replace Abstract: Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computat

tutorialsarxiv-cs-ai

SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE Inference

arXiv:2610.15417v1 Announce Type: new Abstract: Mixture-of-experts (MoE) models activate only a few experts per token, but when their expert weights are offloaded to host memory, host-to-device (H2D)

Agents

What Information Buys in Active Inference

arXiv:2610.19046v1 Announce Type: new Abstract: Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscien

agentsarxiv-cs-lg

Edge-Native Temporal--Graph Dual-Expert Inference for Dexterous Grasp Classification

arXiv:2610.14926v1 Announce Type: new Abstract: Multi-finger hands contact objects at different times, so a global close command can overload early-contact fingers while others are still moving. Dense

researcharxiv-cs-ro

Exploration strategies for Gaussian process bandit optimization with inference regret guarantees

arXiv:2610.16187v1 Announce Type: cross Abstract: Existing cumulative regret guarantees for Bayesian optimization (BO) do not necessarily characterize the quality of the algorithm's final output. This

Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference

arXiv:2605.06046v2 Announce Type: replace Abstract: Auto-regressive token generation in large language models is memory-bound because it requires 'attending to' key and value tensors (KV cache) of all

local-aiarxiv-cs-lg

SCOPE Before You Sample: Conformal Scores Guide Posterior Exploration for Fast and Calibrated Bayesian Predictive Inference

arXiv:2610.17146v1 Announce Type: cross Abstract: Bayesian predictive regions account for parameter uncertainty but can be poorly calibrated under model misspecification, even with expensive full MCMC

Selective Inference for Clustering Under Unknown Covariance

arXiv:2610.17458v1 Announce Type: cross Abstract: Unsupervised clustering is widely used to discover underlying group structure, after which researchers often seek to validate these groups via post-ho

graphsignal-top: a `top` for inference GPUs in the open-source Graphsignal profiler, kernels ranked by time, live, with readable names

A few weeks ago we posted the open-source Graphsignal profiler here: wrap vllm serve / sglang serve / whatever in graphsignal-run and it serves everything it measures as JSON, mainly so an agent can r

local-air-localllama

How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

arXiv:2610.13764v1 Announce Type: cross Abstract: Prior-data fitted networks (PFNs) are trained to approximate the Bayesian posterior predictive distribution (PPD) of a synthetic prior, yet results ab

MHR-Router: Multimodal Heterogeneous Retrieval-Enhanced Adaptive Routing for VLM Inference

arXiv:2610.13278v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit complementary capabilities, yet selecting the most suitable expert for a given image-question query remains challe

Hardware

Run multi-node inference workloads using Ray LLM Deep Learning Containers

Learn how to deploy one large language model across two GPU nodes on Amazon EKS using the multi-node Ray LLM Deep Learning Containers, with separate prefill and decode nodes behind one OpenAI-compatib

hardwareaws-ml-blog

Scalable Sparse Autoregressive Variational Inference for Non-Gaussian Spatial Fields

arXiv:2610.13656v1 Announce Type: cross Abstract: Latent field models are essential for spatial statistics, but the posterior distribution of the latent field, even with a Gaussian process (GP) prior,

Scan-Parallel Variational Inference for event-driven Latent Jump Models

arXiv:2610.12986v1 Announce Type: cross Abstract: Continuous-time latent state-space models are widely used for irregularly observed time series, typically representing uncertainty through a random in

Tutorials

Controllable Exaggeration for Generative Motion Models via Training-Time Adaptation and Inference-Time Guidance

arXiv:2610.12316v1 Announce Type: new Abstract: Recent motion generative models have demonstrated strong capabilities in synthesizing physically plausible character motion, but often overlook establis

tutorialsarxiv-cs-cv

9 Oct 2026

I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]

local-air-machinelearning

ALHR - Adaptive Learnable Hierarchical Routing, uses static binary trees and learnable functions to minimize the amount of keys to be read. It does use a dense teacher while phase 1 of training howeve

9 Oct 2026

Best current R9700 inference engine?

model-releasesr-localllama

There are way too many forks to keep track of, so I've gotten lost. As far as I can tell, Radiance VLLM is best for models that fit in GPUs while some form of llama.cpp is probably best for MOE RAM-sp

8 Oct 2026

Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD

arXiv:2610.09651v1 Announce Type: new Abstract: DP-SGD protects training data by adding Gaussian noise to clipped gradients. The amount of noise is usually chosen by running a numerical privacy accoun

8 Oct 2026

Industry

Pay-per-inference for AI agents: How BlockRun and Incarna use Amazon Bedrock AgentCore payments

Amazon Bedrock AgentCore payments gives AI agents a managed way to pay for services on demand, with spending limits enforced by the infrastructure. See how Incarna's agents pay BlockRun for model infe

industryaws-ml-blog

8 Oct 2026

Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation

arXiv:2610.07716v1 Announce Type: new Abstract: Prefill-only decision models inspired by the Jev model score every candidate in a menu during a single forward pass and never decode, which makes one ca

researcharxiv-cs-cl

7 Oct 2026

VALSE: Vertical Adaptive Layer Skipping for Efficient Inference in Large Language Models

arXiv:2610.07606v1 Announce Type: new Abstract: This paper establishes a theoretical framework for vertical adaptive layer skipping, proving three foundational results: (i) an Expected FLOPs formula (

7 Oct 2026

Where Do Two Populations of Persistence Diagrams Differ? Calibrated Local Inference at a Fixed Budget

arXiv:2610.08292v1 Announce Type: cross Abstract: Many two-sample tests for populations of persistence diagrams assess global differences without identifying the regions of the birth-death plane that

7 Oct 2026

Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models

arXiv:2610.03647v1 Announce Type: cross Abstract: Many machine learning methods aim to approximate the lower-dimensional manifold on which the data lives. A desirable feature of such methods is that t

5 Oct 2026

BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration

arXiv:2610.02800v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, e

5 Oct 2026

Explainable Molecular Structure Inference from GC--MS with Diffusion Models and LLM Reranking

arXiv:2610.03066v1 Announce Type: cross Abstract: GC--EI--MS is an important technique for analyzing volatile and semivolatile compounds in complex samples. However, conventional methods rely heavily

5 Oct 2026

Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs

arXiv:2610.01595v1 Announce Type: cross Abstract: Video Large Language Models (VideoLLMs) receive frames in sequential order and interpret how visual content evolves along the temporal axis, yet tempo

2 Oct 2026

JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces

arXiv:2610.00437v1 Announce Type: new Abstract: LLM agents generate intermediate reasoning and actions token by token, making extended interactions slow and computationally expensive. Jev-style models

2 Oct 2026

Symmetry Discovery in Quantum Learning: Observable-Level and Task-Level Inference from Finite Measurements

arXiv:2610.00157v1 Announce Type: cross Abstract: Symmetry reduces the capacity of a quantum learning model, but the imposed group must match both the measured information and the label transformation

safetyarxiv-cs-lg

2 Oct 2026

Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions