Search: “inference”

GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs

arXiv:2609.35882v1 Announce Type: cross Abstract: Genome foundation models are most useful where sequences are generated, yet the largest models need datacenter accelerators and a place to send privat

30 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

LOCO-AdaMP: Built-in LOCO Inference for Adaptive Minipatch Ensembles with Enhanced Prediction

DGX agent

arXiv:2609.36396v1 Announce Type: cross Abstract: As black-box machine learning models become increasingly common, extracting interpretations with uncertainty quantification has become a critical chal

researcharxiv-cs-lg

30 Sept 2026

Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks

arXiv:2609.36471v1 Announce Type: cross Abstract: World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds

30 Sept 2026

You Cannot Pick a Provider From the Price List: Market-Aware Routing for Open-Weight LLM Inference

arXiv:2609.37902v1 Announce Type: new Abstract: Existing LLM routers choose among models using static per-model costs. We show that open-weight inference markets introduce a second, largely ignored de

30 Sept 2026

Accelerator Choice Is Not Enough: AlphaFold2 Inference on Cloud TPUs

arXiv:2609.34818v1 Announce Type: cross Abstract: AlphaFold2 is written in JAX, so the same inference code compiles and runs unchanged on CPUs, GPUs and Google Cloud TPUs. That portability makes the a

DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference

arXiv:2609.32204v1 Announce Type: cross Abstract: Secure Transformer inference protects sensitive inputs but incurs substantial cryptographic overhead, with nonlinear operations such as Softmax and Ge

Distance-KV: Exploiting Relative Distance for Efficient Long-Context Inference

arXiv:2609.32663v1 Announce Type: new Abstract: The memory usage and decoding latency of LLM inference grow rapidly with context length. To reduce these costs, key-value (KV) cache compression methods

Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference

arXiv:2601.22132v2 Announce Type: replace Abstract: Large Language Models (LLMs) deliver state-of-the-art performance on complex reasoning tasks, but their inference costs limit deployment at scale. S

Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It

arXiv:2609.32444v1 Announce Type: new Abstract: We study training-inference mismatch in reinforcement learning with verifiable rewards (RLVR) for large language models, where rollouts are sampled by a

Spexis: Speculative Lookahead Scheduling for LLM Inference

arXiv:2609.34370v1 Announce Type: new Abstract: Spexis is a multi-GPU LLM inference framework that improves the efficiency of pipeline and tensor parallelism through speculative parallelism. Rather th

Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

arXiv:2609.34130v1 Announce Type: new Abstract: This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work i

Mentored Decoding: Faster Inference meets Boosting

arXiv:2609.30474v1 Announce Type: new Abstract: Speculative decoding is a successful technique speeding up inference of a target autoregressive language model via a fast drafter model. Lossy speculati

28 Sept 2026

Tutorials

Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on Factor Graphs

arXiv:2609.29466v1 Announce Type: cross Abstract: Approximate message passing on factor graphs underlies two dominant families of probabilistic inference algorithms: expectation propagation (EP) and v

tutorialsarxiv-cs-ai

Exploiting answer-invariant redundancies in satellite imagery for efficient VLM inference on edge

arXiv:2609.29029v1 Announce Type: new Abstract: Onboard vision-language models could enable satellites to answer queries directly, but exhaustive tiled inference over high-resolution imagery is slow a

researcharxiv-cs-cv

FB-GDM: Fully-Bayesian Guided Diffusion Models for High-Dimensional Linear Inverse Problems via Unsupervised Variational Inference

arXiv:2609.29216v1 Announce Type: cross Abstract: Diffusion models are powerful priors for linear inverse problems, but the reference guidance methods, Diffusion Posterior Sampling (DPS) and Pseudoinv

Spectral-Guided Diffusion: Accelerating Inference via Static Spectral Layer Scheduling

arXiv:2609.29505v1 Announce Type: new Abstract: Diffusion inference repeatedly evaluates the same large network. We ask whether pretrained weights alone can identify residual branches that need not be

UO-FIE: Combining Exact-Label Supervision with Graded Utility for Factivity Inference

arXiv:2609.28605v1 Announce Type: cross Abstract: The Factivity Inference Evaluation 2026 (FIE2026) classifies Chinese context-hypothesis pairs into nine ordered factivity intervals. Its evaluation me

Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models

arXiv:2609.27373v1 Announce Type: new Abstract: Recurrent language models repeatedly apply shared network blocks to refine latent representations, but standard inference recomputes global attention at

researcharxiv-cs-cl

24 Sept 2026

SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference

arXiv:2609.28064v1 Announce Type: new Abstract: Driving world-action models improve planning by coupling multimodal reasoning with future prediction, but their growing inference cost increasingly conf

24 Sept 2026

CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference

arXiv:2609.26300v1 Announce Type: new Abstract: Despite their strong performance, large language models (LLMs) are bottlenecked by KV cache memory traffic during long-context inference. Sparse attenti

23 Sept 2026

TopoCompress: Topology Aware Token Compression Algorithm for Distributed Edge MoE Inference

arXiv:2609.26061v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) models improve capacity with moderate overhead by sparsely activating experts per token. However, deploying MoE across resour

hardwarearxiv-cs-cl

23 Sept 2026

Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models

arXiv:2609.24422v1 Announce Type: new Abstract: Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs). Bayesian i

SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching

arXiv:2602.24208v2 Announce Type: replace Abstract: Diffusion models achieve state-of-the-art video generation quality, but their inference remains expensive due to the large number of sequential deno

safetyarxiv-cs-cv

Local Ai

Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models

arXiv:2602.01654v2 Announce Type: replace Abstract: Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing a

local-aiarxiv-cs-cl

vla.simd: Efficient CPU Inference for Language-Conditioned Manipulation

arXiv:2609.24274v1 Announce Type: new Abstract: Deploying language-conditioned manipulation without a dedicated GPU requires efficient inference and action chunks that cover the delay between policy q

safetyarxiv-cs-ro

Compressed Active Subspaces for Scalable Bayesian Inference

arXiv:2609.19539v1 Announce Type: cross Abstract: Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference

18 Sept 2026

Adaptive hybrid coupling with operator inference, the overlapping Schwarz alternating method and reinforcement learning

arXiv:2609.17837v1 Announce Type: cross Abstract: Hybrid domain decomposition methods provide a flexible framework for coupling full order models (FOMs) and reduced order models (ROMs), but typically

17 Sept 2026

rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference

arXiv:2609.19104v1 Announce Type: cross Abstract: Factory work is a promising early scenario for embodied AI: assigning repetitive manual jobs to robots has clear economic payoff, and a structured sta

17 Sept 2026

Local Ai

Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guarantees

arXiv:2609.16485v1 Announce Type: cross Abstract: We develop a certified continuation framework for equilibrium computation and for training deep equilibrium networks (DEQs), with training formulated

local-aiarxiv-cs-lg

16 Sept 2026

OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design

arXiv:2609.16898v1 Announce Type: cross Abstract: Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a

16 Sept 2026

BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference

arXiv:2609.13592v1 Announce Type: cross Abstract: GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-b

Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference

arXiv:2609.14507v1 Announce Type: cross Abstract: Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV s

Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models

arXiv:2609.15177v1 Announce Type: new Abstract: Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when paral

safetyarxiv-cs-lg

The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference

arXiv:2606.21869v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different

Dissecting GPU Utilization for LLM Inference on Nvidia Hopper

arXiv:2609.12923v1 Announce Type: cross Abstract: A single SM utilization percentage can make an LLM inference workload look compute-saturated while hiding how much useful work is being done. The prob

14 Sept 2026

Applications

GeoSense-AI: Fast Location Inference from Crisis Microblogs

arXiv:2512.18225v2 Announce Type: replace Abstract: This paper presents an applied AI pipeline for real-time geolocation from noisy microblog streams, unifying statistical hashtag segmentation, part-o

applicationsarxiv-cs-cl

14 Sept 2026

GUIDE: Generative Utility Inference and Decision Engine

arXiv:2609.12137v1 Announce Type: new Abstract: Measuring the preferences of human users remains a fundamental challenge of AI alignment. Existing elicitation approaches struggle to efficiently discov

safetyarxiv-cs-lg

14 Sept 2026

Subgroup Membership Inference Audits of Differentially Private Synthetic Text

arXiv:2609.09848v1 Announce Type: cross Abstract: Synthetic data releases are increasingly proposed in the literature as a means of sharing realistic data replicas in lieu of sensitive private dataset

11 Sept 2026

CausalVerify: An Execution-Grounded Benchmark for LLM Causal Inference Workflows

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recover

10 Sept 2026

Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference

arXiv:2609.08189v1 Announce Type: new Abstract: Dynamic layer routing reduces the inference cost of Large Language Models (LLMs) by learning to skip layers for individual tokens. Existing methods, how

10 Sept 2026

Likelihood-free inference with nuisance parameters through normalizing flows

arXiv:2609.10534v1 Announce Type: cross Abstract: We present a simple decomposition of a neural-network-based normalizing flow that naturally uncovers a pivotal statistic (or something close) in the p

10 Sept 2026

ComVLA: Communication-Aware Split Inference for VLA Models in 6G-Connected Robotics

model-releasesarxiv-cs-ro

arXiv:2609.07838v1 Announce Type: new Abstract: Connected robotics is an emerging 6G application where mobile robots follow natural-language instructions to manipulate physical objects. The Vision-Lan

9 Sept 2026

MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

arXiv:2609.07966v1 Announce Type: new Abstract: Key--value (KV) cache compression is an effective way to reduce the memory overhead of large language model (LLM) inference, particularly for long-conte

9 Sept 2026

Agents

Active Inference for an Intelligent Agent in Autonomous Reconnaissance Missions

arXiv:2510.17450v2 Announce Type: replace Abstract: We develop an active inference route-planning method for the autonomous control of intelligent agents. The aim is to reconnoiter a geographical area

agentsarxiv-cs-ai

7 Sept 2026

When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

arXiv:2609.04490v1 Announce Type: new Abstract: Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized st

7 Sept 2026

Instruction Duplication as an Inference-Time Control Primitive

arXiv:2609.04024v1 Announce Type: new Abstract: Procedural instruction following is a basic requirement for controllable language-model systems, especially when generated trajectories are inspected or

4 Sept 2026

Privacy-Preserving Heterogeneous Multi-LLM Federated Inference for Cognitive Diagnosis

arXiv:2609.02947v1 Announce Type: cross Abstract: Significant challenges remain in AI-driven educational systems in balancing privacy preservation with accurate cognitive diagnosis. To overcome this,

4 Sept 2026

Transfiver: Human-AI Co-Inference through a Shared Editable State

arXiv:2609.03797v1 Announce Type: new Abstract: Long-term human-AI interaction is difficult because the information that guides inference is updated implicitly by the model and is not directly inspect