Search: “inference”

Marginal utility, matrix factorization, and the Key-Value (KV) cache: a unified information-economic framework for sovereign geo-mining inference

arXiv:2609.20068v1 Announce Type: new Abstract: This paper builds a theoretical bridge between the economic notion of marginal utility and two machine-learning constructs, matrix factorization and the

18 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Semantic SLAM in Precision Agriculture using Bayesian Inference

DGX agent

arXiv:2609.20604v1 Announce Type: new Abstract: This paper presents a real-time semantic world modeling framework specialized for precision agriculture using autonomous robots. The framework combines

local-aiarxiv-cs-ro

18 Sept 2026

A Calibrated Instrument for Measuring How Inference Optimizations Affect Output Quality

arXiv:2609.18005v1 Announce Type: new Abstract: Large language model optimization is an active research area, spanning quantization of model weights, early-exit methods for skipping layers, and specul

Long-Lived Characters, Local Inference: Incremental Memory Maintenance for Game NPCs

arXiv:2609.18935v1 Announce Type: new Abstract: A game character should not have to reread its entire life before every conversation. For locally deployed language-model characters, however, revising

Physics-Informed Sylvester Normalizing Flows for Bayesian Inference in Magnetic Resonance Spectroscopy

model-releasesarxiv-cs-lg

arXiv:2505.03590v2 Announce Type: replace-cross Abstract: Magnetic resonance spectroscopy (MRS) is a non-invasive technique to measure the metabolic composition of tissues, offering valuable insights

Principled Koopman Representations with Kalman Inference for Efficient Time-Series Prediction

arXiv:2609.17815v1 Announce Type: cross Abstract: The Koopman operator has been widely used for time-series prediction in dynamical systems. However, prior work that learns latent ``Koopman spaces'' u

researcharxiv-cs-ai

Cross-Domain Inference for Human Localization: Applying Wi-Fi RSSI Data to CSI-Trained Models

arXiv:2609.17204v1 Announce Type: cross Abstract: Wi-Fi signal data can be used to compromise the privacy of individuals. While many existing approaches rely on Channel State Information (CSI), collec

16 Sept 2026

Exact Fusion and Coordinated Exploration in Multi-Robot Active Inference

model-releasesarxiv-cs-ro

arXiv:2609.17384v1 Announce Type: new Abstract: Robot teams that learn a common environment model exchange belief summaries and plan by the expected information gain of their actions. Under conjugate

16 Sept 2026

Beyond Noise: Understanding and Overcoming Temperature Effects in Analog DNN Inference

arXiv:2609.15527v1 Announce Type: new Abstract: The energy efficiency of analog computing makes it one of the most promising candidates for deploying resource-intensive machine learning workloads on c

15 Sept 2026

DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understanding

arXiv:2606.22906v2 Announce Type: replace-cross Abstract: Large language models have shown strong performance on software engineering (SE) tasks, yet understanding large industrial repositories remain

local-aiarxiv-cs-ai

15 Sept 2026

LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference

arXiv:2609.13682v1 Announce Type: cross Abstract: We introduce LayerRoute, a parameter-efficient method for adaptive transformer layer-skipping that combines per-layer hard-gated routing (trained via

15 Sept 2026

BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines

model-releasesarxiv-cs-cv

arXiv:2604.05793v2 Announce Type: replace-cross Abstract: In LLM agent pipelines, prompt privacy risk propagates beyond a single model call: raw user content enters retrieval queries, memory writes, t

14 Sept 2026

Tutorials

SCDM: Spatial-Contextual Disentanglement Mamba via Differential Inference for Efficient Image Classification

arXiv:2609.12825v1 Announce Type: new Abstract: State Space Models (SSMs), particularly VMamba, have emerged as efficient alternatives for modeling long-range dependencies in medical image analysis. H

tutorialsarxiv-cs-cv

14 Sept 2026

CIPHER: Benchmarking Cross-record Inference over Privacy-Hardened Evidence Records

arXiv:2609.07022v1 Announce Type: cross Abstract: Reasoning over privacy-constrained records requires combining structured attributes with evidence from free-text narratives. We introduce CIPHER (Cros

Agents

Inference-Time Graph Engineering for Multi-Agent LLM Workflows

arXiv:2609.05774v1 Announce Type: new Abstract: Recent multi-agent LLM systems increasingly rely on graph-structured communication to coordinate specialized agents. We revisit multi-agent orchestratio

agentsarxiv-cs-ai

Settling: Equilibrium Inference for Non-Convex Validity Sets

arXiv:2609.09682v1 Announce Type: new Abstract: Many learning systems return a single point estimate even when admissible outputs form disconnected or non-convex sets. Under squared loss, an ambiguous

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

arXiv:2609.10355v1 Announce Type: cross Abstract: Video understanding has rapidly evolved toward video large language models (VideoLLMs): systems that couple video representations with pretrained larg

Deep Active Inference with Diffusion Policy and Multiple Timescale World Model for Real-World Exploration and Navigation

arXiv:2510.23258v2 Announce Type: replace-cross Abstract: Autonomous robotic navigation in real-world environments requires exploration to acquire environmental information as well as goal-directed na

safetyarxiv-cs-lg

Distribution-free inference on the number of changepoints

arXiv:2609.08234v1 Announce Type: cross Abstract: Suppose we are given an ordered sequence of independent data whose distribution changes K times at unknown locations, for some unknown K geq 0. In thi

Federated Binary Gating with Server-Side Vision-Language Inference for Surveillance Anomaly Classification

arXiv:2609.07403v1 Announce Type: new Abstract: Privacy-sensitive surveillance systems could benefit from large vision-language models (VLMs), but such models typically require centralized access to r

researcharxiv-cs-cv

Solved: LLM inference on Windows was 2–3x slower when the server window wasn't focused

The fix: run the server detached/headless instead of keeping it attached to a console window. RTX 5090, ~27B NVFP4 model via ninfer: Terminal focused: 130–200 tok/s Terminal unfocused: 50–60 tok/s Cli

local-air-localllama

Benchmarking small LLM inference on SageMaker AI: G7 vs G5 and G6

model-releasesaws-ml-blog

Benchmark two 30B Mixture-of-Experts models, Qwen3-Coder-30B and NVIDIA Nemotron-3-Nano-30B, across G5, G6, G6e, and G7 GPU instances on Amazon SageMaker AI. Compare throughput, latency, and cost-per-

8 Sept 2026

Hardware

Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference

arXiv:2609.04895v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models activate only a small subset of experts per token, but the full expert set often exceeds GPU memory, causing repeated we

hardwarearxiv-cs-cl

CacheWeaver: Cache-Aware Evidence Ordering for Efficient Grounded RAG Inference

arXiv:2606.19667v2 Announce Type: replace Abstract: Retrieval-Augmented Generation (RAG) improves factual grounding, but it also lengthens prompts and raises prefill cost. Prefix caching in serving en

local-aiarxiv-cs-cl

Choosing the Right Language Mode at Inference Time for Multilingual Reliability

arXiv:2609.04653v1 Announce Type: new Abstract: Multilingual large language models often struggle to reason in low- to mid-resource languages. Prior work has shown that translation can improve multili

Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference

arXiv:2609.05275v1 Announce Type: new Abstract: Layer dropout (a.k.a. stochastic depth) has been shown to enable faster training, higher accuracy, and robustness to zero-shot layer pruning in both lan

researcharxiv-cs-ai

Impact of Data Loss in Postprocessing on Training and Inference of Quantum Neural Networks

model-releasesarxiv-cs-lg

arXiv:2609.05060v1 Announce Type: cross Abstract: As quantum hardware scales to larger devices, the classical software layers that interface with it must evolve in step. Postprocessing routines develo

Hardware

Flip, Don't Shuffle: Watermarking LLMs at the Speed of Inference

arXiv:2609.03844v1 Announce Type: cross Abstract: We introduce Stateless Bernoulli Watermarking (SBW), a new statistical watermark for Large Language Models that determines green list membership throu

hardwarearxiv-cs-cl

Semiparametric Inference for Counterfactual Regression under Intervention-Driven Shift

arXiv:2504.02694v3 Announce Type: replace-cross Abstract: We study counterfactual regression, which maps features to outcomes under hypothetical scenarios that differ from those observed in the data.

local-aiarxiv-cs-lg

The Dually Flat Geometry of Planning as Inference

arXiv:2609.04005v1 Announce Type: new Abstract: We present an alternative characterization of the occupancy measure of reinforcement learning, obtained by embedding the planning criterion into the dyn

researcharxiv-cs-ai

We open-sourced Paddock, our Rust/C++ inference engine with its own CUDA kernels (MIT/Apache-2.0)

model-releasesr-localllama

I'm one of the developers. We said in August it would go open source in September and it did last night. MIT or Apache-2.0, pick one. The repo you see is our internal repo, kernels included, so from n

WIDE: Wildcard Inference with Dynamic Expansion for Cross-Modal Generative Retrieval

arXiv:2609.03554v1 Announce Type: cross Abstract: Generative retrieval has demonstrated significant success by unifying representation learning and search into a single sequence-to-sequence generation

Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models

model-releasesarxiv-cs-lg

arXiv:2609.01723v1 Announce Type: cross Abstract: Text-to-Speech (TTS) foundation models are increasingly fine-tuned on private datasets to synthesize highly personalized voices, introducing severe pr

3 Sept 2026

Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds

arXiv:2609.02302v1 Announce Type: new Abstract: A core obstacle to alignment evaluation is evaluation awareness: capable models can tell when they are being tested rather than deployed, weakening the

safetyarxiv-cs-ai

3 Sept 2026

Bandits in Prod: Hyperparameter Optimization at Inference Time

arXiv:2609.01335v1 Announce Type: cross Abstract: Many production systems can assess a configuration only by using it on live requests and observing noisy feedback. Modern agentic systems are a promin

safetyarxiv-cs-ai

Hardware

Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance

arXiv:2609.00363v1 Announce Type: new Abstract: Conformance suites for quantized GEMM kernels ask whether two implementations agree within a tolerance. We measure what such a suite can detect. Injecti

hardwarearxiv-cs-lg

HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inference

arXiv:2609.00450v1 Announce Type: cross Abstract: Block Quantization (BQ) is a promising approach for efficient deployment of large language models (LLMs), enabling low-precision computation with cont

Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetry

arXiv:2609.00873v1 Announce Type: new Abstract: Diffusion language models (DLMs) have recently emerged as an alternative modeling paradigm to autoregressive LMs, offering advantages such as parallel g

researcharxiv-cs-cl

Online simultaneous inference for quantiles via smoothed stochastic gradient descent

arXiv:2505.13299v2 Announce Type: replace-cross Abstract: This paper considers the estimation of quantiles via a smoothed version of the stochastic gradient descent (SGD) algorithm. By smoothing the s

TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models

arXiv:2609.01277v1 Announce Type: cross Abstract: Although pretrained joint audio-visual diffusion models offer rich control over what to generate, they provide no explicit control over when an uttera

Defending Wearable VLMs Against Private Attribute Inference

arXiv:2608.28691v1 Announce Type: cross Abstract: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture: a user asks a task-driven question about the surroundi

Applications

Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation

arXiv:2608.28886v1 Announce Type: cross Abstract: What does a generation loop gain from learning on its own verified successes? In cycles of generate, verify, select and LoRA-consolidate on online bin

applicationsarxiv-cs-ai

Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference

arXiv:2608.28726v1 Announce Type: new Abstract: The remarkable performance of multimodal large language models (MLLMs) comes at the cost of substantial computational overhead, posing significant chall

Question: Why is prefill unbelievably faster in vLLM than other inference engines?

model-releasesr-localllama

I only started using some vLLM forks recently in a 4 x 48GB 4090 system. DS4F - ~5000pp/180tg (DSpark) Qwen3.8 Flash next - ~7500pp/135tg (MTP) This is amazing, like having the API in my house. But it

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

arXiv:2608.30505v1 Announce Type: cross Abstract: Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, an

Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network

arXiv:2305.19366v3 Announce Type: replace Abstract: Generative Flow Networks (GFlowNets), a class of generative models over discrete and structured sample spaces, have been previously applied to the p

safetyarxiv-cs-lg

31 Aug 2026

Active Diffusion-Based Inference for Ill-Posed Inverse Problems under Incomplete Priors

arXiv:2608.27080v1 Announce Type: cross Abstract: Many scientific and engineering applications require estimating unknown parameters from experimentally observable data -- an inverse problem that is i

28 Aug 2026

Agents

Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference