Search: “inference”

Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing

arXiv:2610.09778v1 Announce Type: cross Abstract: Reproducible benchmarking of Large Language Model (LLM) inference is challenging because repeated measurements can vary with execution and system stat

8 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

Agentic Design Space Exploration for Joint Hardware Configuration Selection and Mapping of AI Inference Workloads on Heterogeneous Edge SoCs

DGX agent

arXiv:2610.07191v1 Announce Type: cross Abstract: Modern edge Systems-on-Chip (SoCs) integrate heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, each with distinct performance and ene

agentsarxiv-cs-ai

7 Oct 2026

Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion

arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using pr

High-Dimensional Statistical Inference for Sparse Support Vector Machines

arXiv:2610.08345v1 Announce Type: cross Abstract: Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample si

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

arXiv:2610.07791v1 Announce Type: new Abstract: Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tend

Agents

Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

arXiv:2610.07782v1 Announce Type: new Abstract: Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache

agentsarxiv-cs-ai

AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference via Dynamical Text Guidance

arXiv:2508.06084v3 Announce Type: replace Abstract: Vision-language models (VLMs) have achieved impressive performance on multimodal inference tasks, but the cost remains a significant challenge due t

researcharxiv-cs-cv

6 Oct 2026

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

arXiv:2610.00499v1 Announce Type: new Abstract: As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world extit{serving}, where request manag

2 Oct 2026

Stochastic Rounding in Low-Precision Transformer Inference: A Variable-Precision Emulation Study of a Small GPT-2

arXiv:2610.01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look. W

2 Oct 2026

A polynomial time algebraic solution to exact marginal inference in Markov Random Field models

arXiv:1709.09051v3 Announce Type: replace-cross Abstract: This paper develops on algebraic grounds a polynomial time exact linear solution to the hard combinatorial problem of marginal inference in Ma

1 Oct 2026

Instruction Retrieval at Inference Time for Small Language Models

model-releasesarxiv-cs-ai

arXiv:2510.13935v3 Announce Type: replace-cross Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which nee

1 Oct 2026

Applications

Taipan: A Query-free Transfer-based Multiple Sensitive Attribute Inference Attack Solely from Auxiliary Graphs

arXiv:2602.06700v2 Announce Type: replace-cross Abstract: Graph-structured data underpin a wide spectrum of modern applications, yet their multiple sensitive attributes are not isolated but deeply cou

applicationsarxiv-cs-lg

1 Oct 2026

Calibrating One-Round Membership Inference with Neighbors

arXiv:2609.36331v1 Announce Type: cross Abstract: The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models tra

CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts

arXiv:2609.35883v1 Announce Type: cross Abstract: Genome foundation models are growing into sparse mixture-of-experts (MoE) networks whose expert weights no longer fit on the machines that hold the se

Privacy-Friendly Cohort Determination: Sealed, CSP-Independent In-Browser ML Inference of Professional Segments for Identity-Less Advertising

arXiv:2609.36153v1 Announce Type: cross Abstract: B2B advertising targets a viewer's professional attributes (employer size and industry, function, seniority) and has obtained them by matching identit

Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference

model-releasesarxiv-cs-cl

arXiv:2609.36654v1 Announce Type: cross Abstract: Large language models make weight storage and memory traffic major inference costs, motivating low-precision formats that represent each weight with o

AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

arXiv:2609.34683v1 Announce Type: new Abstract: The optimization of LLM serving engines, such as vLLM and SGLang, is largely benchmark-driven: optimizations, scheduling policies, hardware and system d

From Pixel Generation to Topological Inference: Structural Dual Super-Resolution for Trustworthy Cross-Physical-Domain Trabecular Morphology Learning

arXiv:2609.34716v1 Announce Type: new Abstract: Clinical CT and UHRCT cannot resolve individual trabeculae, whereas synchrotron radiation microCT (SR{mu}CT) provides 3.2{mu}m high-resolution reference

researcharxiv-cs-cv

Same Probe, Different Numbers: Are Activation Probes Robust to Inference-Time Numerical Non-Determinism?

arXiv:2609.31796v1 Announce Type: new Abstract: Activation probes are increasingly used to monitor LLMs in deployment. A probe is typically trained under one inference configuration, then used under w

Tutorials

When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers

arXiv:2606.16694v2 Announce Type: replace Abstract: Transformers perform computations through many successive attention and feedforward blocks, allowing them to learn complex correlations between a la

tutorialsarxiv-cs-lg

Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference

model-releasesarxiv-cs-ai

arXiv:2602.08329v2 Announce Type: replace-cross Abstract: A core bottleneck in large language model (LLM) inference is the cost of attending over the ever-growing key-value (KV) cache. Although near-o

25 Sept 2026

COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference

model-releasesarxiv-cs-ai

arXiv:2609.26913v1 Announce Type: cross Abstract: No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or

24 Sept 2026

BOBA: Dynamic Bayesian Optimization through Bayesian Active Inference

arXiv:2609.26021v1 Announce Type: new Abstract: Dynamic black-box optimization presents significant challenges for Bayesian Optimization (BO), as the objective function evolves over time, causing opti

23 Sept 2026

Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

arXiv:2609.25924v1 Announce Type: cross Abstract: Causal inference guides operational and managerial decisions but remains challenging in high-dimensional panel or tensor settings, where decisions may

23 Sept 2026

Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

model-releasesarxiv-cs-cl

arXiv:2609.24698v1 Announce Type: new Abstract: Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follo

Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Services

arXiv:2609.23321v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) has become a key technology for serving large-scale personalized large language models and diffusion models in the cloud. H

From Articles to Publishers: Aggregating Language Model Predictions for News Source Reliability Inference

arXiv:2609.24219v1 Announce Type: new Abstract: Traditionally, the reliability of news publishers is assessed by expert organisations that evaluate editorial practices, transparency and factual standa

researcharxiv-cs-cl

From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification

arXiv:2609.23966v1 Announce Type: new Abstract: LLMs can generate fluent descriptions from tables, but their outputs may remain logically unsupported by the structured data. We introduce STAT-TO-TEXT,

researcharxiv-cs-cl

Locally Private Inference for Riemannian Stochastic Optimization

arXiv:2609.22642v1 Announce Type: cross Abstract: We develop inference for manifold-valued population minimizers when each observation belongs to a different participant and only locally private messa

Riemannian Simultaneous Inference for Tangent Vector Field Regression

arXiv:2609.21910v1 Announce Type: cross Abstract: We consider nonparametric tangent vector field regression on a Riemannian manifold without boundary. Because responses at different points lie in diff

21 Sept 2026

Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents

arXiv:2603.19375v2 Announce Type: replace-cross Abstract: Membership inference attacks (MIAs), which enable adversaries to determine whether specific data points were part of a model's training datase

18 Sept 2026

Tutorials

Learning Submanifolds for Subsequent Inference on Random Dot Product Graphs, Part 1: Theory

arXiv:2609.19357v1 Announce Type: cross Abstract: We propose a framework for restricted inference on random dot product graphs whose latent positions lie on an unknown low-dimensional support manifold

tutorialsarxiv-cs-lg

18 Sept 2026

Applications

The Organization of Inference: Information, Resource Constraints, and AI Production

arXiv:2609.20449v1 Announce Type: new Abstract: The economic value of inference depends on how capacity and task information are distributed across stages of AI production. We study these organization

applicationsarxiv-cs-ai

18 Sept 2026

ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference

arXiv:2609.17943v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviate

researcharxiv-cs-cl

17 Sept 2026

GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

arXiv:2609.17573v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) are emerging as a promising generative paradigm that complements autoregressive decoding. In long-context sett

17 Sept 2026

Visual Perception Engine: Fast and Flexible Multi-Head Inference for Robotic Vision Tasks

arXiv:2508.11584v3 Announce Type: replace-cross Abstract: Deploying multiple machine learning models on resource-constrained robotic platforms for different perception tasks often results in redundant

hardwarearxiv-cs-ai

17 Sept 2026

Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Accelerators

arXiv:2609.16085v1 Announce Type: cross Abstract: Eight-bit integer (INT8) post-training quantization is the default recipe for edge deployment, under a widely held assumption: INT8 makes inference fa

16 Sept 2026

StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection

arXiv:2609.16841v1 Announce Type: cross Abstract: Increasing image resolution produces ever-longer visual-token sequences in vision-language models (VLMs), substantially raising their inference cost.

local-aiarxiv-cs-ai

16 Sept 2026

VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs

arXiv:2609.16722v1 Announce Type: new Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates con

16 Sept 2026

Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries

arXiv:2609.15021v1 Announce Type: cross Abstract: Shared host-memory caching can avoid repeated prefill when a request moves between inference replicas. Its usefulness depends on both correct state tr

15 Sept 2026

Fundamental Dynamical Units for Physics-Informed Structural Inference from Perturbation Time-Series in Networked Systems

arXiv:2609.11934v1 Announce Type: new Abstract: In networked dynamical systems, the parameter of primary mechanistic interest is signed interaction structure. Recovering this structure from perturbati

Impact of Multiple Non-Invasive Biosignals on Cardiovascular Biomarker Estimation via Simulation-Based Inference

arXiv:2609.11969v1 Announce Type: cross Abstract: As the population ages, the number of patients with cardiovascular diseases continues to increase, highlighting the need for early detection before pr

Not All Speech Is Intent: Adaptive Self-Correcting Inference Layer for Post-ASR False Wake-Up

model-releasesarxiv-cs-cl

arXiv:2609.12469v1 Announce Type: new Abstract: False wake-up activations remain a persistent challenge in conversational AI. Speech phonetically similar to a device's wake word can produce a syntacti

One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training

arXiv:2609.11936v1 Announce Type: cross Abstract: Energy supply and heat dissipation are two of the main challenges with modern GPU deployments. While typically discussed in the context of new datacen

Safety

Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attacks

arXiv:2609.10608v1 Announce Type: cross Abstract: Diffusion models have achieved remarkable success in generative tasks across various areas, however their training process raises significant privacy

safetyarxiv-cs-lg

11 Sept 2026

Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact

arXiv:2609.11915v1 Announce Type: cross Abstract: Generative artificial intelligence changes how firms reach customers, but standard marketing data do not record how often users see and notice a firm'

11 Sept 2026

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires balan

9 Sept 2026

Toward Sustainable Distributed LLM Inference: A Systems Synthesis and Research Agenda for an Energy-, Carbon-, and Cache-Aware llm-d Control Plane

arXiv:2609.05565v1 Announce Type: cross Abstract: Large language model (LLM) sustainability is increasingly a serving-systems problem, not only a training problem. In production, energy and carbon imp