Search: “inference”

SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference

arXiv:2604.21231v1 Announce Type: cross Abstract: Efficient inference for on-device Large Language Models (LLMs) remains challenging due to limited hardware resources and the high cost of the prefill

24 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Subject-level Inference for Realistic Text Anonymization Evaluation

DGX agent

arXiv:2604.21211v1 Announce Type: new Abstract: Current text anonymization evaluation relies on span-based metrics that fail to capture what an adversary could actually infer, and assumes a single dat

model-releasesarxiv-cs-cl

24 Apr 2026

Streaming Structured Inference with Flash-SemiCRF

arXiv:2604.18780v1 Announce Type: new Abstract: Semi-Markov Conditional Random Fields (semi-CRFs) assign labels to segments of a sequence rather than to individual positions, enabling exact inference

22 Apr 2026

Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects

arXiv:2604.05546v2 Announce Type: replace Abstract: Large Vision-Language Models (LVLMs) enable sophisticated reasoning over images and videos, yet their inference is hindered by a systemic efficiency

researcharxiv-cs-cl

15 Apr 2026

Unified Multimodal Uncertain Inference

model-releasesarxiv-cs-cv

arXiv:2604.08701v1 Announce Type: new Abstract: We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calib

13 Apr 2026

Cross-Prompt Inference Mitigates Rare-Class Suppression in Imbalanced VLM Finetuning

arXiv:2610.21470v1 Announce Type: new Abstract: When a vision-language model is finetuned on class-imbalanced data, rare-class recall can fall well below the model's own zero-shot performance. In this

researcharxiv-cs-cv

16 Oct 2026

Applications

A Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inference

arXiv:2610.19607v1 Announce Type: cross Abstract: Rising privacy concerns around large language models have accelerated research in private Transformer inference (PTI). Synthesizing the empirical clai

applicationsarxiv-cs-lg

15 Oct 2026

Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms

arXiv:2610.20136v1 Announce Type: new Abstract: Vision-language models (VLMs) are moving onto edge devices, but little is known about how much energy they draw on single-board computers with NPU or GP

hardwarearxiv-cs-lg

15 Oct 2026

SKG: A Law-Induced Engine from Inference to Induction

arXiv:2610.19505v1 Announce Type: new Abstract: This paper proposes the SKG (Symplectic-Kernel Gradient) reasoning engine for law-induced inference and induction. Laws are a priori; data provide insta

15 Oct 2026

PSI-SINDy: Post-Selection Inference for Sparse Identification of Nonlinear Dynamics

arXiv:2610.11486v1 Announce Type: cross Abstract: Sparse identification of nonlinear dynamics (SINDy) is a data-driven framework for discovering governing dynamics from time-series data by identifying

9 Oct 2026

Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads

arXiv:2610.07094v1 Announce Type: cross Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test

7 Oct 2026

Stepped MoE: Segment-Level Routing with Configurable Inference Complexity

arXiv:2610.07348v1 Announce Type: cross Abstract: Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints

7 Oct 2026

Benchmarking Generative Trajectory Models for Active-Inference Control

model-releasesarxiv-cs-ro

arXiv:2610.05692v1 Announce Type: new Abstract: Learning from trajectory demonstrations offers a route to active-inference control of complex systems whose dynamics are difficult to model explicitly.

Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs

arXiv:2610.05305v1 Announce Type: cross Abstract: Large Language Model (LLM) inference has become the dominant workload in modern AI systems, requiring serving infrastructures to maximize throughput w

hardwarearxiv-cs-lg

Finite-Sample Distribution Theory and Efficient Large-Scale Inference for Online Quantile Regression

arXiv:2610.05869v1 Announce Type: cross Abstract: This paper studies online quantile regression for large-scale and streaming data using Stochastic SubGradient Descent (SSGD) with constant learning ra

FlashMoE: Reducing SSD I/O Bottlenecks via ML-Based Cache Replacement for Mixture-of-Experts Inference on Edge Devices

arXiv:2601.17063v2 Announce Type: replace Abstract: Recently, Mixture-of-Experts (MoE) models have gained attention for efficiently scaling large language models. Although these models are extremely l

Task Inference Beyond Least Squares in Behavioral Foundation Models

arXiv:2610.05350v1 Announce Type: new Abstract: Behavioral Foundation Models (BFMs) aim to solve a wide range of downstream tasks without test-time policy learning by inferring a task vector from the

Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs

model-releasesarxiv-cs-ai

arXiv:2610.00047v1 Announce Type: new Abstract: Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM)

2 Oct 2026

Right In-Place (RiP) Convolution: A Simple, General, and Near-Optimal Strategy for Memory-Efficient CNN Inference

arXiv:2610.00586v1 Announce Type: cross Abstract: Activation memory, not compute, limits CNN inference on constrained hardware such as microcontrollers. Direct in-place convolution removes the dual-bu

researcharxiv-cs-cv

2 Oct 2026

Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure

arXiv:2609.40024v1 Announce Type: cross Abstract: We develop a general method for amortized Bayesian inference on multilevel models of arbitrary structure. Given a generative model specified as a dire

1 Oct 2026

RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models

arXiv:2609.39801v1 Announce Type: new Abstract: Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LL

1 Oct 2026

Can Tabular Foundation Models Amortize Statistical Inference?

arXiv:2609.33114v1 Announce Type: cross Abstract: For decades, statistical inference has largely been developed one problem at a time. Given a scientific target, such as a treatment effect or a regres

29 Sept 2026

Superposed Inference for Hyperdimensional Computing

arXiv:2609.32320v1 Announce Type: new Abstract: Hyperdimensional computing (HDC) is attractive for efficient and robust learning, but conventional inference still encodes every query independently, re

29 Sept 2026

TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference

arXiv:2602.05145v2 Announce Type: replace-cross Abstract: Speculative decoding can substantially accelerate LLM inference, but realizing its benefits in practice is challenging due to evolving workloa

hardwarearxiv-cs-ai

25 Sept 2026

Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

model-releasesarxiv-cs-cl

arXiv:2609.25537v1 Announce Type: new Abstract: Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latenc

23 Sept 2026

Applications

Learning Defensive Policies against Diverse Inference Attacks for Smart Meter Privacy

arXiv:2609.25484v1 Announce Type: new Abstract: Smart meter (SM) data provides fine-grained visibility into household energy consumption, but also exposes users to privacy risks. Inference attacks, kn

applicationsarxiv-cs-lg

23 Sept 2026

Applications

Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data

arXiv:2609.24754v1 Announce Type: new Abstract: We investigate next generation reservoir computing (NGRC) as a data-driven approach for inferring unseen components of dynamical systems. We compare NGR

applicationsarxiv-cs-lg

22 Sept 2026

Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

arXiv:2601.20332v3 Announce Type: replace Abstract: Diffusion language models (DLMs) generate text through iterative denoising, but inference requires full-sequence attention at every iteration, resul

16 Sept 2026

LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents

arXiv:2609.14138v1 Announce Type: cross Abstract: As LLM agents become integrated into increasingly complex workflows, they must continually acquire new capabilities while retaining competence on prev

safetyarxiv-cs-ai

15 Sept 2026

Self-Indexing Attention for Compression-Compatible Sparse Long-Context LLM Inference

arXiv:2609.13205v1 Announce Type: cross Abstract: Sparse long-context inference requires efficient token retrieval in both prefill and decode. Existing methods often use different retrieval strategies

researcharxiv-cs-cl

15 Sept 2026

Agents

Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance

arXiv:2609.10105v1 Announce Type: cross Abstract: Compute governance today is a governance of training: the thresholds, reporting requirements, and frontier-AI regimes now in force attach to training

agentsarxiv-cs-ai

11 Sept 2026

EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression

arXiv:2609.11058v1 Announce Type: new Abstract: Recent advances in multimodal large language mod- els (MLLMs) have opened new opportunities for edge intelligence by enabling reasoning across heterogen

11 Sept 2026

Optimizing AI Inference Across the Deployment Stack

arXiv:2609.10550v1 Announce Type: cross Abstract: AI deployment performance is shaped not by model architecture alone, but by interactions among compression, compiler transformations, and serving poli

11 Sept 2026

Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts

arXiv:2609.09241v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large fo

safetyarxiv-cs-cl

Encrypt What Matters: When Selective Homomorphic Inference Is Efficient

arXiv:2609.09357v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) enables inference on private data without revealing it to the server, but evaluating an entire input under FHE is e

Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

arXiv:2609.10264v1 Announce Type: cross Abstract: Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inf

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pri

local-aiarxiv-cs-cl

Thermodynamic Cyclic Processes with Markov Samplers in Bayesian Inference

arXiv:2609.07660v1 Announce Type: cross Abstract: The concept of Markov chain Monte Carlo (MCMC) cycles, an analogy to cyclic processes in heat engines, is presented in order to examine Bayesian infer

researcharxiv-cs-ai

EcoFair: Energy-Efficient Inference Routing for Edge AI under Data Degradation

arXiv:2603.26483v2 Announce Type: replace Abstract: Medical edge-AI systems must operate under a difficult tension: delivering reliable diagnostic inference while running on devices with limited batte

9 Sept 2026

RACE-AIMC: Selective Inference for Heterogeneous Analog In-Memory Accelerators at the Edge

arXiv:2609.03149v1 Announce Type: cross Abstract: Analog in-memory computing (AIMC) speeds up neural-network inference by doing the arithmetic directly inside a memory array, instead of shuttling weig

4 Sept 2026

Escaping Redundant Reasoning: Structure-Aware Search for Inference-Time LLMs

arXiv:2609.00738v1 Announce Type: new Abstract: Inference-time search with large language models (LLMs) often concentrates on a small set of structurally or semantically similar trajectories, leaving

researcharxiv-cs-ai

2 Sept 2026

Towards unsupervised representation learning for quantum data: quantum models with inference and generation

arXiv:2609.00372v1 Announce Type: cross Abstract: With quantum sensors, simulators and networks emerging, a future of quantum technology may produce quantum states as data---that is, coherently rather

2 Sept 2026

A Generalized Optimization Engine (GOE) for Edge AI Inference Acceleration

arXiv:2608.28652v1 Announce Type: new Abstract: Artificial intelligence (AI) models have demonstrated remarkable capabilities across various domains, yet their widespread deployment is impeded by sign

1 Sept 2026

KV Admission: Learning What to Write for Efficient Long-Context LLM Inference

arXiv:2512.17452v4 Announce Type: replace-cross Abstract: Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear Key-Value (KV) cache growth. Prior approaches miti

1 Sept 2026

H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference

arXiv:2608.28113v1 Announce Type: new Abstract: The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language

local-aiarxiv-cs-cl

31 Aug 2026

CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes

arXiv:2608.27455v1 Announce Type: new Abstract: Recent advances in inference-time scaling have significantly improved the reasoning performance of large language models (LLMs). However, these methods

researcharxiv-cs-cl

28 Aug 2026

FlashVLA: Streaming Action Decoding for Fast and Asynchronous VLA Inference

arXiv:2608.27384v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models are increasingly promising for robotic manipulation, yet their real-world deployment remains bottlenecked by high in

hardwarearxiv-cs-ro

28 Aug 2026

Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency

arXiv:2608.23831v1 Announce Type: cross Abstract: While reinforcement learning (RL) allows generalist robot policies to continually improve during deployment, the large model size of modern generalist