Search: “inference”

Statistical Inference for Misspecified Contextual Bandits

arXiv:2606.22639v1 Announce Type: cross Abstract: Contextual bandit algorithms have transformed modern experimentation by enabling real-time adaptation for personalized treatment. Yet these advantages

23 Jun 2026

Compiler-First State Space Duality and Portable O(1) Autoregressive Caching for Inference

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2603.09555v2 Announce Type: replace-cross Abstract: High-throughput Mamba-2 inference is usually tied to fused CUDA and Triton kernels, limiting portability across accelerator backends. We show

local-aiarxiv-cs-ai

11 Jun 2026

Energy Use of AI Inference, Efficiency Pathways, and Test-Time Scaling

arXiv:2509.20241v2 Announce Type: replace Abstract: As AI inference scales to billions of queries, estimates of per-query energy use are increasingly important for capacity planning, efficiency interv

11 Jun 2026

Accounting for AI Inference in Corporate GHG Inventories: A Four-Tier Methodology for Scope 3 Category 1 Reporting

arXiv:2606.10660v1 Announce Type: cross Abstract: AI inference services -- API subscriptions, enterprise chat tools, and SaaS products with embedded AI features -- fall unambiguously within Scope 3 Ca

hardwarearxiv-cs-ai

10 Jun 2026

Disjoint or Overlapping? Inference Windowing for Reconstruction-Based Time Series Anomaly Detection

arXiv:2606.09874v1 Announce Type: new Abstract: Reconstruction-based methods are widely used for time series anomaly detection, where models are trained to reconstruct subsequences, and anomalies are

10 Jun 2026

Spiking Neural Network inference on FPGAs with hls4ml

arXiv:2606.10008v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) provide a naturally temporal machine-learning framework. Their neurons maintain an internal state and propagate informa

10 Jun 2026

AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference

arXiv:2606.07665v1 Announce Type: cross Abstract: Transformer inference increasingly depends on specialized compiler and runtime support, but real model graphs still require semantic decisions about w

Cross-LLM Consistency in Inference: Evidence from Shared Interactions

arXiv:2606.08129v1 Announce Type: new Abstract: Large language models (LLMs) differ in architecture, training data, and optimization procedures, yet they may still develop similar internal inference p

researcharxiv-cs-ai

Disentangling Latent Risk Pathways via Bayesian Hypergraph Inference

arXiv:2606.07677v1 Announce Type: cross Abstract: Electronic health records (EHR) pose large-scale multi-disease modeling problems in which many outcomes are rare and strongly influenced by shared ris

FuseFSS: Efficient Secure LLM Inference with Function Secret Sharing

arXiv:2606.09551v1 Announce Type: cross Abstract: Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings. Recent GPU systems b

hardwarearxiv-cs-ai

In-Context Learning of Temporal Point Processes with Foundation Inference Models

arXiv:2509.24762v3 Announce Type: replace Abstract: Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamic

LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load

arXiv:2603.23640v2 Announce Type: replace-cross Abstract: Deploying large language models on-device for always-on personal agents demands sustained inference from hardware tightly constrained in power

REMEDI: A Benchmark for Retention and Unlearning Evaluation in Multi-label Clinical Disease Inference

arXiv:2606.07141v1 Announce Type: cross Abstract: Language models trained for clinical disease inference are trained on patient data, which may include sensitive and private information, and data owne

8 Jun 2026

Flow Matching Calibration for Simulation-Based Inference under Model Misspecification

arXiv:2509.23385v5 Announce Type: replace-cross Abstract: Simulation-based inference (SBI) is transforming experimental sciences by enabling parameter estimation in complex non-linear models from simu

4 Jun 2026

Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories

arXiv:2606.04778v1 Announce Type: new Abstract: Safety-aligned Large Language Models (LLMs) remain vulnerable to interventions during inference that redirect generation toward harmful outputs. Recent

safetyarxiv-cs-ai

4 Jun 2026

SharedRequest: Privacy-Preserving Model-Agnostic Inference for Large Language Models

arXiv:2606.05004v1 Announce Type: cross Abstract: With the widespread deployment of public large language models (LLMs) such as ChatGPT, protecting user prompt privacy has become an increasingly criti

researcharxiv-cs-ai

4 Jun 2026

Giving a talk this Sunday June 7th at @agihouse_org talking about DeepSeek V4 on @togethercompute! Come hang and let's talk inference!

model-releasestogether-ai--x

A Together AI representative announced a talk scheduled for Sunday, June 7th at AGI House focusing on DeepSeek V4 and inference optimization on Together Compute's platform. The event was promoted as a

NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference

arXiv:2606.03910v1 Announce Type: cross Abstract: Disaggregated LLM inference forces the KV cache to traverse the datacenter network before decoding begins, so transfer time enters directly into the T

local-aiarxiv-cs-ai

Industry

Powering the Inference Era: Inside the DigitalOcean Data & Learning Layer

DigitalOcean discusses infrastructure and platform capabilities designed to support the inference phase of machine learning, where trained models are deployed to make predictions on new data at scale.

industrydigitalocean

Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference

arXiv:2606.02823v1 Announce Type: new Abstract: Two-bit weight quantization is attractive for memory-efficient LLM inference, but the standard W2 level set {-2,-1,0,+1} often collapses under aggressiv

Resource-Constrained Adaptive Inference for Sequential Pricing

arXiv:2606.03736v1 Announce Type: cross Abstract: Resource-constrained pricing controllers can make fixed-price inference impossible: the controller's resource state may remove the target price neighb

local-aiarxiv-cs-lg

Bandit Simulation for Average Reward Inference

arXiv:2606.00913v1 Announce Type: cross Abstract: Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inferenc

Beyond Task-Agnostic: Task-Aware Grouping for Communication-Efficient Multi-Task MoE Inference

arXiv:2606.01007v1 Announce Type: cross Abstract: Sparsely activated Mixture-of-Experts (MoE) models scale capacity via conditional computation, but distributed inference suffers from cross-GPU expert

local-aiarxiv-cs-ai

Bit-Exact AI Inference Verification Without Performance Tradeoffs

arXiv:2606.00279v1 Announce Type: cross Abstract: Verifying claims about AI workloads is a pre- requisite for credible AI governance of covert adversaries (who comply with monitoring only when detecti

hardwarearxiv-cs-lg

Differentially Private Datastore Generation for Retrieval-Augmented Inference

arXiv:2606.01413v1 Announce Type: cross Abstract: It is crucial for modern on-device AI systems that rely on retrieval-augmented inference to release and share datastores without compromising individu

Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Robotics

arXiv:2606.02562v1 Announce Type: cross Abstract: Autonomous robots that interact with people must make safe and efficient decisions under human-induced uncertainty, such as their preferences, goals,

Agents

Threading Optimization for Vision-Language-Action Model Inference in Low-Cost Smart Agricultural Manipulation

arXiv:2606.00966v1 Announce Type: new Abstract: Vision-Language Action (VLA) models continue to face challenges such as slow inference speed and difficulty performing fine-grained motion adjustments,

safetyarxiv-cs-ro

Causal Evaluation of Membership Inference Attacks

arXiv:2602.02819v3 Announce Type: replace Abstract: Membership Inference Attacks (MIAs) aim to distinguish training points (members) from unseen data (non-members), and are widely used to quantify mem

1 Jun 2026

Industry

DigitalOcean Serverless Inference: A Deep Dive

DigitalOcean's serverless inference offering enables developers to deploy and run machine learning models without managing underlying infrastructure, automatically scaling resources based on demand. T

industrydigitalocean

1 Jun 2026

Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption

arXiv:2605.30089v1 Announce Type: new Abstract: Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of inference-time element corruption. This

29 May 2026

Leave a Window Out: Modifying the Jackknife for Predictive Inference in Time Series

arXiv:2605.30292v1 Announce Type: cross Abstract: Conformal prediction methods enjoy strong theoretical and empirical predictive inference performance, provided the data is exchangeable, and predictor

29 May 2026

Tools

Protecting against inference theft

This article discusses security measures and best practices for protecting AI model inference endpoints from unauthorized access and theft, likely covering topics such as API authentication, rate limi

toolsvercel-blog

29 May 2026

ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference

arXiv:2505.19342v2 Announce Type: replace-cross Abstract: Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwi

Deep Neural Network Training as Random Effects: An Optimization-Inference Duality

arXiv:2605.27991v1 Announce Type: cross Abstract: Deep neural networks (DNNs) have achieved remarkable empirical success, yet their training dynamics remain understood mainly from optimization rather

GenSBI: Generative Methods for Simulation-Based Inference in JAX

arXiv:2605.27499v1 Announce Type: new Abstract: Flow and diffusion generative models have established themselves as widely adopted density estimators for simulation-based inference (SBI), extending na

LLM Zeroth-Order Fine-Tuning is an Inference Workload

arXiv:2605.28760v1 Announce Type: new Abstract: Zeroth-order (ZO) fine-tuning is attractive for large language models because it replaces backpropagation with forward objective evaluations. Existing i

Agents

MRMMIA: Membership Inference Attacks on Memory in Chat Agents

arXiv:2605.27825v1 Announce Type: cross Abstract: Membership inference attacks (MIAs) test whether a target data record belongs to a system's private data, and have become a standard tool to measure p

agentsarxiv-cs-lg

Industry

OpenCode Now Supports DigitalOcean Inference Router for Intelligent Model Routing

OpenCode now integrates with DigitalOcean's Inference Router, enabling intelligent routing of AI model requests across distributed infrastructure. This integration allows developers to optimize model

industrydigitalocean

Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity

arXiv:2605.27526v1 Announce Type: cross Abstract: We develop semiparametrically efficient inference for kernel measures of noise heterogeneity in additive noise models. In many applications, the regre

Stop Suppressing the Tail: Causal Inference for Extreme Events

arXiv:2605.27474v1 Announce Type: cross Abstract: Estimating how an outcome responds to a continuous treatment (the Average Dose-Response Function, or ADRF) is a core causal-inference primitive. Howev

UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training

arXiv:2605.27740v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is bottlenecked by the linear growth of the self-attention key-value (KV) cache. Top-k sparse att

researcharxiv-cs-cl

Whose Is This?: Context-Aware Object Ownership Inference with Uncertainty-Guided Questioning

arXiv:2605.28087v1 Announce Type: new Abstract: Service robots must infer object ownership to correctly interpret instructions such as 'bring me my cup.' However, ownership is a latent attribute that

researcharxiv-cs-ro

Agents

Stateful Inference for Low-Latency Multi-Agent Tool Calling

arXiv:2605.26289v1 Announce Type: new Abstract: Multi-agent tool calling is becoming the dominant interaction pattern for LLM-based systems, yet existing inference frameworks treat each tool call as a

agentsarxiv-cs-lg

27 May 2026

Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction

arXiv:2605.24657v1 Announce Type: new Abstract: Major LLM platforms deploy models in an inference-only configuration: the model serves requests but never updates per-user weights. Users must repeatedl

How does Bayesian Sampling help Membership Inference Attacks?

arXiv:2503.07482v2 Announce Type: replace-cross Abstract: Membership Inference Attacks (MIAs) aim to estimate whether a specific data point was used in the training of a given model. Existing state-of

researcharxiv-cs-ai

Applications

Hypothesis Generation and Inductive Inference in Children and Language Models

arXiv:2605.24528v1 Announce Type: new Abstract: Real world decision-making requires constructing mental models under uncertainty over evidence, over the underlying causal rules, and over the state of

applicationsarxiv-cs-ai

OpenRouter raises $113M to bring order to enterprise AI inference routing

Artificial intelligence inference routing startup OpenRouter Inc. today announced it raised 113 million in new funding led by CapitalG, Alphabet Inc.’s independent growth fund, to push the envelope fo

hardwaresiliconangle