Search: “inference”

Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment

arXiv:2506.00166v2 Announce Type: replace-cross Abstract: Existing paradigms for ensuring AI safety, such as guardrail models and alignment training, often compromise either inference efficiency or de

safetyarxiv-cs-cl

4 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Industry

Powering the Inference Era: Inside the DigitalOcean AI-Native Cloud

DGX agent

DigitalOcean outlines its AI-native cloud infrastructure designed to support the inference phase of AI model deployment, emphasizing tools and services that enable businesses to run and scale AI model

industrydigitalocean

4 May 2026

A Novel Computational Framework for Causal Inference: Tree-Based Discretization with ILP-Based Matching

arXiv:2604.27307v1 Announce Type: cross Abstract: Causal inference is essential for data-driven decision-making, as it aims to uncover causal relationships from observational data. However, identifyin

1 May 2026

Local Ai

Distributional Stability of Tangent-Linearized Gaussian Inference on Smooth Manifolds

arXiv:2602.19179v2 Announce Type: replace Abstract: Gaussian inference on smooth manifolds is central to robotics, but exact marginalization and conditioning are generally non-Gaussian and geometry-de

local-aiarxiv-cs-ro

30 Apr 2026

Graph Property Inference in Small Language Models: Effects of Representation and Reasoning Strategy

arXiv:2603.06635v2 Announce Type: replace Abstract: Recent progress in language modeling has expanded the range of tasks that can be approached through natural language interfaces, including problems

30 Apr 2026

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

arXiv:2604.23904v1 Announce Type: cross Abstract: Synthetic data offers a promising tool for privacy-preserving data release, augmentation, and simulation, but its use in causal inference requires pre

Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inference

arXiv:2503.10666v4 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have become widely used across various domains spanning search engines, code generation, and text creation. Howev

Guided Speculative Inference for Efficient Test-Time Alignment of LLMs

arXiv:2506.04118v3 Announce Type: replace Abstract: We propose Guided Speculative Inference (GSI), a novel algorithm for efficient reward-guided decoding in large language models. GSI combines soft be

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference

arXiv:2505.02922v3 Announce Type: replace Abstract: Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing GPU memory and band

The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage

model-releasesarxiv-cs-cl

arXiv:2508.09603v2 Announce Type: replace Abstract: Membership inference attacks serves as useful tool for fair use of language models, such as detecting potential copyright infringement and auditing

SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

arXiv:2604.22575v1 Announce Type: new Abstract: Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenec

27 Apr 2026

Fairness Evaluation and Inference Level Mitigation in LLMs

arXiv:2510.18914v4 Announce Type: replace-cross Abstract: Large language models often display undesirable behaviors embedded in their internal representations, undermining fairness, inconsistency drif

safetyarxiv-cs-ai

24 Apr 2026

Applications

At 30 trillion tokens a day, we're the largest inference provider outside the frontier labs' own APIs. We're hiring. Come build the infrastr…

applicationsfireworks-ai--x

At 30 trillion tokens a day, we're the largest inference provider outside the frontier labs' own APIs. We're hiring. Come build the infrastructure behind the top AI workloads: https://fireworks.ai/car

23 Apr 2026

Transparent Screening for LLM Inference and Training Impacts

arXiv:2604.19757v1 Announce Type: cross Abstract: This paper presents a transparent screening framework for estimating inference and training impacts of current large language models under limited obs

23 Apr 2026

DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing

arXiv:2604.19351v1 Announce Type: new Abstract: The quadratic computational complexity of the standard attention mechanism constitutes a fundamental bottleneck for large language models in long-contex

researcharxiv-cs-cl

Efficient Autoregressive Inference for Transformer Probabilistic Models

arXiv:2510.09477v2 Announce Type: replace-cross Abstract: Set-based transformer models for amortized probabilistic inference and meta-learning, such as neural processes, prior-fitted networks, and tab

SAGE: Training-Free Semantic Evidence Composition for Edge-Cloud Inference under Hard Uplink Budgets

arXiv:2604.19623v1 Announce Type: cross Abstract: Edge-cloud hybrid inference offloads difficult inputs to a powerful remote model, but the uplink channel imposes hard per-request constraints on the n

researcharxiv-cs-cv

Applications

Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks

arXiv:2604.18820v1 Announce Type: cross Abstract: Recovering latent structure from count data has received considerable attention in network inference, particularly when one seeks both cross-group int

applicationsarxiv-cs-lg

Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space

arXiv:2510.12916v2 Announce Type: replace-cross Abstract: Systems of interacting continuous-time Markov chains are a powerful model class, but inference is typically intractable in high dimensional se

LLM-Extracted Covariates for Clinical Causal Inference: Rethinking Integration Strategies

arXiv:2604.16763v1 Announce Type: new Abstract: Causal inference from electronic health records (EHR) is fundamentally limited by unmeasured confounding: critical clinical states such as frailty, goal

Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon

model-releasesarxiv-cs-lg

arXiv:2604.16957v1 Announce Type: new Abstract: We present Open-TQ-Metal, the first implementation of fused compressed-domain attention on Apple Silicon, enabling 128K-context inference for Llama 3.1

Towards Deep Encrypted Training: Low-Latency, Memory-Efficient, and High-Throughput Inference for Privacy-Preserving Neural Networks

arXiv:2604.16834v1 Announce Type: cross Abstract: Privacy-preserving machine learning (PPML) has become increasingly important in applications where sensitive data must remain confidential. Homomorphi

Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models

arXiv:2510.21783v2 Announce Type: replace-cross Abstract: Diffusion models have demonstrated powerful performance in generating high-quality images. A typical example is text-to-image generator like S

20 Apr 2026

The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference

model-releasesarxiv-cs-ai

arXiv:2604.15409v1 Announce Type: cross Abstract: KV caching is a ubiquitous optimization in autoregressive transformer inference, long presumed to be numerically equivalent to cache-free computation.

20 Apr 2026

Tutorials

ICYMI from a few weeks back, we compiled our learnings around how to achieve Training-Inference Parity in MoE Models. The Fundamental Issue:…

Fireworks AI published a compilation of insights on achieving training-inference parity in Mixture of Experts (MoE) models, addressing fundamental challenges in aligning model behavior between trainin

tutorialsfireworks-ai--x

18 Apr 2026

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference

arXiv:2601.07667v2 Announce Type: replace Abstract: Due to the prevalence of large language models (LLMs), key-value (KV) cache reduction for LLM inference has received remarkable attention. Among num

researcharxiv-cs-cl

17 Apr 2026

QualiaNet: An Experience-Before-Inference Network

arXiv:2604.14193v1 Announce Type: new Abstract: Human 3D vision involves two distinct stages: an Experience Module, where stereo depth is extracted relative to fixation, and an Inference Module, where

researcharxiv-cs-cv

17 Apr 2026

Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge

model-releasesarxiv-cs-lg

arXiv:2604.12737v1 Announce Type: cross Abstract: While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs).

15 Apr 2026

Tutorials

Best practices to run inference on Amazon SageMaker HyperPod

This post explores how Amazon SageMaker HyperPod provides a comprehensive solution for inference workloads. We walk you through the platform’s key capabilities for dynamic scaling, simplified deployme

tutorialsaws-ml-blog

MDP Planning as Policy Inference

arXiv:2602.17375v2 Announce Type: replace Abstract: We cast episodic Markov decision process (MDP) planning as Bayesian inference over policies. A policy is treated as the latent variable and is assig

Quantifying Weighted Morphological Content of Large-Scale Structures via Simulation-Based Inference

arXiv:2511.03636v2 Announce Type: replace-cross Abstract: We perform a simulation-based forecasting analysis to compare the cosmological constraining power of higher-order summary statistics of the la

R2E-VID: Two-Stage Robust Routing via Temporal Gating for Elastic Edge-Cloud Video Inference

arXiv:2604.09681v1 Announce Type: cross Abstract: With the rapid growth of large-scale video analytics applications, edge-cloud collaborative systems have become the dominant paradigm for real-time in

researcharxiv-cs-cv

Local Ai

WebLLM: A High-Performance In-Browser LLM Inference Engine

arXiv:2412.15803v2 Announce Type: replace-cross Abstract: Advancements in large language models (LLMs) have unlocked remarkable capabilities. While deploying these models typically requires server-gra

local-aiarxiv-cs-ai

Nvidia published DWDP (Distributed Weight-Data Parallelism), a new inference parallelism strategy focused on prefill. It sounds slightly ins…

Nvidia published DWDP (Distributed Weight-Data Parallelism), a new inference parallelism strategy focused on prefill. It sounds slightly insane until you remember the target machine is GB200 NVL72. Th

hardwaredylan-patel--x

9 Apr 2026

Identification and Inference for Algorithmic Frontiers with Selective Labels

arXiv:2606.14977v2 Announce Type: replace-cross Abstract: This paper provides identification results to characterize a fairness-accuracy (FA) frontier, and statistical inference tools to test hypothes

16 Oct 2026

Privacy-Preserving Text Inference for Health-Misinformation Classification and Semantic Retrieval

model-releasesarxiv-cs-lg

arXiv:2610.20796v1 Announce Type: cross Abstract: Health-misinformation analysis presents a dual challenge: misleading claims can affect public well-being, while systems designed to detect them may ex

16 Oct 2026

PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference

arXiv:2610.19605v1 Announce Type: cross Abstract: Long-output inference in large language models suffers from severe KV cache memory and bandwidth bottlenecks. While KV cache quantization alleviates t

researcharxiv-cs-cl

15 Oct 2026

Semi-Amortized Bayesian Inference with Reusable Local Posteriors

arXiv:2610.18945v1 Announce Type: cross Abstract: Amortized Bayesian inference uses simulations to perform Bayesian estimation when likelihoods are expensive or analytically unavailable. However, lear

15 Oct 2026

Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes

arXiv:2610.14352v1 Announce Type: new Abstract: Signal Temporal Logic (STL) inference provides interpretable formulas for time series classification. For multiclass classification, however, accurate p

Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models

arXiv:2610.15207v1 Announce Type: new Abstract: Large Language Models (LLMs) have been shown to be computationally expensive and vulnerable to jailbreak attacks, in which adversarial prompts bypass sa

Mode Collapse of Mean-Field Variational Inference

arXiv:2510.17063v2 Announce Type: replace-cross Abstract: Mean-field variational inference (MFVI) is a widely used method for approximating high-dimensional probability distributions by product measur

Risk-Averse Inference-Time Diffusion Model Alignment with Finite-Sample Guarantees

arXiv:2610.17579v1 Announce Type: cross Abstract: Inference-time alignment steers diffusion models toward downstream objectives without costly fine-tuning, but existing methods typically optimize expe

Inference-Time Steering Across Scaffolds and Epitopes for Sample-Efficient De Novo Nanobody Design

arXiv:2610.13844v1 Announce Type: new Abstract: Generative molecular design typically relies on over-generation and post-hoc filtering. Inference-time steering methods like Feynman-Kac (FK) steering i

12 Oct 2026

CoreWeave targets AI inference bottlenecks with full-stack optimization

model-releasessiliconangle

AI inference is fast becoming the workload that decides the economics of the AI boom. Training built the first wave of GPU clouds, but serving models faster and cheaper will define the next. That shif

Tutorials

Neural Decoding as Cognitive Inference

arXiv:2610.11923v1 Announce Type: cross Abstract: The brain maintains stable cognition despite continuously changing neural activity. How to extract stable cognitive states from variable neural observ

tutorialsarxiv-cs-ai

Applications

SafeInferCom: Safe Inference-Time Compute via Verifier-Guided Mid-Generation Intervention for Robotic Task Planning

arXiv:2610.11223v1 Announce Type: cross Abstract: Large Reasoning Language Models (LRLMs) enable multi-step reasoning for robotic task planning, but continued reasoning can overwrite valid intermediat

applicationsarxiv-cs-ai

SpikeSSL: A Universal Spike Inference Framework with Dynamics-Informed State-Space Layers

model-releasesarxiv-cs-ai

arXiv:2610.11456v1 Announce Type: new Abstract: Two-photon calcium imaging is a standard tool for recording large neural populations in vivo, yet inferring spikes accurately across the growing diversi

Democratizing MoE inference on commodity GPUs with CoMoE

arXiv:2610.09424v1 Announce Type: cross Abstract: Deploying Mixture-of-Experts (MoE) models relies heavily on Expert Parallelism, which generates intense inter-GPU communication. Consequently, state-o