Search: “inference”

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models

arXiv:2604.15488v1 Announce Type: cross Abstract: Large language models (LLMs) often exhibit undesirable behaviors, such as safety violations and hallucinations. Although inference-time steering offer

safetyarxiv-cs-ai

20 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

Thermodynamic Diffusion Inference with Minimal Digital Conditioning

DGX agent

arXiv:2604.14332v1 Announce Type: new Abstract: Diffusion-model inference and overdamped Langevin dynamics are formally identical. A physical substrate that encodes the score function therefore equili

hardwarearxiv-cs-lg

17 Apr 2026

Instead of the gold standard, we can imagine an inference standard of exchange, the FLOP. (As opposed to tokens, this accounts for AI abilit…

applicationsethan-mollick--x

Instead of the gold standard, we can imagine an inference standard of exchange, the FLOP. (As opposed to tokens, this accounts for AI ability) With some AI help, I figure 1 buys roughly 10^17 managed-

15 Apr 2026

Robust Federated Inference

arXiv:2510.00310v3 Announce Type: replace Abstract: Federated inference, in the form of one-shot federated learning, edge ensembles, or federated ensembles, has emerged as an attractive solution to co

local-aiarxiv-cs-lg

15 Apr 2026

FIT-GNN: Faster Inference Time for GNNs that 'FIT' in Memory Using Coarsening

arXiv:2410.15001v5 Announce Type: replace Abstract: Scalability of Graph Neural Networks (GNNs) remains a significant challenge. To tackle this, methods like coarsening, condensation, and computation

13 Apr 2026

Active Statistical Inference

arXiv:2403.03208v3 Announce Type: replace-cross Abstract: Inspired by the concept of active learning, we propose active inferencenicode{x2013}a methodology for statistical inference with machine-l

10 Apr 2026

Conditional Mean Flows: Few-Step Densities for Simulation-Based Inference

arXiv:2610.21657v1 Announce Type: cross Abstract: Flow matching has emerged as a powerful generative family for simulation-based inference (SBI). However, sampling and density evaluation require numer

16 Oct 2026

Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference

arXiv:2610.21257v1 Announce Type: cross Abstract: Large Language Models (LLMs) demand efficient inference, where the multiply-accumulate (MAC) operations in linear projections dominate compute and ene

16 Oct 2026

Graph Dirichlet Energy for Fitting-Free Bivariate Causal Inference

arXiv:2610.21892v1 Announce Type: new Abstract: We propose the Graph Dirichlet Energy Causal Inference (GDECI) criterion for distinguishing cause from effect in bivariate settings. For a candidate dir

applicationsarxiv-cs-lg

16 Oct 2026

Cerebras positions inference speed as a competitive edge in agentic AI

Agentic workflows are making speed a condition for whole classes of AI applications to exist, and the fastest tokens will determine which platforms win the next phase of AI. Cerebras Systems Inc., whi

agentssiliconangle

Deep Generative Models for Phylogenetic Inference with Complex Evolutionary Processes

arXiv:2610.19053v1 Announce Type: new Abstract: Inference of phylogenetic trees from sequences plays a central role in understanding evolutionary relationships, with applications ranging from tracking

Seamless Deployment of Machine Learning Inference at the Far Edge: An Empirical AWS IoT Greengrass Architecture for Industrial Predictive Maintenance

arXiv:2610.14815v1 Announce Type: cross Abstract: Industrial IoT sensors produce continuous, high-volume data streams that predictive maintenance needs to analyse in real time. Cloud-centric architect

local-aiarxiv-cs-ai

Two Routes to Interoceptive Prioritisation: Dissociating Likelihood Precision and Preference Reweighting in Active Inference

arXiv:2610.18617v1 Announce Type: new Abstract: An embodied agent must prioritise among competing bodily needs. In active inference, it can respond to an urgent need by increasing the precision of inf

agentsarxiv-cs-ai

Closing the Estimation Gap: Training-Free Transductive Inference for Cross-Domain Few-Shot Action Recognition

arXiv:2610.18209v1 Announce Type: new Abstract: Cross-domain few-shot action recognition (CD-FSAR) aims to recognise unseen action classes in a target domain from a handful of labelled videos. Despite

researcharxiv-cs-cv

Constrained Diffusion Models with Primal-Dual Inference

arXiv:2606.17192v2 Announce Type: replace Abstract: This paper develops constrained diffusion models with primal-dual inference (PDI) to sample from optimal distributions of entropy-regularized optimi

Tools

ICYMI from yesterday's @dealbook: inference startups are ripping as enterprises continue to adopt AI - @togethercompute's annual bookings hi…

ICYMI from yesterday's @dealbook: inference startups are ripping as enterprises continue to adopt AI - @togethercompute's annual bookings hit 1.2 billion as of July - @FireworksAI_HQ crossed 1 billion

toolstogether-ai--x

Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference

arXiv:2601.13155v3 Announce Type: replace-cross Abstract: Long-context inference enhances the reasoning capability of Large Language Models (LLMs), but incurs significant computational overhead. Token

Power Side-Channel Membership Inference Attack on Embedded Machine Learning

arXiv:2610.10909v1 Announce Type: cross Abstract: Membership inference attacks (MIAs) threaten the privacy of machine learning (ML) training data by determining whether a sample was used to train a ta

local-aiarxiv-cs-lg

9 Oct 2026

CARE: Certifying Acceleration for Vision-Language-Action Inference

model-releasesarxiv-cs-cl

arXiv:2610.08917v1 Announce Type: new Abstract: While vision-language-action (VLA) models have advanced rapidly, running them at every control step remains expensive. Prior work accelerates VLA infere

8 Oct 2026

Java vllm-like framework claims 90% of perfomance of llama.cpp on local inference on NVIDIA GPUs by compiling Java to CUDA and cuTile

model-releasesr-localllama

TornadoVM: The Java to CUDA engine: https://github.com/beehive-lab/TornadoVM jitLLM: The inference engine: https://github.com/beehive-lab/jitllm Deep dive talk: https://www.youtube.com/watch?v=HO5CpET

8 Oct 2026

Hierarchy-GBP: Accelerating Factor Graph Inference via Abstraction and Recovery

arXiv:2610.06978v1 Announce Type: new Abstract: Gaussian Belief Propagation (GBP) is a distributed inference algorithm that passes messages in graphical models, making it attractive for scalable spati

researcharxiv-cs-cv

Inference-Time Projection for Physically Valid Biomolecular Diffusion Models

arXiv:2610.07037v1 Announce Type: new Abstract: AlphaFold 3-style cofolding models predict biomolecular complexes with high structural accuracy, yet a large fraction of their outputs are physically in

local-aiarxiv-cs-ai

Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference

arXiv:2610.07587v1 Announce Type: new Abstract: LLM orchestration investigates how an orchestrator coordinates a group of autonomous agents to achieve common goals or maximize collective welfare. The

agentsarxiv-cs-cl

Prediction-powered inference for time series across space

arXiv:2610.08715v1 Announce Type: cross Abstract: The following motif is common in spatiotemporal settings: we have a sequence of covariate and label pairs observed for a relatively short, recent time

ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding

arXiv:2610.08078v1 Announce Type: cross Abstract: Standard causal identification methods often assume no unmeasured confounding and can fail when relevant confounders are unobserved. Proximal causal i

Logit-Aware MIMO AirComp for Distributed Mixture-of-Experts LLM Inference over Wireless Edge Networks

arXiv:2610.03741v1 Announce Type: cross Abstract: Distributed mixture-of-experts (MoE) inference is a promising architecture for deploying large language models (LLMs) at wireless edge networks becaus

6 Oct 2026

TreeWalker: Partial Evaluation for Grouped Tree-Ensemble Inference

arXiv:2610.03939v1 Announce Type: new Abstract: Many inference workloads evaluate a trained tree ensemble on row groups that share feature values: discrete-time survival models expand each patient int

6 Oct 2026

LatticeSMC: Where to Spend Inference-Time Compute in Chunked Sequence Generators

arXiv:2610.02774v1 Announce Type: new Abstract: Long-form generators for music, motion and video produce sequences chunk by chunk, with each chunk generated by iterative denoising while rewards are de

5 Oct 2026

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

model-releasesarxiv-cs-cl

arXiv:2610.02986v1 Announce Type: new Abstract: Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without acces

5 Oct 2026

if you’re building an ai app, you might want to host inference to improve your margins, and if you do that you should support a few models, …

if you’re building an ai app, you might want to host inference to improve your margins, and if you do that you should support a few models, and if you’re doing that you might as well build a router, a

agentsyohei-nakajima--x

4 Oct 2026

TopK-Guided: Adaptive, Budget-Aware Activation Sparsity for Efficient LLM Inference

arXiv:2610.01763v1 Announce Type: new Abstract: Activation sparsity speeds up large language model (LLM) inference by setting unimportant activations to zero so that the corresponding computations can

2 Oct 2026

Embodiment-aware control by inference over the operator: a simulation study

arXiv:2609.38437v1 Announce Type: new Abstract: Teleoperation systems are tuned for channel fidelity, while whether the operator experiences the device as part of the body, the Sense of Embodiment (So

agentsarxiv-cs-ro

1 Oct 2026

Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models

arXiv:2609.39091v1 Announce Type: cross Abstract: Inference-time alignment of flow and diffusion-based models is critical for achieving flexible generative modeling. Theoretically, Doob's h-transform

1 Oct 2026

Hardware

CoreWeave expands its AI stack as inference surges: theCUBE’s Fully Connected keynote analysis

Token economics are becoming a defining measure of artificial intelligence infrastructure efficiency. As AI workloads move beyond training, the economics of producing useful intelligence are starting

hardwaresiliconangle

Hardware

Federation of Experts: Communication Efficient Distributed Inference for Large Language Models

arXiv:2605.06206v2 Announce Type: replace Abstract: Mixture of experts has emerged as the primary mechanism for making Large Language Models (LLMs) computationally efficient. However, in distributed s

hardwarearxiv-cs-lg

FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models

arXiv:2609.35947v1 Announce Type: new Abstract: Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the

Post-Anomaly Detection Inference for Deep SVDD

arXiv:2609.37935v1 Announce Type: cross Abstract: Deep Support Vector Data Description (Deep SVDD) has become a prominent framework for unsupervised anomaly detection by learning latent representation

Fiona: Accelerating FHE Inference with Packing-Aware Ternary Weights

arXiv:2609.35352v1 Announce Type: new Abstract: Fully homomorphic encryption (FHE) enables neural network inference directly on encrypted inputs, but it remains orders of magnitude slower than plainte

29 Sept 2026

Forlinx 20-TOPS M.2 AI accelerator supports PCIe cascading for local LLM inference

Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module us

local-air-localllama

29 Sept 2026

Transfer Calibrated Prediction Powered Inference

arXiv:2609.34156v1 Announce Type: new Abstract: Prediction-powered inference (PPI) and its power-tuned extension (PPI++) improve confidence intervals by combining a small gold-standard labeled sample

applicationsarxiv-cs-lg

29 Sept 2026

2400cc Inference Racer: Dual RTX 3090 motors, NVLink turbo, naked 7840U ThinkPad ECU, VW Golf radiator

model-releasesr-localllama

Today I present a fine piece of engineering, carefully assembled inside a custom chipboard chassis: the 2400cc Inference Racer, a.k.a. my winter heater. Power comes from two second-hand AORUS RTX 3090

26 Sept 2026

Learning the Cost of Reliable Inference

arXiv:2609.28322v1 Announce Type: new Abstract: Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on th

24 Sept 2026

Partner-Specific Affective Precision in Social Active Inference

arXiv:2609.24876v1 Announce Type: new Abstract: In multi-agent social settings, model reliability varies across relationships. Beyond inferring what others will do, an agent must calibrate how confide

local-aiarxiv-cs-ai

23 Sept 2026

RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems

arXiv:2609.12551v2 Announce Type: replace-cross Abstract: AI is beginning to make substantive contributions to LLM inference optimization. Existing AI optimizations are predominantly profiling-based.

23 Sept 2026

Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

arXiv:2609.25145v1 Announce Type: cross Abstract: Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend st

23 Sept 2026

Causal Inference with Unobserved Confounding: A Mixture Learning Perspective

arXiv:2609.23219v1 Announce Type: new Abstract: Unobserved confounding is a fundamental challenge in causal inference from observational data. This article develops a mixture-learning perspective, vie

22 Sept 2026

Hardware

Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing

NVIDIA Confidential Computing (CC) secures large‑language‑model inference by providing encrypted virtual machines, GPUs, and NVLink, enabling processing of sensitive data in trusted hardware. Running

hardwarenvidia-developer

22 Sept 2026

React When You Need To: Event-Triggered Asynchronous Inference for VLA Policies