Search: “inference”

Efficient Best-of-N policy evaluation for inference-time alignment

arXiv:2610.09250v1 Announce Type: new Abstract: Best-of-N (BoN) is a common inference-time alignment method that selects the highest-scoring response among N samples from a reference model. Evaluating

safetyarxiv-cs-lg

8 Oct 2026

Applications

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Shape-Bayes: Bayesian Inference of Structured Shapes under Visual Ambiguity

DGX agent

arXiv:2610.09032v1 Announce Type: new Abstract: Perceiving structured shapes, such as human faces, from pixels is an inherently ambiguous task in real-world conditions. Yet, shape inference is largely

applicationsarxiv-cs-cv

8 Oct 2026

Thinking in Depth: Retrospective Inference for Tabular Foundation Models

arXiv:2610.10317v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are pretrained across diverse tabular tasks and make predictions on a new table at inference time using its labeled exa

8 Oct 2026

Cascadia: Resident 975B MoE Inference on Eleven AI PCs

arXiv:2610.07219v1 Announce Type: new Abstract: Mixture-of-experts models make nearly trillion-parameter capacity accessible with sparse per-token computation, provided that the serving system can dis

7 Oct 2026

Inference and learning in sparse autoencoders as natural gradient flow

arXiv:2610.07389v1 Announce Type: cross Abstract: Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overla

researcharxiv-cs-ai

7 Oct 2026

RoboIRS: Inference-Time Internal Representation Steering for Generalist Robot Policies

arXiv:2610.04681v1 Announce Type: new Abstract: Vision-language-action (VLA) and world-action models (WAMs) often degrade under out-of-distribution task variations despite retaining partial task capab

safetyarxiv-cs-ro

6 Oct 2026

Two-Sample Testing via Path-based Inference

arXiv:2610.05684v1 Announce Type: cross Abstract: Modern deep generative models are primarily studied for their ability to generate realistic samples, yet the generative dynamics they learn can also s

6 Oct 2026

A Missing Latent, Not a Missing Simulator: Radius-Augmented Inference for Real JWST Retrieval

arXiv:2610.02245v1 Announce Type: cross Abstract: Amortized simulation-based inference (SBI), which is trained on radiative-transfer simulators, recovers exoplanet atmospheres accurately on synthetic

5 Oct 2026

LLM Inference Dashboard

model-releasesr-localllama

Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS)

4 Oct 2026

Error-Corrected Inference-Time Scaling for Imperfect Diffusion Models

arXiv:2610.01933v1 Announce Type: cross Abstract: Inference-time scaling adapts pretrained diffusion models to new sampling tasks without additional training. Existing methods rely primarily on Monte

2 Oct 2026

STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling

arXiv:2610.00907v1 Announce Type: cross Abstract: Relational foundation models (RFMs) are pretrained once on a collection of relational databases and prediction tasks, and then applied zero-shot to pr

2 Oct 2026

Agents

A memory-based active inference model of DishBrain-like adaptive behaviour

arXiv:2508.06980v2 Announce Type: replace Abstract: Recent and rapid advances in artificial intelligence (AI) make it increasingly important to understand the foundations of adaptive behaviour in auto

agentsarxiv-cs-ai

DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference

arXiv:2609.38823v1 Announce Type: new Abstract: Multimodal mixture-of-experts (MoE) models combine sparse expert activation with visual-language capabilities, yet their inference remains costly becaus

researcharxiv-cs-cv

QuanVI: Score-based Variational Inference via Quantum Maximally Mixed States

arXiv:2609.39164v1 Announce Type: new Abstract: Score-based variational inference (VI) provides an alternative to Kullback--Leibler (KL)-based VI by minimizing the Fisher divergence between the variat

ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference

arXiv:2602.08005v2 Announce Type: replace-cross Abstract: Efficient long-context inference faces two coupled bottlenecks: KV-cache memory grows linearly with context length, while attention computatio

Agents

Synthesis Without Training: An Inference-Only Pipeline for Tabular, Temporal, and Relational Synthetic Data

arXiv:2609.38414v1 Announce Type: new Abstract: Synthetic data generation is dominated by the fit-then-sample paradigm: a generative model is trained on a private dataset and then sampled from. Despit

agentsarxiv-cs-lg

Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS

arXiv:2609.36307v1 Announce Type: new Abstract: Partial Least Squares (PLS) regression extracts a few outcome-aligned directions in a high-dimensional X and is widely used across applied science, but

GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs

arXiv:2609.35882v1 Announce Type: cross Abstract: Genome foundation models are most useful where sequences are generated, yet the largest models need datacenter accelerators and a place to send privat

Introducing Anthropic models on Amazon Bedrock for in-region inference in Seoul and Singapore

model-releasesaws-ml-blog

Amazon Bedrock now supports Anthropic's Claude Opus 5 and Claude Sonnet 5 with in-region inference in Seoul, and Claude Sonnet 5 in Singapore. If you have local data processing requirements in South K

LOCO-AdaMP: Built-in LOCO Inference for Adaptive Minipatch Ensembles with Enhanced Prediction

arXiv:2609.36396v1 Announce Type: cross Abstract: As black-box machine learning models become increasingly common, extracting interpretations with uncertainty quantification has become a critical chal

Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks

arXiv:2609.36471v1 Announce Type: cross Abstract: World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds

safetyarxiv-cs-ai

You Cannot Pick a Provider From the Price List: Market-Aware Routing for Open-Weight LLM Inference

arXiv:2609.37902v1 Announce Type: new Abstract: Existing LLM routers choose among models using static per-model costs. We show that open-weight inference markets introduce a second, largely ignored de

safetyarxiv-cs-ai

Hardware

Accelerator Choice Is Not Enough: AlphaFold2 Inference on Cloud TPUs

arXiv:2609.34818v1 Announce Type: cross Abstract: AlphaFold2 is written in JAX, so the same inference code compiles and runs unchanged on CPUs, GPUs and Google Cloud TPUs. That portability makes the a

hardwarearxiv-cs-lg

DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference

arXiv:2609.32204v1 Announce Type: cross Abstract: Secure Transformer inference protects sensitive inputs but incurs substantial cryptographic overhead, with nonlinear operations such as Softmax and Ge

Distance-KV: Exploiting Relative Distance for Efficient Long-Context Inference

arXiv:2609.32663v1 Announce Type: new Abstract: The memory usage and decoding latency of LLM inference grow rapidly with context length. To reduce these costs, key-value (KV) cache compression methods

Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference

arXiv:2601.22132v2 Announce Type: replace Abstract: Large Language Models (LLMs) deliver state-of-the-art performance on complex reasoning tasks, but their inference costs limit deployment at scale. S

Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It

arXiv:2609.32444v1 Announce Type: new Abstract: We study training-inference mismatch in reinforcement learning with verifiable rewards (RLVR) for large language models, where rollouts are sampled by a

Hardware

Spexis: Speculative Lookahead Scheduling for LLM Inference

arXiv:2609.34370v1 Announce Type: new Abstract: Spexis is a multi-GPU LLM inference framework that improves the efficiency of pipeline and tensor parallelism through speculative parallelism. Rather th

hardwarearxiv-cs-lg

Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

arXiv:2609.34130v1 Announce Type: new Abstract: This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work i

Mentored Decoding: Faster Inference meets Boosting

arXiv:2609.30474v1 Announce Type: new Abstract: Speculative decoding is a successful technique speeding up inference of a target autoregressive language model via a fast drafter model. Lossy speculati

28 Sept 2026

Tutorials

Direct Message Approximation (DMA): A Consistency-Based Framework for Tractable Approximate Inference on Factor Graphs

arXiv:2609.29466v1 Announce Type: cross Abstract: Approximate message passing on factor graphs underlies two dominant families of probabilistic inference algorithms: expectation propagation (EP) and v

tutorialsarxiv-cs-ai

Exploiting answer-invariant redundancies in satellite imagery for efficient VLM inference on edge

arXiv:2609.29029v1 Announce Type: new Abstract: Onboard vision-language models could enable satellites to answer queries directly, but exhaustive tiled inference over high-resolution imagery is slow a

researcharxiv-cs-cv

FB-GDM: Fully-Bayesian Guided Diffusion Models for High-Dimensional Linear Inverse Problems via Unsupervised Variational Inference

arXiv:2609.29216v1 Announce Type: cross Abstract: Diffusion models are powerful priors for linear inverse problems, but the reference guidance methods, Diffusion Posterior Sampling (DPS) and Pseudoinv

researcharxiv-cs-ai

Spectral-Guided Diffusion: Accelerating Inference via Static Spectral Layer Scheduling

arXiv:2609.29505v1 Announce Type: new Abstract: Diffusion inference repeatedly evaluates the same large network. We ask whether pretrained weights alone can identify residual branches that need not be

UO-FIE: Combining Exact-Label Supervision with Graded Utility for Factivity Inference

arXiv:2609.28605v1 Announce Type: cross Abstract: The Factivity Inference Evaluation 2026 (FIE2026) classifies Chinese context-hypothesis pairs into nine ordered factivity intervals. Its evaluation me

Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models

arXiv:2609.27373v1 Announce Type: new Abstract: Recurrent language models repeatedly apply shared network blocks to refine latent representations, but standard inference recomputes global attention at

researcharxiv-cs-cl

24 Sept 2026

SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference

arXiv:2609.28064v1 Announce Type: new Abstract: Driving world-action models improve planning by coupling multimodal reasoning with future prediction, but their growing inference cost increasingly conf

safetyarxiv-cs-ai

24 Sept 2026

CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference

arXiv:2609.26300v1 Announce Type: new Abstract: Despite their strong performance, large language models (LLMs) are bottlenecked by KV cache memory traffic during long-context inference. Sparse attenti

23 Sept 2026

Applications

Offloaded inference for real-world physical AI robotics

applicationsmicrosoft-research

Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and

23 Sept 2026

Hardware

TopoCompress: Topology Aware Token Compression Algorithm for Distributed Edge MoE Inference

arXiv:2609.26061v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) models improve capacity with moderate overhead by sparsely activating experts per token. However, deploying MoE across resour

hardwarearxiv-cs-cl

23 Sept 2026

Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models

arXiv:2609.24422v1 Announce Type: new Abstract: Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs). Bayesian i

SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching

arXiv:2602.24208v2 Announce Type: replace Abstract: Diffusion models achieve state-of-the-art video generation quality, but their inference remains expensive due to the large number of sequential deno

safetyarxiv-cs-cv

Local Ai

Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models

arXiv:2602.01654v2 Announce Type: replace Abstract: Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing a

local-aiarxiv-cs-cl

vla.simd: Efficient CPU Inference for Language-Conditioned Manipulation

arXiv:2609.24274v1 Announce Type: new Abstract: Deploying language-conditioned manipulation without a dedicated GPU requires efficient inference and action chunks that cover the delay between policy q

safetyarxiv-cs-ro

Local Ai

Qwen3.8-27B, up to 144 tok/sec on M5 Max. Let that sink in. ⚡️✨🚀 Excited to partner with @inco_ai to bring you their Splash inference engin…

Qwen3.8-27B, up to 144 tok/sec on M5 Max. Let that sink in. ⚡️✨🚀 Excited to partner with @inco_ai to bring you their Splash inference engine in LM Studio on day 0! Get it running now: https://lmstudio

local-ailm-studio--x

19 Sept 2026

Compressed Active Subspaces for Scalable Bayesian Inference

arXiv:2609.19539v1 Announce Type: cross Abstract: Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference

18 Sept 2026

Agents

How a global fintech scaled coding agent traffic with Dedicated Model Inference

A global fintech uses Together’s Dedicated Model Inference (DMI) to run its GLM‑5.2 coding assistant, handling spike‑heavy traffic that peaks during engineering hours and cannot be met by static capac

agentstogether-ai-blog

18 Sept 2026

Adaptive hybrid coupling with operator inference, the overlapping Schwarz alternating method and reinforcement learning

arXiv:2609.17837v1 Announce Type: cross Abstract: Hybrid domain decomposition methods provide a flexible framework for coupling full order models (FOMs) and reduced order models (ROMs), but typically