Search: “inference”

Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI

model-releasesarxiv-cs-cl

arXiv:2608.19003v1 Announce Type: new Abstract: We ask whether internal representation statistics can provide useful example-level difficulty signals for adaptive inference in multilingual African NLP

20 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Leveraging the frontier ecosystem We work with neolabs like @trajectorylabs and @EngramLab and inference + post-training infra providers lik…

DGX agent

Leveraging the frontier ecosystem We work with neolabs like @trajectorylabs and @EngramLab and inference + post-training infra providers like @baseten, @FireworksAI_HQ, and @appliedcompute to post-tra

applicationsfireworks-ai--x

11 Aug 2026

Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

arXiv:2608.04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute. The term 'test-time scaling,' however, now c

5 Aug 2026

Driving up Inference Energy on SNNs: Per-Sample and Universal Sponge Attacks

arXiv:2607.27990v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) communicate through sparse binary spike events rather than dense activations, enabling energy-efficient inference on ne

31 Jul 2026

LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference

arXiv:2607.27704v1 Announce Type: cross Abstract: As large language models (LLMs) continue to demonstrate exceptional capabilities across various domains, the challenge of achieving energy-efficient a

local-aiarxiv-cs-lg

31 Jul 2026

Denial of Deadline: Network-Driven Accuracy Collapse in Distributed Inference Pipelines

arXiv:2607.24692v1 Announce Type: cross Abstract: Inference systems increasingly combine a fast path that returns predictions within the application's latency deadline together with a higher-accuracy

28 Jul 2026

Training Language Models to Cooperate with Inference-Time Controllers

arXiv:2607.23771v1 Announce Type: new Abstract: Large language model (LLM) performance increasingly depends not only on the base model, but also on the inference-time controller used to organize reaso

28 Jul 2026

Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques

arXiv:2607.12829v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) offer a theoretical advantage in parallel generation over standard autoregressive models. However, parallel ge

15 Jul 2026

Dynamic Online Processor-Native Inference for State Estimation

arXiv:2607.12095v1 Announce Type: cross Abstract: Sensor-rich data-driven applications increasingly use Bayesian approaches to infer latent states of dynamic systems from noisy sensor measurements and

15 Jul 2026

Safe Inference-Time Alignment via Lagrangian Reward Augmentation

arXiv:2607.02781v1 Announce Type: cross Abstract: Inference-time alignment steers a frozen language model during decoding using auxiliary reward signals, avoiding the cost of repeated weight updates.

safetyarxiv-cs-ai

7 Jul 2026

Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

arXiv:2606.27802v1 Announce Type: new Abstract: Hierarchical predictive coding provides an interpretable framework for perception as error-driven inference in multi-layer generative models, while spar

29 Jun 2026

Lifted Causal Inference

arXiv:2606.28024v1 Announce Type: new Abstract: Lifted inference exploits indistinguishabilities in probabilistic graphical models by using a representative for indistinguishable objects, thereby spee

29 Jun 2026

Gradient-Guided Reward Optimization for Inference-time Alignment

arXiv:2606.09635v1 Announce Type: cross Abstract: Ensuring the reliability of Large Language Models (LLMs) under distribution drift requires inference-time adaptation. While inference-time alignment m

safetyarxiv-cs-lg

10 Jun 2026

Collaborative Edge-to-Server Inference for Vision-Language Models

arXiv:2512.16349v2 Announce Type: replace-cross Abstract: We propose a collaborative edge-to-server inference framework for vision-language models (VLMs) that reduces communication cost while maintain

9 Jun 2026

Applications

UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling

arXiv:2605.30898v1 Announce Type: new Abstract: In real-world deployments of large language models (LLMs), balancing inference quality and computational cost has become a central challenge. Existing a

applicationsarxiv-cs-ai

1 Jun 2026

Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference

model-releasesarxiv-cs-ai

arXiv:2605.29467v1 Announce Type: cross Abstract: Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be pre

29 May 2026

When Does Memory Help Multi-Trajectory Inference for Tool-Use LLM Agents?

arXiv:2605.28224v1 Announce Type: new Abstract: Multi-trajectory inference for tool-use LLM agents - generating multiple reasoning attempts and selecting among them - benefits from transferring knowle

28 May 2026

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement

arXiv:2605.19317v1 Announce Type: cross Abstract: Inference-time scaling has emerged as a major approach for improving reasoning capabilities, and has been increasingly applied to diffusion models. Ho

20 May 2026

The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility

arXiv:2605.19537v1 Announce Type: new Abstract: Progress in LLMs is increasingly measured through standardized benchmarks, where state-of-the-art improvements are often separated by fractions of a per

20 May 2026

Stable but Wrong: An Inference Limit in Galactic Archaeology

arXiv:2604.27368v1 Announce Type: new Abstract: Statistical inference in observational science typically relies on a fundamental assumption: as sample size increases and uncertainties decrease, the in

1 May 2026

Efficient, VRAM-Constrained xLM Inference on Clients

arXiv:2604.26334v1 Announce Type: cross Abstract: To usher in the next round of client AI innovation, there is an urgent need to enable efficient, lossless inference of high-accuracy large language mo

30 Apr 2026

PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference

arXiv:2510.13763v2 Announce Type: replace-cross Abstract: Amortized simulator-based inference offers a powerful framework for tackling Bayesian inference in computational fields such as engineering or

22 Apr 2026

CoLLM: A Unified Framework for Co-execution of LLMs Federated Fine-tuning and Inference

arXiv:2604.16400v1 Announce Type: cross Abstract: As Large Language Models (LLMs) are increasingly adopted in edge intelligence to power domain-specific applications and personalized services, the qua

21 Apr 2026

Tutorials

How to Approximate Inference with Subtractive Mixture Models

arXiv:2604.16714v1 Announce Type: new Abstract: Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance

tutorialsarxiv-cs-lg

21 Apr 2026

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

arXiv:2602.11528v2 Announce Type: replace-cross Abstract: Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generate

researcharxiv-cs-cl

21 Apr 2026

Hardware

Stable Token-level Scheduling for Serverless LLM Inference

arXiv:2610.17689v1 Announce Type: new Abstract: Serverless inference is emerging as an efficient deployment model for Large Language Models (LLMs) due to its ability to elastically scale resources, im

hardwarearxiv-cs-ai

15 Oct 2026

Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

arXiv:2610.15281v1 Announce Type: new Abstract: Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs. However, imp

14 Oct 2026

Integrating AIRL-Based Reward Estimation into a Control Method Using Fuzzy Inference and Policy Gradient

arXiv:2610.14723v1 Announce Type: new Abstract: A method integrating policy gradient with fuzzy inference has been proposed. This method, hereafter referred to as the base model, has high flexibility

safetyarxiv-cs-lg

14 Oct 2026

Physical-Field-Bridged Material Parameter Inference from Multi-View Deformation Videos

model-releasesarxiv-cs-cv

arXiv:2610.17591v1 Announce Type: new Abstract: Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation but challenging because material

14 Oct 2026

DP-MIND: Differentially private memory-guided Long context Inference

arXiv:2610.13132v1 Announce Type: new Abstract: LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment. However, the process raises seve

12 Oct 2026

GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference

Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-

local-air-localllama

9 Oct 2026

The Sample Complexity of Membership Inference and Privacy Auditing

arXiv:2508.19458v2 Announce Type: replace Abstract: A membership-inference attack gets the output of a learning algorithm, and a target individual, and tries to determine whether this individual is a

9 Oct 2026

Running an LLM-driven town with 800+ persistent agents: concurrency, context caching, and inference costs

model-releasesr-localllama

I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-

8 Oct 2026

A perspective note on likelihood approximation and inference for complex simulation models using a chain of aggregated normalizing flows

arXiv:2610.07391v1 Announce Type: cross Abstract: We present a new perspective on the problem of likelihood approximation within the framework of simulation-based inference that promotes scalable and

ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference

model-releasesarxiv-cs-ro

arXiv:2610.08444v1 Announce Type: new Abstract: Vision-language-action (VLA) policies repeatedly invoke inference to control robots, making graphics processing unit (GPU) energy a recurring cost of ta

DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching

arXiv:2610.08268v1 Announce Type: cross Abstract: Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models

Probabilistic Counterfactual Inference for Discrete Outcomes in Gaussian-Process Causal Models

arXiv:2610.08689v1 Announce Type: new Abstract: Counterfactual inference in Gaussian-process structural causal models (GP-SCMs) has been developed primarily for continuous endogenous variables, limiti

Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory

arXiv:2610.07311v1 Announce Type: new Abstract: Agentic memory allows LLM agents to reuse past experience, yet retrieved memories can also distort inference even when they are benign, correctly stored

Hardware

Expanding our enterprise inference capacity with IBM Cloud and NVIDIA

Together AI has partnered with IBM and NVIDIA to launch the first dedicated, large‑scale AI inference cluster on IBM Cloud, consisting of NVIDIA B300 GPUs and Spectrum‑X Ethernet networking engineered

hardwaretogether-ai-blog

6 Oct 2026

Inference Auctions

arXiv:2609.40070v1 Announce Type: cross Abstract: When inference demand exceeds available compute capacity, model providers must decide which requests should be served first. Users have different tole

1 Oct 2026

Mitigating Representation Gaps in Amortized Bayesian Inference with Auxiliary Supervision

arXiv:2609.39525v1 Announce Type: cross Abstract: Casting Bayesian inference as a neural network optimization problem targeting an amortized posterior is attractive, as it extends to otherwise intract

local-aiarxiv-cs-lg

1 Oct 2026

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-c

local-aiarxiv-cs-lg

30 Sept 2026

FLIP: Final Layer Inference-Time Probing for Vision-Language Models

arXiv:2609.30993v1 Announce Type: cross Abstract: We present FLIP, a final-layer inference-time probe for testing whether a logit-facing intervention site in an open-weight vision-language model (VLM)

safetyarxiv-cs-ai

28 Sept 2026

HybridInfer: Thermal-Aware Reinforcement-Learning Tier Routing for On-Device, Edge, and Cloud LLM Inference

arXiv:2609.30270v1 Announce Type: new Abstract: On-device inference with small language models keeps user data local, works offline, and incurs no per-query cost, so the on-device tier is preferred wh

28 Sept 2026

New on Dedicated Model Inference: canary rollouts. Upgrade the model behind a live endpoint without downtime. Traffic moves from your curren…

model-releasestogether-ai--x

New on Dedicated Model Inference: canary rollouts. Upgrade the model behind a live endpoint without downtime. Traffic moves from your current deployment to the new checkpoint in gated steps (default 5

22 Sept 2026

Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

model-releasesarxiv-cs-cl

arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather

21 Sept 2026

Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape

arXiv:2609.20614v1 Announce Type: cross Abstract: Frontier AI models are rapidly gaining the ability to exploit vulnerabilities in complex pieces of software. The risk is not theoretical, as evidenced

18 Sept 2026

Structural Inference under Hidden Agents

arXiv:2609.18045v1 Announce Type: new Abstract: Recovering latent interaction structures from multi-agent dynamics is important for understanding and predicting interacting systems. Trajectory-based s