Search: “inference”

Metropolis-Scale Resilient and Trustworthy Traffic Flow Inference Using Multi-Source Data

arXiv:2605.25004v1 Announce Type: cross Abstract: Inferring network-wide traffic states from sparse observations with high accuracy and trustworthy uncertainty quantification is essential for intellig

26 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

DGX agent

arXiv:2605.24391v1 Announce Type: cross Abstract: As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference. In 2022, the Open Com

local-aiarxiv-cs-ai

26 May 2026

Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment

arXiv:2605.25682v1 Announce Type: cross Abstract: Distributing Transformer inference across embedded edge devices can alleviate individual memory and compute constraints, yet practical benefits on rea

local-aiarxiv-cs-ai

26 May 2026

Scheduling LLM Inference with Uncertainty-Aware Output Length Predictions

arXiv:2604.00499v2 Announce Type: replace Abstract: To schedule LLM inference, the extit{shortest job first} (SJF) principle is favorable by prioritizing requests with short output lengths to avoid he

26 May 2026

Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation

arXiv:2601.22367v2 Announce Type: replace-cross Abstract: Generalized Bayesian Inference (GBI) tempers a loss with a temperature eta > 0 to mitigate overconfidence and improve robustness under model m

25 May 2026

ModeSwitch-LLM: A Lightweight Phase-Aware Controller for Cross-Mode LLM Inference on a Single GPU

arXiv:2605.23057v1 Announce Type: cross Abstract: ModeSwitch-LLM is a lightweight request-boundary controller for improving single-GPU large language model inference efficiency by routing each request

25 May 2026

Understanding and Improving Communication Performance in Multi-node LLM Inference

model-releasesarxiv-cs-lg

arXiv:2511.09557v4 Announce Type: replace-cross Abstract: As large language models (LLMs) continue to grow in size, distributed inference has become increasingly important. Model-parallel strategies m

21 May 2026

Latent Action Reparameterization for Efficient Agent Inference

arXiv:2605.18597v1 Announce Type: new Abstract: Large language model (LLM) agents often rely on long sequences of low-level textual actions, resulting in large effective decision horizons and high inf

agentsarxiv-cs-ai

Lever: Speculative LLM Inference on Smartphones

arXiv:2605.16786v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly needed for interactive mobile applications, but high-quality models exceed the limited DRAM available on s

Principles of frugal inference and control

arXiv:2406.14427v4 Announce Type: replace Abstract: A central challenge for intelligent agents in an uncertain world is striking the right balance between utility maximization and resource use, not on

local-aiarxiv-cs-ai

Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures

arXiv:2605.17850v1 Announce Type: cross Abstract: iffusion-based generative models increasingly rely on inference-time guidance, adding a drift term or reweighting mixture of experts, to improve sampl

safetyarxiv-cs-cv

SNLP: Layer-Parallel Inference via Structured Newton Corrections

arXiv:2605.17842v1 Announce Type: new Abstract: Autoregressive language models execute Transformer layers sequentially, creating a latency bottleneck that is not removed by conventional tensor or pipe

Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents

arXiv:2504.07347v3 Announce Type: replace-cross Abstract: As demand for Large Language Models (LLMs) and AI agents grows rapidly, optimizing systems for efficient LLM inference becomes critical. While

agentsarxiv-cs-lg

Amortized Energy-Based Bayesian Inference

arXiv:2605.15407v1 Announce Type: cross Abstract: We consider amortized Bayesian inference for nonlinear inverse problems in settings where only samples from the joint distribution of parameters and o

18 May 2026

ContextFlow: Context-Aware Flow Matching For Trajectory Inference From Spatial Omics Data

arXiv:2510.02952v3 Announce Type: replace Abstract: Inferring trajectories from longitudinal spatially-resolved omics data is fundamental to understanding the dynamics of structural and functional tis

local-aiarxiv-cs-lg

15 May 2026

TRIO: Token Reduction via Inference-Objective Guidance for Efficient Vision-Language Models

arXiv:2602.04657v3 Announce Type: replace Abstract: Recently, reducing redundant visual tokens in vision-language models (VLMs) to accelerate VLM inference has emerged as a hot topic. However, most ex

local-aiarxiv-cs-cv

15 May 2026

MARLIN: Multi-Agent Game-Theoretic Reinforcement Learning for Sustainable LLM Inference in Cloud Datacenters

arXiv:2605.13496v1 Announce Type: cross Abstract: Large Language Models (LLMs) have become increasingly prevalent in cloud-based platforms, propelled by the introduction of AI-based consumer and enter

agentsarxiv-cs-lg

14 May 2026

Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

model-releasesarxiv-cs-ai

arXiv:2510.18245v3 Announce Type: replace-cross Abstract: Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model (LLM)

14 May 2026

Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inference

arXiv:2605.11581v1 Announce Type: new Abstract: When large language models (LLMs) serve real-time inference in commercial online advertising systems, end-to-end latency must be strictly bounded to the

13 May 2026

KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference

arXiv:2605.12471v1 Announce Type: cross Abstract: We introduce KV-Fold, a simple, training-free long-context inference protocol that treats the key-value (KV) cache as the accumulator in a left fold o

13 May 2026

Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models

arXiv:2605.11854v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive language models, offering stronger global awareness

researcharxiv-cs-cl

13 May 2026

Compute as Teacher: Turning Inference Compute Into Reference-Free Supervision

arXiv:2509.14234v3 Announce Type: replace Abstract: Where do learning signals come from when there is no ground truth in post-training? We show that inference compute itself can serve as supervision.

Fast-MIA: Efficient and Scalable Membership Inference for LLMs

arXiv:2510.23074v2 Announce Type: replace-cross Abstract: We propose Fast-MIA (https://github.com/Nikkei/fast-mia), a Python library for efficiently evaluating membership inference attacks (MIA) again

researcharxiv-cs-cl

Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks

arXiv:2605.09839v1 Announce Type: cross Abstract: We introduce the Free Energy Manifold (FEM), a score-trained conditional energy model specialized for inference in hybrid Bayesian networks with discr

Private Vertical Federated Inference for Time-Series

arXiv:2605.08343v1 Announce Type: new Abstract: Institutions may benefit from collaborative inference on time-series data. In settings where privacy is necessary, multi-party computation (MPC) is a st

Tutorials

PruneTIR: Inference-Time Tool Call Pruning for Effective yet Efficient Tool-Integrated Reasoning

arXiv:2605.09931v1 Announce Type: cross Abstract: Tool-integrated reasoning (TIR) enables large language models (LLMs) to enhance their capabilities by interacting with external tools, such as code in

tutorialsarxiv-cs-ai

Towards Universal Gene Regulatory Network Inference: Unlocking Generalizable Regulatory Knowledge in Single-cell Foundation Models

model-releasesarxiv-cs-ai

arXiv:2605.08128v1 Announce Type: cross Abstract: Gene Regulatory Network (GRN) inference is essential for understanding complex cellular mechanisms, rendered tractable through single-cell transcripto

Variational Inference for Levy Process-Driven SDEs via Neural Tilting

arXiv:2605.10934v1 Announce Type: cross Abstract: Modelling extreme events and heavy-tailed phenomena is central to building reliable predictive systems in domains such as finance, climate science, an

safetyarxiv-cs-ai

CGF-Softmax: A Cumulant-Based Softmax Reformulation for Efficient Inference under Homomorphic Encryption

arXiv:2602.01621v3 Announce Type: replace-cross Abstract: Homomorphic encryption (HE) is a prominent framework for privacy-preserving machine learning, enabling inference directly on encrypted data. H

Inference-Time Attribute Distribution Alignment for Unconditional Diffusion

arXiv:2605.07456v1 Announce Type: new Abstract: Inference-time controllable generation is essential for real-world applications of unconditional diffusion models. However, most existing techniques foc

It Just Takes Two: Scaling Amortized Inference to Large Sets

arXiv:2605.07972v1 Announce Type: cross Abstract: Neural posterior estimation has emerged as a powerful tool for amortized inference, with growing adoption across scientific and applied domains. In ma

Multi-Objective Constraint Inference using Inverse reinforcement learning

model-releasesarxiv-cs-ai

arXiv:2605.06951v1 Announce Type: new Abstract: Constraint inference is widely considered essential to align reinforcement learning agents with safety boundaries and operational guidelines by observin

Stochastic Transition-Map Distillation for Fast Probabilistic Inference

arXiv:2605.07661v1 Announce Type: cross Abstract: Diffusion models achieve strong generation quality, diversity, and distribution coverage, but their performance often comes with expensive inference.

researcharxiv-cs-cv

ContextPilot: Fast Long-Context Inference via Context Reuse

arXiv:2511.03475v4 Announce Type: replace Abstract: AI applications increasingly depend on long-context inference, where LLMs consume substantial context to support stronger reasoning. Common examples

agentsarxiv-cs-lg

7 May 2026

E-MIA: Exam-Style Black-Box Membership Inference Attacks against RAG Systems

arXiv:2605.00955v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) equips large language models (LLMs) with external evidence by retrieving documents at inference time, but it also

6 May 2026

Applications

From Mice to Trains: Amortized Bayesian Inference on Graph Data

arXiv:2601.02241v5 Announce Type: replace-cross Abstract: Graphs arise across diverse domains, from biology and chemistry to social and information networks, as well as in transportation and logistics

applicationsarxiv-cs-lg

5 May 2026

When Attention Collapses: Residual Evidence Modeling for Compositional Inference

arXiv:2605.02323v1 Announce Type: new Abstract: Compositional inference - the decomposition of observations into an unknown number of latent components - is central to perception and scientific data a

5 May 2026

Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference

arXiv:2605.00005v1 Announce Type: new Abstract: The increasing deployment of deep neural networks (DNNs) in cyber-physical systems (CPS) enhances perception fidelity, but imposes substantial computati

local-aiarxiv-cs-lg

4 May 2026

Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment

arXiv:2506.00166v2 Announce Type: replace-cross Abstract: Existing paradigms for ensuring AI safety, such as guardrail models and alignment training, often compromise either inference efficiency or de

safetyarxiv-cs-cl

4 May 2026

A Novel Computational Framework for Causal Inference: Tree-Based Discretization with ILP-Based Matching

arXiv:2604.27307v1 Announce Type: cross Abstract: Causal inference is essential for data-driven decision-making, as it aims to uncover causal relationships from observational data. However, identifyin

1 May 2026

Distributional Stability of Tangent-Linearized Gaussian Inference on Smooth Manifolds

arXiv:2602.19179v2 Announce Type: replace Abstract: Gaussian inference on smooth manifolds is central to robotics, but exact marginalization and conditioning are generally non-Gaussian and geometry-de

local-aiarxiv-cs-ro

30 Apr 2026

Graph Property Inference in Small Language Models: Effects of Representation and Reasoning Strategy

arXiv:2603.06635v2 Announce Type: replace Abstract: Recent progress in language modeling has expanded the range of tasks that can be approached through natural language interfaces, including problems

30 Apr 2026

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

arXiv:2604.23904v1 Announce Type: cross Abstract: Synthetic data offers a promising tool for privacy-preserving data release, augmentation, and simulation, but its use in causal inference requires pre

Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inference

arXiv:2503.10666v4 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have become widely used across various domains spanning search engines, code generation, and text creation. Howev

Guided Speculative Inference for Efficient Test-Time Alignment of LLMs

arXiv:2506.04118v3 Announce Type: replace Abstract: We propose Guided Speculative Inference (GSI), a novel algorithm for efficient reward-guided decoding in large language models. GSI combines soft be

Hardware

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference

arXiv:2505.02922v3 Announce Type: replace Abstract: Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing GPU memory and band

hardwarearxiv-cs-lg

The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage

arXiv:2508.09603v2 Announce Type: replace Abstract: Membership inference attacks serves as useful tool for fair use of language models, such as detecting potential copyright infringement and auditing

Hardware

SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference