Search: “inference”

arXiv:2608.16221v1 Announce Type: new Abstract: Reliable gas source localization (GSL) is critical to safety in industrial and urban environments, yet remains challenging indoors because walls and obs

local-aiarxiv-cs-ro

18 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

QUMem: Personalized Memory for Query-Conditioned User-State Inference in LLM Agents

DGX agent

arXiv:2608.16168v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly use external memory systems to support personalization by drawing on long and evolving interaction hist

researcharxiv-cs-ai

18 Aug 2026

Generative Modeling with Bayesian Sample Inference

arXiv:2502.07580v4 Announce Type: replace Abstract: We present a novel view of diffusion-like generative modeling from the perspective of iterative Gaussian posterior inference. By treating the genera

17 Aug 2026

P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems

arXiv:2608.14094v1 Announce Type: cross Abstract: Cloud-local LLM inference systems have the potential to use the reasoning capability of large cloud models while protecting sensitive user data on per

17 Aug 2026

Small models, big real-world impact. Proud to see Qwen leading local inference in the State of Open Models. Enjoy the sunshine from Qwen.☀️ …

Small models, big real-world impact. Proud to see Qwen leading local inference in the State of Open Models. Enjoy the sunshine from Qwen.☀️ 😎 Appreciate your work! @huggingface The State of Open Model

model-releasesqwen--x

14 Aug 2026

A variational Bayes approach to inference for low-dimensional parameters in high-dimensional linear regression

model-releasesarxiv-cs-lg

arXiv:2406.12659v3 Announce Type: replace-cross Abstract: We propose a scalable variational Bayes method for statistical inference for a single or pre-specified low-dimensional subset of the coordinat

12 Aug 2026

Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference

model-releasesarxiv-cs-cl

arXiv:2608.10288v1 Announce Type: cross Abstract: The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bili

12 Aug 2026

Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling

arXiv:2608.10896v1 Announce Type: cross Abstract: Constant-stepsize temporal-difference (TD) learning is attractive for policy evaluation, but inference from a single Markov trajectory must account fo

12 Aug 2026

A Mean-Field Framework for Inference-Time Distributional Control of Diffusion Models

arXiv:2608.08770v1 Announce Type: cross Abstract: Diffusion models are increasingly used as controllable samplers, whose generations can be steered at inference time according to a chosen reward funct

Hardware

EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference

arXiv:2608.07964v1 Announce Type: cross Abstract: Load Balancing has emerged as a critical problem in expert-parallel distributed inference of Mixture-of-Experts (MoE) models. As routing distributions

hardwarearxiv-cs-ai

GeoRoute: Geometry-Aware Hybrid Inference for Traffic Future-Frame Prediction

model-releasesarxiv-cs-cv

arXiv:2608.09493v1 Announce Type: new Abstract: Long-horizon future-frame prediction is important for autonomous driving, traffic surveillance, and intelligent transportation systems, yet remains chal

Hardware

IBM and Together AI sign a $240M, multi-year deal to build an AI inference cluster on IBM Cloud, using Nvidia's HGX B300 systems, to support open-source models (Anhata Rooprai/Reuters)

Anhata Rooprai / Reuters: IBM and Together AI sign a 240M, multi-year deal to build an AI inference cluster on IBM Cloud, using Nvidia's HGX B300 systems, to support open-source models — IBM (IBM.N) a

hardwaretechmeme

Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference

arXiv:2608.08730v1 Announce Type: new Abstract: Large Language Models are deployed to multiple types of environments, from internet browsers to edge devices, and WebGPU serves as a modern cross-platfo

Nemotron 3.5 Lightning is available on Together AI through Dedicated Model Inference, giving teams reserved capacity and predictable perform…

model-releasestogether-ai--x

Nemotron 3.5 Lightning is available on Together AI through Dedicated Model Inference, giving teams reserved capacity and predictable performance for high-volume agent workloads. Start building: https:

GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes

arXiv:2608.06836v1 Announce Type: new Abstract: We study the problem of inserting new furniture into indoor scene images. Under masked single-view 2D image-plane conditioning, however, the physical sc

10 Aug 2026

Tools

Voyage AI models now run natively on Fireworks, the first and only dedicated inference platform @VoyageAI by @MongoDB has partnered with. Em…

Voyage AI models now run natively on Fireworks, the first and only dedicated inference platform @VoyageAI by @MongoDB has partnered with. Embed, retrieve, rerank, generate: your full retrieval pipelin

toolsfireworks-ai--x

10 Aug 2026

Local Ai

Building a zero-dependency C inference engine for BitNet (1.58-bit) - lessons from hitting 36 tok/s on a Xeon CPU

Over the past few months I have been building a CPU-first inference engine from scratch in pure C99 (no Python, no CUDA, no BLAS, just GCC and make). The focus has been running 1.58-bit ternary models

local-air-localllama

8 Aug 2026

A Unified Causal Inference Framework for the Desirability of Outcome Ranking Paradigm in Benefit-Risk Evaluation

arXiv:2608.05244v1 Announce Type: cross Abstract: We developed a unified covariate-adjusted causal inference framework for estimating the desirability of outcome ranking (DOOR) probability for benefit

7 Aug 2026

Tutorials

Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference

arXiv:2608.06025v1 Announce Type: new Abstract: In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, wher

tutorialsarxiv-cs-lg

7 Aug 2026

SCI-CLIP: Segment-Centric Inference with Reference Memory for Training-Free Open-Vocabulary Segmentation

arXiv:2608.05627v1 Announce Type: new Abstract: Training-free open-vocabulary segmentation remains limited by a missing inference abstraction. Frozen vision-language features are produced at patch lev

7 Aug 2026

Local Ai

Actual is a local inference stack optimized to let you utilize your personal compute. With its low CPU impact while inferencing and multi-pl…

Actual is a local inference stack optimized to let you utilize your personal compute. With its low CPU impact while inferencing and multi-platform portability, it’s a great pair for your local Hermes

local-ainous-research--x

Elbow-Based MoE Routing: A Training-Free Inference Time Plugin for Expert Selection

arXiv:2608.04401v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models enable model scaling while maintaining low inference-time compute by activating only a subset of experts per token. Howe

SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs

arXiv:2604.07396v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) inference on edge Neural Processing Units (NPUs) is fundamentally constrained by limited on-chip memory capacity. A

What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend

arXiv:2608.04714v1 Announce Type: cross Abstract: Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are co

Bridging Prediction and Attribution: Identifying Forward and Backward Causal Influence Ranges Using Assimilative Causal Inference

arXiv:2510.21889v2 Announce Type: replace-cross Abstract: Causal inference identifies cause-and-effect relationships between variables. While traditional approaches rely on data to reveal causal links

Tutorials

Divide-and-Conquer: Towards Generalizable Amortized Bayesian Inference for the Drift Diffusion Model

arXiv:2608.03566v1 Announce Type: cross Abstract: The drift diffusion model (DDM) is a cornerstone of cognitive decision-making research. Although numerous estimation methods exist, researchers contin

tutorialsarxiv-cs-lg

iFAN: Inference-Aware Learning for Plain Mask Transformers

arXiv:2608.03216v1 Announce Type: new Abstract: Query-based mask transformers assemble segmentation outputs through pixel-wise competition among query predictions of the final layer, yet this inferenc

researcharxiv-cs-cv

PRIVEE: Privacy-Preserving Vertical Federated Learning Against Feature Inference Attacks

arXiv:2512.12840v2 Announce Type: replace-cross Abstract: Vertical Federated Learning (VFL) enables collaborative model training across organizations that share common user samples but hold disjoint f

researcharxiv-cs-ai

SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference

arXiv:2608.03335v1 Announce Type: new Abstract: Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost, making inference prohibitive at video-token scales. Th

AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification

model-releasesarxiv-cs-cl

arXiv:2509.24560v2 Announce Type: replace Abstract: Extended Chain-of-Thought (CoT) reasoning has significantly bolstered the capabilities of medical large language models (LLMs). However, current mod

Inference-Time Policy Alignment for Fair Reinforcement Learning

arXiv:2608.00175v1 Announce Type: new Abstract: Deep reinforcement learning (RL) agents achieve strong performance by optimizing scalar reward functions. However, once deployed, the policies of these

Intention Inference Under Execution Noise: Separating Aleatoric and Epistemic Uncertainty in Social Dilemmas

arXiv:2608.02440v1 Announce Type: cross Abstract: In noisy social dilemmas, intended actions are stochastically corrupted before execution, so an observed defection may reflect hostile intent or actio

MedSAM2-Anatomy: Training-Free Inference-Time Optimization for Musculoskeletal Segmentation

arXiv:2608.00195v1 Announce Type: cross Abstract: High-resolution 3D segmentation of hip and shoulder anatomy from CT and MRI is essential for surgical planning, yet frozen segmentation models often f

Meganeura: Portable GPU Training and Inference through Vulkan and Metal

arXiv:2608.01563v1 Announce Type: new Abstract: Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries. Meganeura asks whether one compact native comp

TrimMoE A communication aware and adaptive depth framework for distributed edge inference

model-releasesarxiv-cs-cl

arXiv:2608.00573v1 Announce Type: cross Abstract: Serving Mixture-of-Experts (MoE) large language models across distributed edge servers is bottlenecked by the cross-server expert transmission. The ex

Hardware

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI https://www.latent…

The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI https://www.latent.space/p/inference-eng @Baseten @philipkiely and @waterloo_i

hardwareswyx--x

3 Aug 2026

GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference

model-releasesarxiv-cs-lg

arXiv:2607.27694v1 Announce Type: cross Abstract: Low-bit quantization is essential for efficient LLM inference, and both rotation and fine-grained group quantization have shown individual promise. Ho

31 Jul 2026

LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference

arXiv:2607.27952v1 Announce Type: new Abstract: Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge d

researcharxiv-cs-cv

31 Jul 2026

Applications

WIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning

arXiv:2607.28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs. Existing static structured pruning methods are hardware-friendly and can deliver

applicationsarxiv-cs-cl

31 Jul 2026

Applications

AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

arXiv:2607.25852v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best

applicationsarxiv-cs-cl

29 Jul 2026

Applications

Cerebras and AMD partner to build the world’s fastest disaggregated AI inference solution

Disaggregated AI inference is proving to be more than a complementary answer to the prefill and decode bottleneck slowing enterprise AI at scale, and Cerebras and AMD just announced a partnership to b

applicationssiliconangle

29 Jul 2026

Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

model-releasesarxiv-cs-lg

arXiv:2607.25018v1 Announce Type: new Abstract: Large language model (LLM) cascades reduce inference cost by routing easy queries to a small model and deferring hard queries to a larger one. Productio

29 Jul 2026

DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference

arXiv:2607.24434v1 Announce Type: cross Abstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, bu

28 Jul 2026

Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

arXiv:2607.23394v1 Announce Type: new Abstract: Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data

researcharxiv-cs-ai

28 Jul 2026

MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

arXiv:2607.22556v1 Announce Type: new Abstract: Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments. However

28 Jul 2026

Kimi K3 is now available for inference & training in @FireworksAI_HQ. Crazy how easy they make it to tune frontier open models like K3 using…

Kimi K3 is now available for inference & training in @FireworksAI_HQ. Crazy how easy they make it to tune frontier open models like K3 using LoRA adapters. Best time to figure out how to own your inte

model-releasesdair-ai--x

27 Jul 2026

Kimi K3 will be here on Monday, but how do you know you’re ready to scale it in production? With our latest inference platform updates you c…

model-releasestogether-ai--x

Kimi K3 will be here on Monday, but how do you know you’re ready to scale it in production? With our latest inference platform updates you can: 1/ Run shadow traffic to see how a new model performs on

26 Jul 2026

InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents

arXiv:2607.20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or n