Knowledge catalogue
arXiv:2610.14089v1 Announce Type: cross Abstract: Smart glasses are a promising platform for vision-language assistants, but private split inference remains expensive: protecting visual representation
arXiv:2610.17689v1 Announce Type: new Abstract: Serverless inference is emerging as an efficient deployment model for Large Language Models (LLMs) due to its ability to elastically scale resources, im
Learn how to run OpenAI GPT-6 Sol, Luna, and Astra on Amazon Bedrock through global cross-Region inference from Singapore, Jakarta, Thailand, Malaysia, and Taipei. This post covers the Converse and Op
arXiv:2610.13132v1 Announce Type: new Abstract: LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment. However, the process raises seve
arXiv:2610.12076v1 Announce Type: new Abstract: Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the
Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-
arXiv:2508.19458v2 Announce Type: replace Abstract: A membership-inference attack gets the output of a learning algorithm, and a target individual, and tries to determine whether this individual is a
arXiv:2609.39342v1 Announce Type: cross Abstract: Intrinsic motivation plays a central role in adaptive and goal-directed behavior by conferring agents reward-independent objectives and biases useful
arXiv:2609.38830v1 Announce Type: new Abstract: Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior
arXiv:2609.36950v1 Announce Type: new Abstract: Simulation-based inference is challenging when many heterogeneous observations must be composed, hierarchical latent structure must be preserved, and th
arXiv:2609.34995v1 Announce Type: cross Abstract: Models of quantum systems faithfully map system parameters to observations, but the inverse problem of parameter inference from measurement data prese
The rise of AI-native cloud provider CoreWeave Inc. is part of the greater story emerging around operationalizing AI. As enterprises shift from training to inference, neoclouds such as CoreWeave are p
Reliable, fast inference is essential. As Ted shared at Apsara Conference 2026, Together makes it easy for teams to get access to frontier open models without needing to manage any of the infrastructu
arXiv:2609.23774v1 Announce Type: new Abstract: Probabilistic inference is generally only tractable in low-treewidth graphical models, limiting its effective applicability in high-treewidth settings.
The future of artificial intelligence inference isn’t premium; it’s ubiquitous and commoditized. That may sound odd coming from someone in the AI semiconductor business. Conventional wisdom suggests c
Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (i
Live now: our Inference Engineering Track from AI Engineer World's Fair 2026. A benchmark tool told to run 200 queries a second that ran 38 and reported 200. A model that answered one prompt in a thou
Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cuttin
arXiv:2604.05012v2 Announce Type: replace-cross Abstract: Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value
arXiv:2609.16161v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown impressive capabilities across a range of natural language processing tasks, and LLM inference has emerged as a
Latest version of the benchmark introduces two new tests for emerging AI deployment patterns, including Agentic Inference The post MLCommons Sets Participation Record with New MLPerf Inference v6.1 Be
MLPerf Inference v6.1 introduced the first agentic workload run on datacenter hardware and evaluated a model with more than one trillion parameters (Kimi K2), setting a new benchmark for scale. The te
arXiv:2609.15847v1 Announce Type: cross Abstract: This paper proposes FREDI (Fair Resource Allocation for Edge Dual-Threshold Inference), a secure wireless edge-intelligence framework for event-trigge
arXiv:2609.15039v1 Announce Type: cross Abstract: User prompts provided to large language models (LLMs) may contain sensitive or private information that can be misused by remotely deployed models, su
arXiv:2606.23346v2 Announce Type: replace-cross Abstract: We perform a realistic KiDS-Legacy mock analysis with field-level neural compression and simulation-based inference using just 60 N-body simul
arXiv:2609.09628v1 Announce Type: cross Abstract: Inferring speaker relationships from spoken conversations is an important step towards socially aware speech understanding. However, this task remains
arXiv:2608.28846v1 Announce Type: cross Abstract: Layer-skipping methods for efficient LLM inference decide, at some granularity, which transformer layers to execute for a given input. We present a ri
arXiv:2608.23841v1 Announce Type: cross Abstract: Single-token autoregressive decode on CPUs is bound by memory bandwidth, not arithmetic: a modern CPU sustains roughly 1 TFLOP/s of compute but only a
Since announcing Jalapeño, our first custom inference chip, we’ve been testing it and the system around it. The results show a major advance: more intelligence from every watt and faster responses, de
arXiv:2608.19003v1 Announce Type: new Abstract: We ask whether internal representation statistics can provide useful example-level difficulty signals for adaptive inference in multilingual African NLP
Leveraging the frontier ecosystem We work with neolabs like @trajectorylabs and @EngramLab and inference + post-training infra providers like @baseten, @FireworksAI_HQ, and @appliedcompute to post-tra
arXiv:2608.04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute. The term 'test-time scaling,' however, now c
arXiv:2607.27990v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) communicate through sparse binary spike events rather than dense activations, enabling energy-efficient inference on ne
arXiv:2607.27704v1 Announce Type: cross Abstract: As large language models (LLMs) continue to demonstrate exceptional capabilities across various domains, the challenge of achieving energy-efficient a
arXiv:2607.24692v1 Announce Type: cross Abstract: Inference systems increasingly combine a fast path that returns predictions within the application's latency deadline together with a higher-accuracy
arXiv:2607.23771v1 Announce Type: new Abstract: Large language model (LLM) performance increasingly depends not only on the base model, but also on the inference-time controller used to organize reaso
arXiv:2607.12829v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) offer a theoretical advantage in parallel generation over standard autoregressive models. However, parallel ge
arXiv:2607.12095v1 Announce Type: cross Abstract: Sensor-rich data-driven applications increasingly use Bayesian approaches to infer latent states of dynamic systems from noisy sensor measurements and
arXiv:2607.02781v1 Announce Type: cross Abstract: Inference-time alignment steers a frozen language model during decoding using auxiliary reward signals, avoiding the cost of repeated weight updates.
arXiv:2606.27802v1 Announce Type: new Abstract: Hierarchical predictive coding provides an interpretable framework for perception as error-driven inference in multi-layer generative models, while spar
arXiv:2606.28024v1 Announce Type: new Abstract: Lifted inference exploits indistinguishabilities in probabilistic graphical models by using a representative for indistinguishable objects, thereby spee
arXiv:2606.09635v1 Announce Type: cross Abstract: Ensuring the reliability of Large Language Models (LLMs) under distribution drift requires inference-time adaptation. While inference-time alignment m
arXiv:2512.16349v2 Announce Type: replace-cross Abstract: We propose a collaborative edge-to-server inference framework for vision-language models (VLMs) that reduces communication cost while maintain
arXiv:2605.30898v1 Announce Type: new Abstract: In real-world deployments of large language models (LLMs), balancing inference quality and computational cost has become a central challenge. Existing a
arXiv:2605.29467v1 Announce Type: cross Abstract: Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be pre
arXiv:2605.28224v1 Announce Type: new Abstract: Multi-trajectory inference for tool-use LLM agents - generating multiple reasoning attempts and selecting among them - benefits from transferring knowle
arXiv:2605.19317v1 Announce Type: cross Abstract: Inference-time scaling has emerged as a major approach for improving reasoning capabilities, and has been increasingly applied to diffusion models. Ho
arXiv:2605.19537v1 Announce Type: new Abstract: Progress in LLMs is increasingly measured through standardized benchmarks, where state-of-the-art improvements are often separated by fractions of a per
arXiv:2604.27368v1 Announce Type: new Abstract: Statistical inference in observational science typically relies on a fundamental assumption: as sample size increases and uncertainties decrease, the in
arXiv:2604.26334v1 Announce Type: cross Abstract: To usher in the next round of client AI innovation, there is an urgent need to enable efficient, lossless inference of high-accuracy large language mo
arXiv:2510.13763v2 Announce Type: replace-cross Abstract: Amortized simulator-based inference offers a powerful framework for tackling Bayesian inference in computational fields such as engineering or
arXiv:2604.16400v1 Announce Type: cross Abstract: As Large Language Models (LLMs) are increasingly adopted in edge intelligence to power domain-specific applications and personalized services, the qua
arXiv:2604.16714v1 Announce Type: new Abstract: Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance
arXiv:2602.11528v2 Announce Type: replace-cross Abstract: Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generate
arXiv:2610.15281v1 Announce Type: new Abstract: Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs. However, imp
arXiv:2610.14723v1 Announce Type: new Abstract: A method integrating policy gradient with fuzzy inference has been proposed. This method, hereafter referred to as the base model, has high flexibility
arXiv:2610.17591v1 Announce Type: new Abstract: Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation but challenging because material
I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-
arXiv:2610.07391v1 Announce Type: cross Abstract: We present a new perspective on the problem of likelihood approximation within the framework of simulation-based inference that promotes scalable and
arXiv:2610.08444v1 Announce Type: new Abstract: Vision-language-action (VLA) policies repeatedly invoke inference to control robots, making graphics processing unit (GPU) energy a recurring cost of ta