Search: “inference”

arXiv:2609.12367v1 Announce Type: cross Abstract: Membership inference attacks (MIAs) are the standard tool for auditing the privacy risks of machine learning models. Given a query point, an MIA aims

arXiv:2609.11940v1 Announce Type: cross Abstract: The rapid diffusion of generative artificial intelligence raises privacy, latency, and performance concerns that motivate a shift toward 'local-first'

we broke down the full open model stack for agentic coding and how to swap each layer independently model → inference → gateway/router → harness → tools the stack stays flexible as better models, prov

arXiv:2608.25761v1 Announce Type: new Abstract: One common trade-off in the use of large language models involves reducing the size of the model while increasing the amount of computation at inference

arXiv:2608.23582v1 Announce Type: cross Abstract: We present the Transformer Accelerator (TFA), a synthesizable, parameterizable INT8 memory-to-memory engine for transformer inference. One time-multip

arXiv:2608.24063v1 Announce Type: cross Abstract: While Vision Large Language Models (VLLMs) have achieved remarkable success in multimodal reasoning, their long-context inference remains prohibitivel

arXiv:2608.18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during infere

arXiv:2608.14408v1 Announce Type: cross Abstract: We study online statistical inference for functionals of the return distribution under a fixed policy. The return distribution is estimated by nonpara

arXiv:2512.01031v2 Announce Type: replace-cross Abstract: Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks. However, these models are typically deplo

arXiv:2607.21702v1 Announce Type: new Abstract: Simulation-based inference (SBI) with machine learning is an increasingly important tool for solving inverse problems in science and engineering, includ

arXiv:2607.00501v1 Announce Type: cross Abstract: We present BaseRT, a native Metal inference runtime for large language models (LLMs) on Apple Silicon, and report the highest inference throughput on

arXiv:2605.29979v1 Announce Type: cross Abstract: The behavior of LLMs does not depend solely on the model itself. Components of the inference system, such as the inference engine, attention backend,

DigitalOcean introduced a unified batch inference solution designed to enable scalable and cost-efficient AI model inference workloads. The offering allows users to process multiple inference requests

arXiv:2603.22430v2 Announce Type: replace Abstract: Offline Reinforcement Learning (RL) learns optimal policies from fixed datasets, training a policy once and deploying it at inference time without f

arXiv:2605.05652v2 Announce Type: replace Abstract: Simulation-based inference (SBI) provides amortized Bayesian parameter inference from simulator-generated data without requiring explicit likelihood

arXiv:2604.15499v1 Announce Type: cross Abstract: Cryptographically secure neural network inference typically relies on secure computing techniques such as Secure Multi-Party Computation (MPC), enabli

arXiv:2302.05545v4 Announce Type: replace-cross Abstract: A novel form of inference attack in vertical federated learning (VFL) is proposed, where two parties collaborate in training a machine learnin

Enterprises are rapidly moving AI workloads from experimentation to production on Google Kubernetes Engine (GKE), using its scalability to serve powerful inference endpoints. However, as these models

Learn how to run OpenAI GPT-6 Sol, Luna, and Astra on Amazon Bedrock through global cross-Region inference from Singapore, Jakarta, Thailand, Malaysia, and Taipei. This post covers the Converse and Op

arXiv:2610.12076v1 Announce Type: new Abstract: Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the

arXiv:2609.39342v1 Announce Type: cross Abstract: Intrinsic motivation plays a central role in adaptive and goal-directed behavior by conferring agents reward-independent objectives and biases useful

arXiv:2609.38830v1 Announce Type: new Abstract: Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior

arXiv:2609.36950v1 Announce Type: new Abstract: Simulation-based inference is challenging when many heterogeneous observations must be composed, hierarchical latent structure must be preserved, and th

arXiv:2609.34995v1 Announce Type: cross Abstract: Models of quantum systems faithfully map system parameters to observations, but the inverse problem of parameter inference from measurement data prese

The rise of AI-native cloud provider CoreWeave Inc. is part of the greater story emerging around operationalizing AI. As enterprises shift from training to inference, neoclouds such as CoreWeave are p

Reliable, fast inference is essential. As Ted shared at Apsara Conference 2026, Together makes it easy for teams to get access to frontier open models without needing to manage any of the infrastructu

arXiv:2609.23774v1 Announce Type: new Abstract: Probabilistic inference is generally only tractable in low-treewidth graphical models, limiting its effective applicability in high-treewidth settings.

The future of artificial intelligence inference isn’t premium; it’s ubiquitous and commoditized. That may sound odd coming from someone in the AI semiconductor business. Conventional wisdom suggests c

Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (i

Live now: our Inference Engineering Track from AI Engineer World's Fair 2026. A benchmark tool told to run 200 queries a second that ran 38 and reported 200. A model that answered one prompt in a thou

Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cuttin

arXiv:2604.05012v2 Announce Type: replace-cross Abstract: Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value

arXiv:2609.16161v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown impressive capabilities across a range of natural language processing tasks, and LLM inference has emerged as a

Latest version of the benchmark introduces two new tests for emerging AI deployment patterns, including Agentic Inference The post MLCommons Sets Participation Record with New MLPerf Inference v6.1 Be

MLPerf Inference v6.1 introduced the first agentic workload run on datacenter hardware and evaluated a model with more than one trillion parameters (Kimi K2), setting a new benchmark for scale. The te

arXiv:2609.15847v1 Announce Type: cross Abstract: This paper proposes FREDI (Fair Resource Allocation for Edge Dual-Threshold Inference), a secure wireless edge-intelligence framework for event-trigge

arXiv:2609.15039v1 Announce Type: cross Abstract: User prompts provided to large language models (LLMs) may contain sensitive or private information that can be misused by remotely deployed models, su

arXiv:2606.23346v2 Announce Type: replace-cross Abstract: We perform a realistic KiDS-Legacy mock analysis with field-level neural compression and simulation-based inference using just 60 N-body simul

arXiv:2609.09628v1 Announce Type: cross Abstract: Inferring speaker relationships from spoken conversations is an important step towards socially aware speech understanding. However, this task remains

arXiv:2608.28846v1 Announce Type: cross Abstract: Layer-skipping methods for efficient LLM inference decide, at some granularity, which transformer layers to execute for a given input. We present a ri

arXiv:2608.23841v1 Announce Type: cross Abstract: Single-token autoregressive decode on CPUs is bound by memory bandwidth, not arithmetic: a modern CPU sustains roughly 1 TFLOP/s of compute but only a

Since announcing Jalapeño, our first custom inference chip, we’ve been testing it and the system around it. The results show a major advance: more intelligence from every watt and faster responses, de

arXiv:2608.19003v1 Announce Type: new Abstract: We ask whether internal representation statistics can provide useful example-level difficulty signals for adaptive inference in multilingual African NLP

Leveraging the frontier ecosystem We work with neolabs like @trajectorylabs and @EngramLab and inference + post-training infra providers like @baseten, @FireworksAI_HQ, and @appliedcompute to post-tra

arXiv:2608.04001v1 Announce Type: cross Abstract: Large language models can solve substantially harder reasoning problems with more inference-time compute. The term 'test-time scaling,' however, now c

arXiv:2607.27990v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) communicate through sparse binary spike events rather than dense activations, enabling energy-efficient inference on ne