Search: “inference”

arXiv:2609.16898v1 Announce Type: cross Abstract: Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a

arXiv:2609.13592v1 Announce Type: cross Abstract: GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-b

For years, the JVM has watched the AI revolution from the bench. Every model, AI framework, every breakthrough, built with/for Python. jinfer is an inference engine built for the JVM from first princi

arXiv:2609.14507v1 Announce Type: cross Abstract: Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV s

arXiv:2609.15177v1 Announce Type: new Abstract: Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when paral

arXiv:2606.21869v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different

arXiv:2609.12923v1 Announce Type: cross Abstract: A single SM utilization percentage can make an LLM inference workload look compute-saturated while hiding how much useful work is being done. The prob

arXiv:2512.18225v2 Announce Type: replace Abstract: This paper presents an applied AI pipeline for real-time geolocation from noisy microblog streams, unifying statistical hashtag segmentation, part-o

arXiv:2609.12137v1 Announce Type: new Abstract: Measuring the preferences of human users remains a fundamental challenge of AI alignment. Existing elicitation approaches struggle to efficiently discov

Bryan Shan / SemiAnalysis: Vera Rubin NVL72 inference tests show up to 7x better token throughput per MW vs. Blackwell on a 1.6T DeepSeek model, above Huang's 3x claim for 1T-3T LLMs — Jensen Sandbagg

arXiv:2609.09848v1 Announce Type: cross Abstract: Synthetic data releases are increasingly proposed in the literature as a means of sharing realistic data replicas in lieu of sensitive private dataset

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recover

arXiv:2609.08189v1 Announce Type: new Abstract: Dynamic layer routing reduces the inference cost of Large Language Models (LLMs) by learning to skip layers for individual tokens. Existing methods, how

NVIDIA’s BioNeMo Inference Runtime (BioIR) accelerates biomolecular structure‑prediction models on GPUs while preserving the familiar PyTorch workflow by providing an end‑to‑end pipeline that handles

arXiv:2609.10534v1 Announce Type: cross Abstract: We present a simple decomposition of a neural-network-based normalizing flow that naturally uncovers a pivotal statistic (or something close) in the p

arXiv:2609.07838v1 Announce Type: new Abstract: Connected robotics is an emerging 6G application where mobile robots follow natural-language instructions to manipulate physical objects. The Vision-Lan

arXiv:2609.07966v1 Announce Type: new Abstract: Key--value (KV) cache compression is an effective way to reduce the memory overhead of large language model (LLM) inference, particularly for long-conte

Nice paper to improve inference efficiency. It's been a while we haven't seen good work on efficiency. Here is why it matters: A long-running agent's workspace outgrows its context window long before

arXiv:2510.17450v2 Announce Type: replace Abstract: We develop an active inference route-planning method for the autonomous control of intelligent agents. The aim is to reconnoiter a geographical area

arXiv:2609.04490v1 Announce Type: new Abstract: Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized st

arXiv:2609.04024v1 Announce Type: new Abstract: Procedural instruction following is a basic requirement for controllable language-model systems, especially when generated trajectories are inspected or

arXiv:2609.02947v1 Announce Type: cross Abstract: Significant challenges remain in AI-driven educational systems in balancing privacy preservation with accurate cognitive diagnosis. To overcome this,

arXiv:2609.03797v1 Announce Type: new Abstract: Long-term human-AI interaction is difficult because the information that guides inference is updated implicitly by the model and is not directly inspect

arXiv:2609.01663v1 Announce Type: cross Abstract: Agentic AI systems are increasingly deployed to process sensitive data at inference time, such as healthcare records or financial documents assembled

Modelplane v0.3 adds Vultr Kubernetes Engine (VKE) as a fully‑managed inference cluster provider, allowing users to describe GPU clusters in YAML with region, node pool, instance class, and min/max sc

The job of doing inference is not the same for every model ou there, as it depends on the architecture and configurations and the available machines. Many times we would not have the right machines to

arXiv:2603.05575v2 Announce Type: replace-cross Abstract: We study prediction-powered conditional inference in the setting where labeled data are scarce, unlabeled covariates are abundant, and a black

arXiv:2608.28911v1 Announce Type: cross Abstract: The key-value (KV) cache is the dominant memory bottleneck of long-context large language model (LLM) inference, growing linearly with context length.

arXiv:2602.13551v3 Announce Type: replace Abstract: Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant L

A great deep dive from the @fal team on how they built H3 Max, combining post-training with a co-designed inference stack to improve prompt adherence, visual quality, and speed. We're proud to see Min

Artificial intelligence is changing what storage and data management platforms look like. In collaboration with Super Micro Computer Inc. and Solidigm, DataDirect Networks Inc. has introduced DDN Ente

arXiv:2603.05335v3 Announce Type: replace-cross Abstract: Modern predictive systems combine predictors, sequential monitors, prediction sets, and online strategies, each with a different certificate o

Benchmarking Retrieval-Augmented Generation End to End — from Building the Vector Database to Serving an Iterative, Multi-Hop Question-Answering Inference Pipeline The post Introducing the MLPerf End-

arXiv:2407.11473v2 Announce Type: replace Abstract: Maximum entropy inference and learning of graphical models are pivotal tasks in learning theory and optimization. This work extends algorithms for t

arXiv:2608.22824v1 Announce Type: new Abstract: Membership inference attacks expose whether individual records were used to train a model, yet existing attacks on diffusion models are largely heuristi

arXiv:2608.22915v1 Announce Type: cross Abstract: Inference-time pipelines often sample multiple outputs, filter them with a learned safety model, and return the proxy-feasible output with the highest

arXiv:2501.06926v5 Announce Type: replace-cross Abstract: Double reinforcement learning (DRL) provides efficient off-policy inference for policy values in nonparametric Markov decision processes (MDPs

arXiv:2608.21393v1 Announce Type: new Abstract: Running large language models inside enterprise environments has always bumped up against a practical wall: the data lives in one place, the AI horsepow

arXiv:2512.24407v2 Announce Type: replace Abstract: In many sequential decision-making problems, researchers observe actions but not the rewards that drive behavior, yet still wish to evaluate and com

arXiv:2608.20965v1 Announce Type: new Abstract: We define an atomic generation fact f=(u,tau,omega,z;rho), recording the origin, realized transformation, concrete occurrence, generated result and rela

arXiv:2608.18591v1 Announce Type: new Abstract: Uniformly allocating inference reasoning budgets to LLMs is expensive and prone to over-thinking penalties; especially in document tasks where visual la

arXiv:2608.17452v1 Announce Type: new Abstract: Machine learning methods predict many real-world systems with remarkable accuracy, but they are typically treated as black boxes that offer no insight i

arXiv:2607.09520v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understoo

arXiv:2608.16236v1 Announce Type: new Abstract: Collaborative inference (CI) splits a model between an edge device and a server, whereby the client computes an intermediate activation, transmits it, a

arXiv:2608.16085v1 Announce Type: cross Abstract: Capability development is routinely inferred from behavioural thresholds, from final checkpoints, or from what a decoder can read out of a hidden stat

arXiv:2608.16221v1 Announce Type: new Abstract: Reliable gas source localization (GSL) is critical to safety in industrial and urban environments, yet remains challenging indoors because walls and obs