Knowledge catalogue
arXiv:2606.29526v1 Announce Type: new Abstract: Reinforcement learning (RL) has gained growing attention in large language model (LLM) post-training, yet RL training remains fragile and can suffer fro
arXiv:2606.23001v1 Announce Type: cross Abstract: On-device LLM inference is increasingly attractive for privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs r
arXiv:2606.21712v1 Announce Type: cross Abstract: Batch inference has become a central mode of AI computation, yet existing inference engines still rely on execution models designed for interactive se
arXiv:2606.11207v1 Announce Type: new Abstract: We present SemantiClean, a modular framework for extracting structured semantic signals from e-commerce session data and driving pluggable inference tar
arXiv:2606.10023v1 Announce Type: cross Abstract: Accurate posterior estimation is central to scientific inference, as uncertainties determine what can be reliably learned from observational data. Whi
PSA: Just added a few thousand chips, including B200s and B300s to our Dedicated Model Inference (http://api.together.ai/endpoints). With Dedicated Model Inference, you can now on-click deploy our Bla
arXiv:2606.02982v1 Announce Type: cross Abstract: The rapid growth of large language model (LLM) inference services has increased the demand for efficient multi-tenant GPU scheduling. While modern inf
arXiv:2601.17130v2 Announce Type: replace Abstract: Graph neural networks (GNNs) are widely used for tasks such as node classification and link prediction, but their use in sensitive settings raises c
arXiv:2503.07154v3 Announce Type: replace-cross Abstract: Generative pre-training is often framed through a false dichotomy between autoregressive models for discrete signals and diffusion models for
arXiv:2410.19371v3 Announce Type: replace-cross Abstract: Differential privacy (DP) provides robust privacy guarantees for statistical inference, but this can lead to unreliable results and biases in
arXiv:2605.29249v1 Announce Type: cross Abstract: Many applications require statistically valid inference across many related tasks, while using only a handful of high-quality labels per hypothesis. I
DigitalOcean describes the architecture and engineering approach behind their Inference Router, a system designed to route and manage machine learning inference requests efficiently across distributed
arXiv:2601.14568v2 Announce Type: replace-cross Abstract: Existing video inference (VI) enhancement methods typically aim to improve performance by scaling up model sizes and employing sophisticated n
A peer-to-peer AI inference mesh is a decentralized network architecture where agents connect directly to discover peers and communicate through distributed protocols to share inference workloads . Su
arXiv:2605.11744v1 Announce Type: new Abstract: Transformer-based large language models face severe scalability challenges in long-context generation due to the computational and memory costs of full-
arXiv:2509.21892v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models typically fix the number of activated experts k at both training and inference. However, real-world deployment
arXiv:2605.03780v1 Announce Type: cross Abstract: Transformers are effective at inferring the latent task from context via two inference modes: recognizing a task seen during training, and adapting to
arXiv:2605.00116v1 Announce Type: new Abstract: In this article, we introduce ViLegalNLI, the first large-scale Vietnamese Natural Language Inference (NLI) dataset specifically constructed for the leg
arXiv:2604.28175v1 Announce Type: new Abstract: Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. How
arXiv:2604.21407v1 Announce Type: new Abstract: When approximating an intractable density via variational inference (VI) the variational family is typically chosen as a simple parametric family that v
arXiv:2601.20706v2 Announce Type: replace-cross Abstract: Diffusion-based LLMs (dLLMs) fundamentally depart from traditional autoregressive (AR) LLM inference: they leverage bidirectional attention, b
arXiv:2604.21231v1 Announce Type: cross Abstract: Efficient inference for on-device Large Language Models (LLMs) remains challenging due to limited hardware resources and the high cost of the prefill
arXiv:2604.21211v1 Announce Type: new Abstract: Current text anonymization evaluation relies on span-based metrics that fail to capture what an adversary could actually infer, and assumes a single dat
Today, Amazon SageMaker AI supports optimized generative AI inference recommendations. By delivering validated, optimal deployment configurations with performance metrics, Amazon SageMaker AI keeps yo
arXiv:2604.18780v1 Announce Type: new Abstract: Semi-Markov Conditional Random Fields (semi-CRFs) assign labels to segments of a sequence rather than to individual positions, enabling exact inference
We are excited to have @baseten as a day 0 launch partner for Kimi K2.6! Their inference stack brings KV-aware routing, NVFP4 on Blackwell, multi-modal hierarchical caching, and prefill-decode disaggr
DigitalOcean's Inference Cloud Memory Layer is a technical architecture component designed to optimize database performance by implementing an in-memory caching layer for faster data access and reduce
Speculative decoding is a technique used to accelerate the slow, sequential token generation (decode stage) in LLM inference. This method significantly reduces latency and improves hardware utilizatio
arXiv:2604.05546v2 Announce Type: replace Abstract: Large Vision-Language Models (LVLMs) enable sophisticated reasoning over images and videos, yet their inference is hindered by a systemic efficiency
arXiv:2604.08701v1 Announce Type: new Abstract: We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calib
New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys @lmsysorg and RadixArk @radixark, and taught by Richard Chen @richardczl, a Member of Technical
arXiv:2610.19607v1 Announce Type: cross Abstract: Rising privacy concerns around large language models have accelerated research in private Transformer inference (PTI). Synthesizing the empirical clai
arXiv:2610.20136v1 Announce Type: new Abstract: Vision-language models (VLMs) are moving onto edge devices, but little is known about how much energy they draw on single-board computers with NPU or GP
arXiv:2610.19505v1 Announce Type: new Abstract: This paper proposes the SKG (Symplectic-Kernel Gradient) reasoning engine for law-induced inference and induction. Laws are a priori; data provide insta
Distributed inference splits a large AI model into sub‑components that run concurrently on multiple compute nodes, allowing each node to process only part of the workload. This parallelism boosts thro
Microsoft Corp. today pulled back the curtains on Maia 300, its newest internally developed artificial intelligence accelerator. The chip is optimized for inference. Microsoft estimates that it can pr
Impressive by @river_ai_inc to beat @deepseek_ai rates - cache in particular is tough! Talented team there, full RL + inference stack should be real competition River Inference is live! We're serving
arXiv:2610.11486v1 Announce Type: cross Abstract: Sparse identification of nonlinear dynamics (SINDy) is a data-driven framework for discovering governing dynamics from time-series data by identifying
arXiv:2610.07094v1 Announce Type: cross Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test
arXiv:2610.07348v1 Announce Type: cross Abstract: Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints
Developers now have a much cheaper way to access cutting-edge artificial intelligence inference capabilities with the launch of Antseed today. It’s a new peer-to-peer marketplace that makes it possibl
arXiv:2610.05692v1 Announce Type: new Abstract: Learning from trajectory demonstrations offers a route to active-inference control of complex systems whose dynamics are difficult to model explicitly.
arXiv:2610.05305v1 Announce Type: cross Abstract: Large Language Model (LLM) inference has become the dominant workload in modern AI systems, requiring serving infrastructures to maximize throughput w
arXiv:2610.05869v1 Announce Type: cross Abstract: This paper studies online quantile regression for large-scale and streaming data using Stochastic SubGradient Descent (SSGD) with constant learning ra
arXiv:2601.17063v2 Announce Type: replace Abstract: Recently, Mixture-of-Experts (MoE) models have gained attention for efficiently scaling large language models. Although these models are extremely l
arXiv:2610.05350v1 Announce Type: new Abstract: Behavioral Foundation Models (BFMs) aim to solve a wide range of downstream tasks without test-time policy learning by inferring a task vector from the
Dr. Ian Cutress / More Than Moore: Q&A with OpenAI VP of Hardware Richard Ho on its Jalapeño inference chip co-designed with Broadcom, using internal OpenAI models to design the chip, and more — VP Ha
There seems to be a whole category of extremely narrow inference runtimes appearing: Strata, ninfer, DwarfStar, Splash, llamAmpere, gufo, etc. They deliberately give up the thing llama.cpp/vLLM are gr
arXiv:2610.00047v1 Announce Type: new Abstract: Diversity collapse in parallel chain-of-thought has motivated inference-time interventions built on a natural design: when a process reward model (PRM)
arXiv:2610.00586v1 Announce Type: cross Abstract: Activation memory, not compute, limits CNN inference on constrained hardware such as microcontrollers. Direct in-place convolution removes the dual-bu
arXiv:2609.40024v1 Announce Type: cross Abstract: We develop a general method for amortized Bayesian inference on multilevel models of arbitrary structure. Given a generative model specified as a dire
arXiv:2609.39801v1 Announce Type: new Abstract: Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LL
arXiv:2609.33114v1 Announce Type: cross Abstract: For decades, statistical inference has largely been developed one problem at a time. Given a scientific target, such as a treatment effect or a regres
arXiv:2609.32320v1 Announce Type: new Abstract: Hyperdimensional computing (HDC) is attractive for efficient and robust learning, but conventional inference still encodes every query independently, re
the inference platforms are doing hardcore work, post-training open weight models to solve customer problems: @FireworksAI_HQ ember-1 @fal H3 max more to come;) makes total sense considering their pos
arXiv:2602.05145v2 Announce Type: replace-cross Abstract: Speculative decoding can substantially accelerate LLM inference, but realizing its benefits in practice is challenging due to evolving workloa
arXiv:2609.25537v1 Announce Type: new Abstract: Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latenc
arXiv:2609.25484v1 Announce Type: new Abstract: Smart meter (SM) data provides fine-grained visibility into household energy consumption, but also exposes users to privacy risks. Inference attacks, kn
arXiv:2609.24754v1 Announce Type: new Abstract: We investigate next generation reservoir computing (NGRC) as a data-driven approach for inferring unseen components of dynamical systems. We compare NGR
arXiv:2601.20332v3 Announce Type: replace Abstract: Diffusion language models (DLMs) generate text through iterative denoising, but inference requires full-sequence attention at every iteration, resul