Knowledge catalogue
We're releasing the full slides for our 2 hr deepdive session from the AI Engineer World's Fair. We covered how we build inference engines to serve agentic workloads at trillion token production scale
arXiv:2607.00733v1 Announce Type: cross Abstract: Mechanistic modeling via ordinary differential equations (ODEs) provides interpretable descriptions of complex dynamics and enables inference of under
arXiv:2607.01018v1 Announce Type: cross Abstract: Reading order inference remains a critical bottleneck in the digitization of complex historical manuscripts, where pages contain multiple spatially in
arXiv:2606.30992v1 Announce Type: cross Abstract: Multicollinearity is a long lasting challenge in observational causal inference, especially in regressions -- highly correlated independent variables
arXiv:2606.31519v1 Announce Type: cross Abstract: Long-context Large Language Model inference is severely bottlenecked by the massive Key-Value (KV) cache, yet existing sparse attention methods often
arXiv:2606.29201v1 Announce Type: cross Abstract: Behavior-cloned policies often learn multiple behavior modes from demonstration datasets, including modes that are unsafe or otherwise undesired at de
Etched Inc., a developer of artificial intelligence inference chips, launched today with 800 million in funding. The startup raised the capital over multiple rounds. The most recent investment, which
arXiv:2606.28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, model
arXiv:2510.08762v2 Announce Type: replace Abstract: Causal inference in spatial domains faces two intertwined challenges: (1) unmeasured spatial factors, such as weather, air pollution, or mobility, t
Special new @latentspacepod live recording added to the Expo schedule on Thursday, with @swyx and @robertwachen alongside his Inference track appearance! congrats to Etched on a massive milestone! We'
arXiv:2606.29337v1 Announce Type: new Abstract: We summarize our submission to Sub-Challenge 1: W4A4 Quantization for Inference (HiF4 / MXFP4) of the ICME 2026 Low-Bit-width Large-Model Quantization C
arXiv:2606.23741v1 Announce Type: cross Abstract: Causal reasoning, which encompasses the discovery of causal structures and the inference of causal effects, is fundamental to data-driven decision mak
Speculative decoding is a technique for reducing inference latency in large language models by using a smaller, faster model to generate candidate tokens that are then verified by a larger model in pa
arXiv:2606.24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiti
arXiv:2606.21428v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) language models are often described as ideal for resource-constrained inference. Each token activates only a small su
arXiv:2606.24552v1 Announce Type: new Abstract: Simulator-in-the-loop optimization offers a promising inference-time mechanism for robot manipulation. It uses a physical simulator as a backend rollout
arXiv:2604.03345v2 Announce Type: replace Abstract: Kolmogorov-Arnold Networks (KANs) have recently emerged as a powerful architecture for various machine learning applications. However, their unique
arXiv:2606.23872v1 Announce Type: cross Abstract: As generative models increasingly produce samples that are indistinguishable from human-created content, it becomes difficult to determine whether a g
OpenAI and Broadcom announced a collaboration on a specialized inference chip designed to optimize the performance and efficiency of large language model deployments. The chip, referred to as 'Jalapen
Upbound Inc. today released Modelplane, a new open-source tool for managing artificial intelligence inference clusters. San Francisco-based Upbound is backed by $69 million from Alphabet Inc.’s GV fun
DFlash is an open source block diffusion model for speculative decoding that significantly accelerates LLM inference on NVIDIA Blackwell GPUs by drafting entire token blocks in parallel and verifying
arXiv:2606.23210v1 Announce Type: new Abstract: Embedded global navigation satellite system (GNSS) interference monitoring requires fast and memory-efficient inference to process large volumes of raw
arXiv:2602.15206v2 Announce Type: replace Abstract: Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it rem
arXiv:2606.22958v1 Announce Type: cross Abstract: Inference-time alignment of pretrained text-to-image models is typically performed along a single control axis, such as classifier-free guidance, atte
arXiv:2606.22639v1 Announce Type: cross Abstract: Contextual bandit algorithms have transformed modern experimentation by enabling real-time adaptation for personalized treatment. Yet these advantages
arXiv:2603.09555v2 Announce Type: replace-cross Abstract: High-throughput Mamba-2 inference is usually tied to fused CUDA and Triton kernels, limiting portability across accelerator backends. We show
arXiv:2509.20241v2 Announce Type: replace Abstract: As AI inference scales to billions of queries, estimates of per-query energy use are increasingly important for capacity planning, efficiency interv
arXiv:2606.10660v1 Announce Type: cross Abstract: AI inference services -- API subscriptions, enterprise chat tools, and SaaS products with embedded AI features -- fall unambiguously within Scope 3 Ca
arXiv:2606.09874v1 Announce Type: new Abstract: Reconstruction-based methods are widely used for time series anomaly detection, where models are trained to reconstruct subsequences, and anomalies are
arXiv:2606.10008v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) provide a naturally temporal machine-learning framework. Their neurons maintain an internal state and propagate informa
arXiv:2606.07665v1 Announce Type: cross Abstract: Transformer inference increasingly depends on specialized compiler and runtime support, but real model graphs still require semantic decisions about w
arXiv:2606.08129v1 Announce Type: new Abstract: Large language models (LLMs) differ in architecture, training data, and optimization procedures, yet they may still develop similar internal inference p
arXiv:2606.07677v1 Announce Type: cross Abstract: Electronic health records (EHR) pose large-scale multi-disease modeling problems in which many outcomes are rare and strongly influenced by shared ris
arXiv:2606.09551v1 Announce Type: cross Abstract: Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings. Recent GPU systems b
arXiv:2509.24762v3 Announce Type: replace Abstract: Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamic
arXiv:2603.23640v2 Announce Type: replace-cross Abstract: Deploying large language models on-device for always-on personal agents demands sustained inference from hardware tightly constrained in power
arXiv:2606.07141v1 Announce Type: cross Abstract: Language models trained for clinical disease inference are trained on patient data, which may include sensitive and private information, and data owne
arXiv:2509.23385v5 Announce Type: replace-cross Abstract: Simulation-based inference (SBI) is transforming experimental sciences by enabling parameter estimation in complex non-linear models from simu
arXiv:2606.04778v1 Announce Type: new Abstract: Safety-aligned Large Language Models (LLMs) remain vulnerable to interventions during inference that redirect generation toward harmful outputs. Recent
arXiv:2606.05004v1 Announce Type: cross Abstract: With the widespread deployment of public large language models (LLMs) such as ChatGPT, protecting user prompt privacy has become an increasingly criti
A Together AI representative announced a talk scheduled for Sunday, June 7th at AGI House focusing on DeepSeek V4 and inference optimization on Together Compute's platform. The event was promoted as a
arXiv:2606.03910v1 Announce Type: cross Abstract: Disaggregated LLM inference forces the KV cache to traverse the datacenter network before decoding begins, so transfer time enters directly into the T
DigitalOcean discusses infrastructure and platform capabilities designed to support the inference phase of machine learning, where trained models are deployed to make predictions on new data at scale.
arXiv:2606.02823v1 Announce Type: new Abstract: Two-bit weight quantization is attractive for memory-efficient LLM inference, but the standard W2 level set {-2,-1,0,+1} often collapses under aggressiv
arXiv:2606.03736v1 Announce Type: cross Abstract: Resource-constrained pricing controllers can make fixed-price inference impossible: the controller's resource state may remove the target price neighb
arXiv:2606.00913v1 Announce Type: cross Abstract: Multi-arm bandit algorithms are increasingly used in online platforms, clinical trials, and social science experiments, but valid statistical inferenc
arXiv:2606.01007v1 Announce Type: cross Abstract: Sparsely activated Mixture-of-Experts (MoE) models scale capacity via conditional computation, but distributed inference suffers from cross-GPU expert
arXiv:2606.00279v1 Announce Type: cross Abstract: Verifying claims about AI workloads is a pre- requisite for credible AI governance of covert adversaries (who comply with monitoring only when detecti
arXiv:2606.01413v1 Announce Type: cross Abstract: It is crucial for modern on-device AI systems that rely on retrieval-augmented inference to release and share datastores without compromising individu
arXiv:2606.02562v1 Announce Type: cross Abstract: Autonomous robots that interact with people must make safe and efficient decisions under human-induced uncertainty, such as their preferences, goals,
Perplexity discusses hybrid agentic inference in Perplexity Computer, which likely describes a computational approach that combines processing between local machines and data centers. The concept sugg
arXiv:2606.00966v1 Announce Type: new Abstract: Vision-Language Action (VLA) models continue to face challenges such as slow inference speed and difficulty performing fine-grained motion adjustments,
arXiv:2602.02819v3 Announce Type: replace Abstract: Membership Inference Attacks (MIAs) aim to distinguish training points (members) from unseen data (non-members), and are widely used to quantify mem
DigitalOcean's serverless inference offering enables developers to deploy and run machine learning models without managing underlying infrastructure, automatically scaling resources based on demand. T
arXiv:2605.30089v1 Announce Type: new Abstract: Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of inference-time element corruption. This
arXiv:2605.30292v1 Announce Type: cross Abstract: Conformal prediction methods enjoy strong theoretical and empirical predictive inference performance, provided the data is exchangeable, and predictor
This article discusses security measures and best practices for protecting AI model inference endpoints from unauthorized access and theft, likely covering topics such as API authentication, rate limi
arXiv:2505.19342v2 Announce Type: replace-cross Abstract: Multi-device inference can reduce Transformer latency by parallelizing computation. However, existing methods require high inter-device bandwi
arXiv:2605.27991v1 Announce Type: cross Abstract: Deep neural networks (DNNs) have achieved remarkable empirical success, yet their training dynamics remain understood mainly from optimization rather
arXiv:2605.27499v1 Announce Type: new Abstract: Flow and diffusion generative models have established themselves as widely adopted density estimators for simulation-based inference (SBI), extending na