Search: “inference”

arXiv:2606.07841v1 Announce Type: cross Abstract: Black-box variational inference (BBVI) is a methodology for posterior approximation that relies on stochastic optimization. In practice, the stochasti

arXiv:2604.02121v2 Announce Type: replace-cross Abstract: Stochastic kinetic models are ubiquitous in physics, yet inferring their parameters from experimental data remains challenging. For determinis

Many research labs only consider inference efficiency after the fact. Step 3.7 Flash is a 198B sparse MoE VLM designed by @StepFun_ai for inference from the start. 196B language backbone with a 1.8B v

arXiv:2606.02643v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG)-enhanced LLM systems, while powerful, introduce substantial inference costs due to the inclusion of an extra mult

arXiv:2606.00946v1 Announce Type: cross Abstract: Efficiently serving large language model (LLM) inference tasks is crucial both for user-perceived latency such as time-to-first-token (TTFT) and for G

arXiv:2602.07905v2 Announce Type: replace Abstract: Large Language Models (LLMs) have shown strong potential in complex medical reasoning yet face diminishing gains under inference scaling laws. While

arXiv:2605.25123v1 Announce Type: cross Abstract: We study inference-time alignment for diffusion-based generative models, aiming to steer a base model toward high-reward outputs without updating its

arXiv:2605.21805v1 Announce Type: cross Abstract: State-space models (SSMs) are powerful probabilistic tools for modeling time-varying systems with latent dynamics. Inference in SSMs involves the esti

arXiv:2603.26539v2 Announce Type: replace Abstract: How does the extent to which a model is open or closed impact the scientific inferences that can be drawn from research that involves it? In this pa

arXiv:2511.21223v2 Announce Type: replace-cross Abstract: Variational inference (VI) is a cornerstone of modern Bayesian learning, enabling approximate inference in complex models. However, its formul

arXiv:2605.17889v1 Announce Type: new Abstract: The Mixture-of-Experts (MoE) architecture improves computational efficiency via sparse expert activation, but throughput-oriented inference faces substa

arXiv:2504.03158v2 Announce Type: replace-cross Abstract: In this work, we propose a new particle-based variational inference (ParVI) method for accelerating the Energetic Variational Inference with I

arXiv:2605.13778v1 Announce Type: cross Abstract: Diffusion-based vision-language-action models (dVLAs) are promising for embodied intelligence but are fundamentally limited in real-time deployment by

arXiv:2605.11093v1 Announce Type: new Abstract: Today's inference-time workloads increasingly depend on timely access to a model's internal states. We present DMI-Lib, a high-speed deep model inspecto

arXiv:2605.11869v1 Announce Type: new Abstract: While the overall inference latency of Video Diffusion Transformers (DiTs) can be substantially reduced through model distillation, per-step inference l

arXiv:2505.13770v3 Announce Type: replace-cross Abstract: Reliable causal inference is essential for making decisions in high-stakes areas like medicine, economics, and public policy. However, it rema

arXiv:2605.11638v1 Announce Type: cross Abstract: U-statistics play a central role in statistical inference. In many modern applications, however, acquiring the labels required for U-statistics is cos

arXiv:2605.10005v1 Announce Type: cross Abstract: In this paper, we describe early work on a specification inference tool for the Move Prover that combines a weakest-precondition (WP) analysis over Mo

arXiv:2605.10533v1 Announce Type: new Abstract: In causal inference, confounders are variables that influence both treatment decisions and outcomes. However, unlike as in randomized clinical trials, t

arXiv:2605.10124v1 Announce Type: cross Abstract: The recent growth of on-device Large Language Model (LLM) inference has driven significant interest in device-edge collaborative LLM inference. As a p

arXiv:2605.09357v1 Announce Type: cross Abstract: Running deep neural networks on microcontroller units (MCUs) is severely constrained by limited memory resources. While TinyML techniques reduce model

arXiv:2605.08168v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models offer a promising path to generalist robot control, but their inference latency causes observation staleness when

arXiv:2605.05615v2 Announce Type: replace Abstract: Large language models (LLMs) impose rapidly growing energy demands, creating an emerging energy and carbon crisis driven by large-scale inference. S

arXiv:2605.06055v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) inference requires large-scale token exchange across devices, making dispatch and combine major bottlenecks in both p

arXiv:2605.07384v1 Announce Type: new Abstract: Inferring the evolution of high-dimensional and multi-modal (e.g., spatio-temporal) physical fields from irregular sparse measurements in real time is a

Excited to speak at PyCon US @pycon about production LLM inference systems, runtime optimization, and next-gen engine design — including some of the new ideas behind the recently open-sourced project

arXiv:2605.04595v1 Announce Type: new Abstract: The rapid adoption of large language models (LLMs) has created significant challenges for efficient inference at scale. Unlike traditional workloads, LL

arXiv:2605.04930v1 Announce Type: new Abstract: Despite theoretical advantages, causal methods for Gene Regulatory Network (GRN) inference from single-cell RNA-seq data consistently fail to match or o

arXiv:2605.03710v1 Announce Type: cross Abstract: Bayesian predictive inference propagates parameter uncertainty to quantities of interest through the posterior-predictive distribution. In practice, t

We’ve developed our own inference engine Runtime-Optimized Serving Engine (ROSE) to serve models ranging from embeddings to trillion-parameter LLMs. With CuTeDSL integrated into our inference engine,

arXiv:2601.10915v2 Announce Type: replace-cross Abstract: In the emerging paradigm of edge learning, neural networks (NNs) are partitioned across distributed edge devices that collaboratively perform

Mike Wheatley / SiliconANGLE: Inference cloud startup DeepInfra raised a 107M Series B co-led by 500 Global and Georges Harik; it currently supports more than 190 open models — Dedicated inference clo

arXiv:2604.09718v2 Announce Type: cross Abstract: LLM-driven web agents operating through continuous inference loops -- repeatedly querying a model to evaluate browser state and select actions -- exhi

arXiv:2604.27536v1 Announce Type: new Abstract: In black-box large language model (LLM) services, response reliability is often only partially observable at decision time, while stronger inference pat

arXiv:2604.27892v1 Announce Type: cross Abstract: The rapidly expanding artificial intelligence (AI) industry has produced diverse yet powerful prediction tools, each with its own network architecture

arXiv:2602.17697v2 Announce Type: replace Abstract: Large Language Models (LLMs) are being increasingly used across a wide range of tasks. However, their substantial computational demands raise concer

arXiv:2604.23709v1 Announce Type: new Abstract: Single image dehazing is often constrained by a trade-off between restoration quality and computational efficiency. While efficient, CNN networks strugg

arXiv:2604.21361v1 Announce Type: new Abstract: Distributed AI inference pipelines rely heavily on timestamp-based observability to understand system behavior. This work demonstrates that even small c

arXiv:2604.20735v1 Announce Type: new Abstract: Accurate condition monitoring of industrial equipment requires inferring latent degradation parameters from indirect sensor measurements under uncertain

arXiv:2604.20775v1 Announce Type: new Abstract: Trajectory Inference (TI) seeks to recover latent dynamical processes from snapshot data, where only independent samples from time-indexed marginals are

arXiv:2604.20420v1 Announce Type: cross Abstract: AI research often emphasizes model design and algorithmic performance, while deployment and inference remain comparatively underexplored despite being

This DigitalOcean guide covers strategies and best practices for deploying and optimizing very large language models (600 billion+ parameters) on cloud infrastructure, focusing on inference performanc

arXiv:2601.22002v3 Announce Type: replace Abstract: Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference. This inference can be ma

arXiv:2604.18310v1 Announce Type: cross Abstract: Variational inference (VI) is a central tool in modern machine learning, used to approximate an intractable target density by optimising over a tracta

arXiv:2604.15787v1 Announce Type: cross Abstract: We introduce EVIL (extbf{EV}olving extbf{I}nterpretable algorithms with extbf{L}LMs), an approach that uses LLM-guided evolutionary search to discover

arXiv:2604.15672v1 Announce Type: cross Abstract: Speculative decoding (SD) accelerates language model inference by drafting tokens from a cheap proposal model and verifying them against an expensive