Knowledge catalogue
Ollama is now an inference provider on the Replicas platform, enabling users to deploy and run open‑source models directly within Replicas. The partnership supports popular models such as Claude Code,
arXiv:2610.20836v1 Announce Type: new Abstract: Evaluator-guided reasoning presents a local control problem at every step: trust the current answer, return a fallback, or spend more inference. We form
arXiv:2610.20893v1 Announce Type: cross Abstract: Estimating causal effects on right-censored time-to-event outcomes is challenging under unmeasured confounding. Proximal causal inference addresses un
arXiv:2610.22099v1 Announce Type: cross Abstract: Masked diffusion language models expose an inference-time choice: which positions become visible after each denoising pass. Many existing decoding met
arXiv:2610.20682v1 Announce Type: cross Abstract: We introduce TurboMamba, a language-model architecture for faster inference, faster training, and better token prediction. A global Mamba-3 model runs
arXiv:2402.05203v3 Announce Type: replace Abstract: We introduce Bellman Conformal Inference (BCI), a framework that wraps around any time series forecasting models and provides calibrated prediction
arXiv:2610.20483v1 Announce Type: new Abstract: We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-samplin
arXiv:2610.16312v1 Announce Type: new Abstract: Explicit reasoning in large language models (LLMs) is often associated with reliability, yet it can still pose risks when a model infers harmful goals u
arXiv:2610.19405v1 Announce Type: cross Abstract: Large Language Model (LLM) serving systems must balance latency and throughput while efficiently utilizing distributed GPU resources. This challenge a
arXiv:2610.14307v1 Announce Type: cross Abstract: We propose ConicFlow, which frames constrained sampling from a flow matching model at inference time as an optimal control problem that balances devia
arXiv:2610.16890v1 Announce Type: cross Abstract: We introduce FORGE (Fast On-chip Rotor Generation for Experts), a procedural weight generation system for Sparse Mixture-of-Experts (MoE) inference. R
arXiv:2610.17321v1 Announce Type: cross Abstract: As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in th
arXiv:2411.18954v5 Announce Type: replace Abstract: Maximum a posteriori inference in a finite Markov random field is generally NP-hard. We study an instance-wise optimizer that minimizes the exact mu
arXiv:2610.16820v1 Announce Type: cross Abstract: In simulation-based inference (SBI), neural posterior estimators trained on a bounded range of sample sizes can become inaccurate on larger datasets b
arXiv:2610.18486v1 Announce Type: new Abstract: Membership Inference Attacks (MIAs) are currently the dominant approach to evaluating privacy in machine learning applications. Despite their significan
arXiv:2610.14567v1 Announce Type: cross Abstract: This paper develops strong Gaussian approximations (KMT type) for constant learning rate stochastic gradient descent (SGD). We work with heavy tailed
arXiv:2610.17531v1 Announce Type: new Abstract: While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting impact
arXiv:2610.14530v1 Announce Type: new Abstract: Multi-robot systems (MRS) offer significant advantages over single-robot platforms by enabling information sharing and coordination for different tasks
Dynamic vision, in action Basalt is an inference engine for 3.8 Flash Next, forked from Strata and heavily tuned for Blackwell. It supports vision, and for this to work you need to have the image enco
Tom Warren / The Verge: Microsoft unveils its third-gen in-house AI chip, the Maia 300, built on TSMC's N2P process node and aimed at serving as its “primary AI inference system” — The new Maia 300 ch
arXiv:2610.11748v1 Announce Type: cross Abstract: U-Net inference for brain-tumor segmentation requires billions of multiply-accumulate operations, motivating hardware that can reduce computation dyna
arXiv:2610.11318v1 Announce Type: new Abstract: Single-cell snapshot data can resolve a continuum of cellular states but do not uniquely determine the dynamics governing transitions between them. Howe
arXiv:2610.11252v1 Announce Type: cross Abstract: This study presents the Neuro-Memory Fuzzy Inference System (NeMeFIS), a hierarchical machine learning architecture that asymmetrically models acceler
arXiv:2610.12327v1 Announce Type: cross Abstract: The memory-bound nature of the decoding stage of large language model (LLM) inference incurs significant latency. Layer-wise training-free network pru
arXiv:2412.08843v3 Announce Type: replace-cross Abstract: Allocation stability is often used to justify Gaussian inference from bandit data, but when is it necessary? In this paper, we address this qu
arXiv:2510.19773v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) have emerged as the standard tool for evaluating the privacy risks of AI models. However, state-of-the-art attac
arXiv:2610.09866v1 Announce Type: new Abstract: We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same in
arXiv:2610.09778v1 Announce Type: cross Abstract: Reproducible benchmarking of Large Language Model (LLM) inference is challenging because repeated measurements can vary with execution and system stat
arXiv:2610.07191v1 Announce Type: cross Abstract: Modern edge Systems-on-Chip (SoCs) integrate heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, each with distinct performance and ene
arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using pr
arXiv:2610.08345v1 Announce Type: cross Abstract: Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample si
arXiv:2610.07791v1 Announce Type: new Abstract: Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tend
llama.cpp can distribute inference on heterogeneous devices through the ggml RPC backend It's an advanced setting but I think with time we'll make it more accessible to regular users. It's crazy that
arXiv:2610.07782v1 Announce Type: new Abstract: Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache
The largest agentic LLM Inference dataset has arrived! 206B tokens · 12,002 sessions · 1,186,582 LLM requests · 1,213,347 tool calls The traces include tool names, arguments, durations, and more. Expl
arXiv:2508.06084v3 Announce Type: replace Abstract: Vision-language models (VLMs) have achieved impressive performance on multimodal inference tasks, but the cost remains a significant challenge due t
Clockwork Systems Inc., the data center infrastructure startup that helps to maximize the efficiency of artificial intelligence chip clusters, has raised 31 million in fresh funding and announced the
arXiv:2610.00499v1 Announce Type: new Abstract: As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world extit{serving}, where request manag
arXiv:2610.01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look. W
arXiv:1709.09051v3 Announce Type: replace-cross Abstract: This paper develops on algebraic grounds a polynomial time exact linear solution to the hard combinatorial problem of marginal inference in Ma
Sophia Yang posted that she was attending Day 2 of the AIconference with the FireworksAI_HQ team. She highlighted discussions on model inference and training among other AI topics. The tweet received
arXiv:2510.13935v3 Announce Type: replace-cross Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which nee
arXiv:2602.06700v2 Announce Type: replace-cross Abstract: Graph-structured data underpin a wide spectrum of modern applications, yet their multiple sensitive attributes are not isolated but deeply cou
Volantis Inc. today announced that it has raised a 88 million funding round led by prominent angel investor Lachy Groom and Abstract Ventures. The Series A deal also drew more than a half-dozen other
arXiv:2609.36331v1 Announce Type: cross Abstract: The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models tra
arXiv:2609.35883v1 Announce Type: cross Abstract: Genome foundation models are growing into sparse mixture-of-experts (MoE) networks whose expert weights no longer fit on the machines that hold the se
arXiv:2609.36153v1 Announce Type: cross Abstract: B2B advertising targets a viewer's professional attributes (employer size and industry, function, seniority) and has obtained them by matching identit
arXiv:2609.36654v1 Announce Type: cross Abstract: Large language models make weight storage and memory traffic major inference costs, motivating low-precision formats that represent each weight with o
arXiv:2609.34683v1 Announce Type: new Abstract: The optimization of LLM serving engines, such as vLLM and SGLang, is largely benchmark-driven: optimizations, scheduling policies, hardware and system d
arXiv:2609.34716v1 Announce Type: new Abstract: Clinical CT and UHRCT cannot resolve individual trabeculae, whereas synchrotron radiation microCT (SR{mu}CT) provides 3.2{mu}m high-resolution reference
arXiv:2609.31796v1 Announce Type: new Abstract: Activation probes are increasingly used to monitor LLMs in deployment. A probe is typically trained under one inference configuration, then used under w
arXiv:2606.16694v2 Announce Type: replace Abstract: Transformers perform computations through many successive attention and feedforward blocks, allowing them to learn complex correlations between a la
Distributed Training and Inference both involves having a fundamental understanding of how distributed systems work in general Distributed Parallelism Tensor Parallelism Pipeline Parallelism Model Par
I maintain TensorSharp, an open-source inference engine. It can now run Qwen-Image 2.1 locally for text-to-image generation and image editing, with support for its LoRA adapters. I’ve added configs fo
OpenAI: OpenAI says it paused training, evaluation, and inference with tool-use of its most capable models after a model bypassed internet restrictions during training — Summary — An agent attempting
arXiv:2602.08329v2 Announce Type: replace-cross Abstract: A core bottleneck in large language model (LLM) inference is the cost of attending over the ever-growing key-value (KV) cache. Although near-o
arXiv:2609.26913v1 Announce Type: cross Abstract: No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or
arXiv:2609.26021v1 Announce Type: new Abstract: Dynamic black-box optimization presents significant challenges for Bayesian Optimization (BO), as the objective function evolves over time, causing opti
arXiv:2609.25924v1 Announce Type: cross Abstract: Causal inference guides operational and managerial decisions but remains challenging in high-dimensional panel or tensor settings, where decisions may
arXiv:2609.24698v1 Announce Type: new Abstract: Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follo