Search: “inference”

We’ve rolled out a significant update to Google Kubernetes Engine (GKE) that solves one of the most annoying problems in cloud infrastructure: cold start latency. GKE now has up to 4x faster node star

b9060 is a release build identifier for llama.cpp, an open-source LLM inference framework implemented in C/C++. This build represents a snapshot in the project's ongoing development, containing bug fi

Release b9063 is a build of llama.cpp, a project for LLM inference in C/C++. llama.cpp enables efficient large language model execution on consumer hardware through optimized implementations and quant

arXiv:2505.00020v2 Announce Type: replace Abstract: Using a legally obtained dataset of 34 copyrighted O'Reilly Media books, we apply the DE-COP membership inference attack method to investigate wheth

arXiv:2605.04980v1 Announce Type: cross Abstract: Activation-based steering provides control of LLM behavior at inference time, but the dominant paradigm reduces each concept to a single direction who

arXiv:2605.04507v1 Announce Type: new Abstract: Negotiation agents must infer what their counterpart values, update those beliefs over dialogue turns, and choose actions under uncertainty. End-to-end

arXiv:2605.04535v1 Announce Type: new Abstract: Inferring continuum models directly from video is hampered by two facts: the recorded field is uncalibrated image intensity rather than a physical state

arXiv:2605.04581v1 Announce Type: new Abstract: Light field (LF) image super-resolution benefits from Epipolar Plane Images (EPIs), whose line slopes explicitly encode disparity. However, existing Tra

arXiv:2605.05125v1 Announce Type: new Abstract: Target trial emulation (TTE) enables causal questions to be studied with observational data when randomized controlled trials (RCTs) are infeasible. Yet

arXiv:2510.17903v3 Announce Type: replace-cross Abstract: This paper tackles the challenging problem of jointly inferring time-varying network topologies and imputing missing data from partially obser

Post-training quantization is a technique that reduces model size and improves inference performance by converting weights and activations to lower precision formats after training is complete. NVIDIA

arXiv:2602.02282v2 Announce Type: replace Abstract: Inferring spatial transcriptomics (ST) from histology enables scalable histogenomic profiling, yet current methods are largely restricted to single-

arXiv:2605.04450v1 Announce Type: cross Abstract: Generative Recommender (GR) inference places embedding hot caches (EMB) and KV caches in direct competition for limited GPU HBM: allocating more memor

arXiv:2508.11196v2 Announce Type: replace Abstract: Recent advances in vision-language models (VLMs) have demonstrated strong generalization in natural image tasks. However, their performance often de

arXiv:2603.01097v2 Announce Type: replace Abstract: Continuous knowledge updating for pre-trained large language models (LLMs) is increasingly necessary yet remains challenging. Although inference-tim

arXiv:2602.10144v2 Announce Type: replace-cross Abstract: Minimizing the inference cost and latency of foundation models has become a crucial area of research. Optimization approaches include theoreti

llama.cpp b9041 is a release build of llama.cpp, an LLM inference library written in C/C++ . The build identifier indicates this is an intermediate development release from the ggml-org/llama.cpp proj

b9047 is a build release of llama.cpp, an open-source C/C++ library for running large language model inference on consumer hardware. The release includes pre-compiled binaries for multiple platforms i

arXiv:2605.02290v1 Announce Type: new Abstract: Distilling large reasoning models is essential for making Long-CoT reasoning practical, as full-scale inference remains computationally prohibitive. Exi

arXiv:2601.21684v2 Announce Type: replace Abstract: Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration

arXiv:2505.19841v2 Announce Type: replace-cross Abstract: This work is focussed on the inversion task of inferring the distribution over parameters of interest leading to multiple sets of observations

arXiv:2510.09472v2 Announce Type: replace Abstract: Despite the remarkable progress in neural models, their ability to generalize, a cornerstone for applications such as logical reasoning, remains a c

arXiv:2507.02974v3 Announce Type: replace-cross Abstract: As major progress in LLM-based long-form text generation enables paradigms such as retrieval-augmented generation (RAG) and inference-time sca

arXiv:2602.01842v3 Announce Type: replace Abstract: Inference-time compute has re-emerged as a practical way to improve LLM reasoning. Most test-time scaling (TTS) algorithms rely on autoregressive de

arXiv:2509.12464v2 Announce Type: replace Abstract: Reasoning language models such as DeepSeek-R1 produce long chain-of-thought traces during inference time which make them costly to deploy at scale.

Mixture-of-Experts (MoE) models enable sparse expert activation, meaning that only a subset of the model’s parameters is used during each inference. However, to translate this sparsity into practical

arXiv:2605.03907v1 Announce Type: new Abstract: Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform

arXiv:2605.03609v1 Announce Type: cross Abstract: Large language models often display heterogeneous moral preferences across settings. We study inference-time steering toward a desired ethical framewo

B9030 is a release build of llama.cpp, a C/C++ implementation for LLM inference . This build release would contain bug fixes, performance improvements, and new features added to the llama.cpp project

arXiv:2605.01106v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model and verifying them in parallel with the target.

arXiv:2605.01495v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding responses in external knowledge during inference. However, conve

arXiv:2404.10370v3 Announce Type: replace Abstract: Open set recognition (OSR) is a critical aspect of machine learning, addressing the challenge of detecting novel classes during inference. Within th

arXiv:2605.02521v1 Announce Type: new Abstract: Affective image editing (AIE) aims to edit visual content to evoke target emotions. However, existing methods often overlook inference efficiency and pr

arXiv:2605.01003v1 Announce Type: cross Abstract: Statistical change point (CP) detection methods typically rely on likelihood-based inference and ignore contextual information about plausible CP loca

arXiv:2605.02888v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model (LLM) inference by using a small draft model to propose candidate tokens that a larger target mo

arXiv:2512.22331v2 Announce Type: replace Abstract: Non-invasive inference of molecular tumor characteristics from medical imaging is a central goal of radiogenomics, particularly in glioblastoma (GBM

arXiv:2602.06932v3 Announce Type: replace Abstract: Speculative decoding can significantly accelerate LLM serving, yet most deployments today disentangle speculator training from serving, treating spe

Release b9016 is a build of llama.cpp, an LLM inference implementation in C/C++ . This build designation represents a specific intermediate version in the project's development cycle, similar to other

arXiv:2605.00078v1 Announce Type: cross Abstract: Visual-Language-Action models (VLAs) have advanced generalist robot control by mapping multimodal observations and language instructions directly to a

arXiv:2503.14459v2 Announce Type: replace-cross Abstract: Practical and ethical constraints often require the use of observational data for causal inference, particularly in medicine and social scienc

arXiv:2605.00423v1 Announce Type: new Abstract: In wireless communications, recovering the optimal solution to the multiple-input multiple-output (MIMO) detection problem is NP-hard. Obtaining high-qu

arXiv:2605.00789v1 Announce Type: new Abstract: Key-Value (KV) cache has become a de facto component of modern Large Vision-Language Models (LVLMs) for inference. While it enhances decoding efficiency

AI compute resources should not be treated as interchangeable commodities, as different hardware configurations, providers, and architectures significantly impact training costs, inference latency, an

🤝 OpenHuman x Fireworks AI We are excited to announce one of our very first partnerships with Fireworks AI. OpenHuman is now powered by @FireworksAI_HQ: the fastest inference for Qwen3 outside of Alib

arXiv:2605.00707v1 Announce Type: new Abstract: Image editing instructions are heterogeneous: a color swap, an object insertion, and a physical-action edit all demand different spatial coverage and di

arXiv:2605.00146v1 Announce Type: new Abstract: Real-time object detection on energy-constrained platforms is critical for applications such as UAV-based inspection, autonomous navigation, and mobile