Search: “inference”

arXiv:2602.06019v2 Announce Type: replace Abstract: Existing techniques for accelerating language model inference, such as speculative decoding, require training auxiliary speculator models and buildi

B8906 is a release tag for llama.cpp, an open-source C/C++ library for local large language model inference. llama.cpp enables LLM inference in C/C++ , and the project uses sequential build identifier

This week at Google Cloud Next ‘26, we are sharing the evolution of Google Kubernetes Engine (GKE), delivering leading performance, efficiency, security, and scale for your most demanding and complex

arXiv:2604.17140v1 Announce Type: cross Abstract: We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the

arXiv:2604.17695v1 Announce Type: cross Abstract: KV cache memory is the dominant bottleneck for long-context LLM inference. Existing compression methods each act on a single axis of the four-dimensio

arXiv:2604.17320v1 Announce Type: new Abstract: Deploying Vision-Language Models (VLMs) under aggressive low-bit inference remains challenging because inference cost is dominated by the long visual-to

b8853 is a release of llama.cpp , the open-source C/C++ library for running large language model inference. The project enables LLM inference with minimal setup and state-of-the-art performance on a w

arXiv:2604.15357v1 Announce Type: cross Abstract: Precise estimation of model inference latency is crucial for time-critical mobile edge applications, enabling devices to calculate latency margins aga

arXiv:2604.16027v1 Announce Type: cross Abstract: Post-trained language models produce less varied outputs than their base counterparts. This output diversity collapse undermines inference-time scalin

arXiv:2601.03523v2 Announce Type: replace Abstract: One of the most important queries in knowledge compilation is weighted model counting (WMC), which has been applied to probabilistic inference on va

arXiv:2604.12660v1 Announce Type: new Abstract: In nonmonotonic reasoning from conditional belief bases, an inference operator satisfying syntax splitting postulates allows for taking only the relevan

arXiv:2601.15498v2 Announce Type: replace Abstract: Speculative Decoding (SD) accelerates autoregressive large language model (LLM) inference by decoupling generation and verification. While recent me

arXiv:2604.11080v1 Announce Type: cross Abstract: Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large La

Auto-generated index of all people mentioned across the wiki.

Today, Reflection released Beam, its first open-weight model built to give developers, researchers, and enterprises open control and customization for frontier software engineering and long-horizon ag

arXiv:2610.21204v1 Announce Type: new Abstract: When a speaker says ``some students passed'', listeners infer that not all of them did. Reasoning models now let users decide how long a model thinks be

arXiv:2610.18304v1 Announce Type: new Abstract: Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due

arXiv:2610.19855v1 Announce Type: new Abstract: As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) mo

arXiv:2610.20055v1 Announce Type: new Abstract: The rapid advancement of large vision language models (LVLMs) has made geolocation inference from images highly accurate and accessible. This poses seri

arXiv:2610.19796v1 Announce Type: cross Abstract: In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting o

arXiv:2610.19051v1 Announce Type: cross Abstract: Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wron

arXiv:2610.14571v1 Announce Type: cross Abstract: Large Language Model (LLM) inference is driving a growing share of datacenter carbon emissions. Although carbon-aware scheduling can reduce these emis

Flo Health leveraged Databricks AI Runtime to create open‑source language models that prioritize both safety and practical utility. By integrating Databricks’ scalable training infrastructure with rob

arXiv:2610.12970v1 Announce Type: cross Abstract: The increasing demand for GPU-accelerated machine learning inference has exposed limitations in centralized GPU infrastructures, including high cost,

gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in

arXiv:2610.11362v1 Announce Type: cross Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference c

arXiv:2610.11765v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly ship with explicit 'thinking modes', yet their counterpart, 'no-thinking', has received far less attention. We

arXiv:2610.10368v1 Announce Type: cross Abstract: Adaptive computation aims to improve language-model inference by tailoring execution to each input. For layer programs, oracle evaluations use known a

arXiv:2610.07816v1 Announce Type: new Abstract: Small language models (SLMs) are attractive as local agent controllers because they reduce remote inference, latency, and deployment footprint, yet stru

'It's well worth spending the money on tokens if the business is getting the benefit.' — IBM Cloud GM Alan Peacock We're scaling open-source inference with @IBM + @nvidia on a dedicated B300 cluster o

arXiv:2610.08463v1 Announce Type: new Abstract: Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an en

Disclosure: This unit was provided free of charge by Orange Pi for testing. No editorial review, no preconditions, no script. All data, bottlenecks, and thermal behavior are reported directly from har

arXiv:2610.04093v1 Announce Type: new Abstract: Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Trai

arXiv:2610.03817v1 Announce Type: cross Abstract: Purpose: To evaluate the performance-feasibility tradeoffs of foundation models (FMs) and task-specific convolutional neural networks (CNNs) trained f

arXiv:2610.06751v1 Announce Type: new Abstract: Matrix completion underlies problems from tabular imputation to causal inference, yet existing tabular foundation models treat it as entry-by-entry pred

arXiv:2610.04537v1 Announce Type: new Abstract: On-device assistants run GPU-based LLM inference alongside CPU retrieval on unified-memory systems. Under a saturated local-retrieval workload, four con

arXiv:2610.04912v1 Announce Type: cross Abstract: Language models increasingly serve prompts that carry private data, and secure inference under homomorphic encryption lets a client outsource the comp

arXiv:2505.18757v3 Announce Type: replace Abstract: Visual token pruning, which aims to compress and prune redundant visual tokens, plays a critical role in efficient inference with large vision-langu

arXiv:2610.02831v1 Announce Type: new Abstract: Vision-language models (VLMs) are powerful listwise rerankers for multimodal retrieval, but high inference costs restrict them to evaluating small local

arXiv:2610.02882v1 Announce Type: cross Abstract: Large-scale foundation models achieve strong performance across diverse tasks, but their size makes inference costly, largely due to dense matrix mult

arXiv:2610.02626v1 Announce Type: new Abstract: Vision-language-action (VLA) models increasingly incorporate intermediate reasoning to improve robotic manipulation, yet existing approaches primarily r

arXiv:2610.01034v1 Announce Type: cross Abstract: Bayesian inference increasingly uses informative but implicit priors represented only by samples, such as historical ensembles, simulator outputs, and

arXiv:2610.01785v1 Announce Type: cross Abstract: Processing long videos with Vision-Language Models (VLMs) is bottlenecked by the quadratic cost of visual tokens, making long-form inference prohibiti

arXiv:2609.39329v1 Announce Type: new Abstract: Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compressio

arXiv:2609.37791v1 Announce Type: new Abstract: Every day, people continuously infer situational context and adjust the way they understand and remember the world. Context, signaled by the prefrontal

arXiv:2609.32350v1 Announce Type: new Abstract: Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementa