Search: “inference”

arXiv:2602.06019v2 Announce Type: replace Abstract: Existing techniques for accelerating language model inference, such as speculative decoding, require training auxiliary speculator models and buildi

arXiv:2604.17140v1 Announce Type: cross Abstract: We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the

arXiv:2604.17695v1 Announce Type: cross Abstract: KV cache memory is the dominant bottleneck for long-context LLM inference. Existing compression methods each act on a single axis of the four-dimensio

arXiv:2604.17320v1 Announce Type: new Abstract: Deploying Vision-Language Models (VLMs) under aggressive low-bit inference remains challenging because inference cost is dominated by the long visual-to

arXiv:2604.15357v1 Announce Type: cross Abstract: Precise estimation of model inference latency is crucial for time-critical mobile edge applications, enabling devices to calculate latency margins aga

arXiv:2604.16027v1 Announce Type: cross Abstract: Post-trained language models produce less varied outputs than their base counterparts. This output diversity collapse undermines inference-time scalin

arXiv:2601.03523v2 Announce Type: replace Abstract: One of the most important queries in knowledge compilation is weighted model counting (WMC), which has been applied to probabilistic inference on va

arXiv:2604.12660v1 Announce Type: new Abstract: In nonmonotonic reasoning from conditional belief bases, an inference operator satisfying syntax splitting postulates allows for taking only the relevan

arXiv:2601.15498v2 Announce Type: replace Abstract: Speculative Decoding (SD) accelerates autoregressive large language model (LLM) inference by decoupling generation and verification. While recent me

arXiv:2604.11080v1 Announce Type: cross Abstract: Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large La

arXiv:2610.21204v1 Announce Type: new Abstract: When a speaker says ``some students passed'', listeners infer that not all of them did. Reasoning models now let users decide how long a model thinks be

arXiv:2610.18304v1 Announce Type: new Abstract: Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due

arXiv:2610.19855v1 Announce Type: new Abstract: As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) mo

arXiv:2610.20055v1 Announce Type: new Abstract: The rapid advancement of large vision language models (LVLMs) has made geolocation inference from images highly accurate and accessible. This poses seri

arXiv:2610.19796v1 Announce Type: cross Abstract: In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting o

arXiv:2610.19051v1 Announce Type: cross Abstract: Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wron

arXiv:2610.14571v1 Announce Type: cross Abstract: Large Language Model (LLM) inference is driving a growing share of datacenter carbon emissions. Although carbon-aware scheduling can reduce these emis

arXiv:2610.12970v1 Announce Type: cross Abstract: The increasing demand for GPU-accelerated machine learning inference has exposed limitations in centralized GPU infrastructures, including high cost,

arXiv:2610.11362v1 Announce Type: cross Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference c

arXiv:2610.11765v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly ship with explicit 'thinking modes', yet their counterpart, 'no-thinking', has received far less attention. We

arXiv:2610.10368v1 Announce Type: cross Abstract: Adaptive computation aims to improve language-model inference by tailoring execution to each input. For layer programs, oracle evaluations use known a

arXiv:2610.07816v1 Announce Type: new Abstract: Small language models (SLMs) are attractive as local agent controllers because they reduce remote inference, latency, and deployment footprint, yet stru

arXiv:2610.08463v1 Announce Type: new Abstract: Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an en

arXiv:2610.04093v1 Announce Type: new Abstract: Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Trai

arXiv:2610.03817v1 Announce Type: cross Abstract: Purpose: To evaluate the performance-feasibility tradeoffs of foundation models (FMs) and task-specific convolutional neural networks (CNNs) trained f

arXiv:2610.06751v1 Announce Type: new Abstract: Matrix completion underlies problems from tabular imputation to causal inference, yet existing tabular foundation models treat it as entry-by-entry pred

arXiv:2610.04537v1 Announce Type: new Abstract: On-device assistants run GPU-based LLM inference alongside CPU retrieval on unified-memory systems. Under a saturated local-retrieval workload, four con

arXiv:2610.04912v1 Announce Type: cross Abstract: Language models increasingly serve prompts that carry private data, and secure inference under homomorphic encryption lets a client outsource the comp

arXiv:2505.18757v3 Announce Type: replace Abstract: Visual token pruning, which aims to compress and prune redundant visual tokens, plays a critical role in efficient inference with large vision-langu

arXiv:2610.02831v1 Announce Type: new Abstract: Vision-language models (VLMs) are powerful listwise rerankers for multimodal retrieval, but high inference costs restrict them to evaluating small local

arXiv:2610.02882v1 Announce Type: cross Abstract: Large-scale foundation models achieve strong performance across diverse tasks, but their size makes inference costly, largely due to dense matrix mult

arXiv:2610.02626v1 Announce Type: new Abstract: Vision-language-action (VLA) models increasingly incorporate intermediate reasoning to improve robotic manipulation, yet existing approaches primarily r

arXiv:2610.01034v1 Announce Type: cross Abstract: Bayesian inference increasingly uses informative but implicit priors represented only by samples, such as historical ensembles, simulator outputs, and

arXiv:2610.01785v1 Announce Type: cross Abstract: Processing long videos with Vision-Language Models (VLMs) is bottlenecked by the quadratic cost of visual tokens, making long-form inference prohibiti

arXiv:2609.39329v1 Announce Type: new Abstract: Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compressio

arXiv:2609.37791v1 Announce Type: new Abstract: Every day, people continuously infer situational context and adjust the way they understand and remember the world. Context, signaled by the prefrontal

arXiv:2609.32350v1 Announce Type: new Abstract: Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementa

arXiv:2609.32956v1 Announce Type: new Abstract: Prior information for real-world physical quantities is most elegantly expressed via partial differential equations (PDEs). In this paper, we propose a

arXiv:2609.33965v1 Announce Type: cross Abstract: We describe a methodology for estimating the per-token energy cost of cloud-hosted large language model (LLM) inference, separating between input (pre

arXiv:2609.30566v1 Announce Type: new Abstract: We present a new inference-time sampler for diffusion models that gives a pretrained model a capability it was never trained for: constructing the atlas

arXiv:2609.30747v1 Announce Type: cross Abstract: As artificial intelligence (AI) becomes embedded in everyday life, its environmental footprint, particularly water consumption remains largely invisib

arXiv:2609.31360v1 Announce Type: new Abstract: Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human bra

arXiv:2609.25085v1 Announce Type: cross Abstract: Data assimilation aims to infer the state of complex dynamical systems based on observational data. However, accurate inference of the multimodal post

arXiv:2609.22471v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models are increasingly deployed alongside Speculative Decoding (SD) to accelerate inference, but combining the two is challe

arXiv:2602.13718v2 Announce Type: replace Abstract: Generative policies for robotic manipulation must balance action accuracy with inference latency. We present HybridFlow, a three-stage policy infere

arXiv:2609.23404v1 Announce Type: new Abstract: Point cloud completion aims to infer a complete 3D shape from a partial point cloud and serves as a fundamental building block for downstream tasks such