Search: “inference”

arXiv:2602.04163v2 Announce Type: replace Abstract: Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundament

arXiv:2605.15325v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown strong performance in video anomaly detection (VAD) while providing interpretable predictions. However, existin

arXiv:2605.15725v1 Announce Type: cross Abstract: Latent Action Models (LAMs) enable the learning of world models from unlabeled video by inferring abstract actions between consecutive frames. However

arXiv:2605.15417v1 Announce Type: cross Abstract: In GFlowNets and variational inference, it has been shown that the mean square error between target and model log probabilities is an effective, low v

arXiv:2512.06655v3 Announce Type: replace-cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract activation directions for inference-time steering, but their standard sparsity obj

arXiv:2605.15626v1 Announce Type: new Abstract: Large language models deliver strong performance across language and reasoning tasks, but their storage and compute costs remain major barriers to deplo

llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance jump with these changes is massive and elevates local inference on commodity hardw

arXiv:2605.15621v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but their inference cost grows rapidly with the number of visual tokens, e

arXiv:2605.15572v1 Announce Type: new Abstract: The dynamic range of activations is a first-order constraint for low-bit quantization, activation scaling, and stable LLM inference. Prior work characte

Agentic AI inference at one-tenth the cost per token with NVIDIA Vera Rubin NVL72. Agent sandboxes run 50% faster on NVIDIA Vera than traditional CPUs — while enterprise data queries are up to 3x fast

arXiv:2605.15353v1 Announce Type: cross Abstract: Inferring the structure of directed acyclic graphs (DAGs) from data is a central challenge in causal discovery, particularly in modern high-dimensiona

arXiv:2605.15609v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions i

arXiv:2605.15816v1 Announce Type: cross Abstract: Stipple patterns, point sets whose local density tracks a target image, are traditionally produced by per-density iterative optimizers, which are slow

arXiv:2605.15508v1 Announce Type: cross Abstract: The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Language Model (LLM) inference. This challenge is p

arXiv:2605.15388v1 Announce Type: new Abstract: Stochastic estimators are fundamental to large-scale optimization, where population quantities must be inferred from noisy oracle observations. Although

b9197 is a build release of llama.cpp , an open-source C/C++ implementation that enables efficient large language model inference on various hardware platforms. The release includes cross-platform bin

Setting a GPU power limit using nvidia-smi reduces heat output by approximately 20% with only a 5-8% inference speed loss. Undervolting the GPU can reduce power consumption by 5-15% with zero performa

arXiv:2605.13916v1 Announce Type: cross Abstract: Online Multiple Testing (OMT), a fundamental pillar of sequential statistical inference, traditionally evaluates the False Discovery Rate (FDR) and st

arXiv:2605.14163v1 Announce Type: new Abstract: Can a committee of weak reasoning-model calls reach the performance of much stronger models? We study verifier-backed committee search as inference-time

arXiv:2602.04473v2 Announce Type: replace Abstract: Recently, diffusion models have brought novel insights to pan-sharpening and notably boosted fusion precision. However, most existing models perform

arXiv:2605.14859v1 Announce Type: cross Abstract: As coding agents gain access to shells, repositories, and user files, least-privilege authorization becomes a prerequisite for safe deployment: an age

arXiv:2605.14104v1 Announce Type: new Abstract: Inferring spatially resolved gene expression from histology images offers a cost-effective complement to spatial transcriptomics (ST). However, existing

arXiv:2605.14665v1 Announce Type: new Abstract: Legal reasoning is not semantic similarity search. A court judgment encodes constrained symbolic reasoning: precedent propagation, procedural state tran

arXiv:2605.13997v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost

arXiv:2605.14267v1 Announce Type: cross Abstract: Diffusion models (DMs) have exhibited remarkable efficacy in various image restoration tasks. However, existing approaches typically operate within th

arXiv:2605.14005v1 Announce Type: new Abstract: Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens a

arXiv:2605.14978v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by having a lightweight draft model propose speculative windows of candidate tokens for parallel verifica

arXiv:2605.14894v1 Announce Type: new Abstract: Recent breakthroughs in video diffusion models have significantly accelerated the development of video editing techniques. However, existing methods oft

arXiv:2601.21656v3 Announce Type: replace Abstract: Clustering tabular data is a fundamental yet challenging problem due to heterogeneous feature types, diverse data-generating mechanisms, and the abs

5/5 What this unlocks: - Change one prompt, serve cached results for all other components. - A/B test any component against identical upstream outputs. - Run best-of-N inference without N branches col

arXiv:2605.13128v1 Announce Type: cross Abstract: This paper introduces an algorithm-agnostic approach to feature-based time series clustering via amortized neural inference. By training neural networ

Batching for vision models is now available in Beta with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mod

arXiv:2605.13306v1 Announce Type: new Abstract: Illuminant estimation aims to infer scene illumination from image measurements despite intrinsic ambiguities between surface reflectance and lighting. M

This Reddit post likely compares inference performance metrics across popular language models running on Ollama, measuring tokens per second as a key performance indicator. The post would help users a

arXiv:2605.13790v1 Announce Type: cross Abstract: Partial differential equations (PDEs) are fundamental for modeling complex natural and physical phenomena. In many real-world applications, however, o

arXiv:2605.13151v1 Announce Type: new Abstract: Category-agnostic pose estimation (CAPE) aims to localize keypoints on query images from arbitrary categories, using only a few annotated support exampl

arXiv:2605.13734v1 Announce Type: cross Abstract: LLMs are widely adopted in production, pushing inference systems to their limits. Disaggregated LLM serving (e.g., PD separation and KV state disaggre

arXiv:2505.12741v2 Announce Type: replace Abstract: Language models are increasingly used not only as standalone predictors but also as components in larger inference systems, from test-time reasoning

arXiv:2506.11274v2 Announce Type: replace-cross Abstract: Test-time scaling has emerged as an effective approach for improving language model performance by utilizing additional compute at inference t

arXiv:2605.12682v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as reasoning modules in many applications. While they are efficient in certain tasks, LLMs often stru

arXiv:2506.12876v2 Announce Type: replace Abstract: The rapid scaling of large language models~(LLMs) has made inference efficiency a primary bottleneck in the practical deployment. To address this, s

arXiv:2601.21731v2 Announce Type: replace Abstract: Prior-Data Fitted Networks (PFNs) enable amortized Bayesian inference in a single forward pass, yet their internal representations remain opaque. It

DeepSeek maintains the highest efficiency in terms of active parameters relative to total model size, outperforming competitors in the ratio of parameters actually used during inference versus total t

Most teams can pick frontier models. Fewer can run them at production scale without hitting constraints in latency, throughput, and governance. Fireworks AI on @Azure AI Foundry provides the inference

arXiv:2605.13190v1 Announce Type: cross Abstract: Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrad

arXiv:2603.20334v2 Announce Type: replace-cross Abstract: In high-complexity abstract reasoning, a system must infer a latent rule from a few examples or structured observations and apply it to unseen