Search: “inference”

arXiv:2508.06038v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) incur substantial computational overhead and inference latency due to the large number of vision tokens introduc

arXiv:2605.16928v1 Announce Type: cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention. Existing efficient alternatives often rely ei

arXiv:2605.17093v1 Announce Type: cross Abstract: Distilling vision-language models into faster hybrid architectures, such as 3:1 Mamba-2/attention mixes, is now standard practice for making inference

arXiv:2605.17268v1 Announce Type: new Abstract: We present the first systematic study of faithfulness in Vision-Language-Action (VLA) driving models, analyzing 300 Alpamayo-R1-10B inferences across 10

arXiv:2605.18331v1 Announce Type: new Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly,

arXiv:2605.16941v1 Announce Type: new Abstract: Diffusion Large Language Models (DLLMs) promise fast parallel generation, yet open-source DLLMs still face a severe quality-speed trade-off: acceleratin

arXiv:2604.08432v2 Announce Type: replace-cross Abstract: Photonic neural networks promise ultrafast inference, yet most architectures rely on linear optical meshes with electronic nonlinearities, rei

arXiv:2605.17633v1 Announce Type: cross Abstract: The Segment Anything Model (SAM) achieves strong open-vocabulary segmentation, but its ViT-based image encoders dominate inference latency and memory.

arXiv:2602.08169v2 Announce Type: replace-cross Abstract: Inference-time steering offers a promising way to control language models (LMs) without retraining. However, standard approaches typically rel

arXiv:2605.17745v1 Announce Type: cross Abstract: Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision

b9221 is an intermediate build release from the llama.cpp project, which is a C/C++ implementation enabling efficient LLM inference on consumer hardware. The release includes platform-specific binarie

arXiv:2605.15592v1 Announce Type: new Abstract: Few-step image generation has seen rapid progress, with consistency and meanflow-based methods significantly reducing the number of sampling steps. Desp

arXiv:2605.15492v1 Announce Type: cross Abstract: Generative models such as diffusion and flow matching have become dominant paradigms for visuomotor policy learning, yet their reliance on iterative d

arXiv:2601.23030v2 Announce Type: replace-cross Abstract: Inference in nonlinear continuous stochastic processes on trees is challenging, particularly when observations are sparse and the topology is

arXiv:2605.15208v1 Announce Type: cross Abstract: Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deploy

arXiv:2604.02268v2 Announce Type: replace Abstract: Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a rel

arXiv:2605.15733v1 Announce Type: cross Abstract: Humans abstract experiences into structured representations to facilitate pattern inference and knowledge transfer. While the hippocampal-entorhinal (

arXiv:2605.15051v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified b

b9159 is a release of llama.cpp published on May 14, 2026 . llama.cpp is a C/C++ implementation of large language model inference that enables efficient LLM execution on consumer hardware with minimal

arXiv:2605.14438v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures enhance the efficiency of large language models by activating only a subset of experts per token. However, standa

arXiv:2605.15133v1 Announce Type: new Abstract: Causal inference, estimating causal effects from observational data, is a fundamental tool in many disciplines. Of particular importance across a variet

arXiv:2605.14521v1 Announce Type: new Abstract: Layer normalization (LN) is a fundamental component in modern deep learning, but its per-sample centering and scaling introduce non-negligible inference

arXiv:2508.17588v2 Announce Type: replace Abstract: Generation-driven world models create immersive virtual environments but suffer slow inference due to the iterative nature of diffusion models. Whil

arXiv:2605.14123v1 Announce Type: cross Abstract: Feature sharing via split inference offers a lightweight alternative to federated learning for resource-constrained hospitals, but transmitted feature

arXiv:2605.14704v1 Announce Type: cross Abstract: In real-world scenes, target objects may reside in regions that are not visible. While humans can often infer the locations of occluded objects from c

arXiv:2605.15050v1 Announce Type: new Abstract: Recently, deep generative models have been used for posterior inference in inverse problems, including high-stakes applications in medical imaging and s

arXiv:2605.14110v1 Announce Type: new Abstract: Vision Transformers (ViTs) enable strong multi-view 3D detection but are limited by high inference latency from dense token and query processing across

arXiv:2605.14717v1 Announce Type: cross Abstract: Label-free single-cell imaging offers a scalable, non-invasive alternative to fluorescence-based cytometry, yet inferring molecular phenotypes directl

arXiv:2605.14570v1 Announce Type: new Abstract: Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. S

arXiv:2605.12706v1 Announce Type: new Abstract: RSNet is an open-source R package that provides a resampling-based framework for robust and interpretable network inference, designed to address the lim

The search did not return specific information about the b9140 release. Based on the context, b9140 is a specific build number from the llama.cpp project releases. llama.cpp is an LLM inference implem

llama.cpp is an open source software library that performs inference on various large language models such as Llama. Build b9144 is an intermediate release version of the llama.cpp project from the gg

arXiv:2511.17031v2 Announce Type: replace-cross Abstract: The rapidly growing computational demands of diffusion models for image generation have raised significant concerns about energy consumption a

arXiv:2605.13338v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) are increasingly integrated into systems requiring reliable multi-step inference, yet this growing dependence exposes ne

arXiv:2605.13457v1 Announce Type: new Abstract: Diffusion-based real-world image super-resolution (Real-ISR) has achieved remarkable perceptual quality; however, directly super-resolving images to 4K

This article explains how asynchronous processing can be integrated with continuous batching techniques to improve inference efficiency in machine learning models. Continuous batching allows requests

arXiv:2605.12549v1 Announce Type: new Abstract: Existing training-free approaches for GUI grounding often rely on multiple inference runs, such as iterative cropping or candidate aggregation, to ident

arXiv:2605.10987v1 Announce Type: new Abstract: Modern machine learning deployments increasingly compose specialized models into dynamic inference pipelines, where upstream components produce intermed

arXiv:2605.12309v1 Announce Type: new Abstract: The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. I

arXiv:2605.11381v1 Announce Type: new Abstract: Physical AI is experiencing rapid growth with frontier foundation models increasing its capabilities across general environments. Physical AI tasks are

arXiv:2602.05830v2 Announce Type: replace-cross Abstract: Floating-point neural networks dominate modern machine learning but incur substantial inference costs, motivating emerging interest in Boolean

arXiv:2605.11327v1 Announce Type: new Abstract: Classical deep learning typically operates on individual cases. Despite its success, real-world usage often requires repeated inference to estimate stat

arXiv:2605.12412v1 Announce Type: new Abstract: Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent

arXiv:2605.10991v1 Announce Type: new Abstract: Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process

arXiv:2605.11240v1 Announce Type: cross Abstract: Generative AI models differ from traditional machine learning tools in that they allow users to provide as much or as little information as they choos

💡 Why did @togethercompute choose NVIDIA Blackwell to serve DeepSeek-V4? Because NVIDIA Blackwell is built for the bottlenecks that matter most in long-context inference: → KV-cache pressure during de