Search: “inference”

Release b9956 is a build version of llama.cpp, an open-source project for LLM inference in C/C++ . The project releases frequently, with multiple releases published in a single day , and b9956 represe

arXiv:2607.07953v1 Announce Type: cross Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at

arXiv:2607.07033v1 Announce Type: cross Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are

b9935 is a build release from the llama.cpp project, which implements LLM inference in C/C++. The project doesn't follow traditional release practices, as multiple releases can be published in a singl

llama.cpp b9936 is a continuous build-tagged release from the open-source llama.cpp project , which is an open-source C/C++ inference engine that powers most of the local-AI ecosystem . This release c

arXiv:2607.07409v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash furthe

arXiv:2506.04985v2 Announce Type: replace Abstract: Large language models (LLMs) require substantial compute, and thus energy, at inference time. While quantizing weights and activations is effective

arXiv:2607.07507v1 Announce Type: cross Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidenc

arXiv:2607.07117v1 Announce Type: cross Abstract: In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query

arXiv:2607.06601v1 Announce Type: cross Abstract: Conditional computation can decouple language model quality from per-token inference cost, yet leading techniques act on a single axis in isolation: M

arXiv:2507.05116v5 Announce Type: replace-cross Abstract: Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural languag

Based on the available information, b9905 is a build-tagged release from the llama.cpp project, which is an open-source C/C++ inference engine that powers most of the local-AI ecosystem . The project

Based on available information, b9910 is a release tag from the llama.cpp project, an open-source C/C++ implementation for running large language model inference locally on consumer hardware. Llama.cp

b9933 is a continuous build-tagged release of llama.cpp , the C/C++ inference engine for running large language models locally. This release represents an incremental update in llama.cpp's development

arXiv:2607.06326v1 Announce Type: new Abstract: Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-l

arXiv:2607.06269v1 Announce Type: new Abstract: Current large language models (LLMs) are fundamentally stateless: their behavior is fully determined by input at inference time, and any higher-order co

In the agentic era, AI is evolving from answering questions to reasoning and taking action. Companies who want to lead in this next phase of AI need computing infrastructure that’s designed and optimi

I cannot put into words how stoked I am on this It's not an understatement to say that this blueprint could represent the future of enterprise inference Introducing the NemoClaw Deep Agents Blueprint,

Speculative decoding is a technique implemented in llama.cpp that speeds up inference by using a smaller, faster model to predict multiple tokens ahead, which a larger model then verifies in parallel,

Own the frontier: 'In our evals, Nemotron 3 Ultra with a tuned LangChain Deep Agents harness provides advanced agent performance at a much lower inference cost. The main takeaway is that agent perform

arXiv:2507.23033v2 Announce Type: replace Abstract: Spiking Neural Networks (SNNs) provide an energy-efficient computing paradigm for neural rendering, but existing spike-based Neural Radiance Field (

arXiv:2607.04140v1 Announce Type: cross Abstract: Autoregressive (AR) text-to-speech (TTS) models generate discrete speech tokens sequentially, which makes inference slow and can degrade robustness by

arXiv:2607.04276v1 Announce Type: new Abstract: Diffusion models achieve impressive image-generation quality but remain expensive at inference time. Diffusion distillation reduces sampling steps, yet

arXiv:2512.24780v2 Announce Type: replace Abstract: Neural networks trained with standard objectives exhibit behaviors characteristic of probabilistic inference: soft clustering, prototype specializat

arXiv:2411.00214v2 Announce Type: replace-cross Abstract: Otto's Wasserstein gradient flow of the inclusive (forward) Kullback--Leibler (KL) divergence offers a principled framework for analyzing stat

arXiv:2607.03696v1 Announce Type: new Abstract: Existing Salient Object Detection in Optical Remote Sensing Image (ORSI-SOD) methods mainly adopt the static inference strategy, which uses fixed traine

arXiv:2607.04978v1 Announce Type: cross Abstract: Joint-Embedding Predictive Architectures (JEPAs) underpin a growing family of latent world models for control from raw pixels, but every existing JEPA

For years, enterprise AI has been synonymous with conversational AI — the customer service bots and digital assistants we interact with every day. But today, the market has shifted. We’ve officially m

arXiv:2603.19873v2 Announce Type: replace Abstract: Fine-tuning foundation models for Earth Observation is computationally expensive, with high training time and memory demands for both training and d

arXiv:2604.00660v2 Announce Type: replace-cross Abstract: Modern data warehouses extend SQL with semantic operators that invoke large language models on each qualifying row, making per-row inference o

This is what latency optimization looks like below the API 👇 Together ATLAS, NVIDIA Blackwell, CUDA, TensorRT-LLM, Dynamo, and custom kernels all working together to make inference faster for users. P

arXiv:2603.14504v2 Announce Type: replace-cross Abstract: Optimizing the noise samples of diffusion and flow models is an increasingly popular approach to align these models to target rewards at infer

Release b9892 is a version identifier for llama.cpp, an open-source C/C++ framework for running large language model inference on consumer hardware. This specific build (b9892) represents a snapshot o

Release b9873 is a version of llama.cpp, a C/C++ project designed to enable LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware locally and in the cloud. The

arXiv:2607.01117v1 Announce Type: new Abstract: Video Large Language Models (VideoLLMs) have shown strong progress in video understanding, yet they still suffer from hallucinations that are inconsiste

arXiv:2607.00529v1 Announce Type: new Abstract: Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to genera

arXiv:2607.00447v1 Announce Type: new Abstract: Large language models often produce hallucinated answers that violate prompt-level constraints. A key diagnostic question is whether these failures refl

Release b9858 is a continuous build-tagged release of llama.cpp , the open-source C/C++ project that enables large language model inference with minimal setup and optimized performance across diverse

arXiv:2606.31315v1 Announce Type: new Abstract: Speculative decoding accelerates inference by using a lightweight draft model to generate candidate tokens in parallel, and are then verified by the tar

arXiv:2606.31331v1 Announce Type: new Abstract: Collaborative inference can improve predictive performance by integrating complementary information across agents, but applying collaborative fusion to

arXiv:2606.31226v1 Announce Type: new Abstract: World-model-based autonomous driving endows the model with the ability to understand scene evolution. Yet this promise is undermined by the prevailing i

arXiv:2603.05353v2 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts.

arXiv:2603.13748v3 Announce Type: replace Abstract: Real-world robots often operate in settings where objective priorities depend on the underlying context of operation. When the underlying context is

Multi-token Residual Prediction is a technique for improving language model inference efficiency by predicting multiple tokens simultaneously rather than one at a time, reducing latency and computatio

arXiv:2511.12309v2 Announce Type: replace-cross Abstract: Self-consistency (SC) is a widely used test-time inference technique for improving performance in chain-of-thought reasoning. It consists of g

arXiv:2606.30849v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial infere