Search: “inference”

arXiv:2604.18556v1 Announce Type: new Abstract: Weight quantization has become a standard tool for efficient LLM deployment, especially for local inference, where models are now routinely served at 2-

arXiv:2505.07033v2 Announce Type: replace-cross Abstract: Many classification performance metrics exist, each suited to a specific application. However, these metrics often differ in scale and can exh

arXiv:2604.18199v1 Announce Type: new Abstract: Transformer-based embedding models suffer from quadratic computational and linear memory complexity, limiting their utility for long sequences. We propo

arXiv:2503.16549v2 Announce Type: replace Abstract: Despite strong results on many tasks, multimodal large language models (MLLMs) still underperform on visual mathematical problem solving, especially

open-source community will work their magic! If any inference providers could serve kimi k2.6 at 300+ tps reliably, I would be happy to switch from gpt 5.4 xhigh to kimi for daily tasks. It's a really

arXiv:2604.01348v2 Announce Type: replace Abstract: Test-time scaling has emerged as an effective way to improve language models on challenging reasoning tasks. However, most existing methods treat ea

arXiv:2603.05863v2 Announce Type: replace Abstract: While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward

arXiv:2604.18396v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency

arXiv:2604.16606v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains

B8852 is a release of llama.cpp, an LLM inference library in C/C++ . This build tag likely contains bug fixes, performance improvements, and feature updates from the llama.cpp development cycle, which

Release b8859 is a version of llama.cpp, an open-source project that enables LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware. The llama.cpp project uses a

arXiv:2602.22159v2 Announce Type: replace Abstract: Arbitrary-Scale SR (ASISR) remains fundamentally limited by cross-scale distribution shift: once the inference scale leaves the training range, nois

arXiv:2603.08899v2 Announce Type: replace Abstract: Speculative decoding has emerged as a powerful approach to accelerate large language model (LLM) inference by employing lightweight draft models to

arXiv:2604.15714v1 Announce Type: cross Abstract: Always-on converter health monitoring demands sub-mW edge inference, a regime inaccessible to GPU-based physics-informed neural networks. This work se

arXiv:2604.15728v1 Announce Type: cross Abstract: Large language model (LLM) routing has emerged as a critical strategy to balance model performance and cost-efficiency by dynamically selecting servic

arXiv:2603.03188v2 Announce Type: replace-cross Abstract: We introduce a novel framework for uncertainty quantification in clustering that combines martingale posterior distributions with density-base

arXiv:2604.15613v1 Announce Type: cross Abstract: We present VoodooNet, a non-iterative neural architecture that replaces the stochastic gradient descent (SGD) paradigm with a closed-form analytic sol

b8842 is a release of llama.cpp, a C/C++ implementation for LLM inference. The llama.cpp project publishes multiple releases in a single day as part of its active development cycle. This specific rele

arXiv:2604.14682v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model (LLM) inference. It uses a small draft model to propose a tree of future tokens. A larger target

arXiv:2604.15180v1 Announce Type: cross Abstract: Sparse attention has been proposed as a way to alleviate the quadratic cost of transformers, a central bottleneck in long-context training. A promisin

b8823 is a build release of llama.cpp, the C/C++ implementation for LLM inference that enables efficient language model execution locally and in the cloud. Based on the rapid development cycle of the

arXiv:2604.14612v1 Announce Type: cross Abstract: Self-speculative decoding is an inference technique for large language models designed to speed up generation without sacrificing output quality. It c

arXiv:2604.13959v1 Announce Type: new Abstract: As AI moves from data centers to robots and wearables, scaling ever-larger models becomes insufficient. Physical AI operates under tight latency, energy

arXiv:2604.15244v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose outputs that a stronger target mod

arXiv:2604.14885v1 Announce Type: new Abstract: Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigate

arXiv:2604.14563v1 Announce Type: new Abstract: Vision Transformer (ViT)-based sparse multi-view 3D object detectors have achieved remarkable accuracy but still suffer from high inference latency due

arXiv:2502.21029v2 Announce Type: replace-cross Abstract: Reliable localization of people is fundamental for service and social robots that must operate in close interaction with humans. State-of-the-

arXiv:2512.15925v2 Announce Type: replace Abstract: Reading stories evokes rich interpretive, affective, and evaluative responses, such as inferences about narrative intent or judgments about characte

arXiv:2511.10946v3 Announce Type: replace Abstract: Vision-language models (VLMs) struggle with 3D-related tasks such as spatial cognition and physical understanding, which are crucial for real-world

Build b8808 is an incremental release of **llama.cpp**, the open-source C/C++ library for running large language model (LLM) inference locally. Like all llama.cpp builds, it likely includes bug fixes,

llama.cpp is a C/C++ library for LLM inference that enables running large language models on consumer hardware. Release b8814 is a specific version in the project's continuous release cycle, which fol

arXiv:2603.12725v3 Announce Type: replace Abstract: In-context operator learning enables neural networks to infer solution operators from contextual examples without weight updates. While prior work h

Most Physical AI models recognize patterns. They don’t understand the world. That’s why they fail on edge cases. BADAS 2.0 is a V-JEPA2 world model trained by @getnexar on real-world videos. We used t

arXiv:2604.13403v1 Announce Type: new Abstract: In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations. Despite its success in large language models, the exte

arXiv:2604.12434v1 Announce Type: cross Abstract: In-context learning enables transformers to adapt to new tasks from a few examples at inference time, while grokking highlights that this generalizati

arXiv:2603.18104v3 Announce Type: replace Abstract: Prevailing AI training infrastructure assumes reverse-mode automatic differentiation over IEEE-754 arithmetic. The memory overhead of training relat

Build b8797 is a sequentially numbered incremental release of llama.cpp, the open-source C/C++ library for local LLM inference maintained by ggml-org on GitHub. As part of llama.cpp's rapid, continuou

arXiv:2604.11915v1 Announce Type: cross Abstract: Modern machine learning methods have been proposed to detect life in extraterrestrial samples, drawing on their ability to distinguish biotic from abi

arXiv:2603.25414v2 Announce Type: replace-cross Abstract: A prevailing assumption in machine learning is that model correctness must be enforced after the fact. We observe that the properties determin

arXiv:2604.12176v1 Announce Type: new Abstract: Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables. This ability is central to scienti

arXiv:2512.10226v2 Announce Type: replace Abstract: Recent Vision-Language-Action (VLA) models for autonomous driving explore inference-time reasoning as a way to improve driving performance and safet

arXiv:2604.12651v1 Announce Type: cross Abstract: Knowledge graph embedding (KGE) models perform well on link prediction but struggle with unseen entities, relations, and especially literals, limiting

arXiv:2604.11995v1 Announce Type: new Abstract: The central goal of active learning is to gather data that maximises downstream predictive performance, but popular approaches have limited flexibility

arXiv:2604.12171v1 Announce Type: cross Abstract: Pipeline parallelism (PP) is widely used to partition layers of large language models (LLMs) across GPUs, enabling scalable inference for large models

arXiv:2604.12219v1 Announce Type: cross Abstract: Video Diffusion Transformers have revolutionized high-fidelity video generation but suffer from the massive computational burden of self-attention. Wh

arXiv:2604.12273v1 Announce Type: cross Abstract: Flow matching has emerged as a powerful generative framework, with recent few-step methods achieving remarkable inference acceleration. However, we id