Search: “inference”

A 7B agent that beats GPT-4o. Lossless weight compression that speeds up inference by 177%. An arena where 23 teams battled across 103,000 adversarial rounds. This year at ICLR, Lambda is presenting t

arXiv:2604.20156v1 Announce Type: new Abstract: Mixture-of-Experts models, now popular for scaling capacity at fixed inference speed, switch experts at nearly every token. Once a model outgrows availa

B8881 is a release of llama.cpp, an open-source C/C++ project for LLM inference . The project aims to enable LLM inference with minimal setup and state-of-the-art performance on a wide range of hardwa

B8886 is a release version from llama.cpp, the C/C++ project for LLM inference. The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art performance on a wide rang

At Google Cloud Next, we are announcing innovations across every layer of our storage stacks — performance, intelligence, and management — to ensure your data is as fast and as useful as the AI models

b8870 is a release from llama.cpp, a project for LLM inference in C/C++ . This release represents a commit snapshot in the rapidly-developed open-source project that enables efficient inference of lar

arXiv:2604.15764v1 Announce Type: cross Abstract: Early-exit neural networks enable adaptive computation by allowing confident predictions to exit at intermediate layers, achieving 2-8imes inference s

b8841 is a release of llama.cpp dated April 18, 2026 , a C/C++ library for large language model inference. The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art

arXiv:2604.14552v1 Announce Type: cross Abstract: Modern datacenters increasingly rely on low-power, single-slot inference accelerators to balance performance, energy efficiency, and rack density cons

arXiv:2604.13440v1 Announce Type: new Abstract: Deploying Large Language Models (LLMs) on edge devices faces severe computational and memory constraints, limiting real-time processing and on-device in

arXiv:2604.05672v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often c

if you’re interested in local/edge AI related repos and their trends, these are your guys Launching Inference Radar: our new weekly newsletter that tracks the top 130+ inference repositories, monitori

Vultr offers preemptible cloud instances powered by AMD Instinct GPUs, providing a cost-effective option for running GPU-accelerated workloads such as AI/ML training, inference, and high-performance c

arXiv:2604.12254v1 Announce Type: cross Abstract: SpanKey is a lightweight way to gate inference without encrypting weights or chasing leaderboard accuracy on gated inference. The idea is to condition

arXiv:2604.09621v1 Announce Type: new Abstract: We present an agent-driven approach to the construction of parameter inference pipelines for scientific data analysis. Our method leverages a multi-agen

arXiv:2504.06307v2 Announce Type: replace-cross Abstract: The rapid adoption of large language models (LLMs) has led to significant energy consumption and carbon emissions, posing a critical challenge

arXiv:2604.11484v1 Announce Type: new Abstract: On-the-Fly Category Discovery (OCD) requires a model, trained on an offline support set, to recognize known classes while discovering new ones from an o

arXiv:2509.25835v4 Announce Type: replace Abstract: Test-time scaling improves large language models (LLMs) on long-horizon reasoning tasks by allocating more compute at inference. LLM inference via t

arXiv:2604.06832v2 Announce Type: replace Abstract: Vision-language models (VLMs) predominantly rely on autoregressive decoding, which generates tokens one at a time and fundamentally limits inference

In freeskiing and snowboarding, traditional video replay shows you what happened during a complex aerial maneuver, but it fails to explain the physics of how it was possible. At the speed of the sport

arXiv:2603.11090v2 Announce Type: replace Abstract: Prior-data fitted networks (PFNs) have emerged as powerful foundation models for tabular causal inference, yet their extension to time series remain

arXiv:2502.17421v4 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) can now process extremely long contexts, efficient inference over these extended inputs has become increasingl

arXiv:2604.08536v1 Announce Type: new Abstract: We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward La

arXiv:2604.06277v1 Announce Type: new Abstract: Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retri

The Nous Research Portal (portal.nousresearch.com) is the account and API management hub for Nous Research's inference offerings. Nous Research launched an Inference API serving its open-source mo...

In production systems today, most AI agents operate as lightweight dispatchers. They parse a prompt, make an API call, retrieve a record, and hand back control. Their execution is fast, stateless and

arXiv:2610.22273v1 Announce Type: cross Abstract: Reasoning over long document contexts remains a challenge for large language models (LLMs). Standard retrieval methods are inherently reactive, bottle

arXiv:2610.14136v2 Announce Type: new Abstract: Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noi

arXiv:2603.00578v2 Announce Type: replace-cross Abstract: Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial i

arXiv:2402.03694v3 Announce Type: replace-cross Abstract: Network traffic analysis in cloud and internet service provider (ISP) networks increasingly relies on machine learning (ML) models, as encrypt

arXiv:2610.16182v1 Announce Type: new Abstract: Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit th

arXiv:2610.19712v1 Announce Type: new Abstract: Multi-session agents can switch between LLMs to balance performance and inference cost. Such switches require transferring a history for continuity, yet

arXiv:2610.19048v1 Announce Type: new Abstract: Record linkage and demographic inference from names are routine demands in social science. Most approaches to these problems require labeled training da

arXiv:2610.18143v1 Announce Type: new Abstract: Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in

arXiv:2508.16939v2 Announce Type: replace Abstract: Diffusion models achieve state-of-the-art generative performance but remain computationally expensive at inference, often requiring many discretizat

arXiv:2610.19362v1 Announce Type: new Abstract: Best-of-N sampling is a simple inference-time scaling strategy for large language models, yet its behavior under incomplete reward models remains poorly

arXiv:2610.19074v1 Announce Type: new Abstract: Reliable uncertainty quantification remains an important challenge in AI. Hierarchical Bayesian methods provide a principled framework for this by repre

arXiv:2610.19121v1 Announce Type: cross Abstract: Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reaso

arXiv:2610.16041v1 Announce Type: new Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of language tasks. At their core, autoregressive LLMs are conditional

arXiv:2602.00865v2 Announce Type: replace Abstract: While multi-view 3D reconstruction has shifted toward large-scale foundation models capable of inferring globally consistent geometry, their relianc

arXiv:2610.17686v1 Announce Type: cross Abstract: Longer contexts allow large language models (LLMs) to condition their predictions on more information, but inference costs grow prohibitively with con

arXiv:2610.15921v1 Announce Type: new Abstract: With ever increasing personalized large language model (LLM) services, the next question to answer is how to improve the quality of services/users' expe

arXiv:2610.17151v1 Announce Type: cross Abstract: Variational inference is a flexible tool for modeling experimental data and has proven useful in crystallography. Pathwise gradient estimators provide

arXiv:2508.10688v3 Announce Type: replace Abstract: Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved

arXiv:2610.14519v1 Announce Type: new Abstract: Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge. Existing studies address this c

arXiv:2610.17636v1 Announce Type: new Abstract: The Safety-Aware Denoiser (SAD) is a recent, training-free, safety-guidance framework for text diffusion models. During inference, SAD steers text sampl