Search: “inference”

arXiv:2607.24027v1 Announce Type: new Abstract: Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Traini

How we built the OlmoEarth Platform to fine-tune geospatial models and run continent-scale satellite inference while managing massive data pipelines, distributed compute, and automatically recovering

Kimi K3 is live on Fireworks. Day 0, inference and training. US-hosted, and zero data retention. This is the first frontier open model in the 3 trillion parameter class. It sports 1M context, native v

arXiv:2607.21774v1 Announce Type: new Abstract: Large language models may infer demographic attributes from subtle linguistic cues even when those attributes are not explicitly stated. This pilot stud

We turned the browser into the inference server. @RunAnywhereAI ( @runanywhere/web ) runs models client-side with WebGPU + WASM SIMD, stores them in OPFS and generates with 0 outbound bytes. No backen

arXiv:2607.20723v1 Announce Type: cross Abstract: This work presents LeakyLMs, a set of attacks that leak proprietary model, architecture, and deployment information from production language models. L

arXiv:2607.20708v1 Announce Type: new Abstract: A recent line of work measures causal emergence in reinforcement learning agents through Integrated Information Decomposition, reporting that Phi_r grow

Phoebe Liu / The Information: AMD is working with Cerebras to combine AMD Helios and Cerebras wafer-scale chips for an AI inference solution to make workloads faster; CBRS closed up 4.86% — AMD said T

arXiv:2607.19431v1 Announce Type: cross Abstract: Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding t

arXiv:2607.20129v1 Announce Type: new Abstract: Quantized small autoregressive reasoning models can enter long, repetitive, or unproductive trajectories, yet inference-time compute is usually allocate

arXiv:2607.19704v1 Announce Type: new Abstract: Scaling LLM-based applications to millions of users is bottlenecked by the inference cost and latency of modern foundation models. A natural fix is to c

arXiv:2607.19623v1 Announce Type: cross Abstract: We characterize per-bit-position fault sensitivity in ML inference across 16 workloads -- spanning transformer-based models and attention-free CNNs --

arXiv:2607.19490v1 Announce Type: cross Abstract: Peer-to-peer distributed inference executes a Large Language Model (LLM) on pooled consumer hardware by spreading its layers across many nodes. Every

arXiv:2607.20327v1 Announce Type: new Abstract: Large language models (LLMs) provide strong reasoning capabilities but are expensive to serve at scale, whereas small language models (SLMs) are cheaper

arXiv:2603.07466v2 Announce Type: replace-cross Abstract: Cloud-based infrastructure has become the dominant platform for deploying large models, particularly large language models (LLMs). Fine-tuning

Vera Rubin NVL72 is Nvidia’s second‑generation, rack‑scale Oberon architecture that achieves inference gains through extreme co‑design. Early engineering‑sample data from CoreWeave show DeepSeek R1 de

arXiv:2607.11183v2 Announce Type: replace Abstract: Large language models increasingly operate as tool-using agents, where small format, argument, or function-call errors can invalidate otherwise plau

arXiv:2607.12650v1 Announce Type: cross Abstract: Tool access alone does not make LLM empirical reasoning governable: accepted outputs need not descend from attested evidence, and accepted deductions

We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 se

Local AI is the future. Learning how to run Opensource models (Inference), how to evaluate them systematically (Evals), and how to customize them (Fine-tuning / RL / Post-training) are invaluable skil

Hugging Face Gemma Challenge results are in! 📈 Over 6 days, more than 100 AI agents and humans collaborated to make Gemma 4 inference 5x faster on a single NVIDIA A10G GPU. - Fastest result: 491.8 TPS

This post discusses the conceptual distinction between training-scaling and inference-scaling in machine learning models, while acknowledging related frameworks or concepts (possibly referencing Sol,

arXiv:2607.06776v1 Announce Type: new Abstract: We introduce an efficient Bayesian deep ensemble method for predictive regression designed to enhance interpretability while maintaining competitive pre

In this post, we walk through five capabilities now available in SageMaker HyperPod inference: multi-tier data capture for auditing and model improvement, direct deployment from Hugging Face Hub, loca

arXiv:2607.07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantizati

Grok 4.5 is not yet using our internally developed C/C++ inference software that exact maps to the GB300 hardware. Doubling or more of the current speed is probably achievable. I think AI has just hit

Happy to announce that we’ve enlisted @FireworksAI_HQ as our primary provider for open-source models. In AI inference, quality, latency, reliability, and privacy vary a lot. Fireworks has earned our t

Local AI just got faster. ⚡️ We worked with @ggerganov to add DFlash support in llama.cpp delivering ~2x faster inference. llama.cpp recently added DFlash support to its speculative decoding arsenal.

We tuned the harness for @NVIDIAAI Nemotron 3 Ultra. Benchmark-leading performance. 10x lower inference costs. ✅ An aggregate score of 0.86 at a cost of 4.48 ✅ The closest-performing model: 43.48 http

We're introducing Provisioned Throughput: reserved inference capacity for frontier open models, with token-based pricing and a 99% uptime SLA. Serverless simplicity, guaranteed capacity, up to 90% low

We're releasing ZML/LLMD, our homegrown LLM server built on top of our homegrown high performance heterogeneous inference stack. It ships with 5 architectures out of the box: NVIDIA, AMD, Metal, Intel

arXiv:2607.03876v1 Announce Type: new Abstract: With the rise of small quantized GGUF-based language models and their increasing use for on-device inference tasks, we have seen the growing need for an

arXiv:2607.04740v1 Announce Type: new Abstract: Spatial Transcriptomics (ST) measures gene expression while preserving spatial context, but its high cost and low throughput leave public datasets small

arXiv:2607.04461v1 Announce Type: new Abstract: Inference-time scaling for text-to-image generation has progressed from simple Best-of-N (BoN) sampling to guided search methods that verify and steer c

arXiv:2512.22999v2 Announce Type: replace-cross Abstract: We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain. To this end, we in

arXiv:2607.03039v1 Announce Type: new Abstract: Neural networks are increasingly used to infer hidden physical structure from dynamical observations, yet it remains unclear whether their out-of-distri

arXiv:2511.04805v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models have shown strong potential in scaling language models efficiently by activating only a small subset of expert

Reuters: Sources: DeepSeek is in the early stages of developing its own AI chip designed for inference, in a bid to reduce its reliance on Nvidia and Huawei hardware — Chinese startup DeepSeek is deve

arXiv:2601.02211v2 Announce Type: replace Abstract: Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and

With inference scale and research scale to drive efficiency, and ever-improving frontier models as brains, this would be a way for the Labs to undercut even open weights models. Some companies will st

A big part of how the team at Runway is able to build and train the models we do and serve the inference demand we have is that we've built incredibly robust research infrastructure and tooling for al

GLM-5.2, an open-source model available through Hugging Face, can now be selected and used within Claude Code through Hugging Face Inference Providers and the hf-claude integration. This development d

arXiv:2607.02391v1 Announce Type: cross Abstract: Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requi

arXiv:2607.00870v1 Announce Type: new Abstract: We study inference-time pattern-memory gating in a production-scale clinical natural language processing (NLP) pipeline. The pipeline pairs a generator

arXiv:2601.11046v2 Announce Type: replace Abstract: Machine learning models for climate and Earth science are becoming increasingly capable, yet model deployment into operational use remains a largely

arXiv:2607.00780v1 Announce Type: new Abstract: Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic attention s