Recirculation
arXiv:2608.17981v1 Announce Type: new Abstract: We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across
Knowledge catalogue
arXiv:2608.17981v1 Announce Type: new Abstract: We describe an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across
IIRC, this is the 3rd month in a row that @FireworksAI_HQ is being featured on @tryramp's top software vendor list. This isn't just for inference / model serving (of which we serve 40T+ tokens a day),
arXiv:2608.14490v1 Announce Type: new Abstract: We present a Test-time World-model Inference (Twin) system, in which a frontier coding agent writes an executable world model for completing continual l
arXiv:2608.10362v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to speculate multiple tokens,
arXiv:2607.17188v2 Announce Type: replace Abstract: While test-time scaling improves the problem-solving ability of large reasoning models (LRMs) through additional inference-time computation, it can
Back after ~6 months — rebuilding my spiking language model around CPU-first inference Hey everyone. It’s been around six months since I last posted anything about this project here. Some of you might
I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan is to start with 2x 16GB GPUs = 32
arXiv:2608.03046v1 Announce Type: new Abstract: Text-to-video (T2V) diffusion transformers (DiTs) are trained with detailed video captions, whereas inference often relies on user prompts rewritten by
arXiv:2608.03682v1 Announce Type: new Abstract: Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, a
arXiv:2608.01979v1 Announce Type: cross Abstract: Visual token pruning reduces the inference cost of multimodal large language models, but a fixed token ratio is poorly matched to text-rich inputs. In
arXiv:2608.01034v1 Announce Type: new Abstract: We introduce Opt.Gear, a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability. It includes a den
Built my 1st inference machine and have been tweaking models trying to get the most out of my modest hardware. I think I’m at a good place but I’m testing with my own prompts. I’ve looked into some of
arXiv:2607.27574v1 Announce Type: new Abstract: Activation steering has emerged in large language models as a lightweight alternative for dynamically changing a model's behavior at inference time. How
At Google, AI is a soup-to-nuts endeavor. Obviously, we make leading AI models like Gemini and Nano Banana. We incorporate AI into the tools you use every day (think Gmail, BigQuery, AlloyDB, Google C
arXiv:2607.23362v1 Announce Type: new Abstract: Recent work has shown that subword vocabularies can be trained to optimize compression for a specific inference rule rather than relying on greedy heuri
arXiv:2607.22586v1 Announce Type: new Abstract: Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with co
arXiv:2603.01119v2 Announce Type: replace-cross Abstract: A fundamental challenge in causal inference with observational data is correct specification of a causal model. When there is model uncertaint
b9908 is a build-tagged release from llama.cpp , the open-source C/C++ inference engine for large language models. llama.cpp is an open-source software library that performs inference on various large
arXiv:2607.06216v1 Announce Type: new Abstract: The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High-frame-rate infe
This entry covers the integration of vLLM, a high-performance inference engine, with Hugging Face's transformers library to enable faster language model inference. The native-speed backend allows user
arXiv:2607.05252v1 Announce Type: new Abstract: Simulation-Based Inference (SBI) is critical for scientific discovery, with generative models offering a promising path toward efficient inference. Howe
arXiv:2606.28600v1 Announce Type: cross Abstract: Neuromorphic and edge computing research has focused on reducing the inference cost of neural network controllers, yet in physical closed-loop systems
arXiv:2606.26922v1 Announce Type: cross Abstract: Continuous driver monitoring in automated vehicles requires low-latency inference while avoiding unsafe decisions under uncertain driver states. Large
arXiv:2606.21238v1 Announce Type: cross Abstract: Key-value (KV) caching is a powerful technique for accelerating large language model inference and generation. Inference workloads are large and diver
B9542 is a build/release version of llama.cpp, the C/C++ implementation of large language model inference designed to enable LLM inference with minimal setup and state-of-the-art performance on variou
arXiv:2606.01269v1 Announce Type: new Abstract: Transitive inference is the challenge of inferring that A < C from knowing only adjacent relations (A < B, B < C). It is solved by humans and animals no
arXiv:2604.01802v2 Announce Type: replace Abstract: Real-time inference of inaccessible interior physical fields from sparse boundary observations is a fundamental but unresolved problem in scientific
arXiv:2603.26506v2 Announce Type: replace-cross Abstract: Connectivity structure shapes neural computation, but inferring this structure from population recordings is degenerate: multiple connectivity
arXiv:2601.03471v3 Announce Type: replace-cross Abstract: Reliable epidemiological reasoning requires synthesizing study evidence to infer disease burden, transmission dynamics, and intervention effec
arXiv:2601.01608v2 Announce Type: replace Abstract: Diffusion models deliver high quality in image synthesis but remain expensive during training and inference. Recent works have leveraged the inheren
arXiv:2605.26584v1 Announce Type: new Abstract: Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks
arXiv:2605.26059v1 Announce Type: cross Abstract: Bayesian inverse design provides a principled framework for inferring aerodynamic geometries from sparse flow observations while quantifying uncertain
arXiv:2509.19929v4 Announce Type: replace-cross Abstract: Uncertainty Quantification (UQ) is paramount for inference in engineering. A common inference task is to recover full-field information of phy
B8984 is a build/release version of llama.cpp, an open-source C/C++ library for large language model inference. Llama.cpp enables LLM inference with minimal setup and state-of-the-art performance on a
arXiv:2604.24938v1 Announce Type: cross Abstract: Depth pruning improves the inference efficiency of large language models by removing Transformer blocks. Prior work has focused on importance criteria
Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr
arXiv:2601.00823v2 Announce Type: replace Abstract: Large reasoning models (LRMs) have heterogeneous inference energy costs based on which model is used and how much it reasons. To reduce energy, it i
arXiv:2509.22166v4 Announce Type: replace-cross Abstract: The demand for efficient large language model (LLM) inference has intensified the focus on sparsification techniques. While semi-structured (N
arXiv:2602.06019v2 Announce Type: replace Abstract: Existing techniques for accelerating language model inference, such as speculative decoding, require training auxiliary speculator models and buildi
B8906 is a release tag for llama.cpp, an open-source C/C++ library for local large language model inference. llama.cpp enables LLM inference in C/C++ , and the project uses sequential build identifier
This week at Google Cloud Next ‘26, we are sharing the evolution of Google Kubernetes Engine (GKE), delivering leading performance, efficiency, security, and scale for your most demanding and complex
arXiv:2604.17140v1 Announce Type: cross Abstract: We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the
arXiv:2604.17695v1 Announce Type: cross Abstract: KV cache memory is the dominant bottleneck for long-context LLM inference. Existing compression methods each act on a single axis of the four-dimensio
arXiv:2604.17320v1 Announce Type: new Abstract: Deploying Vision-Language Models (VLMs) under aggressive low-bit inference remains challenging because inference cost is dominated by the long visual-to
b8853 is a release of llama.cpp , the open-source C/C++ library for running large language model inference. The project enables LLM inference with minimal setup and state-of-the-art performance on a w
arXiv:2604.15357v1 Announce Type: cross Abstract: Precise estimation of model inference latency is crucial for time-critical mobile edge applications, enabling devices to calculate latency margins aga
arXiv:2604.16027v1 Announce Type: cross Abstract: Post-trained language models produce less varied outputs than their base counterparts. This output diversity collapse undermines inference-time scalin
arXiv:2601.03523v2 Announce Type: replace Abstract: One of the most important queries in knowledge compilation is weighted model counting (WMC), which has been applied to probabilistic inference on va
arXiv:2604.12660v1 Announce Type: new Abstract: In nonmonotonic reasoning from conditional belief bases, an inference operator satisfying syntax splitting postulates allows for taking only the relevan
arXiv:2601.15498v2 Announce Type: replace Abstract: Speculative Decoding (SD) accelerates autoregressive large language model (LLM) inference by decoupling generation and verification. While recent me
arXiv:2604.11080v1 Announce Type: cross Abstract: Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large La
Auto-generated index of all people mentioned across the wiki.
Today, Reflection released Beam, its first open-weight model built to give developers, researchers, and enterprises open control and customization for frontier software engineering and long-horizon ag
arXiv:2610.21204v1 Announce Type: new Abstract: When a speaker says ``some students passed'', listeners infer that not all of them did. Reasoning models now let users decide how long a model thinks be
arXiv:2610.18304v1 Announce Type: new Abstract: Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due
arXiv:2610.19855v1 Announce Type: new Abstract: As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) mo
arXiv:2610.20055v1 Announce Type: new Abstract: The rapid advancement of large vision language models (LVLMs) has made geolocation inference from images highly accurate and accessible. This poses seri
arXiv:2610.19796v1 Announce Type: cross Abstract: In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting o
arXiv:2610.19051v1 Announce Type: cross Abstract: Coordinators that act on what they infer about teammates are usually certified by the accuracy of those inferences. We argue that accuracy is the wron
arXiv:2610.14571v1 Announce Type: cross Abstract: Large Language Model (LLM) inference is driving a growing share of datacenter carbon emissions. Although carbon-aware scheduling can reduce these emis