Knowledge catalogue
arXiv:2608.08730v1 Announce Type: new Abstract: Large Language Models are deployed to multiple types of environments, from internet browsers to edge devices, and WebGPU serves as a modern cross-platfo
Nemotron 3.5 Lightning is available on Together AI through Dedicated Model Inference, giving teams reserved capacity and predictable performance for high-volume agent workloads. Start building: https:
arXiv:2608.06836v1 Announce Type: new Abstract: We study the problem of inserting new furniture into indoor scene images. Under masked single-view 2D image-plane conditioning, however, the physical sc
Voyage AI models now run natively on Fireworks, the first and only dedicated inference platform @VoyageAI by @MongoDB has partnered with. Embed, retrieve, rerank, generate: your full retrieval pipelin
Over the past few months I have been building a CPU-first inference engine from scratch in pure C99 (no Python, no CUDA, no BLAS, just GCC and make). The focus has been running 1.58-bit ternary models
arXiv:2608.05244v1 Announce Type: cross Abstract: We developed a unified covariate-adjusted causal inference framework for estimating the desirability of outcome ranking (DOOR) probability for benefit
arXiv:2608.06025v1 Announce Type: new Abstract: In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, wher
arXiv:2608.05627v1 Announce Type: new Abstract: Training-free open-vocabulary segmentation remains limited by a missing inference abstraction. Frozen vision-language features are produced at patch lev
Actual is a local inference stack optimized to let you utilize your personal compute. With its low CPU impact while inferencing and multi-platform portability, it’s a great pair for your local Hermes
arXiv:2608.04401v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models enable model scaling while maintaining low inference-time compute by activating only a subset of experts per token. Howe
arXiv:2604.07396v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) inference on edge Neural Processing Units (NPUs) is fundamentally constrained by limited on-chip memory capacity. A
arXiv:2608.04714v1 Announce Type: cross Abstract: Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are co
arXiv:2510.21889v2 Announce Type: replace-cross Abstract: Causal inference identifies cause-and-effect relationships between variables. While traditional approaches rely on data to reveal causal links
arXiv:2608.03566v1 Announce Type: cross Abstract: The drift diffusion model (DDM) is a cornerstone of cognitive decision-making research. Although numerous estimation methods exist, researchers contin
arXiv:2608.03216v1 Announce Type: new Abstract: Query-based mask transformers assemble segmentation outputs through pixel-wise competition among query predictions of the final layer, yet this inferenc
arXiv:2512.12840v2 Announce Type: replace-cross Abstract: Vertical Federated Learning (VFL) enables collaborative model training across organizations that share common user samples but hold disjoint f
arXiv:2608.03335v1 Announce Type: new Abstract: Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost, making inference prohibitive at video-token scales. Th
arXiv:2509.24560v2 Announce Type: replace Abstract: Extended Chain-of-Thought (CoT) reasoning has significantly bolstered the capabilities of medical large language models (LLMs). However, current mod
arXiv:2608.00175v1 Announce Type: new Abstract: Deep reinforcement learning (RL) agents achieve strong performance by optimizing scalar reward functions. However, once deployed, the policies of these
arXiv:2608.02440v1 Announce Type: cross Abstract: In noisy social dilemmas, intended actions are stochastically corrupted before execution, so an observed defection may reflect hostile intent or actio
arXiv:2608.00195v1 Announce Type: cross Abstract: High-resolution 3D segmentation of hip and shoulder anatomy from CT and MRI is essential for surgical planning, yet frozen segmentation models often f
arXiv:2608.01563v1 Announce Type: new Abstract: Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries. Meganeura asks whether one compact native comp
arXiv:2608.00573v1 Announce Type: cross Abstract: Serving Mixture-of-Experts (MoE) large language models across distributed edge servers is bottlenecked by the cross-server expert transmission. The ex
The Inference Engineering Masterclass: 10x faster models, quantization, speculative decoding, Rubin, & self-optimizing AI https://www.latent.space/p/inference-eng @Baseten @philipkiely and @waterloo_i
arXiv:2607.27694v1 Announce Type: cross Abstract: Low-bit quantization is essential for efficient LLM inference, and both rotation and fine-grained group quantization have shown individual promise. Ho
arXiv:2607.27952v1 Announce Type: new Abstract: Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge d
arXiv:2607.28418v1 Announce Type: cross Abstract: Pruning is a promising approach for improving the efficiency of LLMs. Existing static structured pruning methods are hardware-friendly and can deliver
arXiv:2607.25852v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best
Disaggregated AI inference is proving to be more than a complementary answer to the prefill and decode bottleneck slowing enterprise AI at scale, and Cerebras and AMD just announced a partnership to b
arXiv:2607.25018v1 Announce Type: new Abstract: Large language model (LLM) cascades reduce inference cost by routing easy queries to a small model and deferring hard queries to a larger one. Productio
arXiv:2607.24434v1 Announce Type: cross Abstract: Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, bu
arXiv:2607.23394v1 Announce Type: new Abstract: Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data
arXiv:2607.22556v1 Announce Type: new Abstract: Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments. However
Kimi K3 is now available for inference & training in @FireworksAI_HQ. Crazy how easy they make it to tune frontier open models like K3 using LoRA adapters. Best time to figure out how to own your inte
Kimi K3 will be here on Monday, but how do you know you’re ready to scale it in production? With our latest inference platform updates you can: 1/ Run shadow traffic to see how a new model performs on
arXiv:2607.20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or n
arXiv:2607.19438v1 Announce Type: cross Abstract: Apple's M5 generation introduces a redesigned GPU architecture in which every core carries a dedicated Neural Accelerator: on-die matrix units exposed
arXiv:2607.19353v1 Announce Type: new Abstract: Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary mo
Kimi K3 is coming to Together AI on day zero, July 27. Start building with it the moment it launches, powered by Together AI inference for coding, agents, and production workloads. https://www.togethe
arXiv:2606.17930v2 Announce Type: replace Abstract: AI evaluations are shifting toward harder tasks that benefit from longer trajectories involving tool use and iterative problem solving. As a result,
arXiv:2607.12169v1 Announce Type: new Abstract: Intercomprehension refers to partial intelligibility of an unfamiliar language (L2) by a speaker of a related language (L1). How is this zero-shot cross
arXiv:2512.20685v3 Announce Type: replace-cross Abstract: Diffusion models have recently emerged as powerful learners for simulation-based inference (SBI), enabling fast and accurate estimation of lat
arXiv:2607.06982v1 Announce Type: new Abstract: Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of
Fun conversation with @swyx on our journey building the cloud for true elastic inference, sandboxes, and more. And of course, how we're evolving Modal's dev experience to be better for agents. Modal's
arXiv:2511.16137v2 Announce Type: replace Abstract: Existing studies on quality enhancement for compressed video (QECV) predominantly rely on known quantization parameters (QPs), training separate enh
arXiv:2607.06519v1 Announce Type: new Abstract: Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specif
The shift from model training to agentic inference is forcing a fundamental rethink of how artificial intelligence infrastructure is built and which components carry the most strategic weight. What wa
The race to serve AI inference faster and cheaper is exposing the hard limits of conventional chip architecture. As demand for real-time AI responses accelerates, the industry’s standard response — st
arXiv:2607.06306v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex pro
We're still just scratching the surface of how much better, faster, cheaper we can make inference for open-source models! So super excited to see companies like @zml_ai by @steeve come out of stealth
arXiv:2607.05272v1 Announce Type: cross Abstract: Inference serving systems must balance throughput and latency under bursty, heterogeneous workloads, yet the industry standard remains static batching
arXiv:2607.04302v1 Announce Type: cross Abstract: We present HiFA4, a post-training operator-level design that executes both QK^T and PV in FlashAttention as 4-bit HIF4 Cube GEMMs for LLM inference on
arXiv:2607.04339v1 Announce Type: cross Abstract: Large generative models across text-to-text, text-to-image, and image-to-text modalities have been shown to pose significant privacy risks. One fundam
arXiv:2504.08791v3 Announce Type: replace-cross Abstract: On-device inference offers privacy, offline use, and instant response, but consumer hardware restricts large language models (LLMs) to low thr
arXiv:2601.11960v3 Announce Type: replace-cross Abstract: Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide w
DeepSeek Ltd., one of China’s most visible artificial intelligence companies, is pushing to design its own, in-house silicon aimed at inference workloads, according to a report by Reuters today. The r
arXiv:2607.02819v1 Announce Type: cross Abstract: Cloud-edge Large Vision-Language Model (LVLM) inference enables efficient deployment by splitting computation between edge devices and cloud servers.
Most models run on whatever inference engine they ship with. That's rarely the fastest option. We built our own, AGI-RUN. On a phone it beat every model's own native engine, and ran up to 9.9x faster
arXiv:2607.02119v1 Announce Type: cross Abstract: While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe
arXiv:2607.02501v1 Announce Type: new Abstract: Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model