Knowledge catalogue
arXiv:2601.20332v3 Announce Type: replace Abstract: Diffusion language models (DLMs) generate text through iterative denoising, but inference requires full-sequence attention at every iteration, resul
arXiv:2609.14138v1 Announce Type: cross Abstract: As LLM agents become integrated into increasingly complex workflows, they must continually acquire new capabilities while retaining competence on prev
arXiv:2609.13205v1 Announce Type: cross Abstract: Sparse long-context inference requires efficient token retrieval in both prefill and decode. Existing methods often use different retrieval strategies
arXiv:2609.10105v1 Announce Type: cross Abstract: Compute governance today is a governance of training: the thresholds, reporting requirements, and frontier-AI regimes now in force attach to training
arXiv:2609.11058v1 Announce Type: new Abstract: Recent advances in multimodal large language mod- els (MLLMs) have opened new opportunities for edge intelligence by enabling reasoning across heterogen
arXiv:2609.10550v1 Announce Type: cross Abstract: AI deployment performance is shaped not by model architecture alone, but by interactions among compression, compiler transformations, and serving poli
Positron AI Inc., a developer of inference appliances that don’t include HBM memory, today announced that it has raised 875 million in funding. NEA, Atreides Management, Valor Equity Partners, Andra C
arXiv:2609.09241v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large fo
arXiv:2609.09357v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) enables inference on private data without revealing it to the server, but evaluating an entire input under FHE is e
arXiv:2609.10264v1 Announce Type: cross Abstract: Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inf
arXiv:2609.09662v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased pri
arXiv:2609.07660v1 Announce Type: cross Abstract: The concept of Markov chain Monte Carlo (MCMC) cycles, an analogy to cyclic processes in heat engines, is presented in order to examine Bayesian infer
arXiv:2603.26483v2 Announce Type: replace Abstract: Medical edge-AI systems must operate under a difficult tension: delivering reliable diagnostic inference while running on devices with limited batte
arXiv:2609.03149v1 Announce Type: cross Abstract: Analog in-memory computing (AIMC) speeds up neural-network inference by doing the arithmetic directly inside a memory array, instead of shuttling weig
NVIDIA’s PAIR Virtual Inference Router distributes AI inference requests across multiple local machines—supporting Ollama and LM Studio without any agent or harness changes—to mitigate multi‑agent bot
arXiv:2609.00738v1 Announce Type: new Abstract: Inference-time search with large language models (LLMs) often concentrates on a small set of structurally or semantically similar trajectories, leaving
Today we’re open-sourcing Lily, the local inference engine we built for hybrid compute in Perplexity Computer. Lily is specialized for Qwen3.6-35B-A3B on Apple silicon, built so on-device compute does
arXiv:2609.00372v1 Announce Type: cross Abstract: With quantum sensors, simulators and networks emerging, a future of quantum technology may produce quantum states as data---that is, coherently rather
arXiv:2608.28652v1 Announce Type: new Abstract: Artificial intelligence (AI) models have demonstrated remarkable capabilities across various domains, yet their widespread deployment is impeded by sign
for september, we’re cutting the price of Dedicated Inference on H100s from 5.49/hr to 3.99/hr new + existing deployments get the lower price automatically deploy gemma 4, qwen3/3.5, gpt-oss, llama, n
Inference workloads fall into four use‑case buckets (Chatbots/Copilots, Agents, Content Generation, Translation), each with distinct token patterns that dictate the required GPU memory and compute. Si
arXiv:2512.17452v4 Announce Type: replace-cross Abstract: Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear Key-Value (KV) cache growth. Prior approaches miti
arXiv:2608.28113v1 Announce Type: new Abstract: The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language
Folks! We're just 50 PRs away from more faster inference. Hopefully by end of year. Experts!, please chip in there. List of Open/Ongoing PRs(and also Discussions) related to CPU/RAM/Disk/Hybrid: [Disc
arXiv:2608.27455v1 Announce Type: new Abstract: Recent advances in inference-time scaling have significantly improved the reasoning performance of large language models (LLMs). However, these methods
NVIDIA TensorRT Model Connect delivers a set of reference implementations for deploying open‑source models from Hugging Face checkpoints to native C++ inference in two steps: a Python CLI builds a dep
arXiv:2608.27384v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models are increasingly promising for robotic manipulation, yet their real-world deployment remains bottlenecked by high in
Serving automatic speech recognition (ASR) models at scale is costly when each request uses only a fraction of a GPU. Learn how NVIDIA CUDA Multi-Process Service (MPS) with NVIDIA Triton Inference Ser
arXiv:2608.23831v1 Announce Type: cross Abstract: While reinforcement learning (RL) allows generalist robot policies to continually improve during deployment, the large model size of modern generalist
arXiv:2608.23923v1 Announce Type: new Abstract: Slicing-Aided Hyper Inference (SAHI) improves small object detection in high-resolution images but often spends substantial compute on background tiles.
arXiv:2608.24338v1 Announce Type: new Abstract: Inference-time decoding methods improve LLM reasoning by exploring multiple candidate trajectories, yet treat each trajectory as atomic: either retainin
AI storage infrastructure is becoming a more consequential planning issue as organizations move from model training toward agentic AI. As agents reason, act and reassess, they build longer contexts an
OpenAI reports Jalapeño delivers 1.5–1.9× more AI work per watt and 1.7–3.6× lower latency vs comparison systems on InferenceX.
Qwen3.8 27B is now available for both fine-tuning and Dedicated Model Inference Fine-tune it on your own data, then deploy that model on dedicated infrastructure for production, without stitching toge
Chipmaker Nvidia Corp. says its dedicated artificial intelligence inference accelerator Groq 3 LPX has now entered full production as it strives to maintain its dominance in the world of AI compute. T
Inference Hours is back with another Kimi K3 webinar! On Wednesday, August 26, Zain Hasan sits down with Kevin Cui and Austin Silveria from Together AI's inference team on how we actually serve K3 in
arXiv:2608.18294v1 Announce Type: cross Abstract: An increasing number of scholars use AI to measure variables they subsequently include in downstream analyses. Although AI-measured variables are ofte
arXiv:2608.18098v1 Announce Type: cross Abstract: Key-value (KV) caching is essential for efficient autoregressive inference in transformer based dialog systems, yet existing strategies treat all cach
Amazon Bedrock now offers OpenAI GPT-5.6 models (Sol, Terra, and Luna) in more than 25 AWS Regions with cross-Region inference. Learn how US geographic and global inference profiles route requests for
arXiv:2412.04504v2 Announce Type: replace Abstract: As large language models (LLMs) grow in popularity for their diverse capabilities, improving the efficiency of their inference systems has become in
arXiv:2608.16477v1 Announce Type: new Abstract: AI-RAN brings large language model (LLM) serving close to mobile users, but cellular handover can separate an active request from its inference state: t
arXiv:2608.13144v1 Announce Type: cross Abstract: As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sa
arXiv:2608.13426v1 Announce Type: cross Abstract: Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplicati
arXiv:2507.15100v3 Announce Type: replace-cross Abstract: Natural Language Inference (NLI) determines whether a premise entails, contradicts, or is neutral with respect to a hypothesis. The task is of
Counterpoint Research: In Q2 2026, server-led eSSDs reached 48% of NAND flash shipments as AI workloads shifted from training to inference, driving 5x YoY industry revenue growth — - AI inference work
arXiv:2607.17154v2 Announce Type: replace-cross Abstract: Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it rema
arXiv:2603.18446v2 Announce Type: replace Abstract: Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selectio
arXiv:2608.07480v1 Announce Type: new Abstract: Active inference has emerged as a principled framework for modeling adaptive behavior by balancing goal-directed action with uncertainty reduction. It h
arXiv:2608.09512v1 Announce Type: new Abstract: Active inference offers a unified framework for perception, learning, and action, but scaling discrete active-inference models to rich spatial and tempo
Today the most important factor in AI is cost. My idea is to split ML models inference (closed ones, proprietary) across server and edge computing on clients, and I would like to hear what do you thin
Zain (@zainhas) published a detailed article on August 7, 2026 explaining that autoscaling for highly peaky large‑language‑model (LLM) inference is fundamentally different from autoscaling conventiona
arXiv:2507.03897v3 Announce Type: replace Abstract: We introduce GenAI-Powered Inference (GPI), a statistical framework for both causal and predictive inference using unstructured data, including text
arXiv:2608.00720v1 Announce Type: cross Abstract: Mapping neural networks to FPGAs enables low-latency, energy-efficient inference, particularly for lookup table (LUT)-based models that eliminate mult
arXiv:2608.02353v1 Announce Type: new Abstract: Recent advances in agentic workflow optimization automate workflow design through task-specific workflow search or input-conditioned architecture select
arXiv:2607.28848v1 Announce Type: cross Abstract: LLM serving systems are provisioned for peak load to meet strict latency targets, leaving substantial GPU compute idle whenever traffic falls below pe
arXiv:2607.28633v1 Announce Type: cross Abstract: Disaggregated LLM inference creates a datacenter networking problem that no existing system solves correctly. When prefill and decode run on separate
Using optimization at inference time is a foundational concept of Energy-Based Models (EBM) and Objective-Driven AI architectures (ODAI). When the variables to be inferred are continuous, it makes sen
Learn how to build an inference meta-monitoring system for Amazon SageMaker AI endpoints using Amazon Quick. This governance layer sits above production ML inference pipelines to continuously track pr
arXiv:2607.24148v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have demonstrated strong potential for embodied AI, yet their high inference latency on GPUs limits real-time deploy
arXiv:2607.23478v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) provides strong cryptographic guarantees for private inference, but deploying transformer models under FHE remains