Search: “inference”

Inference of Online Newton Methods with Nesterov's Accelerated Sketching

arXiv:2604.23436v1 Announce Type: cross Abstract: Reliable decision-making with streaming data requires principled uncertainty quantification of online methods. While first-order methods enable effici

researcharxiv-cs-lg

Tutorials

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Inverting Foundation Models of Brain Function with Simulation-Based Inference

DGX agent

arXiv:2604.23865v1 Announce Type: cross Abstract: Foundation models of brain activity promise a new frontier for in silico neuroscience by emulating neural responses to complex stimuli across tasks an

tutorialsarxiv-cs-ai

28 Apr 2026

On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontroller

model-releasesarxiv-cs-cv

arXiv:2604.23012v1 Announce Type: cross Abstract: This paper presents a complete, end-to-end on-device vision machine learning pipeline, comprising data acquisition, two-layer CNN training with Adam o

POPI: Personalizing LLMs via Optimized Natural Language Preference Inference

arXiv:2510.17881v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. W

researcharxiv-cs-ai

SMSI: System Model Security Inference: Automated Threat Modeling for Cyber-Physical Systems

model-releasesarxiv-cs-ai

arXiv:2604.23905v1 Announce Type: cross Abstract: Threat modeling for cyber-physical systems (CPS) remains a largely manual exercise. This project presents SMSI (System Model Security Inference), a hy

The DeepSeek V4 garbled output bug in open source inference engine is fixed in SGLang. To everyone affected over the weekend, sorry for the …

The DeepSeek V4 garbled output bug in open source inference engine is fixed in SGLang. To everyone affected over the weekend, sorry for the trouble. Huge thanks to @Ant_Group for landing the fix PR. I

model-releasesollama--x

27 Apr 2026

Hardware

Toward Robust and Efficient ML-Based GPU Caching for Modern Inference

arXiv:2509.20979v2 Announce Type: replace Abstract: In modern GPU inference, cache efficiency remains a major bottleneck, and heuristic policies such as extsc{LRU} can perform far worse than the offli

hardwarearxiv-cs-lg

27 Apr 2026

DeepSeek v4 Flash with local inference after 24h of playing with that: even with the 2 bit selective quantization GGUF, iti is the FIRST t…

model-releasesclem-delangue--x

DeepSeek v4 Flash with *local inference* after 24h of playing with that: even with the 2 bit selective quantization GGUF, iti is the FIRST time I feel I have a frontier model running on my computer. T

26 Apr 2026

2 bit DeepSeek v4 Flash inference. Experts w1/3: IQ2_XXS, w2: Q2_K, all the rest left mostly F16/F32 Final GGUF: 86.18 GiB Runs on a MacBook…

model-releasesclem-delangue--x

2 bit DeepSeek v4 Flash inference. Experts w1/3: IQ2_XXS, w2: Q2_K, all the rest left mostly F16/F32 Final GGUF: 86.18 GiB Runs on a MacBook M3 max with CPU (No Metal backend test as it will crash my

25 Apr 2026

Agents

Inference that never sleeps, for agents that never stop. 'Why cowork when you can delegate?' That's @DhruvBatra_ on @yutori_ai's new Delegat…

Inference that never sleeps, for agents that never stop. 'Why cowork when you can delegate?' That's @DhruvBatra_ on @yutori_ai's new Delegate — an always-on agent that monitors, researches, and acts a

agentstogether-ai--x

25 Apr 2026

Industry

Meta and Amazon reach a multibillion-dollar, multiyear deal for Meta to rent hundreds of thousands of Amazon's Graviton chips for its AI inference needs (Matt Day/Bloomberg)

Matt Day / Bloomberg: Meta and Amazon reach a multibillion-dollar, multiyear deal for Meta to rent hundreds of thousands of Amazon's Graviton chips for its AI inference needs — Amazon.com Inc. and Met

industrytechmeme

24 Apr 2026

In ChatGPT, full-stack inference improvements enable a more capable model at faster speed. This efficiency is a game-changer for GPT-5.5 Pro…

In ChatGPT, full-stack inference improvements enable a more capable model at faster speed. This efficiency is a game-changer for GPT-5.5 Pro, now a much more practical option for demanding tasks, and

model-releasesopenai--x

23 Apr 2026

Membership Inference for Contrastive Pre-training Models with Text-only PII Queries

arXiv:2603.14222v2 Announce Type: replace-cross Abstract: Contrastive pretraining models such as CLIP and CLAP, serve as the ubiquitous perceptual backbones for modern multimodal large models, yet the

safetyarxiv-cs-ai

23 Apr 2026

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference

arXiv:2604.19769v1 Announce Type: cross Abstract: Key-value (KV) caching is critical for efficient inference in large language models (LLMs), yet its memory footprint scales linearly with context leng

researcharxiv-cs-ai

23 Apr 2026

Adaptive Slicing-Assisted Hyper Inference for Enhanced Small Object Detection in High-Resolution Imagery

arXiv:2604.19233v1 Announce Type: new Abstract: Deep learning-based object detectors have achieved remarkable success across numerous computer vision applications, yet they continue to struggle with s

researcharxiv-cs-cv

Assessing VLM-Driven Semantic-Affordance Inference for Non-Humanoid Robot Morphologies

arXiv:2604.19509v1 Announce Type: new Abstract: Vision-language models (VLMs) have demonstrated remarkable capabilities in understanding human-object interactions, but their application to robotic sys

safetyarxiv-cs-ro

Bayesian Event-Based Model for Disease Subtype and Stage Inference

arXiv:2512.03467v2 Announce Type: replace Abstract: Chronic diseases often progress differently across patients. Rather than randomly varying, there are typically a small number of subtypes for how a

applicationsarxiv-cs-lg

Excited to share our work on production-ready W4A8 inference, now integrated in vLLM! By combining 4-bit weights (low memory) with 8-bit act…

Excited to share our work on production-ready W4A8 inference, now integrated in vLLM! By combining 4-bit weights (low memory) with 8-bit activations (high compute), we hit the sweet spot for both deco

applicationscohere--x

Industry

The LLM Inference Trilemma: Throughput, Latency, Cost

This article examines the fundamental trade-offs in large language model inference operations, specifically the competing priorities of maximizing throughput, minimizing latency, and reducing costs. I

industrydigitalocean

Decoding RWA Tokenized U.S. Treasuries: Functional Dissection and Address Role Inference

arXiv:2507.14808v3 Announce Type: replace-cross Abstract: Tokenized U.S. Treasuries have emerged as a prominent subclass of real-world assets (RWAs), offering cryptographically secured, yield-bearing

applicationsarxiv-cs-lg

Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models

arXiv:2602.07794v3 Announce Type: replace Abstract: Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning. While recent work has identified structured conceptual r

researcharxiv-cs-cl

FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time

arXiv:2604.16648v1 Announce Type: new Abstract: In this work, we present FRIGID, a framework with a novel diffusion language model that generates molecular structures conditioned on mass spectra via i

From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models

arXiv:2603.04592v3 Announce Type: replace Abstract: Standard Large Language Models (LLMs) are predominantly designed for static inference with pre-defined inputs, which limits their applicability in d

applicationsarxiv-cs-cl

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference

arXiv:2601.13684v2 Announce Type: replace Abstract: The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods

researcharxiv-cs-cl

LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference

model-releasesarxiv-cs-cv

arXiv:2604.16492v1 Announce Type: new Abstract: Flow Matching models achieve state-of-the-art image generation quality but incur substantial inference cost due to iterative denoising through large Tra

RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design

arXiv:2604.17175v1 Announce Type: new Abstract: We introduce RosettaSearch, an inference-time multi-objective optimization approach for protein sequence optimization. We use large language models (LLM

Bridging the Gap between Learning and Inference for Diffusion-Based Molecule Generation

arXiv:2411.05472v2 Announce Type: replace Abstract: The paradigm shift toward structure-driven molecule generation has been propelled by advances in deep generative models, such as variational auto-en

Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration

model-releasesarxiv-cs-cl

arXiv:2507.02935v2 Announce Type: replace Abstract: Successful human-agent teaming relies on an agent being able to understand instructions given by a (human) principal. In many cases, an instruction

Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion

arXiv:2511.14178v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have demonstrated significant potential in real-world robotic manipulation. However, pre-trained VLA policies st

safetyarxiv-cs-ro

Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching

arXiv:2604.15009v1 Announce Type: cross Abstract: Flow matching retains the generation quality of diffusion models while enabling substantially faster inference, making it a compelling paradigm for ge

researcharxiv-cs-lg

Analog Optical Inference on Million-Record Mortgage Data

arXiv:2604.13251v1 Announce Type: new Abstract: Analog optical computers promise large efficiency gains for machine learning inference, yet no demonstration has moved beyond small-scale image benchmar

Bias-Corrected Adaptive Conformal Inference for Multi-Horizon Time Series Forecasting

arXiv:2604.13253v1 Announce Type: new Abstract: Adaptive Conformal Inference (ACI) provides distribution-free prediction intervals with asymptotic coverage guarantees for time series under distributio

safetyarxiv-cs-lg

Industry

Cloudflare’s AI Platform: an inference layer designed for agents

We're building AI Gateway into a unified inference layer for AI, letting developers call models from 14+ providers. New features include Workers AI binding integration and an expanded catalog with mul

industrycloudflare-ai

Hardware

Scalable Spatiotemporal Inference with Biased Scan Attention Transformer Neural Processes

arXiv:2506.09163v2 Announce Type: replace Abstract: Neural Processes (NPs) are a rapidly evolving class of models designed to directly model the posterior predictive distribution of stochastic process

hardwarearxiv-cs-lg

Tools

We’re back 🔥. Thrilled to be named once again to the @Forbes AI 50. The AI Native Cloud, built for the full AI lifecycle: fast inference, o…

We’re back 🔥. Thrilled to be named once again to the @Forbes AI 50. The AI Native Cloud, built for the full AI lifecycle: fast inference, open models, fine-tuning at scale. https://www.forbes.com/list

toolstogether-ai--x

Active Imitation Learning for Thermal- and Kernel-Aware LFM Inference on 3D S-NUCA Many-Cores

arXiv:2604.11948v1 Announce Type: new Abstract: Large Foundation Model (LFM) inference is both memory- and compute-intensive, traditionally relying on GPUs. However, the limited availability and high

safetyarxiv-cs-lg

Banger paper from NVIDIA. Agentic reasoning needs models that are not just capable, but efficient at long-context inference. The agent model…

Banger paper from NVIDIA. Agentic reasoning needs models that are not just capable, but efficient at long-context inference. The agent model layer is moving toward open, long-context, high-throughput

model-releasesdair-ai--x

Compute constraints are a double bind: On the inference side you need to either (a) raise prices, (b) ration use, and/or (c) serve worse mod…

applicationsethan-mollick--x

Compute constraints are a double bind: On the inference side you need to either (a) raise prices, (b) ration use, and/or (c) serve worse models. This hurts current growth On the training side, you can

Fully Homomorphic Encryption on Llama 3 model for privacy preserving LLM inference

model-releasesarxiv-cs-ai

arXiv:2604.12168v1 Announce Type: cross Abstract: The applications of Generative Artificial Intelligence (GenAI) and their intersections with data-driven fields, such as healthcare, finance, transport

Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling

arXiv:2604.12806v1 Announce Type: new Abstract: Inferring latent interaction structures from observed dynamics is a fundamental inverse problem in many-body interacting systems. Most neural approaches

applicationsarxiv-cs-lg

ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems

arXiv:2604.11943v1 Announce Type: cross Abstract: An OS kernel that runs LLM inference internally can read logit distributions before any text is generated -- and act on them as a governance primitive

Tools

The edge inference implication: memory, not compute, is the binding constraint. Parcae opens a new axis. Scale quality by looping deeper, no…

The edge inference implication: memory, not compute, is the binding constraint. Parcae opens a new axis. Scale quality by looping deeper, not by adding parameters. Bigger-model quality at smaller-mode

toolstogether-ai--x

Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices

model-releasesarxiv-cs-ai

arXiv:2512.06443v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed on edge devices. To meet strict resource constraints, real-world deployment has pushed

Discrete Bayesian Sample Inference for Graph Generation

arXiv:2511.03015v2 Announce Type: replace Abstract: Generating graph-structured data is crucial in applications such as molecular generation, knowledge graphs, and network analysis. However, their dis

researcharxiv-cs-lg

Agents

GMI Cloud and @Zai_org are bringing fast inference + frontier models around the globe. First stop: Singapore. Big congrats to all the builde…

GMI Cloud and @Zai_org are bringing fast inference + frontier models around the globe. First stop: Singapore. Big congrats to all the builders at the GMI x @Zai_org Agent Hackathon in 🇸🇬 100+ on-site,

agentszhipu-ai--x

[Help] Gemma 4 26B LoRA Training on 16GB VRAM: Loss decreases, but inference degenerates into loops (Masking vs. MoE?)

This Reddit thread discusses a user's experience attempting LoRA fine-tuning of the Gemma 4 26B-A4B model on a 16GB VRAM GPU, where training loss decreases normally but the resulting model degenerates

model-releasesr-ollama

Iterative Inference-time Scaling with Adaptive Frequency Steering for Image Super-Resolution

arXiv:2512.23532v2 Announce Type: replace Abstract: Diffusion models have become a leading paradigm for image super-resolution (SR), but existing methods struggle to guarantee both the high-frequency

researcharxiv-cs-cv

Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

arXiv:2602.19509v3 Announce Type: replace-cross Abstract: We observe that LLM cascading and routing implicitly solves an anytime computation problem -- a class of algorithms, well-studied in classical

researcharxiv-cs-ai