Search: “inference”

IBM Research has presented a new approach that scales AI‑driven chip design by employing open “agent factories,” lowering inference cost by 11.5 × compared to fully proprietary setups. The method enab

agentsibm-research

13 Oct 2026

Agents

Agentic data science with the Data Agent Kit

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

Imagine your compliance team notices that payments are taking longer than expected to clear. They suspect fraud, and you’re the data scientist helping figure out which transactions to investigate firs

agentsgoogle-cloud-ai

12 Oct 2026

Amortized Counterfactual Prediction with Graph Neural Networks

model-releasesarxiv-cs-ai

arXiv:2610.13724v1 Announce Type: cross Abstract: Counterfactual inference asks how an observed outcome might have differed under different circumstances, such as whether a patient would have recovere

DeltaLLM: Delta-Key Sparse Attention for Edge LLMs

arXiv:2507.19608v2 Announce Type: replace Abstract: On-device LLM inference is increasingly important for edge platforms but remains challenging due to high computational and memory demands and tight

local-aiarxiv-cs-ai

DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

model-releasesarxiv-cs-ai

arXiv:2610.13767v1 Announce Type: cross Abstract: Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains thes

Introducing DigitalOcean Signals and Simulations: see how your agents and models really behave in production, then test the fix before it ships

model-releasesdigitalocean

You ship an agent. It mostly works. Then the feedback loop becomes a support ticket: a user got the wrong answer, a tool call looped four times, a refund went out that shouldn’t have. Standard observa

Inverse Synthesis of Shock Time Series from Shock Response Spectra Using Ensembles of Conditional Variational Autoencoders

model-releasesarxiv-cs-lg

arXiv:2610.12820v1 Announce Type: new Abstract: The shock response spectrum (SRS) characterizes the peak responses of single-degree-of-freedom systems to transient acceleration. Because the mapping fr

openbmb/MiniCPM-V-4.7-1B

Gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 1B variant. It builds on the family's single-image, multi-image,

Final revision of zzpanic vllm-radiance for 1 x R9700 GPU, qwen3.8-27b

Summary of project status metrics Since StillDeadcode has released his new radiance inference engine, the writing is on the wall for the old vllm-radiance builds, people have moved on to newer and bet

11 Oct 2026

Speedbumps: Rejection Attacks on Speculative Decoding

arXiv:2610.10929v1 Announce Type: cross Abstract: Speculative decoding is a popular technique for increasing the speed and reducing the costs of large language model (LLM) inference by verifying multi

Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation

model-releasesarxiv-cs-lg

arXiv:2603.28744v2 Announce Type: replace Abstract: The linear representation hypothesis states that neural network activations encode high-level concepts as linear mixtures. However, under superposit

TokenRouter: Efficient Serving System for Token-Level LLM Routing

model-releasesarxiv-cs-cl

arXiv:2610.12242v1 Announce Type: new Abstract: Large language model (LLM) routing distributes inference work across different models, advancing the cost-quality Pareto frontier of LLM serving. While

When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry

arXiv:2610.10616v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models produce routing information during inference that may be logged or exposed for monitoring, debugging, load anal

safetyarxiv-cs-lg

SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning

model-releasesarxiv-cs-cl

arXiv:2610.10304v1 Announce Type: cross Abstract: We study whether latent sequence compression of prompt prefixes preserves the capabilities that large language models rely on during inference. We int

8 Oct 2026

Foveated Compression: Selective High-Resolution Preservation for Token-Efficient VLMs

arXiv:2610.07729v1 Announce Type: new Abstract: Visual tokens are a major source of inference cost in vision-language models, yet simple image downsampling remains a surprisingly strong compression ba

local-aiarxiv-cs-cv

Latent space bias directions in LLMs capture confidence, not fairness

arXiv:2610.08559v1 Announce Type: cross Abstract: Activation steering has gained popularity as a lightweight inference-time debiasing technique for large language models. However, prior work reports t

safetyarxiv-cs-ai

REPA-G: Test-Time Conditioning with Representation-Aligned Visual Features

arXiv:2602.03753v2 Announce Type: replace Abstract: While representation alignment with self-supervised models has been shown to improve diffusion model training, its potential for enhancing inference

safetyarxiv-cs-cv

Two Halves are More than One: Phase-wise Velocity Distillation for Fast and High-Quality Image Generation

model-releasesarxiv-cs-cv

arXiv:2610.08070v1 Announce Type: new Abstract: Recent diffusion-based image generation backbones have grown substantially in scale, making the network inference cost increase rapidly. While diffusion

AID: A Framework for AI Infrastructure Dynamics

arXiv:2610.04801v1 Announce Type: cross Abstract: A useful model of AI inference infrastructure must specify the system state, the information available to an observer, and the decisions the model is

safetyarxiv-cs-lg

Deepseek Peak/Off-Peak and Ownership

Maybe I'm mistaken, but wasn't one of the big pros of Ollama Cloud inference that it was all hosted by Ollama? If that's the case, why duplicate the Deepseek on-/off- hour premiums? Surely they do not

model-releasesr-ollama

Tutorials

Diffusion Transformers are Provably Optimal In-context Generators

arXiv:2610.05333v1 Announce Type: new Abstract: Generative foundation models are attracting interest for their ability to produce desired outputs from demonstrations given at inference time, without u

tutorialsarxiv-cs-lg

Learning Interpretable Differentiable Logic Networks for Tabular Regression

arXiv:2505.23615v2 Announce Type: replace Abstract: Neural networks (NNs) achieve outstanding performance in many domains; however, their decision processes are often opaque and their inference can be

LinearPFN: Amortized Variable Selection for Linear Models with Interactions

arXiv:2610.06580v1 Announce Type: new Abstract: Spike-and-slab regression is a standard Bayesian formulation of variable selection: it returns a posterior distribution over which candidate effects are

MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization

model-releasesarxiv-cs-cv

arXiv:2603.27170v2 Announce Type: replace Abstract: Relative camera pose is a basic geometric cue for visual re-localization and scene understanding. When using images alone, visual evidence may not s

OVAL: Output-Aware Local Page Bases for KV Cache Retrieval

model-releasesarxiv-cs-lg

arXiv:2610.06686v1 Announce Type: new Abstract: Long context inference with large language models becomes increasingly expensive as attention must operate over an ever growing KV cache. Page sparse at

Applications

When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA

arXiv:2610.05273v1 Announce Type: new Abstract: Visual token pruning is an effective way to accelerate vision-language models and is especially useful for vision-language-action (VLA) inference, where

applicationsarxiv-cs-cv

Applications

16-bit Precision of Convolutional Neural Networks on Microcontroller Units for 8-bit Costs

arXiv:2610.03402v1 Announce Type: new Abstract: To deploy deep neural networks on edge hardware, highly efficient inference schemes are necessary that retain high accuracy. This work presents W16A16,

applicationsarxiv-cs-lg

Activation Sparsity with Weight Approximation for Faster LLM Decoding on Offloaded Weights

arXiv:2610.02598v1 Announce Type: new Abstract: Deploying LLMs on consumer-grade GPUs with insufficient memory to hold their weights can result in prohibitively slow inference, because decoding repeat

local-aiarxiv-cs-lg

AI-driven Thermal-aware Data Center Capacity Planning

arXiv:2610.02442v1 Announce Type: new Abstract: The emerging of large language models (LLMs) has posed significant challenges to the thermal management of data center. Intense GPU computation for LLMs

local-aiarxiv-cs-lg

Consecutive Posterior Fusion for Diffusive Recovery of Unobservable Image Structures

arXiv:2610.03261v1 Announce Type: cross Abstract: Solving severely ill-posed imaging inverse problems requires recovering image structures that are unobservable or weakly constrained by the measuremen

Contextual Flow Matching: Adaptive Step Selection in Flow Models for Efficient Visual Generation

arXiv:2610.03202v1 Announce Type: cross Abstract: Flow Matching enables high-quality visual generation via continuous-time dynamics, but inference remains costly due to multiple sequential function ev

Deep Bayesian REFoCUS

arXiv:2610.03419v1 Announce Type: new Abstract: In this work we formulate ultrasound multistatic recovery from arbitrary transmit sequences as a Bayesian inference problem. To that end, we train a dee

From Patching to Pruning Visual Computation in Vision Language Models

arXiv:2610.03389v1 Announce Type: new Abstract: Vision language models (VLMs) incur substantial inference cost because every visual token is processed by the attention and MLP projections of every dec

researcharxiv-cs-cv

Lightweight and Resource-Efficient Perception for Robotic Guide Dogs

arXiv:2610.03187v1 Announce Type: cross Abstract: Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placemen

hardwarearxiv-cs-cv

Refinement Buys Intelligibility, Search Buys Identity: What Test-Time Compute Buys in Masked-Diffusion TTS

arXiv:2610.03320v1 Announce Type: new Abstract: Diffusion language models for text-to-speech combine two forms of computation: model depth (parameters) and refinement steps (inference budget). We ask

SpectralCache: Accelerating Diffusion-Based World Models via Spectral Feature Caching

arXiv:2610.02660v1 Announce Type: new Abstract: Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Trans

researcharxiv-cs-cv

Syntheses

Wiki Lint Report — 2026-10-04

linthealth-checkautomated

Automated lint: 19 errors, 64915 warnings, 3 info

4 Oct 2026

Qwen3.5 arch implementation in FPGA fabric for 9B/27B INT4 models on relatively cheap eBay mining hardware

I've been wanting to test out LLM inference on FPGAs for a while now, but didn't have a big reason too since there were no frontier class models at 9B-27B scale (3.6 27B was great still but it didn't

4 Oct 2026

Do Vision Language Models Understand Human Engagement in Games?

arXiv:2603.18480v2 Announce Type: replace-cross Abstract: Inferring human engagement from gameplay video is important for game design and player-experience research, yet it remains unclear whether vis

2 Oct 2026

RACE: Residual-Aware Test-Time Adaptation for Neighbor-Rich Time-Series Foundation Model Forecasting

arXiv:2610.00405v1 Announce Type: new Abstract: Time-series foundation models (TSFMs) perform strongly across forecasting tasks, but their per-series inference is ill-suited to neighbor-rich forecasti

2 Oct 2026

Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is next

I have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor

2 Oct 2026

DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling

arXiv:2605.21603v1 Announce Type: cross Abstract: Intra-device parallelism addresses resource under-utilization in ML inference and training by overlapping the execution of operators with different re

hardwarearxiv-cs-lg

Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs

arXiv:2609.40093v1 Announce Type: cross Abstract: Expert parallelism (EP) enables inference of large Mixture-of-Experts (MoE) models by placing their experts across multiple GPUs, but requires substan

hardwarearxiv-cs-lg

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

model-releasesarxiv-cs-ai

arXiv:2609.38585v1 Announce Type: cross Abstract: Existing Large Language Model (LLM) routing methods score LLMs independently to select top-k models. However, this ignores model correlations and enfo

Tutorials

Hermes: Learning Contextual Reasoning Unlocks Test-Time Scaling

arXiv:2609.38332v1 Announce Type: cross Abstract: Test-time scaling improves model performance by allocating additional compute during inference. Using this compute effectively across multiple context

tutorialsarxiv-cs-ai

How NVIDIA GPUs Help Accelerate OpenAI’s GPT-6 Astra Ultrafast

GPT-6 Astra Ultrafast, running on NVIDIA Blackwell GPUs, is available now in the OpenAI API and to eligible ChatGPT Work and Codex users. Accelerated by inference optimizations through OpenAI’s models

hardwarenvidia-blog

MoEless: Efficient MoE LLM Serving with Serverless Experts

arXiv:2603.06350v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly adopt Mixture-of-Experts (MoE) architectures to scale efficiently under stringent resource constrain

local-aiarxiv-cs-ai

On the Relaxation of Conditional Independence Assumption for Image Segmentation

arXiv:2609.38930v1 Announce Type: cross Abstract: In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation m

local-aiarxiv-cs-ai