Search: “inference”

Fast token generation emerges as the key differentiator as heterogeneous inference takes hold

model-releasessiliconangle

The race for fast token generation has moved from benchmark sheets into production data centers, and the hardware blueprint for winning it is no longer a GPU-only story. As agentic AI use cases multip

9 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Memory Scarcity, Open Models, and the Restructuring of the AI Industry, 2026-2030 -- A quantitative scenario analysis of inference economics, training-cost divergence, and infrastructure solvency

DGX agent

arXiv:2607.07207v1 Announce Type: cross Abstract: We analyze how four forces restructure the AI industry over 2026-2030: the DRAM/HBM price surge, frontier-capable open-weight models (GLM-5.2), rapid

local-aiarxiv-cs-ai

9 Jul 2026

Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

arXiv:2607.05475v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) on mobile devices enhances privacy and reduces latency, but is severely bottlenecked by hardware inefficiency.

8 Jul 2026

Applications

Learning 4D Geometric Priors for Inference-Efficient World Action Models

arXiv:2607.05468v1 Announce Type: cross Abstract: World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action seque

applicationsarxiv-cs-ai

8 Jul 2026

LLM-Guided Measurement Credibility Correction for Trustworthy Industrial Process Inference

arXiv:2607.06111v1 Announce Type: cross Abstract: Industrial prediction and soft sensing depend on credible input measurements. In field deployment, a predictor may receive biased, delayed, stale, or

local-aiarxiv-cs-ai

8 Jul 2026

EVA-Client: A Unified Data Collection, Inference, and Deployment Framework for Embodied Policies on Real Robots

arXiv:2607.02646v1 Announce Type: cross Abstract: We present EVA-Client, an open-source framework for deployment, data collection, and evaluation of trained manipulation policies on real robots. Sitti

safetyarxiv-cs-cv

GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

arXiv:2607.02518v1 Announce Type: cross Abstract: OpenClaw requests are dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the c

PDEFlow: Autonomous Agentic PDE Pipelines for Neural Operator Learning and Solver-Free Inference

arXiv:2607.05134v1 Announce Type: cross Abstract: We present PDEFlow, an autonomous agentic framework that turns user-level ODE and PDE descriptions into solver-backed neural-operator pipelines. The w

Agents

Resilient by Design -- Active Inference for Distributed Continuum Intelligence

arXiv:2511.07202v3 Announce Type: replace-cross Abstract: Failures are the norm in highly complex and heterogeneous devices spanning the distributed computing continuum (DCC), from resource-constraine

agentsarxiv-cs-ai

Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space

arXiv:2607.02166v1 Announce Type: cross Abstract: The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine learning me

Guided Action Flow: Q-Guided Inference for Flow-Matching Vision-Language-Action Policies

arXiv:2607.02092v1 Announce Type: cross Abstract: Flow-matching vision-language-action policies generate robot action chunks through an iterative transport process, creating an opportunity for test-ti

safetyarxiv-cs-ai

Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space

arXiv:2607.01689v1 Announce Type: cross Abstract: Model merging aims to combine existing single-task solutions into a multi-task solution without additional data-driven fine-tuning.~Most existing appr

Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters

arXiv:2607.01893v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by drafting a block of tokens that the target model verifies left-to-right, committing only t

Calibrated Test-Time Guidance for Bayesian Inference

arXiv:2602.22428v2 Announce Type: replace-cross Abstract: Test-time guidance is a widely used mechanism for steering pretrained diffusion models toward outcomes specified by a reward function. Existin

2 Jul 2026

CausalMix: Data Mixture as Causal Inference for Language Model Training

arXiv:2607.01104v1 Announce Type: cross Abstract: In Large Language Model (LLM) training, data mixing plays a pivotal role in determining model performance. Recent methods optimize mixture weights via

2 Jul 2026

Graded strength of comparative illusions is explained by Bayesian inference

arXiv:2511.14642v2 Announce Type: replace Abstract: Like visual processing, language processing is susceptible to illusions in which people systematically misperceive stimuli. In one such case--the co

researcharxiv-cs-cl

2 Jul 2026

Communication-Aware Robot Execution for Cloud Inference under Spatially Heterogeneous Connectivity

arXiv:2606.31497v1 Announce Type: new Abstract: Cloud-hosted foundation models enable robots to use semantic reasoning beyond onboard computational limits. In this setting, the robot executes a curren

researcharxiv-cs-ro

1 Jul 2026

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

arXiv:2606.31270v1 Announce Type: cross Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant atten

1 Jul 2026

Scaling LLM Inference: Multi-Node KV Cache Offloading with GKE & Managed Lustre

model-releasesgoogle-cloud-ai

Significant contributors to this article include Sneha Aradhey, Software Engineer, Google Kubernetes Engine, and Michael MacDonald, Sr Software Engineer, Google Cloud Managed Lustre. Enterprise produc

1 Jul 2026

Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models

arXiv:2606.29275v1 Announce Type: new Abstract: Diffusion Language Models (DLMs) are typically trained under fixed context structures, restricting denoising to predetermined token subsets. This create

30 Jun 2026

DiffRGD: An Inference-Time Diffusion Guidance Through Riemannian Gradient Descent

arXiv:2606.28417v1 Announce Type: new Abstract: Recently, diffusion models have been widely adopted in generative modeling and have served as foundational models for many image generation tasks. To co

researcharxiv-cs-cv

30 Jun 2026

Inference-time optimization for experiment-grounded protein ensemble generation

arXiv:2602.24007v3 Announce Type: replace-cross Abstract: Protein function relies on dynamic conformational ensembles, yet current generative models like AlphaFold3 often fail to produce ensembles tha

30 Jun 2026

Cascaded Multi-Granularity Pruning for On-Device LLM Inference in Industrial IoT

model-releasesarxiv-cs-cl

arXiv:2606.26861v1 Announce Type: new Abstract: Deploying large language models (LLMs) on Industrial Internet of Things (IIoT) edge devices demands extreme compression, yet existing structured pruning

26 Jun 2026

Decentralised AI Training and Inference with BlockTrain

arXiv:2606.24722v1 Announce Type: new Abstract: Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters. This creates a structural advantage for hyper

24 Jun 2026

DR-Mamba: Automatic Inference-Time Domain Adaptation for Document Image Binarization via Sample-Conditioned Detail-Background Suppression

model-releasesarxiv-cs-cv

arXiv:2606.22625v1 Announce Type: new Abstract: Degraded document image binarization is sensitive to domain shifts caused by paper aging, bleed-through, stains, shadows, and uneven illumination, and t

23 Jun 2026

MedFedPure: A Medical Federated Framework with MAE-based Detection and Diffusion Purification for Inference-Time Attacks

arXiv:2511.11625v2 Announce Type: replace Abstract: Artificial intelligence (AI) has shown great potential in medical imaging, particularly for brain tumor detection using Magnetic Resonance Imaging (

local-aiarxiv-cs-lg

23 Jun 2026

Hardware

Inference chip startup Groq raises $650M to grow its cloud platform

Seven months after inking a 20 billion chip licensing deal with Nvidia Corp., Groq Inc. today announced that it has raised 650 million in funding. Growth investment firm Disruptive and hedge fund Infi

hardwaresiliconangle

22 Jun 2026

Hardware

An open handbook on LLM inference at scale (GPU internals, KV cache, batching, vLLM/SGLang/TensorRT-LLM) [P]

hardwarer-machinelearning

This handbook covers the technical aspects of running large language models efficiently at scale, focusing on GPU optimization techniques including GPU internals, key-value (KV) cache management, batc

20 Jun 2026

A Turbo-Inference Strategy for Object Detection and Instance Segmentation

arXiv:2606.12371v1 Announce Type: new Abstract: Object detection and instance segmentation tasks are closely related. Existing top-down instance segmentation methods usually follow a detect-then-segme

local-aiarxiv-cs-cv

11 Jun 2026

NARRAS: Edge-Triggered Distributed Inference for CSI-Based Localization in Vehicular IoT Networks

arXiv:2606.11914v1 Announce Type: cross Abstract: CSI-based localization with spatially distributed antenna arrays exposes a basic resource trade-off. Each array can provide a rich view of the channel

local-aiarxiv-cs-lg

11 Jun 2026

Open Materials Generation with Inference-Time Reinforcement Learning

arXiv:2602.00424v2 Announce Type: replace Abstract: Continuous-time generative models for crystalline materials enable inverse materials design by learning to predict stable crystal structures, but in

11 Jun 2026

Applications

A Comprehensive Inference-Time Augmentation Framework in Physiological Signals: Application to PPG-Based AF Detection

arXiv:2606.10410v1 Announce Type: new Abstract: Objective: Accurate classification of physiological signals in real-world deployments is challenged by sensor noise, motion artifacts, and distribution

applicationsarxiv-cs-lg

10 Jun 2026

Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2

arXiv:2606.10932v1 Announce Type: new Abstract: We present Density Field State Space Models (DF-SSM), a framework for compressing SSMs to a 1-bit scaffold with int8 low-rank correction. Applied to Mam

local-aiarxiv-cs-cl

10 Jun 2026

GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning

arXiv:2507.14725v4 Announce Type: replace-cross Abstract: Prompt-based continual learning (CL) offers a parameter-efficient way to adapt large language models (LLMs) across task sequences. However, ex

10 Jun 2026

Bridged SBI: Correcting Biased Low-Fidelity Posteriors for Cost-Efficient High-Fidelity Inference

model-releasesarxiv-cs-ro

arXiv:2606.09155v1 Announce Type: new Abstract: Accurate calibration of particle-based simulators is crucial for robotic earthwork simulation, but analytical calibration is challenging due to this tas

9 Jun 2026

Applications

In-Context Learning of Stochastic Differential Equations with Foundation Inference Models

arXiv:2502.19049v3 Announce Type: replace Abstract: Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with r

applicationsarxiv-cs-lg

9 Jun 2026

More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

arXiv:2601.21522v2 Announce Type: replace-cross Abstract: The performance of large language models (LLMs) on verifiable tasks is usually measured by pass@k, the probability of answering a question cor

9 Jun 2026

Tutorials

Structure-Preserving Correction Learning for Sparse Bayesian Inference in Brain Source Imaging

arXiv:2606.07196v1 Announce Type: new Abstract: Classical sparse Type-II Bayesian methods for M/EEG brain imaging support joint estimation of source and noise hyperparameters, but rely on fixed iterat

tutorialsarxiv-cs-lg

8 Jun 2026

Emotion Entanglement and Bayesian Inference for Multi-Dimensional Emotion Understanding

arXiv:2604.00819v2 Announce Type: replace-cross Abstract: Understanding emotions in natural language is inherently a multi-dimensional reasoning problem, where multiple affective signals interact thro

4 Jun 2026

Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling

arXiv:2606.05021v1 Announce Type: new Abstract: We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algor

4 Jun 2026

ResCLIP: Residual Attention for Training-free Dense Vision-language Inference

arXiv:2411.15851v2 Announce Type: replace Abstract: While vision-language models like CLIP have shown remarkable success in open-vocabulary tasks, their application is currently confined to image-leve

local-aiarxiv-cs-cv

3 Jun 2026

Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference

arXiv:2606.03332v1 Announce Type: new Abstract: Probabilistic models are typically trained using task-agnostic objectives like log-loss, which can lead to significant errors in downstream estimation.

local-aiarxiv-cs-lg

3 Jun 2026

Tiny Collaborative Inference for Occlusion-Robust Object Detection

arXiv:2606.02894v1 Announce Type: new Abstract: Small edge devices such as IoT surveillance nodes and search-and-rescue (SAR) platforms are increasingly expected to run computer vision locally. On ult

local-aiarxiv-cs-cv

3 Jun 2026

Fair Finetuning Mitigates Distribution Inference Attacks

arXiv:2606.01719v1 Announce Type: cross Abstract: Machine learning models trained on sensitive data can inadvertently leak population-level information about their training distributions -- a threat k

safetyarxiv-cs-ai

Flow-Transformed Implicit Processes for Function-Space Variational Inference

arXiv:2606.01954v1 Announce Type: new Abstract: Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space m

researcharxiv-cs-lg

HypothesisMed: Inference-Time Answer Fusion and Structured Hypothesis-Space Reporting for Biomedical Question Answering

model-releasesarxiv-cs-cl

arXiv:2606.00971v1 Announce Type: new Abstract: Biomedical question answering with large language models is commonly evaluated using answer accuracy, but answer accuracy alone does not indicate whethe

Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

arXiv:2606.00726v1 Announce Type: new Abstract: Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation. Existing methods of

MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference

model-releasesarxiv-cs-lg

arXiv:2606.01563v1 Announce Type: new Abstract: Autoregressive decoding in Transformer-based language models relies on the KV cache, whose memory footprint grows linearly with sequence length and beco