Search: “inference”

MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization

arXiv:2610.18642v1 Announce Type: new Abstract: The Key-Value (KV) cache constitutes a dominant source of memory consumption during large language model (LLM) inference, particularly in long-context a

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

One Model, Multiple Compute Budgets: Multi-Ratio ConceptMoE for Flexible Compute Across Turns

DGX agent

arXiv:2610.14074v1 Announce Type: new Abstract: Large language models typically support different inference budgets through separate model sizes or fixed compute configurations. In multi-turn interact

researcharxiv-cs-lg

14 Oct 2026

PhaseQuant: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

arXiv:2606.12487v2 Announce Type: replace Abstract: Post-training quantization (PTQ) is essential for efficient large language model (LLM) inference, but reliably quantizing activations remains challe

ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter Transfer

arXiv:2610.15124v1 Announce Type: new Abstract: Foundation models need to support diverse inference budgets, yet training a separate model for each budget duplicates pretraining effort. Elastic models

TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

arXiv:2610.17530v1 Announce Type: new Abstract: We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode,

Industry

Vast Data bets stitched-together AI pipelines can’t keep sensitive data in sync

Data sovereignty is becoming an architectural problem as enterprises push artificial intelligence inference toward their most sensitive information. Pipelines assembled from separate tools struggle to

industrysiliconangle

VideoJev: System One Model for Video Understanding and Interaction

arXiv:2610.18615v1 Announce Type: new Abstract: Visual agents must turn video observations into timely decisions to track progress and choose what to do next. Building on the emerging System One parad

Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors

arXiv:2610.15556v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models commonly generate robot actions token by token through Large Language Model (LLM) decoders, causing inference late

Agents

ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation

arXiv:2602.01709v3 Announce Type: replace Abstract: Current test-time scaling (TTS) techniques enhance large language model (LLM) performance by allocating additional computation at inference time, ye

agentsarxiv-cs-cl

Product Monte Carlo: Expectation Under a Product of Densities

arXiv:2610.12702v1 Announce Type: cross Abstract: Aggregating independently computed sub-sample inferences is a well-known challenge in large-scale Bayesian computation, federated learning, and opinio

Scaling RNA Foundation Models through Iterative Refinement

arXiv:2610.13106v1 Announce Type: new Abstract: Biological function in RNA is encoded across scales, from local sequence motifs to long-range structural dependencies. Conventional RNA foundation model

Hardware

STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules

arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures

hardwarearxiv-cs-lg

Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes

arXiv:2610.12850v1 Announce Type: cross Abstract: Bayesian simulation-based inference (SBI) can struggle to recover increasingly concentrated posteriors as observation size grows. We consider learning

Qwen Flash Next @ 137 tok/s & 3,497 tok/s Prefill w/ 512k context on a 5090, 192gb ram, Windows Build, comparing Strata and Infernix

model-releasesr-localllama

Strata vs Infernix on the same model, RTX 5090 — A/B at 262k and 512k context Hi folks, another inference engine post for your feed. I was blown away by Strata, but saw someone else post about Inferni

10 Oct 2026

Agents

Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context Reinforcement Learning

arXiv:2610.11152v1 Announce Type: cross Abstract: LLM agents increasingly improve at inference time by accumulating experience in context rather than by updating parameters. This process is often desc

agentsarxiv-cs-cl

Just Weather Scoring: Efficient End-to-end Nowcasting with Distributional Diffusion

arXiv:2610.12189v1 Announce Type: cross Abstract: Generative diffusion models are well-suited for probabilistic precipitation nowcasting, but existing approaches often rely on separately trained compr

LIME: Link-based User-item Interaction Modeling with Decoupled XOR Attention for Efficient Test Time Scaling

arXiv:2510.18239v4 Announce Type: replace-cross Abstract: Scaling large recommendation systems requires advancing three major frontiers: processing longer user histories, expanding candidate sets, and

Predictive Multiplicity in Cell-Fate Assignment: Label-Free Rashomon Sets and the Limits of Per-Cell Certification

arXiv:2610.11185v1 Announce Type: new Abstract: Single-cell trajectory inference maps transcriptomic measurements onto developmental continua, yet configurations that fit the data equally well can ass

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages

arXiv:2610.10889v1 Announce Type: new Abstract: Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the

safetyarxiv-cs-cv

Uncovering and Fixing Collider Bias in Bayesian PINNs

model-releasesarxiv-cs-ai

arXiv:2610.11737v1 Announce Type: cross Abstract: Bayesian physics-informed neural networks (B-PINNs) are a popular framework for parameter and state inference from sparse or noisy observations. They

Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution

model-releasesarxiv-cs-cl

arXiv:2610.12367v1 Announce Type: new Abstract: Skills, reusable procedural guidance added at inference, can substantially improve LLM downstream performance (Li et al., 2026). Prior work retrieves sk

Causal Posterior Estimation

arXiv:2505.21468v2 Announce Type: replace Abstract: We present Causal Posterior Estimation (CPE), a novel method for Bayesian inference in simulator models, where evaluating the likelihood function is

Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

arXiv:2610.10520v1 Announce Type: new Abstract: GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing meth

From Pareto to Preference: Personalized Test-Time Scaling via Amortized Agentic Policy Discovery

arXiv:2610.09684v1 Announce Type: new Abstract: Test-time scaling (TTS) improves the reasoning capabilities of large language models by allocating additional inference computation. Existing approaches

safetyarxiv-cs-cl

RFPO: Rectified Flow Policy Optimization for Embodied Control

arXiv:2610.10453v1 Announce Type: new Abstract: Flow-based policies provide an expressive framework for continuous robot control, but their iterative ODE integration incurs substantial inference cost.

safetyarxiv-cs-ro

Same Text, Different Prediction: Serving-Context Nondeterminism in Text Classifiers

arXiv:2610.09111v1 Announce Type: new Abstract: Deterministic inference is essential for reliable and trustworthy machine learning. Prior studies of text generation have shown that changing factors su

researcharxiv-cs-cl

SpikingVLA: Asynchronous Spiking Vision-Language-Action Models

arXiv:2610.09710v1 Announce Type: new Abstract: ANN-to-SNN conversion offers a practical route toward energy-efficient spiking Vision-Language-Action (VLA) models by bypassing the substantial cost of

researcharxiv-cs-cl

Steerspeech: Activation Steering For Emotion Control In Generated Speech

arXiv:2610.10415v1 Announce Type: cross Abstract: Pretrained text-to-speech (TTS) models can generate expressive speech, but reliable inference-time emotion control remains challenging: prompts and re

TAP: Efficient Long-Horizon Agent Pruning via Trajectory-Anchored Recovery

arXiv:2610.09074v1 Announce Type: new Abstract: Emerging long-horizon agentic tasks require repeated model calls, worsening the inference cost of already-costly language models. While narrow agentic t

Secure Speculative Decoding for Large Language Models

arXiv:2610.08678v1 Announce Type: cross Abstract: Speculative decoding accelerates inference for a large language model (LLM), referred to as the target model, by first using a smaller model, referred

safetyarxiv-cs-ai

7 Oct 2026

Steering Diffusion Models to Rare Events with Sequential Monte Carlo

arXiv:2610.08652v1 Announce Type: cross Abstract: Diffusion models are increasingly used as surrogates for expensive simulators in weather prediction, molecular dynamics, and materials design. In thes

7 Oct 2026

A Geometric-Transformation Feature-Adaptive Manifold Restoration Method for Open-Vocabulary Semantic Segmentation of Remote Sensing Images

arXiv:2610.04300v1 Announce Type: new Abstract: The semantic information of objects in remote sensing images is typically invariant to geometric transformations from the dihedral group D4. However, SA

local-aiarxiv-cs-cv

Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design

arXiv:2610.05062v1 Announce Type: cross Abstract: Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets a

local-aiarxiv-cs-ro

Closing the Context Gap: Activation Alignment for Tabular In-Context Learning

arXiv:2610.06679v1 Announce Type: new Abstract: Tabular foundation models perform in-context learning (ICL) by conditioning predictions on labeled training examples provided as context. Unlike traditi

Decoupling Time and Space: A Temporally Conditioned Refinement for EEG Source Imaging

arXiv:2610.06726v1 Announce Type: new Abstract: Electroencephalography (EEG) offers millisecond temporal resolution, but inferring underlying neural sources is a severely ill-posed spatial inverse pro

local-aiarxiv-cs-lg

DriftSR: One-Step Real-World Image Super-Resolution via Distribution Drifting

arXiv:2610.04819v1 Announce Type: new Abstract: One-step real-world image super-resolution (Real-ISR) offers efficient inference, but recovering realistic and perceptually rich details often relies on

local-aiarxiv-cs-cv

Dynamic Time Step Prediction in Inverse Heat Dissipation for Blur-Like Image Restoration Tasks

arXiv:2610.03942v1 Announce Type: new Abstract: When using diffusion models to target image restoration problems, diffusion inversion is typically employed to retain relevant image information from th

First-Order Steering: Translating Weight Adaptation into Activation Steering

arXiv:2610.04283v1 Announce Type: new Abstract: Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering

Grounded in Time: A Multi-Source Dataset and Benchmark for Temporal Grounding in Robotic Manipulation

model-releasesarxiv-cs-cv

arXiv:2610.04255v1 Announce Type: cross Abstract: Robotic manipulation often requires inferring task-relevant states from past interactions when the current observation alone is insufficient to determ

Learning Discriminative Geometry for Drifting Models

arXiv:2610.04703v1 Announce Type: cross Abstract: Recently proposed Drifting Models shift iterative distribution refinement from inference to training, enabling effective one-step generation. However,

Tutorials

Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Foundation Models

arXiv:2610.06813v1 Announce Type: new Abstract: Recent progress in 3D foundation models has enabled rapid 3D reconstruction and camera calibration by leveraging learned 3D priors from vast amount of s

tutorialsarxiv-cs-cv

LiFT: Loop Flow Transformers

arXiv:2610.05538v1 Announce Type: new Abstract: We introduce Loop Flow Transformers (LiFT), a family of looped generative models that scales computation by repeatedly applying a shared Diffusion Trans

Hardware

Rethinking Streaming-Perception Evaluation on Heterogeneous Edge Platforms

arXiv:2610.05578v1 Announce Type: cross Abstract: Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placemen

hardwarearxiv-cs-cv

StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation

model-releasesarxiv-cs-cv

arXiv:2610.05664v1 Announce Type: new Abstract: Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs). However,

Applications

Synthetic-to-Real ViT-Based Pose Estimation of a Noncooperative UAV

arXiv:2610.04335v1 Announce Type: new Abstract: Remote pose estimation of noncooperative Unmanned Aerial Vehicles (UAVs) from imagery is critical, as they cannot be influenced or instrumented in advan

applicationsarxiv-cs-cv

Towards causal effect estimation with learned instrument representations

arXiv:2602.10370v3 Announce Type: replace-cross Abstract: Instrumental variable (IV) methods mitigate bias from unobserved confounding in observational causal inference but rely on the availability of

Hardware

Transformers vs RNNs vs SSMs: Where Does Memory Actually Live? [D]

hardwarer-machinelearning

Someone who has always loved looking at the space between different AI techniques, this time I went a little deeper into the memory trade-offs between RNNs, Transformers and SSMs. I found it interesti

Triggering Generalist Reasoning via Predictive Uncertainty for Dual-System VLA

arXiv:2610.05025v1 Announce Type: new Abstract: Dual-system Vision-Language-Action (VLA) models improve real-time robotic control by pairing a slow, reasoning-capable generalist with a fast specialist