Search: “inference”

arXiv:2604.20156v1 Announce Type: new Abstract: Mixture-of-Experts models, now popular for scaling capacity at fixed inference speed, switch experts at nearly every token. Once a model outgrows availa

hardwarearxiv-cs-lg

23 Apr 2026

When Do Early-Exit Networks Generalize? A PAC-Bayesian Theory of Adaptive Depth

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2604.15764v1 Announce Type: cross Abstract: Early-exit neural networks enable adaptive computation by allowing confident predictions to exit at intermediate layers, achieving 2-8imes inference s

researcharxiv-cs-ai

20 Apr 2026

Hardware

DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance

arXiv:2604.14552v1 Announce Type: cross Abstract: Modern datacenters increasingly rely on low-power, single-slot inference accelerators to balance performance, energy efficiency, and rack density cons

hardwarearxiv-cs-lg

17 Apr 2026

A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Models

arXiv:2604.13440v1 Announce Type: new Abstract: Deploying Large Language Models (LLMs) on edge devices faces severe computational and memory constraints, limiting real-time processing and on-device in

local-aiarxiv-cs-lg

16 Apr 2026

A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model

arXiv:2604.05672v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often c

researcharxiv-cs-ro

16 Apr 2026

SpanKey: Dynamic Key Space Conditioning for Neural Network Access Control

arXiv:2604.12254v1 Announce Type: cross Abstract: SpanKey is a lightweight way to gate inference without encrypting weights or chasing leaderboard accuracy on gated inference. The idea is to condition

15 Apr 2026

Competing with AI Scientists: Agent-Driven Approach to Astrophysics Research

arXiv:2604.09621v1 Announce Type: new Abstract: We present an agent-driven approach to the construction of parameter inference pipelines for scientific data analysis. Our method leverages a multi-agen

14 Apr 2026

Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights

arXiv:2504.06307v2 Announce Type: replace-cross Abstract: The rapid adoption of large language models (LLMs) has led to significant energy consumption and carbon emissions, posing a critical challenge

local-aiarxiv-cs-ai

14 Apr 2026

PACO: Proxy-Task Alignment and Online Calibration for On-the-Fly Category Discovery

arXiv:2604.11484v1 Announce Type: new Abstract: On-the-Fly Category Discovery (OCD) requires a model, trained on an offline support set, to recognize known classes while discovering new ones from an o

safetyarxiv-cs-cv

14 Apr 2026

Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search

arXiv:2509.25835v4 Announce Type: replace Abstract: Test-time scaling improves large language models (LLMs) on long-horizon reasoning tasks by allocating more compute at inference. LLM inference via t

safetyarxiv-cs-ai

13 Apr 2026

Agents

Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM

arXiv:2604.06832v2 Announce Type: replace Abstract: Vision-language models (VLMs) predominantly rely on autoregressive decoding, which generates tokens one at a time and fundamentally limits inference

agentsarxiv-cs-cl

13 Apr 2026

Interventional Time Series Priors for Causal Foundation Models

arXiv:2603.11090v2 Announce Type: replace Abstract: Prior-data fitted networks (PFNs) have emerged as powerful foundation models for tabular causal inference, yet their extension to time series remain

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification

arXiv:2502.17421v4 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) can now process extremely long contexts, efficient inference over these extended inputs has become increasingl

RewardFlow: Generate Images by Optimizing What You Reward

arXiv:2604.08536v1 Announce Type: new Abstract: We introduce RewardFlow, an inversion-free framework that steers pretrained diffusion and flow-matching models at inference time through multi-reward La

safetyarxiv-cs-cv

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

arXiv:2604.06277v1 Announce Type: new Abstract: Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retri

Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries

arXiv:2610.22273v1 Announce Type: cross Abstract: Reasoning over long document contexts remains a challenge for large language models (LLMs). Standard retrieval methods are inherently reactive, bottle

16 Oct 2026

Correction Training for Diffusion Models

arXiv:2610.14136v2 Announce Type: new Abstract: Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noi

safetyarxiv-cs-ai

Tutorials

Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs

arXiv:2603.00578v2 Announce Type: replace-cross Abstract: Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial i

tutorialsarxiv-cs-cl

FlowWise: Stateful Fast-Slow Model Serving for Streaming Traffic Intelligence

arXiv:2402.03694v3 Announce Type: replace-cross Abstract: Network traffic analysis in cloud and internet service provider (ISP) networks increasingly relies on machine learning (ML) models, as encrypt

Applications

Label-Centric Knowledge Retrieval for Zero-Shot Text Classification

arXiv:2610.16182v1 Announce Type: new Abstract: Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit th

applicationsarxiv-cs-ai

New Model, Old Habits: Behavior Transfer under Model Swapping in Multi-Session LLM Agents

arXiv:2610.19712v1 Announce Type: new Abstract: Multi-session agents can switch between LLMs to balance performance and inference cost. Such switches require transferring a history for continuity, yet

Pre-Trained Language Models as Zero-Shot Tools for Social Science Research

arXiv:2610.19048v1 Announce Type: new Abstract: Record linkage and demographic inference from names are routine demands in social science. Most approaches to these problems require labeled training da

safetyarxiv-cs-cl

RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving

arXiv:2610.18143v1 Announce Type: new Abstract: Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in

Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter

arXiv:2508.16939v2 Announce Type: replace Abstract: Diffusion models achieve state-of-the-art generative performance but remain computationally expensive at inference, often requiring many discretizat

Tail Competition Explains Scaling in Best-of-N Sampling for Verifiable Problems

arXiv:2610.19362v1 Announce Type: new Abstract: Best-of-N sampling is a simple inference-time scaling strategy for large language models, yet its behavior under incomplete reward models remains poorly

researcharxiv-cs-cl

Theoretical and Practical Advances in Hierarchical Bayesian Deep Learning Methodology

arXiv:2610.19074v1 Announce Type: new Abstract: Reliable uncertainty quantification remains an important challenge in AI. Hierarchical Bayesian methods provide a principled framework for this by repre

Hardware

Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

arXiv:2610.19121v1 Announce Type: cross Abstract: Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reaso

hardwarearxiv-cs-cl

Autoregressive Decoding as a Martingale: Characterizing the Stochastic Dynamics of LLM Generation

arXiv:2610.16041v1 Announce Type: new Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of language tasks. At their core, autoregressive LLMs are conditional

Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware

model-releasesarxiv-cs-cv

arXiv:2602.00865v2 Announce Type: replace Abstract: While multi-view 3D reconstruction has shifted toward large-scale foundation models capable of inferring globally consistent geometry, their relianc

Encoding Context into LLM Weights: What Makes It Context?

arXiv:2610.17686v1 Announce Type: cross Abstract: Longer contexts allow large language models (LLMs) to condition their predictions on more information, but inference costs grow prohibitively with con

Fast and Accurate: Improving Personalized LLM Performance by Semantic Concentration Inspired Workload Split between RAG and Fine-Tuning

arXiv:2610.15921v1 Announce Type: new Abstract: With ever increasing personalized large language model (LLM) services, the next question to answer is how to improve the quality of services/users' expe

Implicit and explicit reparameterizations of the Folded Normal and Rice distributions

arXiv:2610.17151v1 Announce Type: cross Abstract: Variational inference is a flexible tool for modeling experimental data and has proven useful in crystallography. Pathwise gradient estimators provide

Novel View Synthesis using DDIM Inversion

arXiv:2508.10688v3 Announce Type: replace Abstract: Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved

researcharxiv-cs-cv

Query-Adaptive Neuron Steering for LLMs

arXiv:2610.14519v1 Announce Type: new Abstract: Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge. Existing studies address this c

The Tokens Strike Back: Safety-Aware Denoising with Directional Embedding-based Revision

arXiv:2610.17636v1 Announce Type: new Abstract: The Safety-Aware Denoiser (SAD) is a recent, training-free, safety-guidance framework for text diffusion models. During inference, SAD steers text sampl

safetyarxiv-cs-lg

Amortized Counterfactual Prediction with Graph Neural Networks

arXiv:2610.13724v1 Announce Type: cross Abstract: Counterfactual inference asks how an observed outcome might have differed under different circumstances, such as whether a patient would have recovere

DeltaLLM: Delta-Key Sparse Attention for Edge LLMs

arXiv:2507.19608v2 Announce Type: replace Abstract: On-device LLM inference is increasingly important for edge platforms but remains challenging due to high computational and memory demands and tight

local-aiarxiv-cs-ai

DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

arXiv:2610.13767v1 Announce Type: cross Abstract: Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains thes

Inverse Synthesis of Shock Time Series from Shock Response Spectra Using Ensembles of Conditional Variational Autoencoders

arXiv:2610.12820v1 Announce Type: new Abstract: The shock response spectrum (SRS) characterizes the peak responses of single-degree-of-freedom systems to transient acceleration. Because the mapping fr

Speedbumps: Rejection Attacks on Speculative Decoding

arXiv:2610.10929v1 Announce Type: cross Abstract: Speculative decoding is a popular technique for increasing the speed and reducing the costs of large language model (LLM) inference by verifying multi

Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation

arXiv:2603.28744v2 Announce Type: replace Abstract: The linear representation hypothesis states that neural network activations encode high-level concepts as linear mixtures. However, under superposit

TokenRouter: Efficient Serving System for Token-Level LLM Routing

model-releasesarxiv-cs-cl

arXiv:2610.12242v1 Announce Type: new Abstract: Large language model (LLM) routing distributes inference work across different models, advancing the cost-quality Pareto frontier of LLM serving. While

When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry

arXiv:2610.10616v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models produce routing information during inference that may be logged or exposed for monitoring, debugging, load anal

safetyarxiv-cs-lg

SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning

model-releasesarxiv-cs-cl

arXiv:2610.10304v1 Announce Type: cross Abstract: We study whether latent sequence compression of prompt prefixes preserves the capabilities that large language models rely on during inference. We int

8 Oct 2026

Foveated Compression: Selective High-Resolution Preservation for Token-Efficient VLMs

arXiv:2610.07729v1 Announce Type: new Abstract: Visual tokens are a major source of inference cost in vision-language models, yet simple image downsampling remains a surprisingly strong compression ba

local-aiarxiv-cs-cv

Latent space bias directions in LLMs capture confidence, not fairness

arXiv:2610.08559v1 Announce Type: cross Abstract: Activation steering has gained popularity as a lightweight inference-time debiasing technique for large language models. However, prior work reports t

safetyarxiv-cs-ai

REPA-G: Test-Time Conditioning with Representation-Aligned Visual Features

arXiv:2602.03753v2 Announce Type: replace Abstract: While representation alignment with self-supervised models has been shown to improve diffusion model training, its potential for enhancing inference

safetyarxiv-cs-cv

Two Halves are More than One: Phase-wise Velocity Distillation for Fast and High-Quality Image Generation

model-releasesarxiv-cs-cv

arXiv:2610.08070v1 Announce Type: new Abstract: Recent diffusion-based image generation backbones have grown substantially in scale, making the network inference cost increase rapidly. While diffusion