Search: “inference”

arXiv:2605.18796v1 Announce Type: cross Abstract: LLM cascades and model routing promise lower inference cost by sending easy queries to a small model and escalating hard ones to a large model, but mo

hardwarearxiv-cs-cl

20 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

DGX agent

arXiv:2605.17000v1 Announce Type: cross Abstract: Optimization of LLM training and inference configurations, such as hyperparameters, data mixtures, and prompts, is critical to performance, but it is

model-releasesarxiv-cs-ai

19 May 2026

CADS: Conformal Adaptive Decision System for Cost-Efficient Image Classification

arXiv:2605.16401v1 Announce Type: new Abstract: While high-capacity AI models have advanced state-of-the-art performance, their practical deployment is often hindered by high inference costs, environm

Drift Flow Matching

arXiv:2605.17244v1 Announce Type: cross Abstract: Iterative generative models such as Flow Matching and Diffusion models have demonstrated strong test-time scaling behavior, where additional inference

Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network

arXiv:2605.18190v1 Announce Type: cross Abstract: Diffusion models achieve state-of-the-art generative performance but suffer from high computational costs during inference due to the repeated evaluat

Early Stopping Chain-of-thoughts in Large Language Models

arXiv:2509.14004v2 Announce Type: replace Abstract: Reasoning large language models (LLMs) have demonstrated superior capacities in solving complicated problems by generating long chain-of-thoughts (C

Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models

arXiv:2508.06038v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) incur substantial computational overhead and inference latency due to the large number of vision tokens introduc

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605.16928v1 Announce Type: cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention. Existing efficient alternatives often rely ei

HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation

model-releasesarxiv-cs-cl

arXiv:2605.17093v1 Announce Type: cross Abstract: Distilling vision-language models into faster hybrid architectures, such as 3:1 Mamba-2/attention mixes, is now standard practice for making inference

Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation

arXiv:2605.17268v1 Announce Type: new Abstract: We present the first systematic study of faithfulness in Vision-Language-Action (VLA) driving models, analyzing 300 Alpamayo-R1-10B inferences across 10

safetyarxiv-cs-ai

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

arXiv:2605.18331v1 Announce Type: new Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly,

Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers

arXiv:2605.16941v1 Announce Type: new Abstract: Diffusion Large Language Models (DLLMs) promise fast parallel generation, yet open-source DLLMs still face a severe quality-speed trade-off: acceleratin

Small-scale photonic Kolmogorov-Arnold networks using standard telecom nonlinear modules

arXiv:2604.08432v2 Announce Type: replace-cross Abstract: Photonic neural networks promise ultrafast inference, yet most architectures rely on linear optical meshes with electronic nonlinearities, rei

SparseSAM: Structured Sparsification of Activations in Segment Anything Models

arXiv:2605.17633v1 Announce Type: cross Abstract: The Segment Anything Model (SAM) achieves strong open-vocabulary segmentation, but its ViT-based image encoders dominate inference latency and memory.

Spherical Steering: Geometry-Aware Activation Rotation for Language Models

arXiv:2602.08169v2 Announce Type: replace-cross Abstract: Inference-time steering offers a promising way to control language models (LMs) without retraining. However, standard approaches typically rel

StatQAT: Statistical Quantizer Optimization for Deep Networks

arXiv:2605.17745v1 Announce Type: cross Abstract: Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision

Efficient Image Synthesis with Sphere Latent Encoder

arXiv:2605.15592v1 Announce Type: new Abstract: Few-step image generation has seen rapid progress, with consistency and meanflow-based methods significantly reducing the number of sampling steps. Desp

FLASH: Efficient Visuomotor Policy via Sparse Sampling

arXiv:2605.15492v1 Announce Type: cross Abstract: Generative models such as diffusion and flow matching have become dominant paradigms for visuomotor policy learning, yet their reliance on iterative d

safetyarxiv-cs-cv

Tutorials

Neural Backward Filtering Forward Guiding

arXiv:2601.23030v2 Announce Type: replace-cross Abstract: Inference in nonlinear continuous stochastic processes on trees is challenging, particularly when observations are sparse and the topology is

tutorialsarxiv-cs-lg

Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels

arXiv:2605.15208v1 Announce Type: cross Abstract: Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deploy

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

arXiv:2604.02268v2 Announce Type: replace Abstract: Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a rel

safetyarxiv-cs-lg

Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model

arXiv:2605.15733v1 Announce Type: cross Abstract: Humans abstract experiences into structured representations to facilitate pattern inference and knowledge transfer. While the hippocampal-entorhinal (

Applications

An Interpretable Latency Model for Speculative Decoding in LLM Serving

arXiv:2605.15051v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified b

applicationsarxiv-cs-lg

Hardware

BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE

arXiv:2605.14438v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures enhance the efficiency of large language models by activating only a subset of experts per token. However, standa

hardwarearxiv-cs-ai

Causal Foundation Models with Continuous Treatments

arXiv:2605.15133v1 Announce Type: new Abstract: Causal inference, estimating causal effects from observational data, is a fundamental tool in many disciplines. Of particular importance across a variet

Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm

arXiv:2605.14521v1 Announce Type: new Abstract: Layer normalization (LN) is a fundamental component in modern deep learning, but its per-sample centering and scaling introduce non-negligible inference

HERO: Hierarchical Extrapolation and Refresh for Efficient World Models

arXiv:2508.17588v2 Announce Type: replace Abstract: Generation-driven world models create immersive virtual environments but suffer slow inference due to the iterative nature of diffusion models. Whil

Keyed Nonlinear Transform: Lightweight Privacy-Enhancing Feature Sharing for Medical Image Analysis

arXiv:2605.14123v1 Announce Type: cross Abstract: Feature sharing via split inference offers a lightweight alternative to federated learning for resource-constrained hospitals, but transmitted feature

SceneFunRI: Reasoning the Invisible for Task-Driven Functional Object Localization

arXiv:2605.14704v1 Announce Type: cross Abstract: In real-world scenes, target objects may reside in regions that are not visible. While humans can often infer the locations of occluded objects from c

Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems

arXiv:2605.15050v1 Announce Type: new Abstract: Recently, deep generative models have been used for posterior inference in inverse problems, including high-stakes applications in medical imaging and s

SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection

model-releasesarxiv-cs-cv

arXiv:2605.14110v1 Announce Type: new Abstract: Vision Transformers (ViTs) enable strong multi-view 3D detection but are limited by high inference latency from dense token and query processing across

Towards Label-Free Single-Cell Phenotyping Using Multi-Task Learning

arXiv:2605.14717v1 Announce Type: cross Abstract: Label-free single-cell imaging offers a scalable, non-invasive alternative to fluorescence-based cytometry, yet inferring molecular phenotypes directl

Uncertainty Quantification for Large Language Diffusion Models

arXiv:2605.14570v1 Announce Type: new Abstract: Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. S

A Resampling-Based Framework for Network Structure Learning in High-Dimensional Data

arXiv:2605.12706v1 Announce Type: new Abstract: RSNet is an open-source R package that provides a resampling-based framework for robust and interpretable network inference, designed to address the lim

Energy Scaling Laws for Diffusion Models: Quantifying Compute in Image Generation

model-releasesarxiv-cs-cv

arXiv:2511.17031v2 Announce Type: replace-cross Abstract: The rapidly growing computational demands of diffusion models for image generation have raised significant concerns about energy consumption a

Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models

arXiv:2605.13338v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) are increasingly integrated into systems requiring reliable multi-step inference, yet this growing dependence exposes ne

Hardware

OP4KSR: One-Step Patch-Free 4K Super-Resolution with Periodic Artifact Suppression

arXiv:2605.13457v1 Announce Type: new Abstract: Diffusion-based real-world image super-resolution (Real-ISR) has achieved remarkable perceptual quality; however, directly super-resolving images to 4K

hardwarearxiv-cs-cv

What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs

arXiv:2605.12549v1 Announce Type: new Abstract: Existing training-free approaches for GUI grounding often rely on multiple inference runs, such as iterative cropping or candidate aggregation, to ident

Applications

AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines

arXiv:2605.10987v1 Announce Type: new Abstract: Modern machine learning deployments increasingly compose specialized models into dynamic inference pipelines, where upstream components produce intermed

applicationsarxiv-cs-lg

G^2TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

arXiv:2605.12309v1 Announce Type: new Abstract: The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. I

Kairos: A Scalable Serving System for Physical AI

arXiv:2605.11381v1 Announce Type: new Abstract: Physical AI is experiencing rapid growth with frontier foundation models increasing its capabilities across general environments. Physical AI tasks are

researcharxiv-cs-ro

Learning Compact Boolean Networks

model-releasesarxiv-cs-lg

arXiv:2602.05830v2 Announce Type: replace-cross Abstract: Floating-point neural networks dominate modern machine learning but incur substantial inference costs, motivating emerging interest in Boolean

Applications

Neural Statistical Functions

arXiv:2605.11327v1 Announce Type: new Abstract: Classical deep learning typically operates on individual cases. Despite its success, real-world usage often requires repeated inference to estimate stat

applicationsarxiv-cs-lg

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

arXiv:2605.12412v1 Announce Type: new Abstract: Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent

Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures

arXiv:2605.10991v1 Announce Type: new Abstract: Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process

safetyarxiv-cs-lg

When to Ask a Question: Understanding Communication Strategies in Generative AI Tools

arXiv:2605.11240v1 Announce Type: cross Abstract: Generative AI models differ from traditional machine learning tools in that they allow users to provide as much or as little information as they choos

safetyarxiv-cs-lg

Local Ai

Belief or Circuitry? Causal Evidence for In-Context Graph Learning

arXiv:2605.08405v1 Announce Type: new Abstract: How do LLMs learn in-context? Is it by pattern-matching recent tokens, or by inferring latent structure? We probe this question using a toy graph random

local-aiarxiv-cs-ai

12 May 2026

Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration

arXiv:2605.10195v1 Announce Type: new Abstract: Tree-of-Thought (ToT) reasoning structures Large Language Model (LLM) inference as a tree-based search, demonstrating strong potential for solving compl