Search: “inference”

CADS: Conformal Adaptive Decision System for Cost-Efficient Image Classification

arXiv:2605.16401v1 Announce Type: new Abstract: While high-capacity AI models have advanced state-of-the-art performance, their practical deployment is often hindered by high inference costs, environm

Drift Flow Matching

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

arXiv:2605.17244v1 Announce Type: cross Abstract: Iterative generative models such as Flow Matching and Diffusion models have demonstrated strong test-time scaling behavior, where additional inference

researcharxiv-cs-ai

19 May 2026

Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network

arXiv:2605.18190v1 Announce Type: cross Abstract: Diffusion models achieve state-of-the-art generative performance but suffer from high computational costs during inference due to the repeated evaluat

Early Stopping Chain-of-thoughts in Large Language Models

arXiv:2509.14004v2 Announce Type: replace Abstract: Reasoning large language models (LLMs) have demonstrated superior capacities in solving complicated problems by generating long chain-of-thoughts (C

Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models

arXiv:2508.06038v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) incur substantial computational overhead and inference latency due to the large number of vision tokens introduc

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

arXiv:2605.16928v1 Announce Type: cross Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention. Existing efficient alternatives often rely ei

researcharxiv-cs-ai

HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation

model-releasesarxiv-cs-cl

arXiv:2605.17093v1 Announce Type: cross Abstract: Distilling vision-language models into faster hybrid architectures, such as 3:1 Mamba-2/attention mixes, is now standard practice for making inference

Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation

arXiv:2605.17268v1 Announce Type: new Abstract: We present the first systematic study of faithfulness in Vision-Language-Action (VLA) driving models, analyzing 300 Alpamayo-R1-10B inferences across 10

safetyarxiv-cs-ai

Prune, Update and Trim: Robust Structured Pruning for Large Language Models

arXiv:2605.18331v1 Announce Type: new Abstract: Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly,

Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers

arXiv:2605.16941v1 Announce Type: new Abstract: Diffusion Large Language Models (DLLMs) promise fast parallel generation, yet open-source DLLMs still face a severe quality-speed trade-off: acceleratin

Small-scale photonic Kolmogorov-Arnold networks using standard telecom nonlinear modules

arXiv:2604.08432v2 Announce Type: replace-cross Abstract: Photonic neural networks promise ultrafast inference, yet most architectures rely on linear optical meshes with electronic nonlinearities, rei

SparseSAM: Structured Sparsification of Activations in Segment Anything Models

arXiv:2605.17633v1 Announce Type: cross Abstract: The Segment Anything Model (SAM) achieves strong open-vocabulary segmentation, but its ViT-based image encoders dominate inference latency and memory.

researcharxiv-cs-ai

Spherical Steering: Geometry-Aware Activation Rotation for Language Models

arXiv:2602.08169v2 Announce Type: replace-cross Abstract: Inference-time steering offers a promising way to control language models (LMs) without retraining. However, standard approaches typically rel

StatQAT: Statistical Quantizer Optimization for Deep Networks

arXiv:2605.17745v1 Announce Type: cross Abstract: Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision

b9221

b9221 is an intermediate build release from the llama.cpp project, which is a C/C++ implementation enabling efficient LLM inference on consumer hardware. The release includes platform-specific binarie

Efficient Image Synthesis with Sphere Latent Encoder

arXiv:2605.15592v1 Announce Type: new Abstract: Few-step image generation has seen rapid progress, with consistency and meanflow-based methods significantly reducing the number of sampling steps. Desp

FLASH: Efficient Visuomotor Policy via Sparse Sampling

arXiv:2605.15492v1 Announce Type: cross Abstract: Generative models such as diffusion and flow matching have become dominant paradigms for visuomotor policy learning, yet their reliance on iterative d

safetyarxiv-cs-cv

Tutorials

Neural Backward Filtering Forward Guiding

arXiv:2601.23030v2 Announce Type: replace-cross Abstract: Inference in nonlinear continuous stochastic processes on trees is challenging, particularly when observations are sparse and the topology is

tutorialsarxiv-cs-lg

Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels

arXiv:2605.15208v1 Announce Type: cross Abstract: Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deploy

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

arXiv:2604.02268v2 Announce Type: replace Abstract: Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a rel

safetyarxiv-cs-lg

Structure Abstraction and Generalization in a Hippocampal-Entorhinal Inspired World Model

arXiv:2605.15733v1 Announce Type: cross Abstract: Humans abstract experiences into structured representations to facilitate pattern inference and knowledge transfer. While the hippocampal-entorhinal (

Applications

An Interpretable Latency Model for Speculative Decoding in LLM Serving

arXiv:2605.15051v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified b

applicationsarxiv-cs-lg

b9159

b9159 is a release of llama.cpp published on May 14, 2026 . llama.cpp is a C/C++ implementation of large language model inference that enables efficient LLM execution on consumer hardware with minimal

Hardware

BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE

arXiv:2605.14438v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures enhance the efficiency of large language models by activating only a subset of experts per token. However, standa

hardwarearxiv-cs-ai

Causal Foundation Models with Continuous Treatments

arXiv:2605.15133v1 Announce Type: new Abstract: Causal inference, estimating causal effects from observational data, is a fundamental tool in many disciplines. Of particular importance across a variet

Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm

arXiv:2605.14521v1 Announce Type: new Abstract: Layer normalization (LN) is a fundamental component in modern deep learning, but its per-sample centering and scaling introduce non-negligible inference

HERO: Hierarchical Extrapolation and Refresh for Efficient World Models

arXiv:2508.17588v2 Announce Type: replace Abstract: Generation-driven world models create immersive virtual environments but suffer slow inference due to the iterative nature of diffusion models. Whil

Keyed Nonlinear Transform: Lightweight Privacy-Enhancing Feature Sharing for Medical Image Analysis

arXiv:2605.14123v1 Announce Type: cross Abstract: Feature sharing via split inference offers a lightweight alternative to federated learning for resource-constrained hospitals, but transmitted feature

SceneFunRI: Reasoning the Invisible for Task-Driven Functional Object Localization

arXiv:2605.14704v1 Announce Type: cross Abstract: In real-world scenes, target objects may reside in regions that are not visible. While humans can often infer the locations of occluded objects from c

Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems

arXiv:2605.15050v1 Announce Type: new Abstract: Recently, deep generative models have been used for posterior inference in inverse problems, including high-stakes applications in medical imaging and s

SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection

model-releasesarxiv-cs-cv

arXiv:2605.14110v1 Announce Type: new Abstract: Vision Transformers (ViTs) enable strong multi-view 3D detection but are limited by high inference latency from dense token and query processing across

Towards Label-Free Single-Cell Phenotyping Using Multi-Task Learning

arXiv:2605.14717v1 Announce Type: cross Abstract: Label-free single-cell imaging offers a scalable, non-invasive alternative to fluorescence-based cytometry, yet inferring molecular phenotypes directl

researcharxiv-cs-ai

Uncertainty Quantification for Large Language Diffusion Models

arXiv:2605.14570v1 Announce Type: new Abstract: Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. S

A Resampling-Based Framework for Network Structure Learning in High-Dimensional Data

arXiv:2605.12706v1 Announce Type: new Abstract: RSNet is an open-source R package that provides a resampling-based framework for robust and interpretable network inference, designed to address the lim

b9140

The search did not return specific information about the b9140 release. Based on the context, b9140 is a specific build number from the llama.cpp project releases. llama.cpp is an LLM inference implem

b9144

llama.cpp is an open source software library that performs inference on various large language models such as Llama. Build b9144 is an intermediate release version of the llama.cpp project from the gg

Energy Scaling Laws for Diffusion Models: Quantifying Compute in Image Generation

model-releasesarxiv-cs-cv

arXiv:2511.17031v2 Announce Type: replace-cross Abstract: The rapidly growing computational demands of diffusion models for image generation have raised significant concerns about energy consumption a

Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models

arXiv:2605.13338v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) are increasingly integrated into systems requiring reliable multi-step inference, yet this growing dependence exposes ne

Hardware

OP4KSR: One-Step Patch-Free 4K Super-Resolution with Periodic Artifact Suppression

arXiv:2605.13457v1 Announce Type: new Abstract: Diffusion-based real-world image super-resolution (Real-ISR) has achieved remarkable perceptual quality; however, directly super-resolving images to 4K

hardwarearxiv-cs-cv

Tools

Unlocking asynchronicity in continuous batching

This article explains how asynchronous processing can be integrated with continuous batching techniques to improve inference efficiency in machine learning models. Continuous batching allows requests

toolshugging-face

What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs

arXiv:2605.12549v1 Announce Type: new Abstract: Existing training-free approaches for GUI grounding often rely on multiple inference runs, such as iterative cropping or candidate aggregation, to ident

Applications

AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines

arXiv:2605.10987v1 Announce Type: new Abstract: Modern machine learning deployments increasingly compose specialized models into dynamic inference pipelines, where upstream components produce intermed

applicationsarxiv-cs-lg

G^2TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

arXiv:2605.12309v1 Announce Type: new Abstract: The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. I

Kairos: A Scalable Serving System for Physical AI

arXiv:2605.11381v1 Announce Type: new Abstract: Physical AI is experiencing rapid growth with frontier foundation models increasing its capabilities across general environments. Physical AI tasks are

researcharxiv-cs-ro

Learning Compact Boolean Networks

model-releasesarxiv-cs-lg

arXiv:2602.05830v2 Announce Type: replace-cross Abstract: Floating-point neural networks dominate modern machine learning but incur substantial inference costs, motivating emerging interest in Boolean

Applications

Neural Statistical Functions

arXiv:2605.11327v1 Announce Type: new Abstract: Classical deep learning typically operates on individual cases. Despite its success, real-world usage often requires repeated inference to estimate stat

applicationsarxiv-cs-lg

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

arXiv:2605.12412v1 Announce Type: new Abstract: Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent

Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures

arXiv:2605.10991v1 Announce Type: new Abstract: Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process

safetyarxiv-cs-lg