Search: “inference”

Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies

arXiv:2603.27960v2 Announce Type: replace-cross Abstract: Although Large Vision Language Models (LVLMs) have demonstrated impressive multimodal reasoning capabilities, their scalability and deployment

researcharxiv-cs-cl

14 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks

DGX agent

arXiv:2502.02345v2 Announce Type: replace Abstract: Subspace inference for neural networks assumes that a subspace of their parameter space suffices to produce a reliable uncertainty quantification. I

model-releasesarxiv-cs-lg

13 Apr 2026

ASUS UGen300 USB AI Accelerator 8GB for local inference

local-air-stablediffusion

The ASUS UGen300 is a plug-and-play USB AI accelerator featuring a Hailo-10H processor that delivers up to 40 TOPS (INT4) of inference performance and 8GB of dedicated LPDDR4 memory, consuming just...

AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention

arXiv:2604.07815v1 Announce Type: new Abstract: Long-context inference in LLMs faces the dual challenges of quadratic attention complexity and prohibitive KV cache memory. While token-level sparse att

researcharxiv-cs-cl

CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference

arXiv:2604.06036v3 Announce Type: replace-cross Abstract: Video streaming analytics is a crucial workload for vision-language model serving, but the high cost of multimodal inference limits scalabilit

hardwarearxiv-cs-cv

ConfusionPrompt: Practical Private Inference for Online Large Language Models

arXiv:2401.00870v5 Announce Type: replace-cross Abstract: State-of-the-art large language models (LLMs) are typically deployed as online services, requiring users to transmit detailed prompts to cloud

Domain-Contextualized Inference: A Computable Graph Architecture for Explicit-Domain Reasoning

model-releasesarxiv-cs-ai

arXiv:2604.04344v2 Announce Type: replace Abstract: We establish a computation-substrate-agnostic inference architecture in which domain is an explicit first-class computational parameter. This produc

FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling

arXiv:2604.06779v1 Announce Type: new Abstract: We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte

safetyarxiv-cs-ai

Inference-Time Code Selection via Symbolic Equivalence Partitioning

arXiv:2604.06485v1 Announce Type: cross Abstract: 'Best-of-N' selection is a popular inference-time scaling method for code generation using Large Language Models (LLMs). However, to reliably identify

researcharxiv-cs-ai

Instance-Adaptive Parametrization for Amortized Variational Inference

model-releasesarxiv-cs-ai

arXiv:2604.06796v1 Announce Type: cross Abstract: Latent variable models, including variational autoencoders (VAE), remain a central tool in modern deep generative modeling due to their scalability an

UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding

arXiv:2507.22025v4 Announce Type: replace Abstract: The emergence of Multimodal Large Language Models (MLLMs) has driven significant advances in Graphical User Interface (GUI) agent capabilities. Neve

agentsarxiv-cs-ai

Introducing GLM-5.1 from @Zai_org on Together AI. AI natives can now use GLM-5.1 on Together and benefit from reliable inference for product…

Introducing GLM-5.1 from @Zai_org on Together AI. AI natives can now use GLM-5.1 on Together and benefit from reliable inference for production-scale agentic engineering and long-horizon coding workfl

agentstogether-ai--x

8 Apr 2026

.@ollama is now an inference provider on Replicas. Run open models in Claude Code, Codex, OpenCode and Pi

Ollama is now an inference provider on the Replicas platform, enabling users to deploy and run open‑source models directly within Replicas. The partnership supports popular models such as Claude Code,

model-releasesollama--x

17 Oct 2026

Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

arXiv:2610.20836v1 Announce Type: new Abstract: Evaluator-guided reasoning presents a local control problem at every step: trust the current answer, return a fallback, or spend more inference. We form

Flexible Doubly Robust Proximal Causal Inference for Censored Outcomes

arXiv:2610.20893v1 Announce Type: cross Abstract: Estimating causal effects on right-censored time-to-event outcomes is challenging under unmeasured confounding. Proximal causal inference addresses un

TIPS: A Tiny Inference-time Policy for Scaling in Diffusion Language Models

model-releasesarxiv-cs-ai

arXiv:2610.22099v1 Announce Type: cross Abstract: Masked diffusion language models expose an inference-time choice: which positions become visible after each denoising pass. Many existing decoding met

TurboMamba: Faster Inference, Faster Training, and Better Prediction with Lexical Memory

model-releasesarxiv-cs-cl

arXiv:2610.20682v1 Announce Type: cross Abstract: We introduce TurboMamba, a language-model architecture for faster inference, faster training, and better token prediction. A global Mamba-3 model runs

Bellman Conformal Inference: Calibrating Prediction Intervals For Time Series

arXiv:2402.05203v3 Announce Type: replace Abstract: We introduce Bellman Conformal Inference (BCI), a framework that wraps around any time series forecasting models and provides calibrated prediction

Tutorials

Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

arXiv:2610.20483v1 Announce Type: new Abstract: We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-samplin

tutorialsarxiv-cs-lg

Narrative Paradoxes in LLM Inference: Shaping Reasoning Trajectories toward Misalignment

arXiv:2610.16312v1 Announce Type: new Abstract: Explicit reasoning in large language models (LLMs) is often associated with reliability, yet it can still pose risks when a model infers harmful goals u

Scaling LLM Inference via Parallelism: Throughput-Latency Trade-off

arXiv:2610.19405v1 Announce Type: cross Abstract: Large Language Model (LLM) serving systems must balance latency and throughput while efficiently utilizing distributed GPU resources. This challenge a

hardwarearxiv-cs-lg

ConicFlow: constrained Inference-time optimal control for flow matching models

arXiv:2610.14307v1 Announce Type: cross Abstract: We propose ConicFlow, which frames constrained sampling from a flow matching model at inference time as an optimal control problem that balances devia

local-aiarxiv-cs-cv

FORGE: Procedural Weight Generation for Sparse MoE Inference

arXiv:2610.16890v1 Announce Type: cross Abstract: We introduce FORGE (Fast On-chip Rotor Generation for Experts), a procedural weight generation system for Sparse Mixture-of-Experts (MoE) inference. R

hardwarearxiv-cs-lg

GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference

arXiv:2610.17321v1 Announce Type: cross Abstract: As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in th

ReMAP: Lossless Residual Neural Lifting for Scalable MAP Inference in Arbitrary-Order Markov Random Fields

arXiv:2411.18954v5 Announce Type: replace Abstract: Maximum a posteriori inference in a finite Markov random field is generally NP-hard. We study an instance-wise optimizer that minimizes the exact mu

Sample-Size Extrapolation in Simulation-Based Inference via Statistical Scaling

arXiv:2610.16820v1 Announce Type: cross Abstract: In simulation-based inference (SBI), neural posterior estimators trained on a bounded range of sample sizes can become inaccurate on larger datasets b

SoK: Rethinking Membership Inference in Tabular Machine Learning

arXiv:2610.18486v1 Announce Type: new Abstract: Membership Inference Attacks (MIAs) are currently the dominant approach to evaluating privacy in machine learning applications. Despite their significan

Strong Gaussian Approximation of Constant-Learning-Rate SGD: Bias and Trajectory Inference

arXiv:2610.14567v1 Announce Type: cross Abstract: This paper develops strong Gaussian approximations (KMT type) for constant learning rate stochastic gradient descent (SGD). We work with heavy tailed

Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference

arXiv:2610.17531v1 Announce Type: new Abstract: While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting impact

Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validation

model-releasesarxiv-cs-ro

arXiv:2610.14530v1 Announce Type: new Abstract: Multi-robot systems (MRS) offer significant advantages over single-robot platforms by enabling information sharing and coordination for different tasks

Basalt: Fast Blackwell inference engine for Qwen 3.8 Flash Next (and how we got vision for free)

model-releasesr-localllama

Dynamic vision, in action Basalt is an inference engine for 3.8 Flash Next, forked from Strata and heavily tuned for Blackwell. It supports vision, and for this to work you need to have the image enco

13 Oct 2026

Applications

Microsoft unveils its third-gen in-house AI chip, the Maia 300, built on TSMC's N2P process node and aimed at serving as its 'primary AI inference system' (Tom Warren/The Verge)

Tom Warren / The Verge: Microsoft unveils its third-gen in-house AI chip, the Maia 300, built on TSMC's N2P process node and aimed at serving as its “primary AI inference system” — The new Maia 300 ch

applicationstechmeme

12 Oct 2026

DEX: Digit-Level Early Exit for Energy-Efficient MSDF Neural Network Inference

arXiv:2610.11748v1 Announce Type: cross Abstract: U-Net inference for brain-tumor segmentation requires billions of multiply-accumulate operations, motivating hardware that can reduce computation dyna

safetyarxiv-cs-ai

Finsler Flow Matching: Dynamics-Aware Geodesic Interpolation for Single-Snapshot Trajectory Inference

arXiv:2610.11318v1 Announce Type: new Abstract: Single-cell snapshot data can resolve a continuum of cellular states but do not uniquely determine the dynamics governing transitions between them. Howe

Neuro-Memory Fuzzy Inference System for Mimicking Human-like Car Following Behavior

arXiv:2610.11252v1 Announce Type: cross Abstract: This study presents the Neuro-Memory Fuzzy Inference System (NeMeFIS), a hierarchical machine learning architecture that asymmetrically models acceler

safetyarxiv-cs-ai

SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference

model-releasesarxiv-cs-cl

arXiv:2610.12327v1 Announce Type: cross Abstract: The memory-bound nature of the decoding stage of large language model (LLM) inference incurs significant latency. Layer-wise training-free network pru

Allocation Stability and Wald Inference under Variance-Aware UCB

arXiv:2412.08843v3 Announce Type: replace-cross Abstract: Allocation stability is often used to justify Gaussian inference from bandit data, but when is it necessary? In this paper, we address this qu

Estimating Model-Level Membership Inference Vulnerability Without Reference Models

arXiv:2510.19773v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) have emerged as the standard tool for evaluating the privacy risks of AI models. However, state-of-the-art attac

MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation

arXiv:2610.09866v1 Announce Type: new Abstract: We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same in

Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing

arXiv:2610.09778v1 Announce Type: cross Abstract: Reproducible benchmarking of Large Language Model (LLM) inference is challenging because repeated measurements can vary with execution and system stat

Agentic Design Space Exploration for Joint Hardware Configuration Selection and Mapping of AI Inference Workloads on Heterogeneous Edge SoCs

arXiv:2610.07191v1 Announce Type: cross Abstract: Modern edge Systems-on-Chip (SoCs) integrate heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, each with distinct performance and ene

agentsarxiv-cs-ai

Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion

arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using pr

hardwarearxiv-cs-lg

High-Dimensional Statistical Inference for Sparse Support Vector Machines

arXiv:2610.08345v1 Announce Type: cross Abstract: Using a replica-symmetric high-dimensional characterization, we develop an inferential framework for sparse support vector machines when the sample si

Illusory Pattern Perception Drives Spurious Inference in Large Language Models

arXiv:2610.07791v1 Announce Type: new Abstract: Illusory pattern perception is a well-documented human cognitive tendency to infer meaningful relationships in data that is actually random. Such a tend

researcharxiv-cs-ai

llama.cpp can distribute inference on heterogeneous devices through the ggml RPC backend It's an advanced setting but I think with time we'l…

model-releasesgeorgi-gerganov--x

llama.cpp can distribute inference on heterogeneous devices through the ggml RPC backend It's an advanced setting but I think with time we'll make it more accessible to regular users. It's crazy that

Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

arXiv:2610.07782v1 Announce Type: new Abstract: Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache

agentsarxiv-cs-ai

The largest agentic LLM Inference dataset has arrived! 206B tokens · 12,002 sessions · 1,186,582 LLM requests · 1,213,347 tool calls The tra…

The largest agentic LLM Inference dataset has arrived! 206B tokens · 12,002 sessions · 1,186,582 LLM requests · 1,213,347 tool calls The traces include tool names, arguments, durations, and more. Expl

agentsclem-delangue--x

AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference via Dynamical Text Guidance