Search: “inference”

Relational and Sequential Conformal Inference for Energy Time Series over Graphs via Foundation Models

arXiv:2606.31804v1 Announce Type: new Abstract: Accurate energy demand forecasting is essential for the reliable operation and planning of modern sustainable energy systems. Spatial-temporal graph neu

safetyarxiv-cs-lg

1 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference

DGX agent

arXiv:2606.31145v1 Announce Type: new Abstract: Large language models increasingly operate over long contexts, where the KV cache becomes a dominant memory bottleneck: its size grows linearly with seq

hardwarearxiv-cs-cl

1 Jul 2026

DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

arXiv:2606.28932v1 Announce Type: cross Abstract: Large language models have driven recent progress in language and multimodal AI, yet pre-training them at scale is prohibitively expensive. Low-rank p

30 Jun 2026

Representation Learning for Equivariant Inference with Guarantees

arXiv:2505.19809v3 Announce Type: replace-cross Abstract: In many real-world applications of regression, conditional probability estimation, and uncertainty quantification, exploiting symmetries roote

applicationsarxiv-cs-ai

30 Jun 2026

SpecMind: Cognitively Inspired, Interactive Multi-Turn Framework for Postcondition Inference

arXiv:2602.20610v3 Announce Type: replace-cross Abstract: Specifications are vital for ensuring program correctness, yet writing them manually remains challenging and time-intensive. Recent large lang

safetyarxiv-cs-cl

30 Jun 2026

EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction

arXiv:2606.27550v1 Announce Type: new Abstract: Multi-token prediction has been shown to increase data density during training, improve downstream text-generation quality, and serves as the defacto ap

researcharxiv-cs-cl

29 Jun 2026

Context Recycling for Long-Horizon LLM Inference

arXiv:2606.26105v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons due t

26 Jun 2026

Agents

Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models

arXiv:2606.26366v1 Announce Type: new Abstract: Standard chain-of-thought on moral dilemmas exhibits two failure modes: stakeholder collapse (the trace names at most one party with a stake in the outc

agentsarxiv-cs-ai

26 Jun 2026

TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference

arXiv:2606.27161v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of

26 Jun 2026

Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference

arXiv:2606.17165v3 Announce Type: replace-cross Abstract: Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in t

24 Jun 2026

Exploring Accurate and Transparent Domain Adaptation in Predictive Healthcare via Concept-Grounded Orthogonal Inference

arXiv:2602.12542v2 Announce Type: replace-cross Abstract: Deep learning models for clinical event prediction on electronic health records (EHR) often suffer performance degradation when deployed under

IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference

arXiv:2606.09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning. Both KV memory and KV read b

JOIN: Anchor-Grasp-Conditioned Joining via Opposition, Inference, and Navigation for Bimanual Assistive Manipulation

arXiv:2606.11151v1 Announce Type: new Abstract: Assistive mobility and manipulation platforms have received increasing attention as a means of restoring independence to individuals with disabilities.

researcharxiv-cs-ro

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

arXiv:2606.10537v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with contex

researcharxiv-cs-cl

Hardware

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

arXiv:2606.10445v1 Announce Type: cross Abstract: Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup. However, its strict 50% sparsity co

hardwarearxiv-cs-cl

APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing

arXiv:2606.08761v1 Announce Type: cross Abstract: W4A4 quantization promises full utilization of INT4 Tensor Cores, yet group dequantization overhead on CUDA Cores has driven existing systems to mixed

9 Jun 2026

Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder

model-releasesarxiv-cs-cv

arXiv:2606.05981v1 Announce Type: new Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1

5 Jun 2026

Ekka: Automated Diagnosis of Silent Errors in LLM Inference

arXiv:2606.04594v1 Announce Type: cross Abstract: LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations. The rapid development process can introd

4 Jun 2026

UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing

arXiv:2606.04101v1 Announce Type: cross Abstract: Large-scale expert parallelism (EP) is becoming pivotal for training and serving frontier MoE models, but it also amplifies device-level expert load i

applicationsarxiv-cs-lg

4 Jun 2026

Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings

arXiv:2606.04286v1 Announce Type: new Abstract: Online reviews provide valuable insights into the perceived quality of facets of a product or service. While aspect-based sentiment analysis has focused

applicationsarxiv-cs-cl

4 Jun 2026

Hardware

MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency

arXiv:2606.03014v1 Announce Type: new Abstract: Mixture-of-Agents (MoA) systems improve reasoning accuracy by routing each query to multiple expert LLMs and aggregating their outputs. Efficiently exec

hardwarearxiv-cs-lg

3 Jun 2026

RRISE: Robust Radius Inference via a Surrogate Estimator

arXiv:2606.02876v1 Announce Type: new Abstract: Randomized smoothing (RS) uses a smoothed classifier to provide architecture-agnostic certificates of ell_2 classification robustness, but its dependenc

researcharxiv-cs-lg

3 Jun 2026

SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

model-releasesarxiv-cs-cl

arXiv:2606.03284v1 Announce Type: new Abstract: Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NL

3 Jun 2026

Adversarially Robust Control of Conditional Value-at-Risk via Rockafellar-Uryasev Conformal Inference

arXiv:2606.00320v1 Announce Type: new Abstract: We present an online, distribution-free framework for controlling the Conditional Value-at-Risk (CVaR), extending conformal tail risk control to non-sta

safetyarxiv-cs-lg

Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference

arXiv:2606.01711v1 Announce Type: new Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have achieved remarkable success in vision-language tasks, yet the quadratic computation

researcharxiv-cs-cv

Leyline: KV Cache Directives for Agentic Inference

arXiv:2606.01065v1 Announce Type: cross Abstract: Modern KV cache management assumes the chatbot workload: prompts arrive once and the cache grows append-only, so prefix caching and forward-only evict

ProbeScale: Probing Analysis to Optimize Neural Scaling Laws for Efficient Small Language Model Inference

arXiv:2606.01806v1 Announce Type: cross Abstract: Small Language Models (SLMs) offer a balance between capability and computational feasibility. Neural scaling laws inform their optimal training, sugg

CoFiDA-M: Concept-Aware Feature Modulation for Cross-Domain Adaptation with Image-Only Inference

model-releasesarxiv-cs-cv

arXiv:2605.31591v1 Announce Type: new Abstract: Models for AI-based skin cancer screening suffer a severe performance drop when shifting from expert dermoscopic (source) images to consumer-grade clini

IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inference

arXiv:2511.21513v2 Announce Type: replace Abstract: Deploying Transformer models on edge devices is limited by latency and energy budgets. While INT8 quantization effectively accelerates the primary m

researcharxiv-cs-lg

OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference

arXiv:2510.07651v2 Announce Type: replace-cross Abstract: Large language models (LLMs) with extended context windows enable powerful applications but impose significant memory overhead, as caching all

Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization

arXiv:2412.03876v2 Announce Type: replace Abstract: Text-to-Image (T2I) diffusion models are widely recognized for their ability to generate high-quality and diverse images based on text prompts. Howe

safetyarxiv-cs-cv

SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference

arXiv:2602.00942v3 Announce Type: replace Abstract: Modern large language models are increasingly deployed under compute and memory constraints, making flexible control of model capacity a central cha

researcharxiv-cs-lg

Local Ai

AsymVLM: Asymmetric Token Pruning for Efficient Vision-Language Model Inference

arXiv:2605.29535v1 Announce Type: new Abstract: Vision-Language Models (VLMs) process thousands of visual tokens per image alongside comparatively few text tokens, yet existing compression methods tre

local-aiarxiv-cs-lg

29 May 2026

OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning

arXiv:2605.29657v1 Announce Type: cross Abstract: Vision-language models (VLMs) rely on long visual token sequences for visual understanding, making the prefill stage expensive in both computation and

29 May 2026

On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference

model-releasesarxiv-cs-lg

arXiv:2605.29580v1 Announce Type: new Abstract: While parameter-efficient fine-tuning methods like low-rank adaptation (LoRA) are standard for large language models, principled estimation of epistemic

29 May 2026

EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling

arXiv:2510.11170v2 Announce Type: replace-cross Abstract: With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computatio

28 May 2026

Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation

arXiv:2605.28230v1 Announce Type: new Abstract: Modern video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio, a training-fre

researcharxiv-cs-cv

28 May 2026

The Computational Boundary of Inference: Capability Internalization, Training, and the Turing Jump

arXiv:2605.27381v1 Announce Type: cross Abstract: Claims about recursive self-improvement in AI often slide from repeated internal revision to the possibility of qualitatively stronger capability with

28 May 2026

AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents

arXiv:2605.26596v1 Announce Type: new Abstract: The token-level extractive compressors widely used for general LM context are structurally inappropriate for LLM agents: across 17 (env, backbone, metho

27 May 2026

Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing

arXiv:2605.26429v1 Announce Type: cross Abstract: This paper addresses structured out-of-distribution (OOD) testing in high-stakes machine learning applications. Traditional conformal methods rely on

27 May 2026

BODHI: Precise OS Kernel Specification Inference

arXiv:2605.23931v1 Announce Type: new Abstract: The formal verification of operating system kernels requires precise specifications that capture the intended behavior of system calls. Writing these sp

Identifying and Mitigating Systemic Measurement Bias in Production LLM Inference Benchmarks

arXiv:2605.24217v1 Announce Type: new Abstract: As Large Language Models (LLMs) transition from research environments to production deployments, evaluating their performance against strict Service Lev

INSIGHT: INference-time Sequence Introspection for Generating Help Triggers in Vision-Language-Action Models

arXiv:2510.01389v2 Announce Type: replace-cross Abstract: Recent Vision-Language-Action (VLA) models show strong generalization capabilities, yet they lack introspective mechanisms for anticipating fa

Overcoming 'Physics Shock' in Earth Observation A Heteroscedastic Uncertainty Framework for PINN-based Flood Inference

arXiv:2605.24106v1 Announce Type: cross Abstract: Rapid and accurate flood extent mapping from Remote Sensing data, such as Synthetic Aperture Radar (SAR), is critical for operational disaster respons

applicationsarxiv-cs-ai

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

model-releasesarxiv-cs-cl

arXiv:2605.25758v1 Announce Type: new Abstract: Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the re

Archimedean Copula Inference via Taylor-Mode AD

model-releasesarxiv-cs-lg

arXiv:2605.23134v1 Announce Type: new Abstract: No existing nested Archimedean copula tool handles all three of (a) arbitrary per-variable (right-)censoring in survival analysis, (b) arbitrary nesting

25 May 2026

Agents

From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data

arXiv:2605.22897v1 Announce Type: new Abstract: A persistent challenge in machine learning for scientific applications is jointly achieving prediction and understanding. Statistical models excel on st

agentsarxiv-cs-lg

25 May 2026

SciHorizon-GENE: Benchmarking LLM for Life Sciences Inference from Gene Knowledge to Functional Understanding

arXiv:2601.12805v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown growing promise in biomedical research, particularly for knowledge-driven interpretation tasks. Howeve