Search: “inference”

When Do Fewer Visual Tokens Accelerate Multimodal Inference? A Break-Even Study Across Decision Locations and Hardware

model-releasesarxiv-cs-cv

arXiv:2608.03649v1 Announce Type: new Abstract: Fewer visual tokens do not guarantee lower end-to-end latency. We evaluate break-even with a reproducible protocol that accounts for decision overhead,

5 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Augmented Inverse Hybrid Weighting: Robust Inference under Deterministic and Random Distribution Shifts

DGX agent

arXiv:2608.00701v1 Announce Type: cross Abstract: Reweighting source samples to match a target covariate distribution is a standard response to distribution shift when generalizing evidence from one p

model-releasesarxiv-cs-lg

4 Aug 2026

PeCA: Palette Context Assisted Inference for Test-Time Paint-Bucket Colourisation on Animation Videos

arXiv:2608.00903v1 Announce Type: new Abstract: In animation production, paint-bucket colourisation for hand-drawn animation is a labour-intensive procedure that assigns each enclosed region in line s

applicationsarxiv-cs-cv

4 Aug 2026

Inference-time Trajectory Optimization for Structure-Preserving Manga Image Editing

model-releasesarxiv-cs-cv

arXiv:2603.27790v2 Announce Type: replace Abstract: We present a lightweight, training-free trajectory correction method that adapts a pretrained image editing model to each input manga image using on

3 Aug 2026

ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA

arXiv:2607.28442v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a ke

applicationsarxiv-cs-cv

31 Jul 2026

The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text

arXiv:2607.26309v1 Announce Type: cross Abstract: Estimating causal effects of linguistic properties from observational text is difficult because the same document can contain both the treatment of in

safetyarxiv-cs-cl

30 Jul 2026

When benchmark inferences do not compose: Projectibility in AI evaluation

arXiv:2607.26159v1 Announce Type: cross Abstract: An AI benchmark result rarely reaches a consequential claim in one step. Evaluators generalize it to further cases, interpret it as evidence of capabi

30 Jul 2026

Enabling Fully Integer-Only Inference for Lightweight Detection Transformers

arXiv:2607.24981v1 Announce Type: new Abstract: Vision Transformer detectors now approach the accuracy of CNNs but remain difficult to deploy on NPUs and microcontrollers because key components, inclu

applicationsarxiv-cs-cv

29 Jul 2026

GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference

arXiv:2607.24788v1 Announce Type: new Abstract: As Large Language Models scale to increasingly long contexts, the memory I/O and computational overhead of the Key-Value (KV) cache during decoding emer

29 Jul 2026

Local Ai

Vendor-agnostic ML inference on production edge devices [R]

local-air-machinelearning

I work on PostSlate, a video editing tool, and this comes out of our own work. We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about t

29 Jul 2026

Local Ai

DRC-Aid: Design-Rule Correction via Agentic Framework utilizing Inference-Time Large Language Models

arXiv:2607.22761v1 Announce Type: cross Abstract: Resolving Design Rule Violations (DRVs) in layouts entails an iterative loop of geometric edits and verification. We present DRC-Aid, a closed-loop ag

local-aiarxiv-cs-lg

28 Jul 2026

Semantic Field Theory: Historical Origin, Higher-Order Interaction, and Stabilized Semantic Inference

arXiv:2607.20451v1 Announce Type: new Abstract: Semantic Field Theory (SFT) has developed from a philosophical critique of strong anti-formalist readings of language games into a proposed computationa

researcharxiv-cs-cl

24 Jul 2026

Statistical Inference for Generative Model Comparison

arXiv:2501.18897v4 Announce Type: replace-cross Abstract: Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty qua

24 Jul 2026

Kernel Ridge Regression Inference

arXiv:2302.06578v4 Announce Type: replace-cross Abstract: We provide uniform confidence bands for kernel ridge regression (KRR), a widely used nonparametric regression estimator for nonstandard data s

researcharxiv-cs-lg

23 Jul 2026

LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models

arXiv:2607.16339v2 Announce Type: replace Abstract: Diffusion-based Large Language Models(DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation. However, current

23 Jul 2026

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy

arXiv:2607.19864v1 Announce Type: cross Abstract: Diabetic retinopathy is a leading cause of preventable blindness; its early lesions are small, low contrast, and easily missed in manual screening. Mo

23 Jul 2026

Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMs

arXiv:2607.13080v1 Announce Type: cross Abstract: Autonomous coding agents force engineering organizations to choose between API-based frontier models -- strong reasoning at high token cost -- and on-

16 Jul 2026

PUe: Biased Positive-Unlabeled Learning Enhancement by Causal Inference

arXiv:2607.13428v1 Announce Type: new Abstract: Positive-Unlabeled (PU) learning aims to achieve high-accuracy binary classification with limited labeled positive examples and numerous unlabeled ones.

safetyarxiv-cs-lg

16 Jul 2026

Physically Consistent Parameter Inference: Transparent Machine Learning Emulation in High Energy Physics and Cosmology

arXiv:2607.12726v1 Announce Type: cross Abstract: Global fits in high energy physics and cosmology often face the challenge of exploring high-dimensional parameter spaces with computationally expensiv

15 Jul 2026

Tutorials

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions

arXiv:2603.02204v2 Announce Type: replace Abstract: Selective conformal prediction can yield substantially tighter uncertainty sets when we can identify calibration examples that are exchangeable with

tutorialsarxiv-cs-lg

10 Jul 2026

Agents

Akashic: A Low-Overhead LLM Inference Service with MemAttention

arXiv:2607.05708v1 Announce Type: new Abstract: Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows. Replaying

agentsarxiv-cs-ai

8 Jul 2026

Biologically Informed Deep Neural Networks for Multi-Omic Integration, Pathway Activity Inference and Risk Stratification in Cancer

arXiv:2607.05306v1 Announce Type: new Abstract: Integrating complex, multi-omics data presents significant challenges. Existing approaches often face a trade-off between model interpretability and rep

researcharxiv-cs-lg

7 Jul 2026

SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

arXiv:2607.03333v1 Announce Type: cross Abstract: LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: th

hardwarearxiv-cs-ai

7 Jul 2026

Tile-Level Activation Overlap for Efficient LLM Inference

arXiv:2607.02521v1 Announce Type: cross Abstract: SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time.

7 Jul 2026

MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support

arXiv:2607.01814v1 Announce Type: new Abstract: Traditional Chinese Medicine (TCM) diagnosis, particularly through tongue inspection, faces persistent challenges in subjectivity and reproducibility. T

3 Jul 2026

Proxy-GS: Unified Occlusion Priors for Training and Inference in Structured 3D Gaussian Splatting

arXiv:2509.24421v5 Announce Type: replace Abstract: 3D Gaussian Splatting (3DGS) has emerged as an efficient approach for achieving photorealistic rendering. Recent MLP-based variants further improve

researcharxiv-cs-cv

1 Jul 2026

Relational and Sequential Conformal Inference for Energy Time Series over Graphs via Foundation Models

arXiv:2606.31804v1 Announce Type: new Abstract: Accurate energy demand forecasting is essential for the reliable operation and planning of modern sustainable energy systems. Spatial-temporal graph neu

safetyarxiv-cs-lg

1 Jul 2026

SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference

arXiv:2606.31145v1 Announce Type: new Abstract: Large language models increasingly operate over long contexts, where the KV cache becomes a dominant memory bottleneck: its size grows linearly with seq

hardwarearxiv-cs-cl

1 Jul 2026

DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training

arXiv:2606.28932v1 Announce Type: cross Abstract: Large language models have driven recent progress in language and multimodal AI, yet pre-training them at scale is prohibitively expensive. Low-rank p

How NVIDIA’s Inference Software Stack Powers the Lowest Token Cost

As organizations move from AI pilots to production AI factories, infrastructure decisions have shifted from peak chip specifications to cost per token: how many useful tokens they can deliver per doll

hardwarenvidia-blog

Representation Learning for Equivariant Inference with Guarantees

arXiv:2505.19809v3 Announce Type: replace-cross Abstract: In many real-world applications of regression, conditional probability estimation, and uncertainty quantification, exploiting symmetries roote

applicationsarxiv-cs-ai

SpecMind: Cognitively Inspired, Interactive Multi-Turn Framework for Postcondition Inference

arXiv:2602.20610v3 Announce Type: replace-cross Abstract: Specifications are vital for ensuring program correctness, yet writing them manually remains challenging and time-intensive. Recent large lang

safetyarxiv-cs-cl

EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction

arXiv:2606.27550v1 Announce Type: new Abstract: Multi-token prediction has been shown to increase data density during training, improve downstream text-generation quality, and serves as the defacto ap

researcharxiv-cs-cl

29 Jun 2026

Context Recycling for Long-Horizon LLM Inference

arXiv:2606.26105v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons due t

26 Jun 2026

Agents

Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models

arXiv:2606.26366v1 Announce Type: new Abstract: Standard chain-of-thought on moral dilemmas exhibits two failure modes: stakeholder collapse (the trace names at most one party with a stake in the outc

agentsarxiv-cs-ai

26 Jun 2026

TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference

arXiv:2606.27161v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of

26 Jun 2026

Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference

arXiv:2606.17165v3 Announce Type: replace-cross Abstract: Organizations and researchers show increasing interest in using large language models (LLMs) in place of human participants in A/B tests, in t

safetyarxiv-cs-ai

24 Jun 2026

Exploring Accurate and Transparent Domain Adaptation in Predictive Healthcare via Concept-Grounded Orthogonal Inference

arXiv:2602.12542v2 Announce Type: replace-cross Abstract: Deep learning models for clinical event prediction on electronic health records (EHR) often suffer performance degradation when deployed under

safetyarxiv-cs-ai

IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference

arXiv:2606.09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning. Both KV memory and KV read b

JOIN: Anchor-Grasp-Conditioned Joining via Opposition, Inference, and Navigation for Bimanual Assistive Manipulation

arXiv:2606.11151v1 Announce Type: new Abstract: Assistive mobility and manipulation platforms have received increasing attention as a means of restoring independence to individuals with disabilities.

researcharxiv-cs-ro

Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

arXiv:2606.10537v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with contex

researcharxiv-cs-cl

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

arXiv:2606.10445v1 Announce Type: cross Abstract: Semi-structured 2:4 sparsity is widely supported by modern accelerators, providing up to a 2x theoretical speedup. However, its strict 50% sparsity co

hardwarearxiv-cs-cl

APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing

arXiv:2606.08761v1 Announce Type: cross Abstract: W4A4 quantization promises full utilization of INT4 Tensor Cores, yet group dequantization overhead on CUDA Cores has driven existing systems to mixed

9 Jun 2026

Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder

model-releasesarxiv-cs-cv

arXiv:2606.05981v1 Announce Type: new Abstract: Aggressive distillation of the diffusion U-Net inverts the per-frame bottleneck of real-time text-to-image pipelines: once the denoiser is a 4-step or 1

5 Jun 2026

Ekka: Automated Diagnosis of Silent Errors in LLM Inference

arXiv:2606.04594v1 Announce Type: cross Abstract: LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations. The rapid development process can introd

4 Jun 2026

UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancing

arXiv:2606.04101v1 Announce Type: cross Abstract: Large-scale expert parallelism (EP) is becoming pivotal for training and serving frontier MoE models, but it also amplifies device-level expert load i

applicationsarxiv-cs-lg

4 Jun 2026

Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings

arXiv:2606.04286v1 Announce Type: new Abstract: Online reviews provide valuable insights into the perceived quality of facets of a product or service. While aspect-based sentiment analysis has focused

applicationsarxiv-cs-cl

4 Jun 2026

MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency