Search: “inference”

Alignment Defends LLMs from Property Inference Attacks

arXiv:2606.10217v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets that may contain sensitive, dataset-level properties. Recent work h

safetyarxiv-cs-lg

AnyMod-LLVE: Low-Light Video Enhancement with Modality-Agnostic Inference

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2606.11186v1 Announce Type: new Abstract: Low-light video enhancement (LLVE) remains a challenging task due to severe information degradation under low-illumination conditions. Recent multimodal

applicationsarxiv-cs-cv

10 Jun 2026

CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

arXiv:2606.10935v1 Announce Type: cross Abstract: Large language model inference is bottlenecked by autoregressive decoding, where each token requires a full forward pass. Multi-token prediction (MTP)

Tutorials

Learn how @cursor_ai partnered with Together AI to deliver real-time inference for AI-powered coding in this article from @ce_zhang and @rea…

Learn how @cursor_ai partnered with Together AI to deliver real-time inference for AI-powered coding in this article from @ce_zhang and @realDanFu. Cursor's in-editor agents generate code while develo

tutorialstogether-ai--x

Operator Fusion for LLM Inference on the Tensix Architecture

arXiv:2606.09879v1 Announce Type: new Abstract: This study addresses on-device inference bottlenecks of Transformer models on Tenstorrent's Tensix architecture and proposes an operator fusion strategy

local-aiarxiv-cs-lg

RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

arXiv:2606.09937v1 Announce Type: cross Abstract: We introduce RKSC (Reasoning-Aware KV Cache Sharing), a training-free inference framework that eliminates two structural redundancies in multi-branch

AUCp: Pseudo-AUC for Inference Model Selection with Unlabeled Validation Data in Abnormality Detection

arXiv:2606.08742v1 Announce Type: new Abstract: Abnormality detection is a crucial yet challenging task in medical image analysis. Distinguishing abnormalities from normal data by learning to reconstr

researcharxiv-cs-cv

Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy

arXiv:2606.09080v1 Announce Type: new Abstract: Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove compu

BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inference

model-releasesarxiv-cs-lg

arXiv:2606.09514v1 Announce Type: new Abstract: Large language models (LLMs) incur high inference cost due to their depth and parameter scale. Depth pruning can reduce latency by skipping redundant Tr

Tutorials

C^3ache: Accelerating World Action Models with Cross Inference Chunk Cache

arXiv:2606.08962v1 Announce Type: cross Abstract: World Action Models (WAMs) generalize better than standard Vision-Language-Action (VLA) policies to novel motions and environments, because a video-mo

tutorialsarxiv-cs-cv

Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization

arXiv:2510.10028v2 Announce Type: replace-cross Abstract: The rapid advancement of Low-Altitude Economy Networks (LAENets) has enabled a variety of applications, including aerial surveillance, environ

From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

arXiv:2606.09508v1 Announce Type: new Abstract: Existing sparse attention and KV cache compression methods for long-context LLM inference typically apply fixed sparsity patterns or uniform budgets acr

Inference-Time Conformal Reasoning with Valid Factuality Control for Large Language Models

arXiv:2606.08831v1 Announce Type: new Abstract: Large language models (LLMs) increasingly perform multi-step reasoning, where intermediate claims form implicit directed acyclic graphs whose node corre

Intrinsic Selection and Particle Resampling for Inference-Time Scaling Beyond Domain Verifiability

arXiv:2606.08850v1 Announce Type: cross Abstract: Inference-Time Scaling (ITS) has largely succeeded in verifiable domains like math and coding, where cheap verification enables scalable output select

Sequential statistical inference for Large Language Models: Representation, validity, and monitoring

arXiv:2606.07624v1 Announce Type: new Abstract: This discussion argues that sequential statistical inference can naturally contribute to LLM trustworthiness. In deployment, LLM systems are queried rep

safetyarxiv-cs-lg

sGPO: Trading Inference FLOPs for Training Efficiency in RLVR

arXiv:2606.08854v1 Announce Type: cross Abstract: Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each

safetyarxiv-cs-ai

Three MLX videos dropped at WWDC: Running agents locally by @angeloskath https://www.youtube.com/watch?v=wykPErJ8M-8 Distributed inference a…

Three MLX videos dropped at WWDC: Running agents locally by @angeloskath https://www.youtube.com/watch?v=wykPErJ8M-8 Distributed inference and training by Tatiana Likhomanenko https://www.youtube.com/

local-aiollama--x

Visual Template Inference for Data Extraction from Documents

model-releasesarxiv-cs-cv

arXiv:2501.06659v2 Announce Type: replace-cross Abstract: Many templatized documents are programmatically generated from structured data following a visual template. Such documents include invoices, t

vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models

arXiv:2606.08094v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) policies are typically shipped as Python/PyTorch stacks that assume a workstation-class GPU, a mismatch for the hardware

When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference

arXiv:2606.08098v1 Announce Type: new Abstract: Majority voting over sampled answers is the dominant unsupervised aggregator for multi-sample LLM inference. We show that piping the signals every sampl

Introducing 'Signals of AI' - DeepAgent powered by - @nebiustf for Inference - @NVIDIAAI Nemotron 3 Ultra - @tavilyai for Web Search - @Lang…

model-releasesharrison-chase--x

Introducing 'Signals of AI' - DeepAgent powered by - @nebiustf for Inference - @NVIDIAAI Nemotron 3 Ultra - @tavilyai for Web Search - @LangChain for the core deepagent architecture! An ideal engine f

8 Jun 2026

Industry

We are releasing our first quantized checkpoints for the Qwen3.5 series of models, co-designed jointly with our inference engine to achieve …

We are releasing our first quantized checkpoints for the Qwen3.5 series of models, co-designed jointly with our inference engine to achieve maximum possible performance on Apple hardware Starting from

industryclem-delangue--x

8 Jun 2026

GITCO: Gated Inference-Time Context Optimization in TSFMs

arXiv:2606.05332v1 Announce Type: new Abstract: Patch-based Time Series Foundation Models (TSFMs) suffer from context poisoning: structurally anomalous patches capture disproportionate attention and s

6 Jun 2026

Four weeks, three enterprise customers, same pattern across all three. Most inference spend goes to retrieval loops. A generic index carries…

Four weeks, three enterprise customers, same pattern across all three. Most inference spend goes to retrieval loops. A generic index carries no knowledge of domain, query types, or task structure, so

applicationspinecone--x

5 Jun 2026

Multi-Granularity Reasoning for Natural Language Inference

arXiv:2606.05181v1 Announce Type: new Abstract: Natural Language Inference (NLI) is a fundamental task in natural language understanding that requires determining the logical relationship between a pr

researcharxiv-cs-cl

5 Jun 2026

Statistically Reliable LLM-Based Ranking Evaluation via Prediction-Powered Inference

model-releasesarxiv-cs-cl

arXiv:2606.05308v1 Announce Type: cross Abstract: With PRECISE, we extended Prediction-Powered Inference to produce bias-corrected estimates of ranking evaluation metrics by combining a small human-la

5 Jun 2026

Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge

arXiv:2606.04581v1 Announce Type: cross Abstract: Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs). In this work, we propos

4 Jun 2026

Hardware

SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

arXiv:2606.04511v1 Announce Type: new Abstract: Sparse attention reduces compute and memory bandwidth for long-context LLM inference. However, two key challenges remain: (1) KV cache capacity still gr

hardwarearxiv-cs-cl

4 Jun 2026

Hardware

Together AI provides the inference stack behind both: high-throughput serving on the latest NVIDIA Blackwell GPUs for agentic workloads, and…

Together AI provides the inference stack behind both: high-throughput serving on the latest NVIDIA Blackwell GPUs for agentic workloads, and TensorRT engines plus event-driven streaming I/O for low-la

hardwaretogether-ai--x

4 Jun 2026

Day 2 at #MSBuild is about what it takes to move beyond generic foundation models. Think customization, inference performance, and getting p…

applicationsfireworks-ai--x

Day 2 at #MSBuild is about what it takes to move beyond generic foundation models. Think customization, inference performance, and getting production-ready AI deployed at scale. @chahvivi will lead a

Fast-dLLM++: Frechet Profile Decoding for Faster Diffusion LLM Inference

arXiv:2606.02955v1 Announce Type: cross Abstract: Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely co

Fine-tuning to production inference is the gap where teams get stuck. At #MSBuild today, our own Rob Ferguson, @danielhanchen (@UnslothAI) a…

applicationsfireworks-ai--x

Fine-tuning to production inference is the gap where teams get stuck. At #MSBuild today, our own Rob Ferguson, @danielhanchen (@UnslothAI) and @marksaroufim (@coreautoai) discuss: model customization

Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics

arXiv:2606.02981v1 Announce Type: new Abstract: Best-of-N inference scaling (drawing N candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by

researcharxiv-cs-cl

Spike-Aware C++ INT8 Inference for Sparse Spiking Language Models on Commodity CPUs

arXiv:2606.03026v1 Announce Type: cross Abstract: Spiking language models expose activation sparsity that dense Transformer runtimes do not directly exploit. This paper studies that property from a sy

The Impact of Temporal Granularity on Socio-Demographic Inference from Household Load Profiles

arXiv:2606.03358v1 Announce Type: new Abstract: Smart meter data can reveal sensitive socio-demographic characteristics of households, raising privacy concerns. While this risk has been demonstrated a

Using open models and inference clouds (which serve open models) is a leading indicator of what is to come. The advantage of open weights is…

model-releasesclem-delangue--x

Using open models and inference clouds (which serve open models) is a leading indicator of what is to come. The advantage of open weights is that you can train, serve, and continually improve your own

Efficient Test-time Inference for Generative Planning Models

arXiv:2606.00618v1 Announce Type: new Abstract: Generative models have emerged as a powerful paradigm for AI planning, yet their performance remains constrained by the training data distribution. One

Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference

arXiv:2603.12037v2 Announce Type: replace Abstract: Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an i

safetyarxiv-cs-lg

Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry

arXiv:2511.01064v3 Announce Type: replace-cross Abstract: Variational inference (VI) approximates a target density p by the best match q in a family of tractable distributions. The best variational ap

GLIDE: Graph-guided Leap Inference for Diffusion Estimation of Spatio-Temporal Point Processes

arXiv:2606.01273v1 Announce Type: new Abstract: Spatio-temporal point processes (STPPs) provide a principled framework for modeling asynchronous events in continuous time and space. Recent diffusion-b

local-aiarxiv-cs-lg

Tutorials

MINES: Explainable Anomaly Detection through Web API Invariant Inference

arXiv:2512.06906v2 Announce Type: replace-cross Abstract: Detecting the anomalies of web applications, important infrastructures for running modern companies and governments, is crucial for providing

tutorialsarxiv-cs-lg

Move from test to production by running high-performance inference directly on Foundry. At #MSBuild, we demoed an end-to-end workflow showin…

applicationsfireworks-ai--x

Move from test to production by running high-performance inference directly on Foundry. At #MSBuild, we demoed an end-to-end workflow showing how unified infrastructure improves latency, reduces cost,

Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference

arXiv:2606.02430v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into high-performance computing (HPC) workflows, accelerating scientific discovery through di

Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time

model-releasesarxiv-cs-cl

arXiv:2606.01923v1 Announce Type: new Abstract: Large Language Models (LLMs) frequently exhibit 'contextual disregard' when faced with input evidence that conflicts with their internal parametric memo

Score imes Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation

arXiv:2606.00739v1 Announce Type: new Abstract: Large language models hallucinate even when the answer lies within their parameters. While inference-time scaling can surface this latent knowledge, the

Tools

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

MiniMax-M3 is a large language model capable of handling 1 million token contexts and multimodal inputs while maintaining efficient inference performance. Together AI's blog post discusses techniques

toolstogether-ai-blog

Hardware

Threshold-Based Exclusive Batching for LLM Inference

arXiv:2606.00516v1 Announce Type: new Abstract: Mixed batching (MB)--interleaving prefill and decode in a single batch--has become the standard scheduling strategy for large language model (LLM) infer

hardwarearxiv-cs-ai

Today we're announcing that hybrid agentic inference is coming to Perplexity Computer. Computer can split tasks between a local model runnin…

Today we're announcing that hybrid agentic inference is coming to Perplexity Computer. Computer can split tasks between a local model running on your machine and frontier models in the cloud. This kee

local-aiperplexity--x