Search: “inference”

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

arXiv:2506.06295v2 Announce Type: replace-cross Abstract: Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of d

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning

DGX agent

arXiv:2606.03113v1 Announce Type: new Abstract: Large Language Models suffer from slow autoregressive inference. While self-speculative decoding accelerates this process, its efficiency is hampered by

model-releasesarxiv-cs-cl

3 Jun 2026

Fixing FOLIO and MALLS: Verified Annotations and an LLM-assisted Framework to Focus Human Relabeling

arXiv:2606.02837v1 Announce Type: cross Abstract: Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), m

Local Ai

Generalizing Graph Foundation Models via Hyperbolic Retrieval-Augmented Generation

arXiv:2606.03307v1 Announce Type: cross Abstract: Graph foundation models (GFMs) emerged as a dominant paradigm in graph representation learning by leveraging large-scale pre-training for cross-domain

local-aiarxiv-cs-ai

Learning without training: The implicit dynamics of in-context learning

arXiv:2507.16003v4 Announce Type: replace Abstract: One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context. Namely at inference time an LLM is able to l

tutorialsarxiv-cs-cl

Pathway-Structured Privileged Distillation for Deployable Computational Pathology

arXiv:2606.02877v1 Announce Type: new Abstract: Integrating transcriptomics and histopathology can improve cancer risk modelling, yet practical use is constrained by the limited availability of RNA pr

safetyarxiv-cs-cv

StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

arXiv:2606.03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks. However, these systems are highly sensitive to

The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Size

arXiv:2606.02646v1 Announce Type: cross Abstract: Inference-time multi-agent LLM scaling lacks a shared unit: counting nominal agents conflates cost with independent evidence. We derive a two-paramete

Unveiling the Structure of Do-Calculus Reasoning via Derivation Graphs

arXiv:2606.03719v1 Announce Type: new Abstract: The do-calculus defines a general system of inference for interventional queries, allowing causal quantities to be transformed through successive applic

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

arXiv:2606.03971v1 Announce Type: new Abstract: Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segm

tutorialsarxiv-cs-cv

When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics

arXiv:2606.03569v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference. While vis

safetyarxiv-cs-ai

A Hypertoroidal Covering for Perfect Color Equivariance

arXiv:2603.04256v3 Announce Type: replace Abstract: When the color distribution of input images changes at inference, the performance of conventional neural network architectures drops considerably. A

researcharxiv-cs-cv

Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation

arXiv:2512.16310v3 Announce Type: replace-cross Abstract: LLM-based agents increasingly use multiple external tools to complete complex tasks. We study Tools Orchestration Privacy Risk (TOP-R): an age

AGENTCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

arXiv:2606.02461v1 Announce Type: new Abstract: Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future e

Agents

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

arXiv:2606.01363v1 Announce Type: new Abstract: Model-based reinforcement learning (MBRL) infers information about the environment from a learned dynamics model and bears the potential to address open

agentsarxiv-cs-lg

Cost-Aware Diffusion Draft Trees for Speculative Decoding

arXiv:2606.01813v1 Announce Type: new Abstract: Speculative decoding accelerates inference by having a lightweight drafter propose tokens verified in parallel by the target language model. Block diffu

Cross-Environment Neural Reranking for Sample-Efficient Action Selection in Text-Based Agents

model-releasesarxiv-cs-cl

arXiv:2606.02204v1 Announce Type: new Abstract: Large language model agents achieve strong performance on text-based benchmarks but incur prohibitive inference costs, motivating the use of compact neu

Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap

arXiv:2512.10236v2 Announce Type: replace-cross Abstract: Modern ML workloads demand distributing training and inference across multiple GPUs. However, these parallelization techniques often suffer fr

hardwarearxiv-cs-lg

DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding

arXiv:2606.02091v1 Announce Type: new Abstract: Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in

DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing

arXiv:2511.04791v2 Announce Type: replace Abstract: Modern LLM serving systems must sustain high throughput while meeting strict latency SLOs across two distinct inference phases: compute-intensive pr

hardwarearxiv-cs-lg

EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models

arXiv:2606.01756v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong performance on image and video understanding tasks, but their inference efficiency is constrained by

researcharxiv-cs-cv

FLARE: Diffusion for Hybrid Language Model

arXiv:2606.01774v1 Announce Type: cross Abstract: Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-laten

hardwarearxiv-cs-ai

From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection

arXiv:2602.03018v2 Announce Type: replace Abstract: Outlier detection (OD) is widely used in practice; but its effective deployment on new tasks is hindered by lack of labeled outliers, which makes al

researcharxiv-cs-lg

FROST-STA: Frozen Dense Features for the Ego4D Short-Term Object Interaction Anticipation

arXiv:2606.00694v1 Announce Type: new Abstract: Short-term anticipation in egocentric video requires more than recognizing the current scene: a system must infer which object the camera wearer will co

safetyarxiv-cs-cv

HERO'S JOURNEY: Testing Complex Rule Induction with Text Games

model-releasesarxiv-cs-cl

arXiv:2606.02556v1 Announce Type: new Abstract: We introduce HERO'S JOURNEY, a benchmark for rule induction in goal-directed episodic tasks, where agents must infer hidden rules from demonstrations an

HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression

arXiv:2606.01934v1 Announce Type: cross Abstract: Large language models achieve remarkable performance via extended chain-of-thought (CoT) reasoning, yet this lengthy process incurs substantial infere

safetyarxiv-cs-cl

LERD: Latent Event-Relational Dynamics for Neurodegenerative Classification

arXiv:2602.18195v2 Announce Type: replace-cross Abstract: Alzheimer's disease (AD) alters brain electrophysiology and disrupts multichannel EEG dynamics, making accurate and clinically useful EEG-base

tutorialsarxiv-cs-ai

LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

arXiv:2602.23881v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate t

Applications

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

arXiv:2606.01336v1 Announce Type: new Abstract: As real-world applications increasingly require processing inputs of 100k+ tokens, the gap between context length and inference efficiency has become a

applicationsarxiv-cs-cl

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling

arXiv:2606.00888v1 Announce Type: cross Abstract: Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we fin

Motion-aware Event Suppression for Event Cameras

model-releasesarxiv-cs-cv

arXiv:2602.23204v3 Announce Type: replace Abstract: In this work, we introduce the first framework for Motion-aware Event Suppression, which learns to filter events triggered by IMOs and ego-motion in

Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving

arXiv:2606.01839v1 Announce Type: cross Abstract: LLM-based agents resolve a user task through many turns of dependent inference and tool calls, producing a workload whose total cost is unknown when t

hardwarearxiv-cs-lg

Agents

OctoT2I: A Self-Evolving Agentic Text-to-Image Router

arXiv:2606.01803v1 Announce Type: new Abstract: The explosive growth of Text-to-Image (T2I) models, from large-scale versions to lightweight, real-time ones, now faces diminishing marginal returns fro

agentsarxiv-cs-ai

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

arXiv:2606.01682v1 Announce Type: cross Abstract: Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small mo

On the Collapse of Generative Paths: A Criterion and Correction for Diffusion Steering

arXiv:2512.10339v2 Announce Type: replace Abstract: Inference-time steering adapts pretrained diffusion and flow models to new tasks without retraining, often utilizing ratio-of-densities construction

On the evolution of the concept of probability as a mirror of the evolution of reason

arXiv:2606.00102v1 Announce Type: new Abstract: Over the centuries, probability theory has grown from the calculus of games of chance into a central framework for reasoning under uncertainty. This art

Private and Stable Test-Time Adaptation with Differential Privacy

arXiv:2606.01908v1 Announce Type: cross Abstract: Test-time adaptation (TTA) can reduce error on new and different data by updating the model on these inputs during inference. However, these updates r

researcharxiv-cs-cv

SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding

arXiv:2606.00021v1 Announce Type: cross Abstract: Speculative Decoding (SD) accelerates Large Language Model (LLM) inference by employing a lightweight draft model to propose candidate tokens, which a

SimSD: Simple Speculative Decoding in Diffusion Language Models

arXiv:2606.02544v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference throug

Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards

arXiv:2510.01167v2 Announce Type: replace-cross Abstract: Aligning large language models to human preferences is inherently multidimensional, yet most pipelines collapse heterogeneous signals into a s

safetyarxiv-cs-ai

SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training

arXiv:2606.02355v1 Announce Type: new Abstract: Long-horizon LLM agents can benefit from reusable skills, yet existing skill-based methods often rely on external skill generators during training or pe

safetyarxiv-cs-ai

STARFISH: faST Accuracy Recovery in pruned networks From Internal State Healing

arXiv:2606.01126v1 Announce Type: cross Abstract: Pruning is a process designed to reduce the number of weights in a large neural network. This can substantially speed up inference but might cause a c

StreamingVLM: Real-Time Understanding for Infinite Video Streams

arXiv:2510.09608v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) could power real-time assistants and autonomous agents, but they face a critical challenge: understanding near-i

Strong Stochastic Flow Maps

arXiv:2606.01086v1 Announce Type: cross Abstract: Flow and diffusion models generate high-quality samples in many modalities; however, many network evaluations are required during inference due to num

tutorialsarxiv-cs-ai

ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks

arXiv:2606.01080v1 Announce Type: cross Abstract: Large language models often improve on difficult tasks by spending inference-time compute on a reasoning trace before producing the final answer. That

Training-Free Imitation Learning with Closed-Form Diffusion Policies

arXiv:2606.01238v1 Announce Type: cross Abstract: While diffusion-based policies have impressive performance and expressivity, their long offline training slows down the data collection and policy dep

safetyarxiv-cs-lg

Training Prompt Matters: State-Adaptive Optimization for Robust Fine-Tuning

arXiv:2606.01967v1 Announce Type: new Abstract: While prompt engineering is instrumental in maximizing the capabilities of Large Language Models (LLMs) during inference, the role of prompts during tra

Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention

model-releasesarxiv-cs-cl

arXiv:2606.01243v1 Announce Type: new Abstract: Latent reasoning enables Large Language Models (LLMs) to perform multi-step inference within continuous hidden states, offering efficiency gains over ex