Search: “inference”

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

arXiv:2606.04446v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model

researcharxiv-cs-lg

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Depth-Attention: Cross-Layer Value Mixing for Language Models

DGX agent

arXiv:2606.05014v1 Announce Type: new Abstract: Self-attention selects information freely across the sequence, but across depth, Transformers merely add each layer's output to the residual stream, so

researcharxiv-cs-cl

4 Jun 2026

GroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs

model-releasesarxiv-cs-cv

arXiv:2606.04184v1 Announce Type: new Abstract: True general intelligence requires not only a model of the physical world but also a social world model: the capacity to infer how individual mental sta

Applications

Long-Term and Short-Term Transistor Aging in Deep Neural Networks: Impact and Mitigation

arXiv:2606.04266v1 Announce Type: cross Abstract: Deep neural networks (DNNs) are used in a variety of real-world applications including, for example, image classification and speech recognition. The

applicationsarxiv-cs-lg

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606.04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typical

local-aiarxiv-cs-ai

Applications

SSSD: Simply-Scalable Speculative Decoding

arXiv:2411.05894v3 Announce Type: replace-cross Abstract: Speculative Decoding has emerged as a popular technique for accelerating inference in Large Language Models. However, most existing approaches

applicationsarxiv-cs-ai

Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling

arXiv:2606.04920v1 Announce Type: cross Abstract: Quantizing deep neural networks is essential for efficient inference on resource-constrained devices. However, most existing methods are designed for

safetyarxiv-cs-cv

Transmuting prompts into weights

arXiv:2510.08734v3 Announce Type: replace Abstract: A growing body of research has demonstrated that the behavior of large language models can be effectively controlled at inference time by directly m

researcharxiv-cs-lg

Applications

Uncertainty-Aware End-to-End Co-Design of Neural Network Processors: From Training and Mapping to Fabrication

arXiv:2606.04850v1 Announce Type: cross Abstract: Designing a neural network processor is an end-to-end co-design problem: network architecture and training budget determine the inference workload; ha

applicationsarxiv-cs-ai

WAM-Nav: Asymmetric Latent World-Action Modeling for Unified Visual Navigation

arXiv:2606.04907v1 Announce Type: new Abstract: Visual navigation requires generating smooth and collision-free trajectories under complex geometric and physical constraints. Existing reactive policie

safetyarxiv-cs-ro

Are we really tilting? The mechanics of reward guidance in flow and diffusion models

arXiv:2606.02884v1 Announce Type: cross Abstract: Reward guidance algorithms steer a learned generative process toward the reward-tilted measure at inference time. While empirically powerful, these me

safetyarxiv-cs-ai

b9491

b9491 is a release of llama.cpp , a C/C++ implementation of large language model inference that enables running LLMs locally with minimal dependencies. This release likely contains bug fixes, feature

b9493

B9493 is a release of llama.cpp, an LLM inference framework in C/C++ . This release includes updates to model support, such as centralized hidden activation mappings and additions for granite embeddin

b9495

The search didn't return specific information about release b9495. Based on the available information about llama.cpp releases, here's a summary: llama.cpp enables LLM inference with minimal setup and

Denoising Tells When to Replan: Denoising-Variance Adaptive Chunking for Flow-Based Robot Policies

arXiv:2606.03847v1 Announce Type: new Abstract: Action chunking has become a common inference strategy for flow-based robot policies, improving action coherence by modeling multi-step temporal depende

safetyarxiv-cs-ro

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

arXiv:2506.06295v2 Announce Type: replace-cross Abstract: Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of d

researcharxiv-cs-ai

Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning

model-releasesarxiv-cs-cl

arXiv:2606.03113v1 Announce Type: new Abstract: Large Language Models suffer from slow autoregressive inference. While self-speculative decoding accelerates this process, its efficiency is hampered by

Fixing FOLIO and MALLS: Verified Annotations and an LLM-assisted Framework to Focus Human Relabeling

arXiv:2606.02837v1 Announce Type: cross Abstract: Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), m

Generalizing Graph Foundation Models via Hyperbolic Retrieval-Augmented Generation

arXiv:2606.03307v1 Announce Type: cross Abstract: Graph foundation models (GFMs) emerged as a dominant paradigm in graph representation learning by leveraging large-scale pre-training for cross-domain

local-aiarxiv-cs-ai

Tutorials

Learning without training: The implicit dynamics of in-context learning

arXiv:2507.16003v4 Announce Type: replace Abstract: One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context. Namely at inference time an LLM is able to l

tutorialsarxiv-cs-cl

MiniMax M3 arrives with MiniMax Sparse Attention (MSA), 15.6x faster decoding at 1M tokens. We're partnering with @MiniMax_AI to power the i…

model-releasesfireworks-ai--x

MiniMax M3 arrives with MiniMax Sparse Attention (MSA), 15.6x faster decoding at 1M tokens. We're partnering with @MiniMax_AI to power the inference behind this week's launch. Head to http://minimax.i

Pathway-Structured Privileged Distillation for Deployable Computational Pathology

arXiv:2606.02877v1 Announce Type: new Abstract: Integrating transcriptomics and histopathology can improve cancer risk modelling, yet practical use is constrained by the limited availability of RNA pr

safetyarxiv-cs-cv

StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

arXiv:2606.03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks. However, these systems are highly sensitive to

The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Size

arXiv:2606.02646v1 Announce Type: cross Abstract: Inference-time multi-agent LLM scaling lacks a shared unit: counting nominal agents conflates cost with independent evidence. We derive a two-paramete

Unveiling the Structure of Do-Calculus Reasoning via Derivation Graphs

arXiv:2606.03719v1 Announce Type: new Abstract: The do-calculus defines a general system of inference for interventional queries, allowing causal quantities to be transformed through successive applic

researcharxiv-cs-ai

Tutorials

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

arXiv:2606.03971v1 Announce Type: new Abstract: Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segm

tutorialsarxiv-cs-cv

When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics

arXiv:2606.03569v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference. While vis

safetyarxiv-cs-ai

A Hypertoroidal Covering for Perfect Color Equivariance

arXiv:2603.04256v3 Announce Type: replace Abstract: When the color distribution of input images changes at inference, the performance of conventional neural network architectures drops considerably. A

researcharxiv-cs-cv

Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation

arXiv:2512.16310v3 Announce Type: replace-cross Abstract: LLM-based agents increasingly use multiple external tools to complete complex tasks. We study Tools Orchestration Privacy Risk (TOP-R): an age

AGENTCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

arXiv:2606.02461v1 Announce Type: new Abstract: Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future e

Agents

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

arXiv:2606.01363v1 Announce Type: new Abstract: Model-based reinforcement learning (MBRL) infers information about the environment from a learned dynamics model and bears the potential to address open

agentsarxiv-cs-lg

b9468

B9468 is an intermediate build release of llama.cpp, the open-source C/C++ library that enables large language model inference on consumer hardware. This release continues the project's rapid developm

b9474

B9474 is a release version of llama.cpp, a tool for LLM inference in C/C++ . The release likely contains bug fixes, feature updates, and performance improvements to the llama.cpp project, which enable

b9478

b9478 is a release of llama.cpp, a project for LLM inference in C/C++ . This release represents one of the project's frequent build updates, as the llama.cpp project releases new versions regularly wi

b9480

B9480 is a release of llama.cpp, an LLM inference project in C/C++ . The release likely contains updates, bug fixes, or feature improvements to the llama.cpp codebase for running large language models

b9483

Build b9483 is an intermediate release of llama.cpp, the C/C++ inference engine for running large language models locally. This build represents a specific commit snapshot from the ggml-org llama.cpp

Cost-Aware Diffusion Draft Trees for Speculative Decoding

arXiv:2606.01813v1 Announce Type: new Abstract: Speculative decoding accelerates inference by having a lightweight drafter propose tokens verified in parallel by the target language model. Block diffu

researcharxiv-cs-cl

Cross-Environment Neural Reranking for Sample-Efficient Action Selection in Text-Based Agents

model-releasesarxiv-cs-cl

arXiv:2606.02204v1 Announce Type: new Abstract: Large language model agents achieve strong performance on text-based benchmarks but incur prohibitive inference costs, motivating the use of compact neu

Hardware

Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap

arXiv:2512.10236v2 Announce Type: replace-cross Abstract: Modern ML workloads demand distributing training and inference across multiple GPUs. However, these parallelization techniques often suffer fr

hardwarearxiv-cs-lg

DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding

arXiv:2606.02091v1 Announce Type: new Abstract: Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in

researcharxiv-cs-cl

Hardware

DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing

arXiv:2511.04791v2 Announce Type: replace Abstract: Modern LLM serving systems must sustain high throughput while meeting strict latency SLOs across two distinct inference phases: compute-intensive pr

hardwarearxiv-cs-lg

EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models

arXiv:2606.01756v1 Announce Type: new Abstract: Large vision-language models (LVLMs) achieve strong performance on image and video understanding tasks, but their inference efficiency is constrained by

researcharxiv-cs-cv

Hardware

FLARE: Diffusion for Hybrid Language Model

arXiv:2606.01774v1 Announce Type: cross Abstract: Autoregressive (AR) large language models (LLMs) have achieved broad practical success, but sequential decoding remains a key bottleneck for low-laten

hardwarearxiv-cs-ai

From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection

arXiv:2602.03018v2 Announce Type: replace Abstract: Outlier detection (OD) is widely used in practice; but its effective deployment on new tasks is hindered by lack of labeled outliers, which makes al

researcharxiv-cs-lg

FROST-STA: Frozen Dense Features for the Ego4D Short-Term Object Interaction Anticipation

arXiv:2606.00694v1 Announce Type: new Abstract: Short-term anticipation in egocentric video requires more than recognizing the current scene: a system must infer which object the camera wearer will co

safetyarxiv-cs-cv

HERO'S JOURNEY: Testing Complex Rule Induction with Text Games

model-releasesarxiv-cs-cl

arXiv:2606.02556v1 Announce Type: new Abstract: We introduce HERO'S JOURNEY, a benchmark for rule induction in goal-directed episodic tasks, where agents must infer hidden rules from demonstrations an

HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression

arXiv:2606.01934v1 Announce Type: cross Abstract: Large language models achieve remarkable performance via extended chain-of-thought (CoT) reasoning, yet this lengthy process incurs substantial infere

safetyarxiv-cs-cl

Tutorials

LERD: Latent Event-Relational Dynamics for Neurodegenerative Classification

arXiv:2602.18195v2 Announce Type: replace-cross Abstract: Alzheimer's disease (AD) alters brain electrophysiology and disrupts multichannel EEG dynamics, making accurate and clinically useful EEG-base

tutorialsarxiv-cs-ai