Search: “inference”

Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

arXiv:2608.08829v1 Announce Type: cross Abstract: Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the i

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Describe-to-Score: A text-guided framework for image complexity assessment

DGX agent

arXiv:2509.16609v2 Announce Type: replace Abstract: Accurately assessing image complexity (IC) is essential for many vision tasks, yet existing approaches rely almost exclusively on visual features an

model-releasesarxiv-cs-cv

11 Aug 2026

Goedel-Code-Prover: Hierarchical Proof Search for Open State-of-the-Art Code Verification

model-releasesarxiv-cs-ai

arXiv:2603.19329v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can generate plausible code but offer limited guarantees of correctness. Formally verifying that implementations

Goose: Anisotropic Speculation Trees for Training-Free Speculative Decoding

arXiv:2604.02047v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple candidate tokens and verifying them in a single forward p

Graph-Guided Safe Diffuser: Topological Graph Guidance for Safe Diffusion Planning

arXiv:2608.09484v1 Announce Type: new Abstract: Many diffusion-based planners enforce safety through inference-time guidance, but such interleaved trajectory deformations often degrade kinematic feasi

safetyarxiv-cs-ro

High-Layer Attention Pruning with Rescaling

model-releasesarxiv-cs-cl

arXiv:2507.01900v3 Announce Type: replace Abstract: Pruning is a highly effective approach for compressing large language models (LLMs), significantly reducing inference latency. However, conventional

I gave DeepSeek V4 Flash basic vision by training a 40M connector on 100K examples

model-releasesr-localllama

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs

arXiv:2608.08588v1 Announce Type: cross Abstract: Hamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradient

Applications

Length-MAX Tokenizer for Language Models

arXiv:2511.20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed

applicationsarxiv-cs-ai

Tutorials

Leveraging generative models to assist Monte Carlo sampling

arXiv:2608.07648v1 Announce Type: cross Abstract: Sampling high-dimensional probability distributions is a central task in scientific computing, with applications ranging from Bayesian inference to st

tutorialsarxiv-cs-lg

LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization

arXiv:2608.08721v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically dete

Applications

LineGraph2Road: Structural Graph Reasoning on Line Graphs for Road Network Extraction

arXiv:2602.23290v2 Announce Type: replace Abstract: Extracting routable road networks from satellite imagery requires accurate topology recovery beyond pixel-level segmentation. Recent methods decompo

applicationsarxiv-cs-cv

NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents

model-releasesnvidia-developer

NVIDIA Nemotron 3.5 Lightning is an open‑source mixture‑of‑experts language model totaling 30 B parameters with only 3 B active during inference, designed to serve high‑volume, low‑latency execution f

Agents

Population-Scalable Multi-Agent World Modeling

arXiv:2608.08600v1 Announce Type: cross Abstract: World models have recently achieved impressive progress in visual prediction and interactive generation, but extending them to multi-agent environment

agentsarxiv-cs-ai

RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

model-releasesarxiv-cs-ai

arXiv:2608.08684v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by KV cache memory, yet distributing a limited cache budget across layers remains challenging. Existing met

Agents

SAKE: Structured Agentic Knowledge Extrapolation for Complex LLM Reasoning via Reinforcement Learning

arXiv:2505.15062v5 Announce Type: replace-cross Abstract: Knowledge extrapolation is the process of inferring novel information by combining and extending existing knowledge that is explicitly availab

agentsarxiv-cs-ai

SplitGaussian: Reconstructing Dynamic Scenes via Visual Geometry Decomposition

arXiv:2508.04224v2 Announce Type: replace Abstract: Reconstructing dynamic 3D scenes from monocular video remains fundamentally challenging due to the need to jointly infer motion, structure, and appe

researcharxiv-cs-cv

Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing

arXiv:2608.08528v1 Announce Type: new Abstract: Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries,

💡The world needs an open-source platform, and that’s exactly what we’re building to give our customers more choice and the flexibility to c…

model-releasesmistral-ai--x

💡The world needs an open-source platform, and that’s exactly what we’re building to give our customers more choice and the flexibility to choose the right model for the right task. As part of this, we

An AI4AI Framework for Visual Token Pruning

arXiv:2608.07193v1 Announce Type: cross Abstract: Visual-token pruning can substantially reduce the inference cost of multimodal large language models (MLLMs), yet existing methods largely rely on fix

safetyarxiv-cs-cv

Applications

Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry

arXiv:2608.06849v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by quadratic attention computation and growing KV-cache costs. Existing sparse attention and KV-compression

applicationsarxiv-cs-ai

Conformal Fusion Under Missing Modalities

arXiv:2608.07183v1 Announce Type: new Abstract: Multimodal fusion architectures typically assume all modalities are available at inference, yet sensor failures, acquisition variability, and cost const

researcharxiv-cs-lg

ControlRef: Efficient Layout-Guided Multi-Instance Generation via Anchored 4D-RoPE

arXiv:2608.06878v1 Announce Type: new Abstract: Layout-guided multi-instance generation is essential for controllable image synthesis in Multi-Modal Diffusion Transformers (MM-DiTs). However, integrat

local-aiarxiv-cs-cv

Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks

arXiv:2608.06839v1 Announce Type: new Abstract: Artificial Neural networks (ANNs) are often treated as black-box models, making explainability a central challenge in deep learning. Many engineering me

researcharxiv-cs-lg

omlab/VLX-Seek-1.5-10B · Hugging Face

VLX-Seek-1.5-10B VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical setting

local-air-localllama

Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation

arXiv:2608.06632v1 Announce Type: new Abstract: Industrial recommendation systems predominantly adopt a passive ranking paradigm that infers user preferences from implicit behavioral signals (e.g., cl

Hardware

Summarize First, Download Later: Onboard VLMs for Bandwidth-Efficient Earth Observation

arXiv:2608.06959v1 Announce Type: new Abstract: Modern Earth observation (EO) satellites carry increasingly advanced sensors that produce vast volumes of high-resolution, multispectral data, yet downl

hardwarearxiv-cs-cv

The Sparsity Whisperer

arXiv:2608.06630v1 Announce Type: new Abstract: Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We

[2606.05682] Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation

model-releasesr-localllama

Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization

9 Aug 2026

M3 16GB running Ollama (Qwen 9B) is extremely slow (10-12 mins per task). Am I doing something wrong?

Hey everyone, I constantly see high praise for M3 and M4 Macs for local LLM inference, even the base/16GB models. However, my experience has been quite different, and I'm trying to figure out if I hav

model-releasesr-ollama

9 Aug 2026

MI25 for 80-100€ worth it?

seems to be about as good as a vega 56 with 16Gb of VRAM, is it worth it? (don’t want to deal with NVIDIA drivers on Linux, already have an rx6650xt and might simply use vulkan for llamacpp inference)

local-air-localllama

8 Aug 2026

A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization

arXiv:2608.05217v1 Announce Type: cross Abstract: Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets.

researcharxiv-cs-cv

Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay

arXiv:2608.05784v1 Announce Type: new Abstract: Computer-use agents pay full frontier inference to re-derive routines their user has already performed, because an agent's memory today records what the

local-aiarxiv-cs-ai

Agents

AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games

arXiv:2608.06362v1 Announce Type: cross Abstract: Deciding which of two agents is stronger means playing games until skill outweighs luck, and every game costs money, model inference, or expert time.

agentsarxiv-cs-ai

DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding

arXiv:2608.05448v1 Announce Type: new Abstract: Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to

researcharxiv-cs-cl

Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding

arXiv:2608.05780v1 Announce Type: new Abstract: Recent advancements in MLLM-based long-form video understanding have mitigated inference-time computational cost and limited context lengths by selectin

safetyarxiv-cs-cv

parakeet.wgsl – Fast, accurate ASR in the browser, via raw WebGPU & SIMD WASM

High-performance inference of NVIDIA's Parakeet TDT 0.6B V2 English transcription model, in the browser. Check out the live demo: https://parakeet.narcotic.sh/ A fully custom, dependancy-free implemen

local-air-localllama

QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

arXiv:2608.05326v1 Announce Type: cross Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache. A dominant line of work

researcharxiv-cs-cl

Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning

arXiv:2608.05643v1 Announce Type: new Abstract: Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollou

Scaffold-Mediated Post-Training: Co-Evolving Model Parameters and Procedural Scaffold Graphs

model-releasesarxiv-cs-cl

arXiv:2608.05156v1 Announce Type: new Abstract: Post-training of large language models optimizes only parameters, while inference-time procedural scaffolds are typically designed independently of para

Tutorials

STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

arXiv:2602.14265v3 Announce Type: replace Abstract: Inference-Time-Compute (ITC) methods like Best-of-n and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diver

tutorialsarxiv-cs-cl

Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet

arXiv:2509.06861v3 Announce Type: replace Abstract: Test-time scaling increases inference-time computation through longer reasoning chains and has shown strong performance gains across many domains. H

The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images

arXiv:2608.06270v1 Announce Type: new Abstract: The 'thinking-with-images' paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations o

Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation

arXiv:2608.06177v1 Announce Type: new Abstract: Binary neural networks are very attractive for constrained deployment, enabling small footprint and low-power inference. For binary activations, the dot

Timestep-Conditioned Transformers for Global Weather Forecasting

arXiv:2608.06241v1 Announce Type: new Abstract: Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a f

researcharxiv-cs-lg

When Drafts Evolve: Speculative Decoding Meets Online Learning

arXiv:2603.12617v2 Announce Type: replace-cross Abstract: Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model

When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories

model-releasesarxiv-cs-ai

arXiv:2608.06057v1 Announce Type: new Abstract: Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain struct

b10293