Search: “inference”

Speculative Probing: LLM Monitoring at Speculative-Decoding Cost

model-releasesarxiv-cs-cl

arXiv:2608.28099v1 Announce Type: cross Abstract: Real-time classification during language model inference is valuable for safety filtering, behavioral analysis, and model monitoring, but current appr

31 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

TokenPilot: Cache-Efficient Context Management for LLM Agents

DGX agent

arXiv:2606.17016v2 Announce Type: replace Abstract: As LLM agents are deployed in long-horizon sessions, context accumulation drives up inference costs. Existing approaches utilize text pruning or dyn

researcharxiv-cs-cl

31 Aug 2026

Affix Cache for Diffusion Large Language Models

arXiv:2608.26140v1 Announce Type: new Abstract: Diffusion Large Language Models (DLLMs) enable non-autoregressive decoding and bidirectional context modeling, but efficient inference remains challengi

researcharxiv-cs-cl

BREAKING: SpaceXAI has released a major new update for Grok Build (v1.0.13). The update improves reliability with automatic retries and reco…

model-releaseselon-musk--x

BREAKING: SpaceXAI has released a major new update for Grok Build (v1.0.13). The update improves reliability with automatic retries and recovery from truncated responses, inference failures, oversized

Tutorials

Cross-lingual Representation Learning via Centroid Intervention Fusion

arXiv:2608.26357v1 Announce Type: new Abstract: Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages. Inference-time intervention o

tutorialsarxiv-cs-cl

Dynamical phase selection controls compute scaling in looped transformers

model-releasesarxiv-cs-lg

arXiv:2608.26556v1 Announce Type: cross Abstract: A looped transformer performs inference by iterating a weight-tied map, making its computation a dynamical process whose cost is set by the resulting

TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding

arXiv:2608.26112v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve

researcharxiv-cs-cl

TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

model-releasesarxiv-cs-cl

arXiv:2608.27128v1 Announce Type: new Abstract: Long-context inference is bottlenecked by the memory footprint of the key-value (KV) cache, especially for small models under tight resource budgets. Ex

Ultra Low-Power, Lightweight, Probabilistic RSS-Based Path Reconstruction: A System for Landscape-Scale Bee Tracking

arXiv:2608.27152v1 Announce Type: new Abstract: Applications in fields such as movement ecology, Internet of Things or robotics share the need for systems that localize devices that are too small and

local-aiarxiv-cs-lg

A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards

arXiv:2608.24935v1 Announce Type: new Abstract: Accurate identification of early-stage apple fruitlet anatomical structures, including the calyx, fruitlet body, and peduncle, is essential for robotic

local-aiarxiv-cs-cv

Controllable Affective Generation via Latent Vector Steering

arXiv:2608.25569v1 Announce Type: new Abstract: Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive application

safetyarxiv-cs-cl

GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Prediction

arXiv:2601.13207v2 Announce Type: replace Abstract: Geo-localization aims to infer the geographic location where an image was captured using observable visual evidence. Traditional methods achieve imp

Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More

arXiv:2608.25332v1 Announce Type: new Abstract: Vision-Language Models (VLMs) have exhibited impressive performance across diverse visual scenarios. However, this success comes at the cost of explosiv

researcharxiv-cs-cv

SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs

arXiv:2608.25068v1 Announce Type: new Abstract: Depth pruning removes entire Transformer blocks to reduce the inference cost of large language models, but disrupts the hidden-state distributions expec

StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models

arXiv:2608.26067v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a sin

[Megathread] GLM-5.3-Flash - former ox-alpha

model-releasesr-localllama

Megathread for discussing the release of GLM-5.3-Flash. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuration Experiences, Benchmarks & Model Comparisons We'll try

MoRF-AST: Calibrated Probabilistic Virtual Sensing for Structural Monitoring under Changing Operating Conditions

model-releasesarxiv-cs-lg

arXiv:2608.24531v1 Announce Type: cross Abstract: Probabilistic full-field reconstruction provides uncertainty-aware response evidence for structural reliability assessment, yet inference from sparse

PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control

arXiv:2608.24115v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) can integrate long visual histories, reason under partial observability, and infer behavior from a few exampl

TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal

Tutorials

ZODIAC: Zero-shot Octree-based Diffusion for Anatomical Completion

arXiv:2608.24422v1 Announce Type: new Abstract: Recovering the full 3D spine anatomy from intraoperative ultrasound is an ill-posed inverse problem, as the complete structure must be inferred from inc

tutorialsarxiv-cs-cv

A Latency-Aware Framework for Visuomotor Policy Learning on Industrial Robots

arXiv:2602.14255v2 Announce Type: replace Abstract: Industrial robots are increasingly deployed in construction and manufacturing tasks, where the deployment of end-to-end visuomotor policies is chall

safetyarxiv-cs-ro

DynaContext: Self-Improving Dynamic Contextualization of Optimized Prompts for Heterogeneous Parameter Extraction

arXiv:2608.22014v1 Announce Type: new Abstract: Automated prompt and skill optimization typically produces a single static instruction that is reused across inference instances until the next optimiza

LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization

arXiv:2608.21836v1 Announce Type: new Abstract: Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a

Power-Performance Characterization of TinyML Systems

arXiv:2608.21646v1 Announce Type: cross Abstract: TinyML systems are enabling machine learning (ML) inference at the edge. However, there is little quantitative analysis of such systems. This paper pr

Proxy reliance in large language model decisions is uncalibrated to predictive evidence

arXiv:2608.22887v1 Announce Type: new Abstract: Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy

The Imitator Game: Benchmarking Robot Imitative Ability Beyond Action Prediction

arXiv:2608.22301v1 Announce Type: cross Abstract: Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at hand

VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting

arXiv:2603.14659v2 Announce Type: replace-cross Abstract: Video reasoning requires models to locate and track question-relevant evidence across frames. While reinforcement learning (RL) with verifiabl

DiffVC-ONE: Diffusion-based Generative Video Compression with One-Step Video Diffusion Transformer

arXiv:2608.20515v1 Announce Type: new Abstract: Generative video compression can recover rich visual details at low bitrates, but simultaneously achieving high temporal consistency and low inference c

researcharxiv-cs-cv

Hardware

Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores

arXiv:2608.20725v1 Announce Type: cross Abstract: Convolution is a principal computational bottleneck in deep neural networks, and its efficiency depends on tight integration between algorithms and GP

hardwarearxiv-cs-cv

GeoExplain: Multimodal Reasoning based on Hierarchy of Visual Information in Street View

arXiv:2506.16633v3 Announce Type: replace-cross Abstract: Multimodal reasoning is a process of understanding, integrating and inferring information across different data modalities. It has recently at

Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers

model-releasesarxiv-cs-lg

arXiv:2608.20758v1 Announce Type: new Abstract: Bayesian Neural Networks (BNNs) with Bayesian output layers provide a principled and tractable framework for quantifying predictive uncertainty, yet the

28 TPS on Qwen2.5-7B across two separate cloud regions over public WAN using speculative decoding + CUDA Graphs [P]

model-releasesr-machinelearning

been building ShardFlow for the past few months, a distributed LLM inference framework that splits any HuggingFace transformer across N GPU machines and uses neural speculative decoding to deal with W

23 Aug 2026

“The All Spark” Cluster: Upgrading from 16 - 36 DGX Sparks

Earlier this year I posted about building what at the time I believe was the first 16x DGX Spark Cluster. I’m now adding 20 more Sparks to the cluster in my homelab server rack, giving me 4.6TB of uni

local-air-localllama

23 Aug 2026

Agents

FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills

arXiv:2607.21596v2 Announce Type: replace Abstract: Large language model agents can adapt to complex tasks by constructing workflows at inference time, but procedures discovered in one episode are usu

agentsarxiv-cs-ai

Learning how to Forget: Fine-tuning for Long-Context Sparse Attention

arXiv:2608.19920v1 Announce Type: new Abstract: A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer langua

safetyarxiv-cs-cl

Hardware

Starcloud raises $250M to build AI data centers in orbit

Artificial intelligence hardware startup Starcloud Inc. today announced that it has raised 250 million in funding at a 2.3 billion valuation. Investment firm Manhattan West led the deal. It was joined

hardwaresiliconangle

Write Once, Run Everywhere: The Axon DSL for Shape-Safe and Framework-Agnostic LLM Architectures

arXiv:2608.19889v1 Announce Type: new Abstract: The entire ecosystem of open-source language models effectively relies on a single platform. What if this platform was forced to shut down tomorrow? Imp

local-aiarxiv-cs-ai

GEAR: Generative Expansion and Real Anchoring for Two-Stage Distillation of Tabular Foundation Models

arXiv:2608.18849v1 Announce Type: new Abstract: Tabular foundation models (TFMs) achieve strong performance through in-context learning, but context-dependent inference imposes substantial latency and

researcharxiv-cs-lg

20 Aug 2026

Hardware

LLM-Powered Predictive Decision-Making for Sustainable Data Center Operations

arXiv:2608.18503v1 Announce Type: new Abstract: The growing demand for AI-driven workloads, particularly from Large Language Models (LLMs), has raised concerns about the significant energy and resourc

hardwarearxiv-cs-lg

20 Aug 2026

Low-Power, Neuromorphic, Acoustic Anomaly Detection for Persistent Machine Monitoring

arXiv:2608.18341v1 Announce Type: cross Abstract: Persistent acoustic monitoring can detect machine faults without physical contact, but always-on inference is constrained by power, latency, and deplo

20 Aug 2026

G-ReAct: Graph-Guided Deep Search via Structure-State Co-Evolution

arXiv:2608.01324v2 Announce Type: replace Abstract: Deep search has become a fundamental capability of large language models (LLMs) for solving open-domain complex tasks. However, existing approaches

19 Aug 2026

When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models

arXiv:2606.21802v2 Announce Type: replace Abstract: Standard tokenwise diffusion LMs keep training corruption and inference commitment at token granularity throughout denoising. At high noise, this le

local-aiarxiv-cs-cl

19 Aug 2026

Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL

arXiv:2608.15389v1 Announce Type: new Abstract: LLM-based Text-to-SQL progress is reported across heterogeneous benchmarks, backbones, and inference protocols, making cross-system comparison fragile.

CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs

arXiv:2608.14791v1 Announce Type: new Abstract: Abductive reasoning, often characterized as inference to the best explanation, is central to explanation under uncertainty, from everyday sense-making a

Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention

arXiv:2608.15522v1 Announce Type: new Abstract: Recent audio-visual generation models can synthesize synchronized video and sound in a unified diffusion process, but their inference cost remains high

researcharxiv-cs-cv

Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning

arXiv:2608.14706v1 Announce Type: cross Abstract: Standard autoregressive video generation algorithms based on Diffusion and Flow Matching rely on rigid training objectives and static sampling schedul

Probabilistic Circuits as Reasoning Machines in Artificial Intelligence (Part I)

arXiv:2608.16565v1 Announce Type: new Abstract: This cumulative habilitation thesis studies probabilistic circuits (PCs) as a powerful and tractable framework for reasoning and learning under uncertai

StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding

arXiv:2608.16320v1 Announce Type: new Abstract: Streaming video understanding demands direct responses from the causally observed prefix of an unfolding video. Existing systems add inference-time memo

safetyarxiv-cs-cv