Search: “inference”

ExpBoN: Exponential-Noise Best-of-n for Efficient Test-Time LLM Alignment

arXiv:2609.21899v1 Announce Type: new Abstract: Best-of-n (BoN) sampling is a simple yet effective inference-time alignment method, but hard maximization provides only coarse control over the trade-of

safetyarxiv-cs-lg

21 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generation

DGX agent

arXiv:2609.21474v1 Announce Type: new Abstract: Fast-WAM shows that video-action co-training improves control without generating future video at inference, making the representation from a single vide

safetyarxiv-cs-cv

21 Sept 2026

Tutorials

Shake to Learn: Dynamic Interrogation of Hidden Object Physics for Robotic Manipulation with Physical Reservoir Computing

arXiv:2609.20970v1 Announce Type: new Abstract: Many physical properties relevant to robotic manipulation are hidden from vision. A sealed object, for example, may reveal little about its center of ma

tutorialsarxiv-cs-ro

21 Sept 2026

Local Ai

The Weight Is Over - Interactive Diffusion on Consumer GPUs

arXiv:2609.21849v1 Announce Type: cross Abstract: On-device inference is booming, but the momentum is almost all in language models. Diffusion pipelines are memory hungry, latency-sensitive, and requi

local-aiarxiv-cs-cv

21 Sept 2026

Syntheses

Wiki Lint Report — 2026-09-20

linthealth-checkautomated

Automated lint: 14 errors, 54840 warnings, 3 info

20 Sept 2026

I turned an asymetric pair of Tesla V100s PCIe both (16 GB + 32 GB) into a surprisingly capable local LLM lab — 1.38k prompt tok/s, 40 decode tok/s with qwen3.8 27B Q6 and Q8...

model-releasesr-localllama

TL;DR: I run a mismatched Tesla V100-PCIE pair—one 16 GB card and one 32 GB card, 48 GB total—in a Proxmox/LXC-based local-inference lab. The practical winner so far is a recent CUDA build of llama.cp

19 Sept 2026

Training a Neural Network on AMD MI50s Using Vulkan: Proof of ConceptOr: Why I Stopped Listening and Just Did It

model-releasesr-localllama

Note: This writeup was put together with the help of AI. So I've got dual AMD MI50 32GB cards. If you know these cards, you know the story - AMD dropped official ROCm support for gfx906 after ROCm 5.7

19 Sept 2026

A Learning Algorithm for Threshold Boolean Networks with Prescribed Fixed Points

arXiv:2609.20298v1 Announce Type: new Abstract: We present a learning algorithm for inferring threshold Boolean networks (TBNs) with a prescribed set of fixed points. The proposed method employs a cus

Design of the IBM Granite 5.0 TurboCTC ASR Model

model-releasesarxiv-cs-cl

arXiv:2609.20104v1 Announce Type: new Abstract: We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an e

On-Demand Attention: Language Models Know When to Recall

model-releasesarxiv-cs-cl

arXiv:2609.20734v1 Announce Type: new Abstract: Reasoning and agentic workloads increasingly demand efficient long-context inference. Yet full-attention decoding reads the growing history at every ste

Reproducibility is not construct validity: LLM measurement of institutionally situated communication

arXiv:2609.19866v1 Announce Type: new Abstract: High annotation reproducibility does not necessarily imply that an LLM-inferred measure captures the construct it is intended to measure. We test this d

safetyarxiv-cs-ai

SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership

arXiv:2609.19610v1 Announce Type: new Abstract: Understanding humans over long horizons requires agents to infer not only what people need in the moment, but also how routines form, why they repeat, a

super pumped for this convo w/ @huntlovell and @allietheicon on all things langchain 🤝 jev

model-releasesharrison-chase--x

super pumped for this convo w/ @huntlovell and @allietheicon on all things langchain 🤝 jev Hot topic livestream: Learn about Jev A buzzy new model Jev, by @typesafeai, reports up to 200x faster infere

Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It

arXiv:2609.18849v1 Announce Type: cross Abstract: An agentic request spends substantial wall-clock time waiting for tools, and its KV cache holds GPU memory the whole time. Serving systems decide whet

Agents

Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning

arXiv:2609.18417v1 Announce Type: new Abstract: Multi-turn agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both traini

agentsarxiv-cs-cl

Local Ai

HyQuant: Hybrid-Precision Quantization for LLM Attention

arXiv:2608.27875v3 Announce Type: replace Abstract: Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the atten

local-aiarxiv-cs-ai

Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

arXiv:2609.18842v1 Announce Type: new Abstract: The scaling laws hold that a language model grows more capable with more parameters and more training data, and Mixture-of-Experts (MoE) architectures h

OBC-Prune: Outcome-Based Calibration for Large Reasoning Model Pruning

arXiv:2609.17890v1 Announce Type: new Abstract: Large reasoning models (LRMs) generate long chain-of-thought traces before answering, creating significant inference overhead. Pruning can reduce this c

Applications

Pose2Muscle: Structured Spatio-Temporal Decoding for Discrete Muscle Activity Estimation from Human Pose

arXiv:2609.18336v1 Announce Type: new Abstract: Muscle activity is fundamental to human movement, and understanding its patterns is critical for injury prevention and rehabilitation. Conventional musc

applicationsarxiv-cs-cv

TabPFN-3.5: Technical Report

arXiv:2609.17895v1 Announce Type: new Abstract: We introduce TabPFN-3.5, our new flagship Tabular Foundation Model. It significantly outperforms its predecessor, TabPFN-3, and all existing baselines a

Local Ai

WISE: A Lightweight, Weakly-Supervised Model for Onboard Fire Smoke Detection and Localization

arXiv:2609.18227v1 Announce Type: new Abstract: Wildfire smoke detection from satellite imagery is critical for early warning and rapid response. For onboard satellite deployment, detection systems mu

local-aiarxiv-cs-cv

Hardware

Accelerated Decoding of Centroid Positional Encoding for Instance Segmentation

arXiv:2609.16874v1 Announce Type: new Abstract: Beyond model inference, the decoding stage, which converts raw network outputs into task-level representations, constitutes a significant portion of the

hardwarearxiv-cs-cv

ANIMASK: What the Model Contributes to Role Play in Simulated Story Worlds

arXiv:2609.16667v1 Announce Type: new Abstract: When a language model plays a character, the observed behavior reflects both the assigned persona and the default dispositions of the actor model itself

Beyond the Name: Demographic Leakage in De-Identified Resumes and Evaluation Artifacts in LLM Bias Audits

arXiv:2609.16501v1 Announce Type: cross Abstract: De-identified resume screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leak

safetyarxiv-cs-ai

Causal Discovery via Transformed Low-Rank Quantile Surfaces

arXiv:2609.16931v1 Announce Type: cross Abstract: We propose Low-Rank Quantile Surfaces (LRQS), a bivariate causal model in which, in the causal direction, an unknown monotone transformation of the co

Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling

arXiv:2609.16450v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative

ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding

arXiv:2609.17241v1 Announce Type: new Abstract: While draft-model-free speculative decoding offers a promising path to efficient LLM inference, it is frequently constrained by stale draft candidates a

researcharxiv-cs-cl

I built a native Vulkan training backend for 143 modern Transformer architectures — no CUDA or PyTorch required

model-releasesr-localllama

I've been working on something that started as the training backend for my Hierarchos architecture, but it has grown into a much broader project: https://github.com/necat101/Hierarchos-Native Hierarch

b10981

model-releasesllama-cpp-releases

OpenVINO: optimize stateful decode and GPU MoE inference (#28638) exclude GPU/NPU failing POOL_2D case Fix pool case ggml-openvino: fix stateful decode for Gemma-4 per-layer-type head sizes ggml-openv

Disentangling Topology and Diversity in Multi-Agent LLMs for Multilingual Low-Resource Emotion Detection

arXiv:2609.14570v1 Announce Type: cross Abstract: Multi-agent LLM systems combine multiple inference calls, but prior work often confounds how calls are connected with how they are diversified. We stu

Applications

Noise-Adaptive Conformal Classification with Marginal Coverage

arXiv:2501.18060v2 Announce Type: replace-cross Abstract: Conformal inference provides a rigorous statistical framework for uncertainty quantification in machine learning, enabling well-calibrated pre

applicationsarxiv-cs-lg

Hardware

OpWeave: Flexible Operator Disaggregation for Heterogeneous LLM Serving

arXiv:2609.14237v1 Announce Type: cross Abstract: LLM serving systems increasingly disaggregate inference into finer-grained stages, with recent approaches separating attention from FFN or MoE executi

hardwarearxiv-cs-ai

Reconstructing Is Not Acting: Action-Centric Latent Dynamics Modeling

arXiv:2609.15189v1 Announce Type: new Abstract: Latent action models (LAMs) learn action representations from unlabeled videos by inferring latent actions from visual transitions and reconstructing fu

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

arXiv:2609.15028v1 Announce Type: new Abstract: In-context learning (ICL) enables models to infer tasks from demonstrations, but existing benchmarks generally lack matched text and image versions need

When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

arXiv:2609.15106v1 Announce Type: new Abstract: Large language models can hallucinate even when the knowledge required for a correct answer is already available. We study this failure through a latent

researcharxiv-cs-cl

Adaptive AI: Energy Efficient Multi-exit TinyML on Intelligent Vision Systems at the Edge

arXiv:2609.11939v1 Announce Type: cross Abstract: Traditional TinyML systems for edge devices achieve high accuracy by relying on fixed-depth models that require a constant number of multiply-accumula

researcharxiv-cs-cv

Attention Quantization for Tabular Foundation Models

arXiv:2609.13031v1 Announce Type: new Abstract: With the recent rise and adoption of tabular foundation models, optimizing their inference performance becomes an emerging field for efficiency research

GLaMoR: Consistency Checking of OWL Ontologies using Graph Language Models

arXiv:2504.19023v2 Announce Type: replace-cross Abstract: Semantic reasoning aims to infer new knowledge from existing knowledge, with OWL ontologies serving as a standardized framework for organizing

Hardware

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

arXiv:2609.12412v1 Announce Type: cross Abstract: Foundation models, including large language models, vision-language models, and time-series foundation models, are increasingly deployed on embedded a

hardwarearxiv-cs-lg

RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

arXiv:2609.12552v1 Announce Type: new Abstract: Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left th

Semantically Aligned Gradient-Driven Context-Preserving Image Editing

arXiv:2609.12691v1 Announce Type: new Abstract: Instruction-guided image editing has a training-time blind spot. Generative editors are never required to semantically verify whether their outputs actu

Syntheses

Wiki Lint Report — 2026-09-13

linthealth-checkautomated

Automated lint: 12 errors, 54839 warnings, 3 info

13 Sept 2026

Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration

arXiv:2609.11446v1 Announce Type: new Abstract: Heterogeneous model collaboration seeks to exploit the complementary strengths of different models to balance predictive performance and inference cost.

12 Sept 2026

Routing by Reasoning Need: Trajectory-Aware Decoding Control for Diffusion Vision-Language Models

arXiv:2609.11315v1 Announce Type: new Abstract: Diffusion vision-language models generate answers through iterative refinement, exposing intermediate answer trajectories that can be inspected and cont

12 Sept 2026

X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation

arXiv:2609.11412v1 Announce Type: cross Abstract: Reducing audio-encoder depth lowers the inference cost of speech large language models, but removing complete blocks perturbs the embeddings consumed

safetyarxiv-cs-ai

12 Sept 2026

A Multi-Stage Rule-Chaining Framework for Compositional and Interpretable Cognitive Reasoning

arXiv:2609.10654v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC) benchmarks cognitive generalization, the ability to infer and apply abstract rules from limited examples. T

11 Sept 2026

IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies

arXiv:2609.10915v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies leverage pretrained vision-language backbones to achieve strong cross-task generalization. A leading design coup

11 Sept 2026

Longitudinal Risk Prediction in Mammography with Privileged History Distillation

arXiv:2603.15814v2 Announce Type: replace Abstract: Longitudinal mammography screening has become an important source of information for improving future breast cancer risk prediction. However, the pe