Search: “inference”

A Generative Model of Complex Networks Using Graphons and Neural Inverse Operators

arXiv:2610.02439v1 Announce Type: new Abstract: Generative graph models are central to understanding and simulating complex networks. However, existing approaches have complementary strengths and limi

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

Anthropic Subscriptions Offer 5x+ More Value Than OpenAI

DGX agent

Anthropic’s paid subscription plans deliver more than five‑times the perceived value of comparable OpenAI offerings when evaluated by cost per inference unit and overall revenue contribution. The heav

hardwaresemianalysis

5 Oct 2026

Dynamic LLM Routers are Often Misguided

arXiv:2610.02762v1 Announce Type: new Abstract: Dynamic LLM routers promise to cut inference costs by sending each query to the cheapest model that can answer it correctly. We analyze six commercial r

Exact Memory-Time Optimization for Prefix-Cached Language Model Serving

arXiv:2610.02766v1 Announce Type: new Abstract: Retaining language-model prefix states trades recomputation against storage time. Optimizing each cached block independently can overcount savings: a re

One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras

arXiv:2610.02675v1 Announce Type: cross Abstract: Single-photon avalanche diode (SPAD) cameras operate fundamentally differently from conventional cameras due to their photon-counting nature. Each fra

Hardware

Practical limit hit. Decoding so fast that tool calls (cpu) starting to become real limit not decode or prefill. Single RTX5090. Porting Kenshi to Godot project.

Hi folks, LIVE PROJECT PAGE TLDR: Moral of the story. You need better CPU to do actual agentic coding doing real work... I've been on a mission to make my RTX5090 go brrr for past 2 months so much so

hardwarer-localllama

Quantum parameter estimation with uncertainty quantification from continuous measurement data using neural network ensembles

arXiv:2509.10756v4 Announce Type: replace-cross Abstract: We show that ensembles of deep neural networks, called deep ensembles, can be used to perform quantum parameter estimation while also providin

Quoting Felix Rieseberg

model-releasessimon-willison

The 'old' version of Cowork runs model inference in the cloud, executing tool calls in an Anthropic-provided VM we shipped to your computer. We added the VM for capability, safety, and security reason

What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute

arXiv:2610.02491v1 Announce Type: new Abstract: Large language models spend the same amount of computation on every token they generate, regardless of how difficult each token is to produce. Methods s

[Discussion] A 5KB pure x86-64 assembly engine for Gemma-2B (FP16, 4.6 tok/s on CPU)

model-releasesr-localllama

Hi everyone, Sharing a personal project exploring the minimal bare-metal footprint required to run an autoregressive LLM. Instead of relying on large runtimes or compiler abstractions, I wrote an infe

4 Oct 2026

Capturing In-Context Learning Dynamics with Task Operators

arXiv:2610.01054v1 Announce Type: cross Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requir

Does Scaling Reinforcement Learning Really Require More Training?

arXiv:2610.01133v1 Announce Type: cross Abstract: Scaling reasoning typically spends more compute on reinforcement learning (RL) or on inference. We show that a completed RL training history can yield

From Discovery to Decision: Finite-Budget Recoverability in LLM Voting

arXiv:2610.01014v1 Announce Type: new Abstract: Voting over multiple LLM responses is a common primitive in test-time scaling and ensemble inference. Collecting more responses can expand the candidate

In-context Learning of Single-index Targets: Comparing Kernel and Feature Learners

arXiv:2610.01712v1 Announce Type: new Abstract: In-context learning (ICL) enables a pretrained model to infer a task from demonstrations without updating its parameters. While much of the existing the

Kinematic MeanFlow: One-Step Action Generation Policy for Robotic Foundation Models

arXiv:2610.00864v1 Announce Type: cross Abstract: In this paper, we study how to achieve one-step action generation in Robotic Foundation Models (RFMs), aiming to overcome the high inference latency o

local-aiarxiv-cs-ai

Large Language Bayes Is Not Reparameterisation-Invariant

arXiv:2610.00265v1 Announce Type: new Abstract: Large Language Bayes (LLB) answers an informal modelling question by sampling candidate probabilistic programs from a language model, running approximat

MapLightning: Online Vectorized HD Map Construction with 1D Map Tokens

model-releasesarxiv-cs-cv

arXiv:2610.01905v1 Announce Type: new Abstract: Online vectorized HD map construction is essential for scaling safe autonomous driving and requires accurate, real-time inference. Prior methods typical

Applications

Multi-Task Anti-Causal Learning for Reconstructing Urban Events from Residents' Reports

arXiv:2603.11546v2 Announce Type: replace Abstract: Many real-world machine learning tasks are anti-causal: they require inferring latent causes from observed effects. In practice, we often face multi

applicationsarxiv-cs-lg

MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs

arXiv:2610.01434v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) incur substantial inference costs when processing long visual-textual sequences. While existing operation com

One Basis to Animate Them All: Gaussian Blendshape Distillation for Real-Time Avatars

arXiv:2610.02207v1 Announce Type: cross Abstract: 3D Gaussian avatars support fast rendering, however, their real-time animation is often challenged by the costly neural inference. We address this bot

local-aiarxiv-cs-ai

Redundancy Meets Synergy: Dependency-aware Expert Selection for MoE via Submodular Optimization

arXiv:2610.00558v1 Announce Type: new Abstract: While Mixture-of-Experts (MoE) models effectively scale model capacity through sparse activation, their deployment is often bottlenecked by prohibitive

Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces

arXiv:2610.00751v1 Announce Type: cross Abstract: Recent theoretical work identified fundamental properties of representation geometry that shape inference ability of deep neural networks. These inclu

Safety

TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety

arXiv:2610.01323v1 Announce Type: new Abstract: Safety-aligned large language models (LLMs) often refuse a harmful request but comply once the same goal is spread over several turns. Preference object

safetyarxiv-cs-ai

When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

arXiv:2610.00373v1 Announce Type: new Abstract: Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a l

Acceleration of Diffusion Language Model through Discrete Average Generator

arXiv:2609.38364v1 Announce Type: cross Abstract: Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few-

Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems

arXiv:2609.38806v1 Announce Type: cross Abstract: Next-token prediction has driven remarkable progress in large language models, yet a growing body of evidence suggests that they can struggle on probl

tutorialsarxiv-cs-cl

Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure

arXiv:2609.38697v1 Announce Type: cross Abstract: We present Cascadia, a system for serving large language models on fleets of commodity Intel AIPCs using their CPU, integrated-GPU, and NPU resources.

Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models

arXiv:2609.39134v1 Announce Type: new Abstract: Visual-token compression improves the efficiency of large vision-language models, but can expose failures that full-token evaluation misses. We study ad

Safety

Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models

arXiv:2609.39859v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) have shown strong potential for faster inference through parallel token generation when combined with confidenc

safetyarxiv-cs-lg

GraphMAS: A Systematic Benchmark of Multi-Agent Coordination for Graph Learning

arXiv:2609.39777v1 Announce Type: new Abstract: LLM-based multi-agent systems coordinate specialized reasoning through aggregation, interaction, and adaptive control, yet their potential for graph lea

Mitigating Memorization In Language Models

arXiv:2410.02159v3 Announce Type: replace-cross Abstract: Language models (LMs) can 'memorize' information, i.e., encode training data in their weights in such a way that inference-time queries can le

local-aiarxiv-cs-ai

Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions

arXiv:2609.38593v1 Announce Type: cross Abstract: Skills are external artifacts that Large Language Models (LLMs) consume at inference time to improve their performance on specialized domains by incor

Abductive World Modeling via Causal Representation Learning

arXiv:2609.36985v1 Announce Type: cross Abstract: The central challenge of world modeling is to learn representations that capture how the world evolves. However, existing world models predominantly r

tutorialsarxiv-cs-ai

Amazon Bedrock expands Claude model availability to in-country inferencing in India

model-releasesaws-ml-blog

Anthropic's Claude Opus 5, Claude Sonnet 5, and Claude Haiku 4.5 are now available in India through Amazon Bedrock geographic cross-Region inference. You can access these models while processing data

ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction

arXiv:2609.36835v1 Announce Type: new Abstract: Long-context large language model inference is bottlenecked by KV caches that grow linearly with sequence length. This burden is especially severe for l

Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models

arXiv:2609.35875v1 Announce Type: new Abstract: Multi-agent debate (MAD) reportedly improves reasoning and factuality over single-model inference, but prior work treats agents as symmetric peers, leav

Applications

Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment

arXiv:2501.18157v2 Announce Type: replace-cross Abstract: Building reliable speech systems often requires combining multiple modalities, like audio and visual cues. While such multimodal solutions fre

applicationsarxiv-cs-cv

Estimating the Causal Effects of T Cell Receptors

arXiv:2410.14127v2 Announce Type: replace-cross Abstract: A central question in human immunology is how a patient's T cell receptors impacts disease. Here, we introduce a method to infer the causal ef

Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling

arXiv:2609.38104v1 Announce Type: new Abstract: Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models

From Retrieval to Reasoning: Agentic Mechanism Prediction from Cell Painting Profiles

arXiv:2609.36406v1 Announce Type: new Abstract: Cell Painting is a high-content morphological profiling assay widely used for phenotype-based biological inference, with mechanism of action (MOA) predi

GleanVID: Complementary Token Selection for Efficient Video Large Language Models

arXiv:2609.37042v1 Announce Type: new Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the large

local-aiarxiv-cs-cv

IronLLM: Forging Compact Edge-Native Language Models for Real-Time Embodied Intelligence

arXiv:2609.36860v1 Announce Type: new Abstract: We present IronLLM-0.6B, a 654M-parameter language model designed for efficient on-device inference. IronLLM-0.6B combines a hybrid attention architectu

Learning via Self-Consistency for Diffusion-based Video Reasoning

arXiv:2609.36826v1 Announce Type: new Abstract: Video generation models have demonstrated emerging zero-shot capabilities for visual reasoning, perception, and other vision tasks. However, diffusion-b

More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses

arXiv:2609.35873v1 Announce Type: new Abstract: Automated generation of LLM harnesses promises to improve inference through task specialization. Yet additional answer coverage can arise from repeated

tutorialsarxiv-cs-ai

Hardware

Purlin: Separating Orchestration from the Datapath of Collectives

arXiv:2609.36954v1 Announce Type: cross Abstract: Distributed inference depends on GPU collective communication that must keep pace with evolving hardware and specialized workloads. However, existing

hardwarearxiv-cs-ai

Applications

Bayesian Active Learning for Intent Disambiguation in Interactive Robot Planning

arXiv:2609.34270v1 Announce Type: new Abstract: Interactive robot planning requires robots to infer and execute human intentions from natural language instructions that are often ambiguous, incomplete

applicationsarxiv-cs-ro

29 Sept 2026

Do World Models Learn Global Understanding?

arXiv:2609.34058v1 Announce Type: new Abstract: AI systems often feel brittle and fragmented. A large language model (LLM) may correctly explain a concept but fail to apply it, or follow safety instru

local-aiarxiv-cs-lg

29 Sept 2026

Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs