Search: “inference”

Reasoning as Data: Representation-Computation Unity and Its Implementation in a Domain-Algebraic Inference Engine

arXiv:2604.10908v1 Announce Type: new Abstract: Every existing knowledge system separates storage from computation. We show this separation is unnecessary and eliminate it. In a standard triple is_a(A

researcharxiv-cs-ai

14 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

We've tested new OSS models the moment they're released for a while at Lindy. Inference is our #1 cost by a lot (more than payroll) — cuttin…

DGX agent

We've tested new OSS models the moment they're released for a while at Lindy. Inference is our #1 cost by a lot (more than payroll) — cutting it by 2-5x would be transformative. Last year, OSS models

model-releasesharrison-chase--x

14 Apr 2026

Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning

arXiv:2509.16606v5 Announce Type: replace-cross Abstract: In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained comm

local-aiarxiv-cs-lg

13 Apr 2026

Hardware

CSAttention: Centroid-Scoring Attention for Accelerating LLM Inference

arXiv:2604.08584v1 Announce Type: cross Abstract: Long-context LLMs increasingly rely on extended, reusable prefill prompts for agents and domain Q&A, pushing attention and KV-cache to become the domi

hardwarearxiv-cs-ai

13 Apr 2026

QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference

arXiv:2604.08585v1 Announce Type: cross Abstract: Cache fusion accelerates generation process of LLMs equipped with RAG through KV caching and selective token recomputation, thereby reducing computati

local-aiarxiv-cs-ai

13 Apr 2026

And if you are interested in the answer, Claude and ChatGPT 5.4 Pro think running OpenClaw with local inference on your Mac uses more total …

model-releasesethan-mollick--x

And if you are interested in the answer, Claude and ChatGPT 5.4 Pro think running OpenClaw with local inference on your Mac uses more total power. Gemini doesn't think so (but appears to have not thou

12 Apr 2026

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference

model-releasesarxiv-cs-cl

arXiv:2604.07394v1 Announce Type: cross Abstract: The quadratic computational complexity of standard attention mechanisms presents a severe scalability bottleneck for LLMs in long-context scenarios. W

10 Apr 2026

See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs

arXiv:2604.05650v2 Announce Type: replace Abstract: Video Large Language Models (Video-LLMs) excel in video understanding but suffer from high inference latency during autoregressive generation. Specu

researcharxiv-cs-cl

10 Apr 2026

Tools

A common theme at @aiDotEngineer @swyx @steipete 🦞 Lot’s of fun and meeting great people, my talk about model inference at @superlinked is …

Fardis Makraduli ([@f_makraduli](https://x.com/f_makraduli)) shared a post about attending the AI Engineer ([@aiDotEngineer](https://x.com/aiDotEngineer)) event, noting shared themes and networking...

toolsswyx--x

9 Apr 2026

Tutorials

The founder of a $4B inference company says that if you're building agents, foundational models could become your IP. According to @lqiao, 9…

The founder of a $4B inference company says that if you're building agents, foundational models could become your IP. According to @lqiao, 90% of the world's data is still private and locked inside ap

tutorialssonya-huang--x

9 Apr 2026

Agents

EvoFlux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents

arXiv:2606.12674v2 Announce Type: replace Abstract: Small language models (1.5B-4B parameters) offer low-cost, low-latency tool use but often produce plausible plans that fail during execution due to

agentsarxiv-cs-ai

Physics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference

arXiv:2610.21833v1 Announce Type: new Abstract: Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and evaluate power flow orders of magnitude faster

RTX 2080 Ti 22GB modded card for local LLM inference – anyone tested one?

model-releasesr-localllama

Title: I'm building a small local AI server to validate a project I've developed. The application currently runs Qwen 9B and the server will only be used for an initial pilot with a few users. I have

Scene Reconstruction Using Ubiquitous Wireless Signals via Diffusion Posterior Inference Over physics-Grounded Representations

arXiv:2610.22120v1 Announce Type: new Abstract: Knowing the layout of the physical world matters for many applications, yet the sensors we rely on for it (e.g., cameras and LiDAR) can fail in darkness

Accelerating VLA Inference with Perturbation Informed Execution

arXiv:2610.19656v1 Announce Type: new Abstract: Action-chunking vision-language-action (VLA) models predict sequences of future robot actions. Under fixed-horizon execution, the robot executes a fixed

local-aiarxiv-cs-ro

AdaptFuse: Training-Free Sequential Preference Inference via Externalized Bayesian Computation

model-releasesarxiv-cs-cl

arXiv:2604.03925v2 Announce Type: replace Abstract: Large language models (LLMs) struggle to accumulate evidence across multiple rounds of user interactions, failing to update their beliefs in a manne

Safety

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

arXiv:2610.19817v1 Announce Type: new Abstract: Human values shape how we expect language models to behave. Dominant alignment approaches treat values as global objectives, although the appropriate va

safetyarxiv-cs-cl

Neural Wasserstein Splines: Representation-Invariant Second-Order Priors for Trajectory Inference

arXiv:2610.19878v1 Announce Type: new Abstract: Population dynamics are often observed only as unpaired snapshots, which leave the evolution between measurements underdetermined. Second-order regulari

Accelerating Cross-Model LLM Inference via Attention Map Reuse

arXiv:2610.14688v1 Announce Type: new Abstract: LLM-augmented data workflows often involve multiple heterogeneous models operating over substantial shared context, such as schemas, tool specifications

ASM-6D: Fast Certified Pose-Shape Inference with Complete Correspondence Flow Maps

arXiv:2610.15287v1 Announce Type: new Abstract: Category-level 6D pose and shape estimation requires recovering an object's pose and geometry from partial observations without an exact instance model.

researcharxiv-cs-cv

Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback

arXiv:2610.18656v1 Announce Type: new Abstract: Language model systems repeatedly make test-time decisions by comparing similarity: they retrieve exemplars, rank candidates, and revise responses. Yet

COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration

arXiv:2610.17332v1 Announce Type: new Abstract: Vision large language models (VLLMs) are deployed in privacy-sensitive domains, raising training-data exposure concerns. Existing gray-box membership in

Hardware

Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference

arXiv:2610.14277v1 Announce Type: cross Abstract: Tensor parallelism ends every layer with an all-reduce that sits between two dependent matrix multiplications, so its cost cannot be hidden and grows

hardwarearxiv-cs-lg

Embodiment-Aware Inference-Time Search for Human Reaction Selection in Humanoid Retargeting

model-releasesarxiv-cs-ro

arXiv:2610.15274v1 Announce Type: new Abstract: Training humanoid robots to learn human behaviors requires expensive data collection, and human-robot interaction data can be even more difficult and co

EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference

arXiv:2610.14324v1 Announce Type: new Abstract: Foundation models like V-JEPA are increasingly deployed across a heterogeneous edge-cloud continuum spanning smart glasses, phones, robots, and servers.

local-aiarxiv-cs-cv

Safety

MOSAIC: Mode- and Support-Aware Inference-Time Control of Robot Diffusion Models

arXiv:2610.17001v1 Announce Type: new Abstract: Generative policies such as diffusion models provide expressive data-driven priors for robot behavior, yet the situations a robot faces at execution tim

safetyarxiv-cs-ro

Privacy-Preserving GNN Inference on Financial Transaction Graphs via Homomorphic Encryption

arXiv:2610.18054v1 Announce Type: cross Abstract: Anti-money laundering (AML) in financial networks requires modeling transaction patterns that span multiple institutions. In practice, however, banks

Agents

When Does Opponent Modeling Pay Off? Spatial Inference Stabilises Reinforcement Learning under Partial Observability

arXiv:2610.18229v1 Announce Type: cross Abstract: Deep reinforcement learning agents in partially observable competitive games must act on beliefs about an unseen opponent's intent. We study a spatial

agentsarxiv-cs-lg

Amortised Inference through One-Step Implicit Sampling

arXiv:2610.12679v1 Announce Type: new Abstract: Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms. Multiple

12 Oct 2026

Deploying RDMA for Large-Scale Inference and Training Workloads

arXiv:2610.12815v1 Announce Type: new Abstract: RDMA can reduce communication overhead in distributed machine learning systems, but realizing these benefits in production requires more than replacing

12 Oct 2026

FastJEV: Understanding Redundancy for Compact JEV Inference

arXiv:2610.11379v1 Announce Type: new Abstract: JEV models make multimodal decisions by directly scoring candidates. Although the common context is encoded once, candidate evaluation can still repeat

researcharxiv-cs-cv

9 Oct 2026

[Paper] Stepped MoE: Segment-Level Routing with Configurable Inference Complexity

Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures

local-air-localllama

9 Oct 2026

Poster: A Preliminary Study of LLM Distillation Inference

arXiv:2610.12137v1 Announce Type: cross Abstract: Unauthorized model distillation, in which a model is trained on the outputs of a proprietary large language model (LLM), is a growing threat to model

9 Oct 2026

Co-Optimizing Graph Sparsification and Approximate Computing for Energy-Efficient FPGA-Based GCN Inference

arXiv:2610.06138v1 Announce Type: new Abstract: Graph Convolutional Networks (GCNs) have emerged as a powerful framework for learning from graph-structured data, yet their deployment on resource-const

Tutorials

GeoFunFlow: Geometric function flow matching for joint probabilistic inference of physical fields and complex geometries

arXiv:2509.24117v2 Announce Type: replace Abstract: Inverse problems governed by partial differential equations (PDEs) arise widely in science and engineering, but are often ill-posed and limited by s

tutorialsarxiv-cs-lg

Safety

Recon2Servo: Robotic Ultrasound Visual Servoing via Learned Image-to-Motion Inference

arXiv:2610.06010v1 Announce Type: new Abstract: Ultrasound visual servoing is essential for autonomous robotic ultrasound, yet 6-DoF probe control from 2D B-mode images remains challenging due to limi

safetyarxiv-cs-ro

Systematic Hazard Sampling: Minimal-Variance Inference for Discrete Diffusion and Flow Models

arXiv:2601.02799v3 Announce Type: replace Abstract: Uniform-noise discrete diffusion and flow models generate sequences non-autoregressively through iterative, context-dependent token replacements. Ho

AIR-LLM: Broadcasting AI Weights over Radio for Memory-Free Edge LLM Inference via RF Computing

arXiv:2610.00465v1 Announce Type: cross Abstract: Next-generation large language models (LLMs) are expanding from the cloud to ubiquitous edge devices. However, edge devices typically either lack the

2 Oct 2026

Fold'EM: Direct atomic structure inference from Cryo-EM particles

arXiv:2610.01358v1 Announce Type: cross Abstract: Single-particle cryo-electron microscopy (cryo-EM) has become a widely adopted technique for biomolecular structure determination. The conventional cr

researcharxiv-cs-ai

2 Oct 2026

MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models

arXiv:2602.03537v2 Announce Type: replace Abstract: Matryoshka Quantization (MatQuant), Any-Precision-LLM (AP) and AnyBCQ (AB) are recent quantization approaches showing that a single integer-quantize

1 Oct 2026

Agents

SparseEngine: Sparse-First Inference Engine

arXiv:2609.39068v1 Announce Type: new Abstract: Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these

agentsarxiv-cs-lg

1 Oct 2026

Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference

arXiv:2609.36437v1 Announce Type: new Abstract: Zero-knowledge proofs are emerging as a promising approach for enabling private, verifiable LLM governance and auditing, where regulators, users, and au

researcharxiv-cs-ai

30 Sept 2026

Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference

arXiv:2609.37090v1 Announce Type: new Abstract: Large multimodal models (LMMs) support diverse visual understanding and reasoning tasks but are often impractical to run entirely on resource-constraine

local-aiarxiv-cs-cv

30 Sept 2026

ConRAG: Lightweight inference of multi-hop relations

arXiv:2609.35193v1 Announce Type: new Abstract: Understanding how two entities are connected often requires tracing multi-hop relations across documents to identify intermediate entities and supportin

From Data to Program: Fast & Direct Generative Program Inference from Empirical Data

arXiv:2609.35348v1 Announce Type: new Abstract: Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI, a pretrained data

Posterior Regimes and Latent Deception: Variational Bayesian Inference in Hidden Markov Models for Sequential Fraud Detection in Financial Transactions

arXiv:2609.34031v1 Announce Type: new Abstract: We present a three-tier progression of Hidden Markov Models: maximum-likelihood (Baum-Welch), variational Bayesian (VBEM), and a neural variational exte

Applications

RAVEL: Asynchronous Rolling Inference for Flow-Based Vision-Language-Action Models

arXiv:2609.34170v1 Announce Type: new Abstract: Flow-based vision-language-action (VLA) models are highly effective for generalist robot manipulation, yet their reliance on computationally expensive V

applicationsarxiv-cs-ro

Simulation-Based Inference for Plate Reverb System Identification

arXiv:2609.35295v1 Announce Type: cross Abstract: We address Task A of the 1st DAFx Parameter Estimation Challenge, which aims to retrieve the physical parameters of a plate model from an impulse resp