Search: “inference”

CPU-only inference on a Celeron N5095 SBC: 6 models from 0.6B to 8B, benchmarked

model-releasesr-localllama

I wanted to know how cheap you can go and still run local models, so I ran Ollama CPU-only on a Youyeetoo X1S. It's a single-board x86 machine with a Celeron N5095 (Jasper Lake, 4C/4T, 15W), 16GB of R

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Evolving Cache Schedules for Fast Diffusion Policy Inference

DGX agent

arXiv:2607.20293v1 Announce Type: new Abstract: Diffusion policies achieve strong visuomotor control by iteratively denoising action chunks, but repeated denoising makes real-time deployment computati

safetyarxiv-cs-cv

23 Jul 2026

How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF

arXiv:2607.19712v1 Announce Type: new Abstract: In RLHF pipelines, reward scoring blocks policy updates. Slow scoring bottlenecks the entire loop, since no update runs until every rollout gets a score

Membership Inference Attacks for Unseen Classes

arXiv:2506.06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is data auditing, i.e., using statistical tools to determine whether harmful content may have been used in t

Statistical Inference for Rank Allocation in Low-Rank Adaptation

arXiv:2607.20205v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models. Since different modules and laye

Optimizing Binary and Ternary Neural Network Inference on RRAM Crossbars using CIM-Explorer

arXiv:2505.14303v3 Announce Type: replace-cross Abstract: Using Resistive Random Access Memory (RRAM) crossbars in Computing-in-Memory (CIM) architectures offers a promising solution to overcome the v

16 Jul 2026

The SIGReg Objective as Variational Free Energy: A Theoretical Active-Inference Account of JEPA World Models

arXiv:2607.13612v1 Announce Type: cross Abstract: Joint-Embedding Predictive Architectures (JEPAs) are the dominant design for latent world models, yet they are usually justified by empirical performa

safetyarxiv-cs-ai

16 Jul 2026

Globalized Constrained Stein Variational Inference for Diverse Feasible Robot Motion Planning

arXiv:2607.12732v1 Announce Type: new Abstract: Robot motion planning is inherently multimodal, yet classical planners typically return only a single solution. Probabilistic formulations address this

hardwarearxiv-cs-ro

15 Jul 2026

Agents

Behavioral Privacy Leakage in Agentic Negotiation: Formalizing and Mitigating Inference Attacks via Randomized Policies

This paper was accepted at the AI4TCI (Workshop on AI for Secure and Trustworthy Critical Infrastructure Systems) Workshop at the International Conference on Availability, Reliability and Security (AR

agentsapple-ml-research

PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLM

arXiv:2505.16456v3 Announce Type: replace Abstract: Recent advances in 3D content generation have amplified demand for dynamic models that are both visually realistic and physically consistent. Howeve

safetyarxiv-cs-cv

Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning

arXiv:2607.08444v1 Announce Type: cross Abstract: In this paper, we study quantile-based distributional reinforcement learning from the perspective of statistical efficiency. We focus on distributiona

Workload-Preserving Differentially Private Synthetic Data for Causal Inference via Maximum-Entropy Calibration

arXiv:2607.08122v1 Announce Type: new Abstract: Workload-based differentially private (DP) synthetic data methods privately measure aggregate queries and post-process the noisy answers into synthetic

Amortized Inference for Correlated Discrete Choice Models via Equivariant Neural Networks

arXiv:2603.24705v3 Announce Type: replace-cross Abstract: Discrete choice models are fundamental tools in management science, economics, and marketing for understanding and predicting decision-making.

InferNet: Exploiting Aggregate GPU Profiles as Side-Channel for DNN Architecture Inference

arXiv:2304.03388v2 Announce Type: replace Abstract: Deep Neural Networks (DNNs) have become ubiquitous for their ability to solve problems across various domains, including computer vision, natural la

hardwarearxiv-cs-lg

Refine Thought: A Test-Time Inference Method for Embedding Model Reasoning

arXiv:2511.13726v2 Announce Type: replace-cross Abstract: We propose RT (Refine Thought), a method that can enhance the semantic reasoning ability of text embedding models. The method obtains the fina

Trajectory Inference of Human Aging from Cross-Sectional DNA Methylation Data

arXiv:2607.06583v1 Announce Type: cross Abstract: DNA methylation (DNAm) serves as one of the most robust molecular biomarkers of biological aging. While conventional epigenetic clocks accurately pred

FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision

arXiv:2511.21245v3 Announce Type: replace Abstract: Monocular 3D face reconstruction estimates a 3D morphable model (3DMM) representation from a single image, providing geometry-aware expression codes

researcharxiv-cs-cv

8 Jul 2026

Inference chip startup SambaNova valued at 11B in 1B funding round

Chip startup SambaNova Inc. today announced that it has raised 1 billion in funding at a 11 billion valuation. General Atlantic led the Series F round with contributions from more than a dozen others.

hardwaresiliconangle

8 Jul 2026

Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B

arXiv:2607.04244v1 Announce Type: new Abstract: This report describes our approach to the Efficient Qwen Competition, where the goal is to enable low-latency serving of Qwen3.5-4B on a resource-constr

7 Jul 2026

Recursive Entropic Variational Inference for Nonlinear State-Space Models

arXiv:2511.15409v2 Announce Type: replace Abstract: We present a class of algorithms for state estimation in nonlinear, non-Gaussian state-space models. Our approach is based on a variational Lagrangi

7 Jul 2026

Black-Box Inference of LLM Architectural Properties with Restrictive API Access

arXiv:2607.01313v1 Announce Type: cross Abstract: In practice, most commercial LLM providers do not publicly release details of underlying LLM architectures. However, prior work has shown that given l

Embedding Inference Attack

arXiv:2607.01276v1 Announce Type: cross Abstract: Embedding models are essential components of modern Information Retrieval (IR) systems, yet they are typically hidden behind APIs. Recent works have s

ProCal: Inference-Time Proposal Calibration for Open-Vocabulary Object Detection

arXiv:2607.01759v1 Announce Type: cross Abstract: Open-vocabulary object detection aims to localize and classify objects beyond the fixed set of categories seen dur ing training. Recent open-vocabular

local-aiarxiv-cs-ai

Separating Expert Retention from Autonomous Source Inference in Raw-ECG-Replay-Free Continual ECG Deployment

arXiv:2607.01674v1 Announce Type: new Abstract: In multi-source ECG deployment, models may need to incorporate new data sources when earlier raw ECGs cannot be retained or replayed. Freezing a pretrai

VaSST: Variational Inference for Symbolic Regression using Soft Symbolic Trees

arXiv:2602.23561v2 Announce Type: replace-cross Abstract: Symbolic regression (SR) has gained recent traction in AI-driven scientific discovery for learning closed-form physical laws. Yet existing met

VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon

arXiv:2607.01804v1 Announce Type: new Abstract: Vision-Language-Action (VLA) foundation models have recently achieved strong progress in embodied intelligence. To reduce policy-call frequency while pr

local-aiarxiv-cs-ro

Personalized Object Identification and Localization via In-Context Inference with Vision-Language Models

arXiv:2607.00357v1 Announce Type: new Abstract: Personalized object localization (POL) localizes an object instance in a query image based on a few reference images with bounding-box annotations and a

local-aiarxiv-cs-cv

2 Jul 2026

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

arXiv:2606.29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.

Federated Learning with Energy-Based Structured Probabilistic Inference

arXiv:2606.30161v1 Announce Type: cross Abstract: Federated learning typically aggregates client updates using fixed or heuristic weighting rules, which can be suboptimal when clients have heterogeneo

Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence

arXiv:2507.00719v3 Announce Type: replace-cross Abstract: Typically, numerical simulations of Earth systems are coarse, and Earth observations are sparse and gappy. We apply four generative diffusion

I-BBS: Coordinate-Free Inference of Latent Sub-Manifolds Using Random Distance Matrix Theory

arXiv:2606.29675v1 Announce Type: new Abstract: Bogomolny, Bohigas and Schmit (BBS) found that the spectrum of the pairwise distance matrix on N points sampled from a smooth d-dimensional manifold enc

Tutorials

ScaleErasure: Inference-Time Minimal Intervention for Precise Concept Erasure in Next-Scale Autoregressive Image Generation

arXiv:2606.29282v1 Announce Type: new Abstract: Concept erasure aims to prevent image generative models from producing unsafe content while preserving their general generative capability. Meanwhile, n

tutorialsarxiv-cs-cv

StackingNet: Collective Inference Across Independent AI Foundation Models

arXiv:2602.13792v2 Announce Type: replace Abstract: Artificial intelligence built on large foundation models has transformed language understanding, computer vision, and reasoning, yet these systems r

safetyarxiv-cs-ai

Applications

KISS-IMU: Self-supervised Inertial Odometry with Motion-balanced Learning and Uncertainty-aware Inference

arXiv:2603.06205v2 Announce Type: replace Abstract: Inertial measurement units (IMUs), which provide high-frequency linear acceleration and angular velocity measurements, serve as fundamental sensing

applicationsarxiv-cs-ro

Ontology-Guided Evidence Path Inference for Multi-hop Knowledge Graph Question Answering

arXiv:2606.28076v1 Announce Type: new Abstract: Knowledge graph question answering (KGQA) aims to answer natural-language questions by reasoning over structured facts. Existing multi-hop KGQA methods

The Simulacrum: Decision-Theoretic Pretraining for Near-Optimal Time-Series Forecasting and Inference

arXiv:2606.27711v1 Announce Type: cross Abstract: We introduce a neural network-based framework for learning time series estimators through a process we term decision-theoretic pretraining. Analysts s

WattLayer: Get Layers Right to Estimate Inference Energy of Neural Networks

arXiv:2606.27841v1 Announce Type: cross Abstract: The widespread adoption of Artificial Intelligence (AI) has led to increasing concerns about energy consumption, yet there is a lack of standardized m

Agents

Rethinking Training & Inference for Forecasting: Linking Winner-Take-All back to GMMs

arXiv:2606.26424v1 Announce Type: cross Abstract: Trajectory forecasting for autonomous driving has advanced rapidly, yet representative models often produce uninformative posteriors over forecast mod

agentsarxiv-cs-cv

26 Jun 2026

SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference

arXiv:2606.26587v1 Announce Type: cross Abstract: Low-bit floating-point formats and semi-structured sparsity are increasingly supported by modern accelerators, yet combining them for LLM activation c

26 Jun 2026

TGHE: Template-based Graph Homomorphic Encryption for Privacy-Preserving GNN Inference in Edge-Cloud Systems

arXiv:2606.26664v1 Announce Type: cross Abstract: Existing homomorphic encryption (HE)-based GNN systems adopt a graph-centric paradigm that couples per-query cost to global graph size, limiting evalu

local-aiarxiv-cs-ai

26 Jun 2026

Semantic Allocation in Ordered Bottlenecks: Predictive Residual Inference for Visual Representation Learning

arXiv:2606.25232v1 Announce Type: cross Abstract: Ordered bottlenecks aim to provide utility at flexible budgets by assigning coarse information to early tokens and task-relevant detail to later ones.

researcharxiv-cs-cv

25 Jun 2026

Applications

TRACER: Training-Free Closed-Loop Structured Inference for Traffic Accident Reconstruction

arXiv:2606.25002v1 Announce Type: new Abstract: Traffic accident reconstruction is a forensic inverse problem that requires recovering physically consistent motion from sparse and heterogeneous eviden

applicationsarxiv-cs-lg

25 Jun 2026

MERGE: Minimal Expression-Replacement GEneralization Test for Natural Language Inference

arXiv:2510.24295v2 Announce Type: replace Abstract: As many benchmarks have become saturated, it has become increasingly important to create new datasets that evaluate the generalization capacity of c

researcharxiv-cs-cl

OpenAI, Broadcom debut custom Jalapeño chip for AI inference

OpenAI Group PBC today revealed a custom chip called Jalapeño that it will use to power its large language models. The processor is the fruit of a collaboration with Broadcom Inc., which is no strange

hardwaresiliconangle

Agents

Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents

arXiv:2511.07397v2 Announce Type: replace Abstract: Voice agents face a fundamental tension: the reasoning, retrieval, and tool use that make foundation models capable are iterative and slow, while co

agentsarxiv-cs-cl

Towards Spec Learning: Inference-Time Alignment from Preference Pairs

arXiv:2606.24004v1 Announce Type: cross Abstract: Steering a large language model (LLM) toward a desired behavior typically relies on an iterative process of hand-crafting a prompt based on a careful

Concordia: JIT-Compiled Persistent-Kernel Checkpointing for Fault-Tolerant LLM Inference

arXiv:2606.23521v1 Announce Type: cross Abstract: Long-running LLM agents keep valuable state resident on GPUs: KV caches, request schedulers, communication state, and sometimes online adapters. Losin

hardwarearxiv-cs-lg

23 Jun 2026

Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead