Search: “inference”

VERA: Variational Inference Framework for Jailbreaking Large Language Models

arXiv:2506.22666v3 Announce Type: replace-cross Abstract: The rise of API-only access to state-of-the-art LLMs highlights the need for effective black-box jailbreak methods to identify model vulnerabi

safetyarxiv-cs-cl

2 Jun 2026

Tutorials

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Haptic Sorter: A Unified Planning Framework for Online Shape Estimation and Real-Time Pose Inference

DGX agent

arXiv:2605.31352v1 Announce Type: new Abstract: Robotics manipulation usually assumes that the shape and pose of the object are known to the robot prior to motion planning. However, precise geometric

tutorialsarxiv-cs-ro

1 Jun 2026

Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode

arXiv:2605.30571v1 Announce Type: cross Abstract: Physical AI systems, including robots, autonomous vehicles, embodied agents and edge copilots, often run a different inference workload from cloud LLM

1 Jun 2026

Tools

MiniMax M3 is live and Together AI is powering its inference 🚀 Tomorrow at 6pm PT we're going live on X Spaces with the teams behind the mo…

MiniMax M3 is live and Together AI is powering its inference 🚀 Tomorrow at 6pm PT we're going live on X Spaces with the teams behind the model and the infrastructure to give you a deep dive. https://x

toolstogether-ai--x

1 Jun 2026

Parallel Tempering Initial Sampling in Inference-Time Reward Alignment

arXiv:2605.30991v1 Announce Type: cross Abstract: Inference-time reward alignment steers pretrained diffusion and flow-based generative models to satisfy user-specified rewards without retraining. Rec

safetyarxiv-cs-cv

1 Jun 2026

what a wonderful project: parakeet.cpp https://github.com/mudler/parakeet.cpp GGML based parakeet inference pipeline that's 2x faster than m…

what a wonderful project: parakeet.cpp https://github.com/mudler/parakeet.cpp GGML based parakeet inference pipeline that's 2x faster than my ONNX parakeet pipeline on Apple Silicon! (Needed a few loc

local-aijeremy-howard--x

31 May 2026

A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning

arXiv:2605.29454v1 Announce Type: new Abstract: While Membership Inference Attacks (MIAs) are the prevailing method for identifying training data, their application has expanded into privacy auditing

Adaptive Exponential Integration for Stable Gaussian Mixture Black-Box Variational Inference

arXiv:2601.14855v3 Announce Type: replace Abstract: Black-box variational inference (BBVI) with Gaussian mixture families offers a flexible approach for approximating complex posterior distributions w

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

arXiv:2605.29233v1 Announce Type: cross Abstract: Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to s

local-aiarxiv-cs-ai

Tutorials

Having Grok Build sub-agents to iterate several ideas for me on data loading, batching, inference, and writing results to files for dense da…

Having Grok Build sub-agents to iterate several ideas for me on data loading, batching, inference, and writing results to files for dense datasets before I went to sleep. It gave me a nice summary of

tutorialselon-musk--x

MarginGate: Sparse Margin-Triggered Verification for Batch-Invariant LLM Inference

arXiv:2605.30218v1 Announce Type: new Abstract: Temperature-zero BF16 LLM inference is often treated as reproducible, yet the same request can emit different tokens when decoded alone or inside a larg

Sources: ByteDance has partnered with chipmaker InnoStar to develop an AI inference chip modeled after Groq's LPUs, which are built to run AI models at low cost (The Information)

The Information: Sources: ByteDance has partnered with chipmaker InnoStar to develop an AI inference chip modeled after Groq's LPUs, which are built to run AI models at low cost — TikTok owner ByteDan

hardwaretechmeme

Wasserstein Contraction of Coordinate Ascent Variational Inference

arXiv:2605.30253v1 Announce Type: cross Abstract: We study the contraction in Wasserstein distance of the coordinate ascent variational inference algorithm. This is shown to hold under a transport-inf

CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models

arXiv:2605.28792v1 Announce Type: new Abstract: Electroencephalography (EEG) is a critical, non-invasive method to monitor electrical brain activity. EEGs can span anywhere from a couple seconds to mu

researcharxiv-cs-ai

Agents

E^3-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

arXiv:2605.27428v1 Announce Type: new Abstract: Edge deployments of generative inference increasingly face two practical realities: per-device per-model performance is often unknown at deployment time

agentsarxiv-cs-lg

Applications

Every LLM API call depends on the inference engine underneath it. Tokenization, scheduling, prefill, decode, KV cache, batching, and streami…

applicationstogether-ai--x

Every LLM API call depends on the inference engine underneath it. Tokenization, scheduling, prefill, decode, KV cache, batching, and streaming determine whether the experience is fast, scalable, and p

Applications

From Causal Discovery to Dynamic Causal Inference in Neural Time Series

arXiv:2603.20980v2 Announce Type: replace-cross Abstract: Time-varying causal models provide a powerful framework for studying dynamic scientific systems, yet most existing approaches assume that the

applicationsarxiv-cs-ai

glad to know Mythos' safety concerns have been addressed right as Anthropic also secured tens of billions in inference compute 👍

model-releasesjeremy-howard--x

glad to know Mythos' safety concerns have been addressed right as Anthropic also secured tens of billions in inference compute 👍 JUST IN: Anthropic announces it will roll out Claude Mythos “in the com

Next will be writing the inference stack in C for simultaneous high-speed RL across a large block of GB300s. (We do use a little C++ tbh, bu…

Elon Musk indicated that the next development phase involves implementing an inference stack in C (with some C++) to enable high-speed reinforcement learning operations simultaneously across a large c

industryelon-musk--x

TinyDejaVu: Smaller RAM and Faster Inference with Neural Networks on MCUs for Sensor Data Streams

arXiv:2512.09786v2 Announce Type: replace Abstract: Examples of embedded intelligence include a wide variety of tiny neural networks used on-board wireless sensors and actuators, which are expected to

AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference

model-releasesarxiv-cs-cl

arXiv:2512.11280v2 Announce Type: replace Abstract: Large language models (LLMs) have achieved remarkable performance across a wide range of tasks, but their increasing parameter sizes significantly s

Assessing Per-Sample Membership Inference Vulnerability without Retraining

arXiv:2602.15919v2 Announce Type: replace-cross Abstract: Recent work in the privacy literature shows that sample-targeted membership inference attacks (MIAs) significantly outperform untargeted appro

researcharxiv-cs-ai

DEI: Diversity in Evolutionary Inference for Quality-Diversity Search

arXiv:2605.27130v1 Announce Type: cross Abstract: We present DEI: Diversity in Evolutionary Inference, a distributed Quality-Diversity (QD) search framework that assigns heterogeneous large language m

i own 0 A100s but i can run cloud inference with sub 80ms network latency by renting one in Kansas (0% action queue starvation) this is an e…

i own 0 A100s but i can run cloud inference with sub 80ms network latency by renting one in Kansas (0% action queue starvation) this is an early checkpoint of smolvla trained on 50 episodes for ~20k s

industryclem-delangue--x

MATT-CTR: Unleashing a Model-Agnostic Test-Time Paradigm for CTR Prediction with Confidence-Guided Inference Paths

arXiv:2510.08932v2 Announce Type: replace Abstract: Recently, a growing body of research has focused on either optimizing CTR model architectures to better model feature interactions or refining train

MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration

arXiv:2605.26546v1 Announce Type: new Abstract: Mobile graphical user interface (GUI) agents enable AI models to autonomously operate smartphones on behalf of users. However, most existing systems foc

Super happy that a bunch of people are finding marlin useful. Thank you for the inference support @ZeroGPU_AI @huggingface 🥰🤝 We’ve got ha…

Super happy that a bunch of people are finding marlin useful. Thank you for the inference support @ZeroGPU_AI @huggingface 🥰🤝 We’ve got hands on more compute now so we’ll also release a series of blog

industryclem-delangue--x

Train with autoregression & convert weights to diffusion for inference.

Train with autoregression & convert weights to diffusion for inference. Most researchers agree that autoregression is best when memory bandwidth is cheap and diffusion is best when FLOPS are cheap. Th

industryemad-mostaque--x

Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models

arXiv:2605.25230v1 Announce Type: new Abstract: Recent work on recursive architectures has shown that tiny neural networks can be surprisingly powerful on structured reasoning tasks. The trick is to m

local-aiarxiv-cs-ai

Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference

arXiv:2511.16449v4 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have shown great potential for embodied AI by integrating visual perception, language understanding, and a

researcharxiv-cs-ai

ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference

arXiv:2510.02361v2 Announce Type: replace-cross Abstract: Transformer-based large models excel in natural language processing and computer vision, but face severe computational inefficiencies due to t

Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models

arXiv:2605.24799v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a wide range of vision language tasks. However, when applied to

local-aiarxiv-cs-ai

EPPC-OASIS: Ontology-Aware Adaptation and Structured Inference Refinement for Electronic Patient-Provider Communication Mining in Secure Messages

arXiv:2605.24172v1 Announce Type: new Abstract: Secure patient-provider messages contain clinically important communication behaviors that are difficult to characterize manually at scale. The Electron

safetyarxiv-cs-ai

From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlas

arXiv:2602.00491v2 Announce Type: replace Abstract: Public health reasoning requires population level inference grounded in scientific evidence, expert consensus, and safety constraints. However, it r

safetyarxiv-cs-cl

Inference Time Context Sparsity: Illusion or Opportunity?

arXiv:2605.24168v1 Announce Type: new Abstract: Sparsity has long been a central theme in LLM efficiency, but its role in context processing remains unresolved. As LLM workloads shift toward longer co

hardwarearxiv-cs-ai

Inference Time Optimization with Confidence Dynamics

model-releasesarxiv-cs-cl

arXiv:2605.25244v1 Announce Type: new Abstract: Inference time optimization techniques, such as repeated sampling, have significantly advanced the reasoning capabilities of Large Language Models (LLMs

Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning

arXiv:2605.25916v1 Announce Type: new Abstract: Federated edge learning (FEEL) has recently emerged as a promising paradigm for achieving edge intelligence (EI) via enabling collaborative model traini

safetyarxiv-cs-lg

On Reliability of Efficient Membership Inference Vulnerability Evaluation

arXiv:2605.25819v1 Announce Type: new Abstract: Membership inference attacks (MIAs) are popular methods for empirically assessing the leakage of sensitive information in the training data through mode

safetyarxiv-cs-lg

Physics-Guided Concentration Inference from Resistance Transients in a Mixed-Phase SnO-SnO_2 Carbon Monoxide Sensor with p-n Switching

arXiv:2605.23971v1 Announce Type: cross Abstract: This work presents a physics-guided machine-learning framework for carbon monoxide concentration inference from experimentally measured resistance tra

Source: AI inference provider Baseten is in talks to raise 1B at a post-money valuation of 11B, up from 5B after its 300M Series E announced in January (The Information)

The Information: Source: AI inference provider Baseten is in talks to raise 1B at a post-money valuation of 11B, up from 5B after its 300M Series E announced in January — AI startup Baseten has recent

industrytechmeme

TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation

arXiv:2605.25547v1 Announce Type: new Abstract: Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size. We instead explore inferenc

safetyarxiv-cs-ro

Safe Reinforcement Learning with Preference-based Constraint Inference

arXiv:2603.23565v2 Announce Type: replace-cross Abstract: Safe reinforcement learning (RL) is a standard paradigm for safety-critical decision making. However, real-world safety constraints can be com

safetyarxiv-cs-ai

25 May 2026

XWind: A Cross-site Router for Large Language Model Inference Serving at Renewable Energy Farms

arXiv:2605.23348v1 Announce Type: cross Abstract: AI power demand is growing at an unprecedented rate while power grids are often ailing and struggle to keep up. Grid expansion comes with high capital

hardwarearxiv-cs-ai

25 May 2026

Our inference stack, optimized for Blackwells, with a novel attention kernel and many new optimizations has started rolling out! It's alread…

Our inference stack, optimized for Blackwells, with a novel attention kernel and many new optimizations has started rolling out! It's already charting on Artificial Analysis, eg: #1 speed and latency

hardwaretogether-ai--x

24 May 2026

Decision-Aware Quadratic ReLU Replacement for HE-Friendly Inference

arXiv:2605.22237v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) supports only additions and multiplications, so FHE-only neural-network inference typically replaces ReLU with poly

Large-scale Score-based Variational Posterior Inference for Bayesian Deep Neural Networks

arXiv:2602.05873v2 Announce Type: replace Abstract: Bayesian (deep) neural networks (BNN) are often more attractive than the vanilla point-estimate deep learning in various aspects including uncertain

Rule-State Inference (RSI): A Bayesian Framework for Compliance Monitoring in Rule-Governed Domains

arXiv:2603.21610v2 Announce Type: replace Abstract: Compliance monitoring in rule-governed domains (tax administration, clinical protocol adherence, environmental regulation) faces three structural ob

Three Costs of Amortizing Gaussian Process Inference with Neural Processes

arXiv:2605.21798v1 Announce Type: new Abstract: Neural processes amortize Gaussian process inference, replacing the exact O(n^3) posterior with a learned O(n) map from context sets to predictive distr