Search: “inference”

Trajectory Prediction via Bayesian Intention Inference under Unknown Goals and Kinematics

model-releasesarxiv-cs-ro

arXiv:2509.24928v2 Announce Type: replace Abstract: This work introduces an adaptive Bayesian algorithm for real-time trajectory prediction via intention inference, where a target's intentions and mot

14 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Reliable Inference in Edge-Cloud Model Cascades via Conformal Alignment

DGX agent

arXiv:2510.17543v3 Announce Type: replace Abstract: Edge intelligence enables low-latency inference via compact on-device models, but assuring reliability remains challenging. We study edge-cloud casc

model-releasesarxiv-cs-lg

13 Aug 2026

Transit Destination Inference from Tap-In-Only Bus Smart-Card Data: A Hierarchical Bayesian Approach

arXiv:2608.11223v1 Announce Type: cross Abstract: Entry-only automatic fare collection systems record boardings but not alightings, preventing direct construction of origin-destination (OD) matrices.

13 Aug 2026

Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems

arXiv:2509.05624v3 Announce Type: replace-cross Abstract: How much information about an agent's underlying values can be recovered from its observable behavior? This question matters for any approach

12 Aug 2026

Expedia recently moved its ranking models to a state-of-the-art Keras 3 setup. Results: 30% faster training, and inference latency decreased…

researchfrancois-chollet--x

Expedia recently moved its ranking models to a state-of-the-art Keras 3 setup. Results: 30% faster training, and inference latency decreased by 70%. Read their writeup about the upgrade: https://mediu

12 Aug 2026

EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins

arXiv:2607.08793v4 Announce Type: replace-cross Abstract: Sepsis is a leading cause of mortality, yet optimal treatment policies remain contested. Existing reinforcement learning (RL) approaches learn

safetyarxiv-cs-ai

11 Aug 2026

Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

arXiv:2608.09225v1 Announce Type: cross Abstract: The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests.

11 Aug 2026

☁️Mistral is bringing together the inference infrastructure, open models, and long-term commitments Europe needs to control its AI future, a…

model-releasesmistral-ai--x

☁️Mistral is bringing together the inference infrastructure, open models, and long-term commitments Europe needs to control its AI future, and setting a roadmap for the world. 🧵: https://mistral.ai/ne

11 Aug 2026

A primer on optimal transport for causal inference with observational data

arXiv:2503.07811v3 Announce Type: replace-cross Abstract: The theory of optimal transportation has developed into a powerful and elegant framework for comparing probability distributions, with wide-ra

researcharxiv-cs-ai

CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM

arXiv:2504.17584v2 Announce Type: replace-cross Abstract: Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.g., CPUs, HBM-

CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights

arXiv:2608.06763v1 Announce Type: new Abstract: Weight quantization for large-language-model inference must balance adaptive reconstruction levels with representations regular enough for efficient GPU

hardwarearxiv-cs-lg

Fast and Accurate: An Adaptive VLA Inference Framework through Environment-aware Model Selection

arXiv:2608.06434v1 Announce Type: cross Abstract: Embodied intelligence demands both long-horizon reasoning and real-time closed-loop responsiveness. Recent dual-system Vision-Language-Action (VLA) ar

Agents

absolute @FireworksAI_HQ domination for anybody wondering what happens to the inference platforms when hyperscalers compete... here's a nice…

absolute @FireworksAI_HQ domination for anybody wondering what happens to the inference platforms when hyperscalers compete... here's a nice narrative violation to chew on :) 📣 @Kimi_Moonshot's Kimi K

agentssonya-huang--x

7 Aug 2026

LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction

arXiv:2608.05600v1 Announce Type: cross Abstract: Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE), whereas online reinforcement learn

safetyarxiv-cs-ai

7 Aug 2026

Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selection

arXiv:2608.04667v1 Announce Type: cross Abstract: Selective inference (SI) provides statistically valid p-values for hypotheses selected by applying an algorithm to the data, correcting for the bias t

Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models

model-releasesarxiv-cs-cv

arXiv:2608.04404v1 Announce Type: new Abstract: World Action Models (WAMs) improve robot manipulation by learning how the environment evolves beyond the current observation. However, existing approach

I ported vLLM's serving stack to C++20: 66 MiB binary, no Python at inference, output checked token-for-token against vLLM

model-releasesr-localllama

I'm the author, so discount the enthusiasm accordingly. This is an unaffiliated community port, not endorsed by the vLLM project, which it uses to verify its correctness. What started it: I love vLLM,

I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a 'harness'), running at inference …

researchfrancois-chollet--x

I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a 'harness'), running at inference time, orchestrating thousands of calls to a neural network f

Applications

Together Serverless Inference gives developers a managed production path for bringing FLUX 3 into creative products and automated media work…

applicationstogether-ai--x

Together Serverless Inference gives developers a managed production path for bringing FLUX 3 into creative products and automated media workflows. Start building: https://www.together.ai/models/flux-3

Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models

arXiv:2608.03112v1 Announce Type: cross Abstract: Vision-language models excel at image and video understanding but suffer from high inference latency due to the need to process thousands of tokens pe

5 Aug 2026

Tutorials

Causal Inference with Unstructured Outcomes

arXiv:2608.03085v1 Announce Type: cross Abstract: Causal inference has traditionally centered on scalar outcomes: whether a patient recovers, how much a worker earns, or how many visits a website rece

tutorialsarxiv-cs-lg

5 Aug 2026

Trajectory inference via Acceleration Matching

arXiv:2608.03916v1 Announce Type: new Abstract: Trajectory inference is a fundamental problem in many scientific domains: given a collection of unpaired snapshots of observations at discrete time poin

5 Aug 2026

Tutorials

Causal Inference with Unstructured Treatments

arXiv:2608.00657v1 Announce Type: cross Abstract: Causal inference usually concerns a scalar treatment, yet in many problems the treatment is unstructured: a text, an image, or a sequence of clinical

tutorialsarxiv-cs-lg

Characterizing Bias in Post-Bandit Inference under Index Algorithms

arXiv:2608.01069v1 Announce Type: new Abstract: Bandit algorithms generate data for downstream inference, but adaptive sampling biases post-bandit sample means. We analyze this bias for stable index a

Decrease the power limit of your 5090 to at least 480W - the performance penalty for inference is negligible.

model-releasesr-localllama

I run my inference machine in the living room, so noise and heat output are a significant concern. Ran a quick test using my daily driver model (Qwen 3.6-27b) and at 480W, the card outputs only 2.1% l

Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning

arXiv:2410.06814v2 Announce Type: replace Abstract: Over-parameterized models are typically vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in

HapticVLA: Contact-Rich Manipulation via Vision-Language-Action Model without Inference-Time Tactile Sensing

arXiv:2603.15257v2 Announce Type: replace Abstract: Tactile sensing is a crucial capability for Vision-Language-Action (VLA) architectures, as it enables dexterous and safe manipulation in contact-ric

safetyarxiv-cs-ro

Learning-Based Collaborative MEC for LLM Inference with Soft-Deadline Awareness via Transformer-Enhanced PPO

arXiv:2608.02031v1 Announce Type: cross Abstract: This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints. In

Local Ai

Spatiotemporal Proximal Causal Inference under Hidden Confounding and Interference

arXiv:2608.01352v1 Announce Type: new Abstract: Estimating causal effects from real-world spatiotemporal data is challenging due to hidden confounders and interference. Standard causal identification

local-aiarxiv-cs-lg

FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models

arXiv:2607.29602v1 Announce Type: cross Abstract: Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are alre

3 Aug 2026

Studying quantization trade-offs for efficient inference deployment in machine translation

arXiv:2607.29397v1 Announce Type: new Abstract: Deploying large language models in realistic server environments poses challenges, as the system needs to provide high-quality responses with low latenc

hardwarearxiv-cs-cl

3 Aug 2026

How well do multiple GPUs scale for LLM inference? (Trying to understand the basics)

model-releasesr-localllama

Hi everyone, I’m fairly new to the multi-GPU side of local LLMs and I’m trying to understand how inference actually scales across multiple GPUs. Suppose I have a model running on a single GPU and then

2 Aug 2026

Autoscaling endpoints for LLM inference

GPU utilization can read healthy while your queue backs up, and a new replica takes minutes to warm. Here's how to pick autoscaling metrics, tune scale-up/down windows, and budget for cold starts on d

hardwaretogether-ai-blog

Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference

The article shows that dense‑attention performance in long‑context inference is governed by group size (query heads per KV head), head dimension, and sequence length, with prefill being compute‑bound

hardwarenvidia-developer

Doubly Robust Functional Representation Learning for Longitudinal Causal Inference with Irregular Histories

arXiv:2607.28567v1 Announce Type: cross Abstract: Longitudinal causal studies often record histories as irregular functional fragments: laboratory values, physiologic signals, sensor streams, and imag

researcharxiv-cs-lg

From Expert Reduction to Behavioral Divergence: Tracing Numerical State through Sparse MoE Inference

arXiv:2607.28097v1 Announce Type: new Abstract: Mathematically equivalent expert-reduction orders can produce observably different sparse-MoE executions. We isolate this effect in native DeepSeek-V4-F

The first thing we learned building autoscaling for dedicated inference is that CPU-style metrics don't tell the whole picture. A GPU can re…

The first thing we learned building autoscaling for dedicated inference is that CPU-style metrics don't tell the whole picture. A GPU can read 60% busy while the engine's queue is already backing up,

hardwaretogether-ai--x

Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Time

arXiv:2607.26647v1 Announce Type: new Abstract: While text-to-image diffusion models achieve impressive visual quality, they frequently struggle to maintain precise alignment with complex compositiona

safetyarxiv-cs-cv

30 Jul 2026

Lilith: Backdoor Generalization under Training-Inference Trigger Shift

arXiv:2607.26099v1 Announce Type: cross Abstract: Machine-learning services increasingly rely on public data, third-party providers, and outsourced training, creating opportunities for data-poisoning

30 Jul 2026

sure we lose money on every inference but we make it up in volume

model-releasesgary-marcus--x

sure we lose money on every inference but we make it up in volume We are committed to pushing the model frontier across cost efficiency, capability, and speed. Starting today, we are reducing prices f

30 Jul 2026

A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods

arXiv:2504.05431v3 Announce Type: cross Abstract: Variational inference, as an alternative to Markov chain Monte Carlo sampling, has played a transformative role in enabling scalable computation for c

researcharxiv-cs-lg

Agents

Agentic inference wastes GPUs on KV cache thrashing. ThunderAgent fixes it at the scheduler level: 2.5x higher single-node throughput and ~1…

Agentic inference wastes GPUs on KV cache thrashing. ThunderAgent fixes it at the scheduler level: 2.5x higher single-node throughput and ~10x lower P50 latency at high concurrency. ThunderAgent was a

agentstogether-ai--x

At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

arXiv:2607.25504v1 Announce Type: cross Abstract: Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference f

CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention

arXiv:2607.25291v1 Announce Type: new Abstract: The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical

researcharxiv-cs-cl

DeepSeek V4 Flash isn't just for inference anymore. Fine-tune it on Fireworks with supervised fine-tuning, preference tuning, and combined p…

model-releasesfireworks-ai--x

DeepSeek V4 Flash isn't just for inference anymore. Fine-tune it on Fireworks with supervised fine-tuning, preference tuning, and combined preference optimization from the managed UI. Reinforcement le

Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors

arXiv:2607.25335v1 Announce Type: cross Abstract: Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains ques

researcharxiv-cs-ai

Local Ai

Mondrian: On-Device High-Performance Video Analytics with Compressive Packed Inference

arXiv:2403.07598v2 Announce Type: replace Abstract: In this paper, we present Mondrian, an edge system that enables high-performance object detection on high-resolution video streams. Many lightweight

local-aiarxiv-cs-cv

Agents

ThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale

ThunderAgent is a program-aware scheduler for agentic inference. By treating each agent workflow as a schedulable program, it eliminates KV cache thrashing to deliver more than 2x single-node throughp

agentstogether-ai-blog