Knowledge catalogue
arXiv:2610.08268v1 Announce Type: cross Abstract: Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models
arXiv:2610.08689v1 Announce Type: new Abstract: Counterfactual inference in Gaussian-process structural causal models (GP-SCMs) has been developed primarily for continuous endogenous variables, limiti
arXiv:2610.07311v1 Announce Type: new Abstract: Agentic memory allows LLM agents to reuse past experience, yet retrieved memories can also distort inference even when they are benign, correctly stored
Together AI has partnered with IBM and NVIDIA to launch the first dedicated, large‑scale AI inference cluster on IBM Cloud, consisting of NVIDIA B300 GPUs and Spectrum‑X Ethernet networking engineered
arXiv:2609.40070v1 Announce Type: cross Abstract: When inference demand exceeds available compute capacity, model providers must decide which requests should be served first. Users have different tole
arXiv:2609.39525v1 Announce Type: cross Abstract: Casting Bayesian inference as a neural network optimization problem targeting an amortized posterior is attractive, as it extends to otherwise intract
arXiv:2609.38090v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-c
arXiv:2609.30993v1 Announce Type: cross Abstract: We present FLIP, a final-layer inference-time probe for testing whether a logit-facing intervention site in an open-weight vision-language model (VLM)
arXiv:2609.30270v1 Announce Type: new Abstract: On-device inference with small language models keeps user data local, works offline, and incurs no per-query cost, so the on-device tier is preferred wh
New on Dedicated Model Inference: canary rollouts. Upgrade the model behind a live endpoint without downtime. Traffic moves from your current deployment to the new checkpoint in gated steps (default 5
arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather
arXiv:2609.20614v1 Announce Type: cross Abstract: Frontier AI models are rapidly gaining the ability to exploit vulnerabilities in complex pieces of software. The risk is not theoretical, as evidenced
arXiv:2609.18045v1 Announce Type: new Abstract: Recovering latent interaction structures from multi-agent dynamics is important for understanding and predicting interacting systems. Trajectory-based s
Today’s LLMs still write like typewriters: one token at a time. This sequential process creates a hard inference bottleneck. We're introducing Uno, a diffusion-augmented LLM that delivers autoregressi
In this analysis, MLPerf Inference Working Group chairs Miro Hodak and Frank Han break down a record field of 30 submitters and 120 systems, the arrival of agentic and end-to-end benchmarks, and what
arXiv:2609.16742v1 Announce Type: cross Abstract: Convolutional Neural Networks (CNNs) are increasingly deployed in safety-critical edge applications, where soft errors can silently corrupt inference
arXiv:2609.17193v1 Announce Type: new Abstract: Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed e
arXiv:2609.14213v1 Announce Type: cross Abstract: Modern mobile inference runs on heterogeneous platforms combining mobile GPUs with multiple CPU core clusters. Existing optimizations typically exploi
arXiv:2609.12421v1 Announce Type: cross Abstract: We study an online sketched Newton method that approximates the Newton direction at each step via a state-of-the-art sketching solver, called the gene
arXiv:2609.11334v1 Announce Type: new Abstract: Natural Language Inference processes pairs of sentences to extract their semantic relations. NLI has been a hot research topic, integrated as a main com
arXiv:2609.11799v1 Announce Type: cross Abstract: Large language models are often fine-tuned, shared, or downloaded from third parties, so a deployed model may carry a hidden backdoor that behaves nor
arXiv:2609.09504v1 Announce Type: cross Abstract: We perform field-level likelihood-free inference of the matter density parameter Omega_m from simulated galaxy catalogs using machine learning models
Amazon SageMaker Inference now offers prefix-aware routing, a routing strategy that sends requests sharing the same prompt prefix to the same instance so the KV cache stays warm. In benchmarks on Llam
arXiv:2608.31025v1 Announce Type: new Abstract: Inferring object dynamics from visual observations is essential for intelligent agents to reason about and interact with the physical world, yet remains
arXiv:2310.02423v3 Announce Type: replace Abstract: We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call Delta-amortized inference (Delta-A
arXiv:2608.28576v1 Announce Type: cross Abstract: Synthetic data can improve statistical inference when real data are scarce, but naively treating synthetic samples as real data can introduce bias and
arXiv:2608.27206v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) demonstrate exceptional visual reasoning capabilities, yet their inference costs escalate rapidly with the proliferation
arXiv:2608.25096v1 Announce Type: new Abstract: The growing adoption of large language models (LLMs) has raised increasing concerns about the energy consumption and environmental impact of inference.
arXiv:2608.22646v1 Announce Type: new Abstract: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference. This makes generation
Mike Wheatley / SiliconANGLE: Nvidia says its inference accelerator Groq 3 LPX has entered full production and Nebius has signed on as the first customer, and SpaceX will deploy Vera CPUs — Chipmaker
The next era of AI inference won’t be defined by a single breakthrough chip, network or system. It’ll be defined by how every layer of the AI factory works together. That’s why NVIDIA is extending Ver
arXiv:2608.19377v1 Announce Type: cross Abstract: Bayesian inference via Markov Chain Monte Carlo (MCMC) provides effective parameter estimation, but its real-time application in complex physical syst
arXiv:2603.19997v2 Announce Type: replace Abstract: We investigate the separation of literal interpretation from contextual inference in a collaborative block-building tasks, where an agent must resol
arXiv:2608.17616v1 Announce Type: new Abstract: We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retrain
AI inference infrastructure is becoming a system-level challenge as organizations move generative and agentic applications into production. Graphics processing unit performance remains essential, but
arXiv:2608.15016v1 Announce Type: cross Abstract: Network incident response remains slow and labor-intensive as the defender must infer multi-stage attacks from partial observations and translate reco
A few weeks after closing a 300 million round backed by Nvidia Corp., Etched Inc. today announced that it has raised an additional 700 million in funding. Investment firm Jane Street led the deal. It
arXiv:2608.12973v1 Announce Type: cross Abstract: In this paper, we study how to perform statistical inference for quantile temporal difference learning (QTD) in distributional reinforcement learning.
ai is the ultimate grow the pie event cheaper inference = better gross margins for application companies, AND beautiful cohort expansion for model and inference companies Sequoia Capital GP @sonyatwee
Together Serverless Inference gives developers a managed, high-throughput path for running Qwen3.8-2.4T-A95B across coding and agentic workloads. Start building: https://www.together.ai/models/qwen3-8
As inference becomes the dominant workload in AI infrastructure, multi-tier storage architectures are emerging as a key method for cost control and enhanced performance. These architectures combine fl
arXiv:2608.06723v1 Announce Type: cross Abstract: The rapid scaling of Large Language Models (LLMs) has significantly increased computational cost, energy consumption, and inference latency, making ac
arXiv:2608.05791v1 Announce Type: cross Abstract: Large language model (LLM)-driven multi-agent systems typically require multiple model invocations and complex coordination during inference, and thei
Tobias Mann / The Register: AMD acquires Toronto-based Taalas, which integrates model weights directly into silicon to boost inference performance, for an undisclosed sum — Early tech demos show model
arXiv:2608.02378v1 Announce Type: new Abstract: Private inference protects both user inputs and server models during neural network inference, but existing solutions remain too slow for practical depl
arXiv:2608.00059v1 Announce Type: new Abstract: The success of connectome mapping now shifts the challenge of understanding the nervous system to the interpretation of neural circuits. Here, we devise
arXiv:2608.01975v1 Announce Type: cross Abstract: Large language model (LLM) inference has evolved from an offline workload into a continuously operated software service, yet root-cause analysis remai
arXiv:2602.05307v2 Announce Type: replace Abstract: Large reasoning models (LRMs) achieve strong performance by producing long chains of thought, but their inference costs are high and often generate
arXiv:2607.26571v1 Announce Type: new Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprin
The Together AI platform’s dedicated inference architecture consists of three immutable entities: **configs** (engine, GPU type/count, parallelism and optimization profile), **deployments** (a specifi
arXiv:2607.23915v1 Announce Type: cross Abstract: We examine how prompt tone affects both accuracy of the LLM answers and inference cost as reflected in output-token consumption. Experiments were perf
This is what strong inference economics unlock. @rox_ai built its own search agent, ran it in production for 6+ months, and reached 91.3% accuracy at 1.03¢ per query. Together AI is proud to help powe
arXiv:2607.12376v1 Announce Type: cross Abstract: Causal inference using front door intervention and multi-instance learning (MIL) has advanced the analysis of Whole Slide Images (WSI) in digital path
arXiv:2607.12287v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment
Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at native speed, often matching or beating hand-written implementations. Until now, every new architecture
arXiv:2607.07840v1 Announce Type: cross Abstract: Gradient estimation -- the task of computing the gradient of the expected value of a probabilistic program -- has diverse applications in scientific c
arXiv:2502.18447v2 Announce Type: replace Abstract: Existing approaches to reward inference typically assume that humans provide demonstrations according to specific behavior models. However, humans o
This post likely discusses FireworksAI's fast inference capabilities and their implications or surprising effectiveness in the AI inference space. The post is from Sonya Huang, associated with SemiAna
arXiv:2607.01831v1 Announce Type: cross Abstract: Long-context inference is increasingly common in large language model (LLM) serving, driven by retrieval-augmented generation and agentic systems. In
ThunderAgent is an agentic inference system designed for fast and efficient execution of AI agent programs, developed by Together AI. The system appears to optimize program-aware inference by leveragi