Search: “inference”

arXiv:2609.14144v1 Announce Type: new Abstract: A transformer language model is trained to respond to any prompt, but each deployment asks only a narrow range of questions: a support assistant sees de

arXiv:2609.14850v1 Announce Type: new Abstract: Large language models (LLMs) demonstrate impressive capabilities, but their deployment presents significant efficiency challenges. Autoregressive decodi

arXiv:2609.12448v1 Announce Type: new Abstract: Sensitive attributes such as age, income, and occupation can be inferred from user-generated content by aggregating indirect cues across many ordinary p

**Summary for AI knowledge base** Vera Rubin NVL72 is a co‑designed platform combining the Rubin GPU, Vera CPU, NVLink 6 Switch, ConnectX‑9, BlueField‑4, and Spectrum‑6, targeting agentic inference wo

Spoke with many providers who lurk in this sub, plus met folks who work in inference. For a 10k monthly revenue, a provider was able to retain only 200 dollars in profit due to GPU costs. Their custom

arXiv:2609.08082v1 Announce Type: new Abstract: Preference-based fine-tuning methods such as RLHF and DPO require substantial compute and large preference datasets. They also need direct access to the

arXiv:2609.09981v1 Announce Type: cross Abstract: We study offline inference for the optimal value in reinforcement learning. Two new nuisances are derived as fixed points of a self-induced Bellman eq

Amazon SageMaker HyperPod now supports model caching for inference, which pre-loads model weights and container images onto cluster nodes so pods read from local NVMe storage instead of downloading ov

Ari Levy / CNBC: Qualcomm signs a deal with Amazon to provide “multiple generations of customized silicon” to help build AWS' AI infrastructure for inference; QCOM jumps ~5% — Qualcomm shares rose 4%

arXiv:2609.05073v1 Announce Type: new Abstract: Conformal counterfactual inference enables network operators to use logged telemetry to reliably answer 'what-if' questions about network operation. The

arXiv:2609.05262v1 Announce Type: new Abstract: Computational phylogenetics has become an essential tool in historical linguistics, yet its application at a global scale remains constrained by two fac

arXiv:2609.01679v1 Announce Type: new Abstract: The ability of AI systems to improve their behavior during deployment is becoming increasingly important. As inference moves beyond the static execution

arXiv:2609.02160v1 Announce Type: cross Abstract: Diffusion models achieve high sample quality but remain expensive at inference time because sampling requires many sequential neural function evaluati

Australian teams can now access OpenAI GPT-5.6 Sol, Terra, and Luna models on Amazon Bedrock with global cross-Region inference from the Asia Pacific (Sydney) and Asia Pacific (Melbourne) Regions. Thi

arXiv:2602.03708v3 Announce Type: replace Abstract: Large Language Models (LLMs) achieve strong performance across many tasks but suffer from high inference latency due to autoregressive decoding. The

Speculative decoding speeds LLM inference by letting a lightweight draft model propose multiple tokens that the larger target model verifies in parallel, reducing decoding iterations while preserving

arXiv:2609.00407v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models enable efficient scaling of large language model (LLM) inference but suffer from substantial data-movement overhead wh

arXiv:2609.01024v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. However

arXiv:2609.01232v1 Announce Type: cross Abstract: Vision Transformers (ViTs) are increasingly used in split-inference systems, where edge devices transmit intermediate token representations to a remot

Hi, everyone, I’ve just released VoxGen, a lightweight native inference engine for VoxCPM2, written in Rust and using Vulkan compute instead of Python/PyTorch/CUDA. Why VoxGen? The main reason I start

arXiv:2608.30295v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong capabilities in handling long-context tasks, but processing such long contexts remains challengi

arXiv:2512.16391v2 Announce Type: replace-cross Abstract: Attention is the dominant source of latency during long-context LLM inference, an increasingly popular workload with reasoning models and RAG.

arXiv:2608.28327v1 Announce Type: cross Abstract: Practitioners defend large language models (LLMs) by stacking defenses, assuming the layers compound. A stack is an ensemble, and ensembles compound o

arXiv:2608.26324v1 Announce Type: new Abstract: Best-of-N (BoN) sampling is the simplest and most widely deployed inference-time alignment strategy, but it suffers from two distinct problems: reward h

Learn how Salesforce used Amazon SageMaker AI Inference Component placement (the SchedulingConfig parameter) to distribute model copies across multiple Availability Zones, meeting their Multi-AZ high

Together AI announced a broadcast titled “Serving Kimi K3 at scale: An inference deep dive” scheduled for 7 pm on August 26, 2026 (UTC‑adjusted time shown via X’s timestamp formatting). The streamed s

arXiv:2608.22115v1 Announce Type: new Abstract: Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensiona

arXiv:2509.04719v3 Announce Type: replace-cross Abstract: The widespread adoption of diffusion models for image generation necessitates efficient parallel inference to manage their substantial computa

arXiv:2608.22179v1 Announce Type: cross Abstract: Membership inference asks whether a text was used to train a language model, whereas AI-generated text detection asks whether it was generated by a la

arXiv:2608.21070v1 Announce Type: cross Abstract: Inferring continuous system evolution from sparse temporal snapshots is a key challenge in generative modeling and single-cell omics. While Optimal Tr

arXiv:2608.20123v1 Announce Type: cross Abstract: We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level r

arXiv:2608.18374v1 Announce Type: cross Abstract: We address two open questions in streaming PCA via Oja's algorithm: sharp operator-norm convergence for general rank under sub-Gaussian data, and dist

arXiv:2608.19115v1 Announce Type: new Abstract: Modern pretrained encoders make representations from heterogeneous views increasingly reusable, but the procedure that determines view utility and combi

arXiv:2608.18884v1 Announce Type: new Abstract: Reinforcement-learning training of reasoning LLMs (e.g., GRPO) is expensive and requires a controllable environment, committing every contribution to a

Counterpoint: “If alignment issues are becoming big enough that OpenAI is willing to commit 20% of research inference compute to chain-of-thought monitoring [suggesting] that suggests that alignment i

Robbie Whelan / Wall Street Journal: AI inference chip startup Etched raised 700M led by its new server rack customer Jane Street at a 21B valuation, up from 10.3B after raising 300M in July — Etched,

arXiv:2608.15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and

arXiv:2608.15402v1 Announce Type: new Abstract: Generative model alignment has received broad interest, and significant progress has been made in supervised fine-tuning and inference-time computation.

Qwen3.8-Max is live on DigitalOcean Serverless Inference. Launching side by side with DigitalOcean as our Day 0 launch partner. Big model. Smooth sailing. Now on DigitalOcean.🌊🏄‍♀️ @digitalocean Now a

arXiv:2608.13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by h

arXiv:2608.11840v1 Announce Type: cross Abstract: Growing demand for artificial intelligence (AI) inference services requires scalable infrastructure, yet centralized serving costs rise with demand. W

arXiv:2608.10506v1 Announce Type: cross Abstract: Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resourc

arXiv:2601.20819v2 Announce Type: replace-cross Abstract: Machine learning predictions are increasingly used to supplement incomplete or costly-to-measure outcomes in fields such as biomedical researc

arXiv:2608.09444v1 Announce Type: cross Abstract: A main promise of looped language models (LMs) is depth-adaptive inference. By iterating a block of shared layers a variable number of times, the mode

arXiv:2502.11027v5 Announce Type: replace Abstract: Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhanc

Together AI is teaming up with @IBM and @nvidia to bring enterprise-grade AI inference to IBM Cloud. A dedicated NVIDIA B300 cluster. Spectrum-X networking. First of its kind on IBM Cloud, powered by