Search: “inference”

Enterprise AI scalability is running into a hard budget wall for some customers. As agentic workloads multiply, cloud token bills are consuming allocations faster than planned, prompting companies to

Coding agents have changed how fast teams ship, and more of the software they ship now includes agents. A production agent spans model calls, routing decisions, tools, and the infrastructure underneat

Microsoft is announcing its third-generation Maia in-house AI chip today, designed to be the 'primary AI inference system' for Microsoft's products and services. Maia 300 not only improves the cost an

Reflection AI Inc. today introduced Beam, an open-source large language model with 501 billion parameters. The launch comes a few months after the startup raised funding at a 25 billion valuation. Aro

Cloudflare AI Gateway now supports native web search API integration in partnership with Ceramic.ai, Exa, and Linkup. Developers can now inject real-time web context into model inference calls via AI

AI agent infrastructure is evolving to support systems that move continuously among inference, feedback and training. Cognition AI Inc.’s Devin now assists throughout the software development lifecycl

The Nvidia Deploy repository provides open‑source C++ samples that pair ONNX Runtime with the TensorRT RTX execution provider to accelerate local inference on Windows and Linux. It separates model exp

Generative recommender systems model recommendation as high‑cardinality event streams, turning user actions, context, and candidate items into tokens for sequence modeling over user behavior. NVIDIA’s

Written by: Robin Grunewald, Supriya Mazumdar, Kelli Vanderlee Introduction Google Threat Intelligence Group (GTIG) examines vulnerability disclosure and exploitation statistics to evaluate the impact

GPT‑6.1 Sol is now generally available on Amazon Bedrock, providing enhanced reasoning for coding, computer use, and professional workflows. It runs on a high‑performance inference engine optimized fo

Learn how to run SkyRL, an open-source reinforcement learning framework, on Amazon SageMaker HyperPod to post-train a Qwen3-VL-8B vision-language model with GRPO. This walkthrough covers building the

SWE‑Serve evaluates AI coding agents on 53 inference‑engineering tasks derived from SGLang pull requests across six functional domains. Among 19 tasks that launch a live server, patches that pass loca

Following a successful private preview, we’re thrilled to open DigitalOcean Managed Agents to everyone. Teams can now deploy their preferred agent harness (like OpenCode, Codex CLI) or bring their own

Operating large data pipelines requires an orchestration layer that scales smoothly as workloads expand. When your pipelines process millions of complex data points every day to feed predictive models

Amazon SageMaker AI now offers instance preference lists for training and processing jobs. Specify an ordered list of up to five instance types, and SageMaker AI automatically launches on the first ty

The telecommunications industry is currently undergoing a paradigm shift, moving from traditional manual human-driven operations to fully Autonomous Network Operations. Modern networks have grown incr

We are excited to share that Google Cloud was named a Leader and received the highest score in the ‘current offering’ category in the Forrester Wave™: Public Cloud Platforms, Q3 2026 report, which exa

AI inference chipmaker d-Matrix today announced it will use NVIDIA NVLink Fusion to connect its next-generation Raptor XPUs to NVIDIA’s AI infrastructure platform — joining a growing roster of ecosyst

Catch up on July and August in Microsoft Foundry: stable Hosted Agent and Toolbox SDK paths, Claude tools on Azure-hosted deployments, Model Router updates, local inference improvements, and the SDK m

Managed MLflow on Amazon SageMaker AI now syncs richer model metadata (training metrics, evaluation results, inference specs, and lineage) into the SageMaker AI Model Registry, with lifecycle stage pr

GPT-6 Astra from OpenAI is now generally available on Amazon Bedrock. It brings deeper reasoning and sharper judgment to your most demanding tasks, running on the Amazon Bedrock inference engine built

The era of AI inference has arrived. Imagine a healthcare system analyzing millions of data points in real time to accelerate life-saving medical research, or an intelligent assistant instantly resolv

Nvidia is announcing its new Personal AI Router (PAIR), a free tool that syncs up your home computers for tackling local AI inference tasks with tools like Ollama and LM Studio. Let's get the obvious

AI factory deployment is becoming a workload-sizing exercise rather than a custom infrastructure project. Enterprises need infrastructure spanning edge inference and large-model training. Broadcom Inc

Aranya Inc., a startup that says it can transform raw bare-metal servers into custom, production-ready graphics processing unit clusters for artificial intelligence inference in less than two days, la

Enterprise infrastructure is being rebuilt around agentic AI, and the center of gravity is shifting back toward private environments where data, cost and control sit under one roof. As inference workl

Self-hosted speech AI carries an observability trade-off: the numbers that drive capacity planning and cost management stay locked inside the vendor container. Deepgram closes that gap on Amazon SageM

NVIDIA cuML and cuVS now support multi‑GPU all‑neighbors kNN graph construction in UMAP, enabling end‑to‑end scaling of the dimensionality reduction algorithm to hundreds of billions of vectors. The i

NVIDIA Nemotron 3.5 Lightning is an open‑source mixture‑of‑experts language model totaling 30 B parameters with only 3 B active during inference, designed to serve high‑volume, low‑latency execution f

Physical AI is forcing the technology industry to rethink the entire computing stack. Robots, autonomous systems and intelligent devices need economical inference, secure data access and infrastructur

The market for data center GPUs is evolving beyond individual chip specifications into a contest over fully integrated rack-scale systems. As AI workloads scale into the gigawatt range, buyers increas

OpenAI GPT-5.6 Sol, Terra, and Luna are now generally available on Amazon Bedrock. Learn how to select a model, run inference through the Responses API on the bedrock-mantle endpoint, reduce cost with

This post covers Opus 5’s improvements and practical guidance for AI engineers integrating the model into agentic systems and production inference workloads on Amazon Bedrock. See the documentation fo

The math behind reinforcement learning (RL) post-training for large language models (LLMs) is notoriously unforgiving. As frontier AI labs push the boundaries of reasoning and coding models using RL p

When Advanced Micro Devices Inc. held its earnings call in May, Chief Executive Lisa Su told analysts that the current ratio of 4.5 GPUs to 1 CPU will compress toward 1 to 1 as AI agents and inference

NVIDIA’s Rubin GPU, the core of the Vera Rubin platform, delivers up to 10× the agentic inference throughput per watt compared with previous generations, using 336 billion transistors, 224 SMs, 896 Te

Today, GPT-5.6 Sol, Terra, and Luna from OpenAI are generally available on Amazon Bedrock, bringing the smartest family of models from OpenAI yet to Amazon Bedrock’s next-generation inference engine b

As AI infrastructure investment scales globally and inference workloads multiply, cache storage is emerging as the critical data layer that makes AI factories functional, persistent and economically v

As organizations scale modern workloads — from high-throughput databases and network/security appliances to real-time analytics and AI/ML inference — network and block storage performance can quickly

Provisioned Throughput gives you reserved inference capacity for frontier open models like MiniMax M3 and GLM-5.2. Token-based pricing, a 99% uptime SLA, and up to 90% lower cost than proprietary APIs

Chinese AI startup DeepSeek is developing its own inference chips to reduce Nvidia dependence and break U.S. export controls, as China moves to protect its AI ecosystem. DeepSeek has been quietly deve

In this post, we walk through how to get started with MiniMax models on Amazon Bedrock, including the capabilities supported by these models, the service tiers available, how on-demand inference scale

In this post, you learn how to use the new MLflow integration with Amazon SageMaker AI optimized inference recommendation jobs and Amazon SageMaker AI benchmark jobs to automatically stream experiment

NVIDIA Confidential Computing addresses data privacy and security concerns for AI workloads by protecting data during inference and engagement with models, offering high-performance protection in loca

Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference. O

As AI moves from model development to production inference, compute demand is accelerating and shifting toward continuously operating AI factories that generate tokens at scale. This shift requires ac