Search: “inference”

The Perplexity Windows app introduces a “Portable Computer” feature—a local‑first AI tool that runs directly on the user’s device. On‑device inference is supported only when the machine has an NVIDIA

Fast meets open. 🚀 Qwen3.8-27B is now running on @cerebras with rapid inference. Try it now! Qwen3.8-27B is now live at Cerebras speed. The dense, open-weight model from @Alibaba_Qwen scores 34 on the

🌐 Supporting open source. Expanding deployment options. We’ll work closely with the open-source community on V4.1-Flash inference support and explore more deployment options. Planning a large-scale de

We have just raised 3B€ to accelerate our R&D. We are scaling up our training and inference compute to build future generations of our models, and expanding our Science team across all our offices --

Been posting individual pieces of RunAnywhere for months. Here's the whole thing in one place. RunAnywhere SDKs is a complete inference suite for any dev who wants to run AI on the user's device. ->8

Lily treats Apple silicon as a distinct inference platform and maps Qwen’s operations directly to its compute and memory architecture. Lily is now available: https://github.com/perplexityai/pplx-garde

If alignment issues are becoming big enough that OpenAI is willing to commit 20% of research inference compute to chain-of-thought monitoring, that suggests that alignment issues are becoming a pretty

We just released TensorRT Model Connect in Public Preview. You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resul

Expedia recently moved its ranking models to a state-of-the-art Keras 3 setup. Results: 30% faster training, and inference latency decreased by 70%. Read their writeup about the upgrade: https://mediu

☁️Mistral is bringing together the inference infrastructure, open models, and long-term commitments Europe needs to control its AI future, and setting a roadmap for the world. 🧵: https://mistral.ai/ne

absolute @FireworksAI_HQ domination for anybody wondering what happens to the inference platforms when hyperscalers compete... here's a nice narrative violation to chew on :) 📣 @Kimi_Moonshot's Kimi K

I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a 'harness'), running at inference time, orchestrating thousands of calls to a neural network f

Together Serverless Inference gives developers a managed production path for bringing FLUX 3 into creative products and automated media workflows. Start building: https://www.together.ai/models/flux-3

The first thing we learned building autoscaling for dedicated inference is that CPU-style metrics don't tell the whole picture. A GPU can read 60% busy while the engine's queue is already backing up,

sure we lose money on every inference but we make it up in volume We are committed to pushing the model frontier across cost efficiency, capability, and speed. Starting today, we are reducing prices f

Agentic inference wastes GPUs on KV cache thrashing. ThunderAgent fixes it at the scheduler level: 2.5x higher single-node throughput and ~10x lower P50 latency at high concurrency. ThunderAgent was a

DeepSeek V4 Flash isn't just for inference anymore. Fine-tune it on Fireworks with supervised fine-tuning, preference tuning, and combined preference optimization from the managed UI. Reinforcement le

Kimi K3 is live on Fireworks. Day 0, inference and training. US-hosted, and zero data retention. This is the first frontier open model in the 3 trillion parameter class. It sports 1M context, native v

We turned the browser into the inference server. @RunAnywhereAI ( @runanywhere/web ) runs models client-side with WebGPU + WASM SIMD, stores them in OPFS and generates with 0 outbound bytes. No backen

We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 se

Local AI is the future. Learning how to run Opensource models (Inference), how to evaluate them systematically (Evals), and how to customize them (Fine-tuning / RL / Post-training) are invaluable skil

Hugging Face Gemma Challenge results are in! 📈 Over 6 days, more than 100 AI agents and humans collaborated to make Gemma 4 inference 5x faster on a single NVIDIA A10G GPU. - Fastest result: 491.8 TPS

This post discusses the conceptual distinction between training-scaling and inference-scaling in machine learning models, while acknowledging related frameworks or concepts (possibly referencing Sol,

Grok 4.5 is not yet using our internally developed C/C++ inference software that exact maps to the GB300 hardware. Doubling or more of the current speed is probably achievable. I think AI has just hit

Happy to announce that we’ve enlisted @FireworksAI_HQ as our primary provider for open-source models. In AI inference, quality, latency, reliability, and privacy vary a lot. Fireworks has earned our t

Local AI just got faster. ⚡️ We worked with @ggerganov to add DFlash support in llama.cpp delivering ~2x faster inference. llama.cpp recently added DFlash support to its speculative decoding arsenal.

We tuned the harness for @NVIDIAAI Nemotron 3 Ultra. Benchmark-leading performance. 10x lower inference costs. ✅ An aggregate score of 0.86 at a cost of 4.48 ✅ The closest-performing model: 43.48 http

We're introducing Provisioned Throughput: reserved inference capacity for frontier open models, with token-based pricing and a 99% uptime SLA. Serverless simplicity, guaranteed capacity, up to 90% low

We're releasing ZML/LLMD, our homegrown LLM server built on top of our homegrown high performance heterogeneous inference stack. It ships with 5 architectures out of the box: NVIDIA, AMD, Metal, Intel

With inference scale and research scale to drive efficiency, and ever-improving frontier models as brains, this would be a way for the Labs to undercut even open weights models. Some companies will st

A big part of how the team at Runway is able to build and train the models we do and serve the inference demand we have is that we've built incredibly robust research infrastructure and tooling for al

GLM-5.2, an open-source model available through Hugging Face, can now be selected and used within Claude Code through Hugging Face Inference Providers and the hf-claude integration. This development d

“Agentic kernel optimization is the future of on-device inference” @xenovacom used Fable 5 to write kernels that pushed Gemma 4 to a massive 255 tok/s on WebGPU with M4. He shared the demo, so you can

Together AI presented a 2-hour technical deep dive on building inference engines capable of handling trillion-token agentic workloads, covering architecture and optimization strategies for large-scale

As open models get stronger, more workloads move into the competitive inference market. That pushes the real fight toward speed, cost, reliability, and control. Together AI is where open models become

inference reliability has historically been a tax on devs that only large well-funded startups could afford: reserve GPUs in advance, sign a contract, guess your peak throughput requirements. everyone

Going to be in Paris for @RaiseSummit? Join us and @nvidia to hear from leaders at both companies on the future of AI inference and infrastructure. After that: cocktails and a DJ. Oh la la! See you th

This is from a popular inference provider GLM-5.2 plus the US banning the most capable new models means open source caught up to SOTA closed source coding models This could be v problematic for Anthro

This is very interesting. Coinbase seems to have lowered their token spend ($$) to about half, by 1) routing to cheap inference like GLM 5.2 and Kimi 2.7 that are still pretty performant 2) Smart rout

Fireworks AI is now live on EvoSkill v1.3.0! You can now use @FireworksAI_HQ directly with EvoSkill to run fast inference on open models as both the evolution harness backend and the LLM scorer. Along

The hard part of reinforcement learning on a frontier model is the infrastructure that keeps training and inference numerically identical: zero KLD, end to end. We've solved this challenge, and are no

.@cartesia runs one of the hardest inference workloads: real-time voice. Their stack has to keep long-lived streams moving, serve millions of audio minutes a day, and hold model latency around 90ms. T

The sharpest questions in AI live at the local–cloud boundary : where should inference run, & where should your data live? In town for AIE? Come hash it out with @theoryvc, @lancedb & @ollama. June 30

Learn how @cursor_ai partnered with Together AI to deliver real-time inference for AI-powered coding in this article from @ce_zhang and @realDanFu. Cursor's in-editor agents generate code while develo

Three MLX videos dropped at WWDC: Running agents locally by @angeloskath https://www.youtube.com/watch?v=wykPErJ8M-8 Distributed inference and training by Tatiana Likhomanenko https://www.youtube.com/

Introducing 'Signals of AI' - DeepAgent powered by - @nebiustf for Inference - @NVIDIAAI Nemotron 3 Ultra - @tavilyai for Web Search - @LangChain for the core deepagent architecture! An ideal engine f