Search: “inference”

On Aug 13 2026, OpenAI announced a preview of its “Ultrafast” mode for the GPT‑5.6 model, called Sol, which claims inference speeds up to 14× faster than current releases. The feature is initially rol

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B

💡The world needs an open-source platform, and that’s exactly what we’re building to give our customers more choice and the flexibility to choose the right model for the right task. As part of this, we

The math still ain’t mathing. Recently we estimated global (ex China) AI revenues of around 200bn annualised, based on four different sources. Just come across a fifth, based on Nvidia inference sales

two weeks ago i went on @swyx's pod and said some things that i... should not have said. a lot has happened since then, i owe you all an apology. i'm sorry that i was right about every single thing. a

I'm actually fairly bearish on frontier lab valuations. I've never seen the reasons articulated to my satisfaction, so before I go to sleep, I wanted to quickly jot down my thinking here. The basic is

We ran a large-scale distillation attack on the Kimi K3 technical report by reading it in parallel at the Hugging Face Journal Club :) https://youtu.be/MW8-kqd2SD8?si=jSKDogcUWbJ8N2k7 Our main takeawa

Kimi K3 is now available directly from its Hugging Face model page through Together AI, give it a try! Kimi K3 in @huggingface Inference Providers is live via @togethercompute 3/M input tokens, 15/M o

Kimi K3 is now live on @togethercompute! Happy to have Together AI as our day0 launch partner, giving developers immediate access to K3 through high-throughput inference optimized for coding agents an

NVIDIA AI has released Cosmos 3 Edge, a 4‑billion‑parameter open‑source world model designed for on‑device inference. It enables robots to learn and act, autonomous vehicles to interpret road scenes a

SSI should open-source an opus-tier model. 1. it nearly closes the us/china gap on the os frontier 2. they don't want to waste compute allocation on inference, releasing os will enable continued full

Trending AND fastest-growing in the same month? Benchmarks change daily, but only @tryramp has the database of real receipts to track this. We can confirm: demand for open-weight inference and trainin

New model for AMD Strix Halo users: My 198B Step 3.7 Flash release was a big hit, but this one may be even better: 298B-parameter Hy3, now running on a new 2-bit FPX codebook designed to map efficient

Together AI announced provisioned throughput capabilities, a feature that allows users to reserve dedicated model inference capacity at predictable costs. This offering enables organizations to optimi

This is a big update that I’m excited to finally share. We launched our first dev platform for AI media in 2020, offering a simple API and a diverse set of models. Back then, we built a powerful SDK a

// ReContext // Models now support 128K context windows and still fail to use evidence that is already in the prompt. Where is the gap? New paper introduces ReContext, a training-free inference harnes

Don't forget, we'll be in Paris for @RaiseSummit. Will we see you there? Going to be in Paris for @RaiseSummit? Join us and @nvidia to hear from leaders at both companies on the future of AI inference

TTT-Discover is a method that enables models to learn and discover patterns during test time rather than only during training, allowing for adaptation to new data distributions at inference. The appro

Aurora is a unified training-serving system that integrates reinforcement learning with adaptive speculative training to optimize large language model inference and training efficiency. The system dyn

Untied Ulysses is a memory-efficient technique for context parallelism that processes attention heads in chunks rather than sequences, reducing memory overhead during transformer inference and trainin

Together AI announced a Series C funding round and shared details about their company vision, current capabilities, and future direction in open-source AI development and inference optimization. The a

After 18 months of writing, coding, and experimenting, Build a Reasoning Model (From Scratch) is finally out! My first copies just arrived! 📚 440 full-color pages. Inference scaling, reinforcement lea

There's a big difference between a single model call and serving an agent at scale. @ZainHasan6 breaks down what actually changes. Catch our team this Monday at 9 a.m. PST for their open-source infere

VCs are now sharing screenshots in group chats of Claude discouraging investment in open-source AI infra startups and models. Obviously there is an absolute EXPLOSION of pitches in inference companies

NVIDIA's GLM-5.2-NVFP4 is a quantized version of a large language model optimized for inference efficiency using NVIDIA's proprietary quantization format. The model is hosted on Hugging Face and repre

400T tokens is what production adoption looks like. Teams are moving real workloads to open models because they want frontier quality, better tokenomics, and more control over inference. Together AI g

5.2 could be better with more RL ... Deepswe's benchmark results are my own experience. I've used all models, GLM 5.2 ≈ Claude Opus 4.6–4.7. Kimi 2.7 code more like inference optimization. Looking for

The Top AI Papers of the Week (June 14 - June 21): - PreAct - SpatialClaw - Back on Track - OpenClaw-Skill - From Trainee to Trainer - Compositional Skill Routing - Can LLM Agents Infer World Models?

This is what the AI engineering ecosystem looks like in one room. 60+ Gold Sponsors are joining us at AI Engineer World's Fair, spanning infra, inference, agents, data, eval, and observability. The wh

deepagent for competitive analysis Introducing 'Signals of AI' - DeepAgent powered by - @nebiustf for Inference - @NVIDIAAI Nemotron 3 Ultra - @tavilyai for Web Search - @LangChain for the core deepag

MiniMax M3 arrives with MiniMax Sparse Attention (MSA), 15.6x faster decoding at 1M tokens. We're partnering with @MiniMax_AI to power the inference behind this week's launch. Head to http://minimax.i

Make sure to join our live Spaces chat on MiniMax M3 starting in 4 hours. You can pre-submit questions by replying to this tweet. MiniMax M3 is live and Together AI is powering its inference 🚀 Tomorro

We'll get into @MiniMax_AI M3's model performance, the MSA architecture and what it means for long context, and how Together is optimizing inference and KV-cache for this new architecture. Set your re

love it! @ggerganov 's llama.cpp delivering! pibot is now running fully local, using parakeet for STT, qwen3-tts for TTS, and Qwen 3.6 as the local multi-modal LLM via llama.cpp. The STT and TTS infer

pibot is now running fully local, using parakeet for STT, qwen3-tts for TTS, and Qwen 3.6 as the local multi-modal LLM via llama.cpp. The STT and TTS inference engines are Rust/mlx-c based. Ported fro

Took me a while to figure out what all the ESMFold2 rage was about. At first, the benchmarking data didn't look super remarkable to me but it turns there are many impressive aspects: - Fully open sour

Fast, faster, Qwen. 🚀 Thrilled to see Qwen3.5 reaching a record-breaking 580 tps for agentic workloads on the TokenSpeed engine! This milestone wouldn't be possible without our incredible partners. Hu

Hugging Face has released 1-bit and Ternary Bonsai Image 4B, a new family of lightweight image-generation models optimized for efficient diffusion inference. These models are designed to deliver high-

RWKV-7 G1g is here: the world's best pure RNN LLM, and a competitive LLM in general. Try https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2 for bsz16 7B inference. G1h in June 🙂 p.s. const 15000+tps

Nathan's @cursor_ai team didn't prompt-engineer their way to Composer 2.5. They trained it. The massive RL program runs RL rollouts on Fireworks, alongside production inference. 'Comment 🔥 to see my p

llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance jump with these changes is massive and elevates local inference on commodity hardw

5/5 What this unlocks: - Change one prompt, serve cached results for all other components. - A/B test any component against identical upstream outputs. - Run best-of-N inference without N branches col

Batching for vision models is now available in Beta with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mod

DeepSeek maintains the highest efficiency in terms of active parameters relative to total model size, outperforming competitors in the ratio of parameters actually used during inference versus total t

Most teams can pick frontier models. Fewer can run them at production scale without hitting constraints in latency, throughput, and governance. Fireworks AI on @Azure AI Foundry provides the inference

We published new research on how we serve post-trained Qwen3 235B models on NVIDIA GB200 NVL72 Blackwell racks. GB200 is a major step up over Hopper for high-throughput inference on large MoE models,