Search: “inference”

Opportunistic Expert Activation (OEA) is a batch-aware expert routing technique for mixture-of-experts models that enables faster decoding without requiring model retraining. The method optimizes whic

This post from Ollama's official X account discusses performance improvements for the Gemma 4 model, likely covering optimizations related to MLX (Machine Learning eXperimental framework) and MTP (Mul

ParallelKernelBench is a benchmark tool or methodology developed by Together AI for evaluating the performance of parallel kernel execution in machine learning systems. The benchmark likely measures m

Thrilled to announce the Wearable AI Workshop at ECCV 2026 🎉 that we're organizing with an awesome group of folks across Meta Reality Labs, AMI Labs, HKUST, Georgia Tech, UCF, and U. of Edinburgh. If

Fireworks AI announced GLM 5.2 Fast, an optimized version of their GLM 5.2 model that maintains the same quality and capabilities as the standard version while delivering significantly faster inferenc

We're coming out of stealth. We've built our first racks after a successful A0 tapeout, 1B+ in customer contracts, and 800m raised. Early customer tests show us achieving SOTA throughput, latency, and

Chess engines tell you the best move. But grandmasters are human, they don’t always play it. So I built 'Kibitz': a human move predictor for chess broadcasts. I trained this model on my Nvidia RTX 508

Llama.cpp has merged DFLASH support into its main branch, which is a development update related to optimization or acceleration technology for running large language models locally. The post expresses

MASSIVE NEWS has partnered with NVIDIA to advance local AI deployment as a default option, addressing the shift toward on-device and edge AI processing rather than cloud-dependent models. This collabo

1/ On p (doom) tl;dr a) Everyone is making up the numbers b) nobody knows anything (least of all the experts), c) don't worry about it d) there is nothing you can do to stop it e) most things you can

As token usage explodes, model choice becomes product strategy. Teams are already testing models like GLM-5.2 because they want frontier quality, better tokenomics, and more control over cost, data, a

Many companies could build their own Workday or whatever, too, using open source alternatives. They don't do that for a wide variety of reasons. Same dynamics apply in AI, only more so (upgrade cycles

NVIDIA has released GLM-5.2 NVFP4, a new quantization format for large language models. The announcement invites comparison with alternative quantization methods for performance and efficiency evaluat

In HF GGUF section of models, we are emphasizing MTP heads with its own sign 𝗠𝗧𝗣 llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance j

lots of folks prepping talks next week (congrats!). Some thoughts from RLing on thousands of hours of engineer- and researcher- focused talks: - AI generated svgs > AI generated imgs. MAXIMUM 4 ai slo

Video models are hard. Real-time video models are even harder. Loved this breakdown by @itunpredictable on how we productionized Runway Characters, our real-time interactive avatar model. We had to fi

While we eagerly await Fable 5's return, our agentic WebGPU kernel optimization framework kept running. Opus 4.8 picked up where Fable left off, pushing Liquid AI's new LFM2.5 230M to an unbelievable

GLM-5.2 achieved token generation speeds of 318 tokens per second, surpassing the previously reported 280 tokens per second benchmark, with indications that further performance improvements are possib

AI21 Labs announced achieving the #1 ranking on DeepResearch Bench II without developing any new agents, suggesting they improved performance through optimization of existing systems or methodology ra

.@cristianoamon and @ClementDelangue drop a 'one more thing' at @Qualcomm investor day: Qualcomm and @huggingface partner to bring open models to Qualcomm platforms and beyond. - Users will be able to

I decided to run my knowledge base inside a @Cloudflare Worker👀 How? Surprisingly easily: • Spin up a @qdrant_engine cluster and grab the credentials • Process incoming files at the edge: parse with @

Krea 2 is back and this time the weights are OPEN. This open source release ships with two models designed to work together — @krea_ai 2 RAW and Krea 2 Turbo. RAW is for training, Turbo is for inferen

What should a world model for agile quadrotor control actually provide? 📄 Arxiv: https://arxiv.org/pdf/2606.23444 🌐 Project: https://pratyaksh10.github.io/skyjepa-project-page/ 💻 Code: https://github.

A year ago this would have been an obvious closed-model task. Now GLM-5.2 can read the issue, reason through the scene, patch the code, and keep moving on Together AI. @togethercompute + @Zai_org GLM

// Evolving Meta-Skill for Multi-Agent Systems // Can a multi-agent system get better at orchestration without touching a single weight? Automatic MAS generation has been stuck between two bad options

We, along with @zhijianliu_ and @jianchen1799, are excited to share these models on @huggingface for the entire community to use. We look forward to getting speculative decoding speedups into the hand

Very pleased to hear Anthropic have walked back this policy https://simonwillison.net/2026/Jun/11/anthropic-walks-back-policy/ BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks yo

As vertically integrated platforms start to dominate they lock out third party access to the most valuable portions of the platform. Of course, Anthropic is has the right to implement whatever policy

Coding agents break when models are 'almost' bug-free. But almost valid JSON is just not the same valid JSON. Fun piece here from @akshay_pachaar shows why SFT can't fix this, and how GRPO trains agai

In good faith and with no judgment (mistakes happen), I truly hope that Anthropic will hear the feedback and change course on this. Anthropic is a company that has been raising awareness about AI mani

BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice. We

I'm really not interested in testing a new closed-weight model in the expensive tier. What's the point? I know what will happen: 1. They present the model. 2. The model beats the competition on benchm

blast from the past 3.5 years ago; some things have changed (esp. coding and math, via neurosymbolic techniques) but many haven’t: Bottom line: From the outset Large Language Models like GPT-3 have gr

We're building this at LangChain Fleet lets you create and manage a fleet of agents. Each agent specializes in a workflow, e.g. inbox management, blog writing, competitor research, candidate recruitin

VLA-JEPA just dropped in LeRobot 🤖 What makes this model special is that it does not just learn what action to take from a given observation, it also leverages a JEPA world model to learn action-relev

Your margin is my opportunity: AI version… The biggest surprise of 2026 is that the capability gap between the best open-weight/source models and the best closed models has narrowed much faster than t

Highlights: 👉 Design-first generation for ads, posters, packaging, and product visuals 👉 Strong typography and multilingual text rendering 👉 Precise layout and color-palette control for brand workflow

Introducing Ideogram 4 from @ideogram_ai on Together AI, an open image model built for design with strong text rendering, layout control, and native 2K image generation. AI natives can now use Ideogra

Token costs are why there will be no saas apocalypse / good dev tools are cached intelligence for agents! The popular theory goes: agents can write code, so they'll just rebuild every tool from scratc

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise work

NEW: NVIDIA ships 550B MoE open model for long-running agents. Very exciting times to see more open models to support local long-running coding agents. Today we're shipping Nemotron 3 Ultra. A 550B Mo

@nvidia @nebiustf More info on the new Nemotron 3 Ultra: https://x.com/NVIDIAAI/status/2062521325076299981?s=20 Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built

NVIDIA’s Nemotron 3 Ultra is available on Ollama’s cloud! Try it 👇 Claude Code: ollama launch claude --model nemotron-3-ultra:cloud Hermes Agent: ollama launch hermes --model nemotron-3-ultra:cloud Op

Today I'm launching a new project called SynthTraces 🔥 It is a minimal codebase to generate synthetic coding agent session traces using Pi (from @badlogicgames) I wanted a large number of coding-agent

Harvey just published a study showing a hybrid setup, open source GLM 5.1 as primary worker, routing to Opus 4.7 only when needed beats pure Opus 4.7 on quality and costs less. This is the multi-model

This post highlights four recent improvements to the ecosystem of open-weight large language models designed to run efficiently on consumer hardware, covering developments that make local LLM deployme