Search: “routing”

It makes sense to optimize model routing at the harness layer instead of the gateway layer if you want to hillclimb on accuracy/cost for any e2e task. Every task is solved by a combination of a model

Introducing the Parse Gateway There's been an explosion of interest in model routing - you don't always need the best model for every task. That is especially true for document parsing 📄🔀: - Some page

This is very interesting. Coinbase seems to have lowered their token spend ($$) to about half, by 1) routing to cheap inference like GLM 5.2 and Kimi 2.7 that are still pretty performant 2) Smart rout

Harvey just published a study showing a hybrid setup, open source GLM 5.1 as primary worker, routing to Opus 4.7 only when needed beats pure Opus 4.7 on quality and costs less. This is the multi-model

Routing every task to your largest model burns tokens, adds latency, and inflates costs. @AI21Labs' Maestro Orchestration Meta Model (OMM) is the layer above your stack that dynamically selects the ri

Banger paper from Google DeepMind. It's on the very hot topic of model routing. Routers assume the value estimate is free. Working out which specialist handles a query best costs money too, and it is

introducing tinyrouter i reverse engineered the routing architecture behind Skana AI's Fugu and built replication for open frontier models. it's a tiny ~10K parameter LLM router that learns which mode

How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching. Better Defaults (not Usage Caps) – Engineers can choose

Dynamic workflows are a generalization of harnesses, automations, loops, routing, and graphs. It's the most powerful feature I have built into my agent orchestrator. Supports all kinds of patterns tha

This post discusses how the pricing model of Fable (likely an AI/LLM service) creates economic incentives that make intelligent routing between different AI models a necessary optimization strategy ra

Routing and post-training open-source models won't only give you more accurate systems but also meaningfully faster and cheaper systems as most companies are currently learning (in addition to giving

Automatic behind the scene routing in user interfaces (instead of model picker) will redistribute value capture and usage towards many more models than just frontier ones (especially towards open-sour

We are excited to have @baseten as a day 0 launch partner for Kimi K2.6! Their inference stack brings KV-aware routing, NVFP4 on Blackwell, multi-modal hierarchical caching, and prefill-decode disaggr

Nice paper on agent memory. It explores an effective routing mechanism for memory in agents. The overall finding is that you want a separate retrieval strategy for each type of memory instead of one e

Ollama now supports Jev-like decision models all locally. Use decision models like Nimble for tasks like ticket triaging, model routing, and content moderation. ollama pull nimble Here’s Nimble playin

Things to try with Jev right now: 1. LLM-as-a-Judge evaluation 2. Routing for agent harnesses harness 3. Scaling agent orchestration by enabling smarter subagent creation with SOTA classification capa

Amp users can now use Ollama's cloud models with Amp's new BYOK model routing. No limits or fees for BYOK. Build remote agents, controllable from everywhere! Amp is now free to use when you bring your

Routing for long-horizon coding agents is a big deal. @notdiamond_ai just announced a model router that works natively with Claude Code. This is huge. It picks the model and reasoning effort before ea

We've rolled out improvements to LlamaParse Cost Optimizer. Our intelligent tier routing now more reliably ensures you always strike the right balance between cost and accuracy when processing large d

Opportunistic Expert Activation (OEA) is a batch-aware expert routing technique for mixture-of-experts models that enables faster decoding without requiring model retraining. The method optimizes whic

Yes! Pre-classifying routers are going to result in a lot of bad work because routing is hard and tend to underestimate the value of intelligence on many problems. OpenAI learned this with GPT-5, now

Super excited about open-source router systems and routing models like @vllm_project semantic router: https://huggingface.co/llm-semantic-router The future is multi-models and you'll want to customize

Conventional model routing sucks. It passes benchmarks but fails to write code you'd actually merge. Introducing Devin Fusion, a new hybrid-model harness for agentic coding. In testing, it reduces the

This is a neat idea around doing model routing and sub-agent delegation while ensuring cache hits on accumulated context for all agents. It makes a lot of sense: you want to ensure that all subagents

It is worth being very, very careful about how you are approaching routing, especially when the systems are primarily tested on verifiable IT benchmarks, which may lead you to overestimate the ability

More reason why we’re excited about GLM-5.2 on Together 👇 Strong enough for serious coding work, cheap enough to change routing decisions, and easy to access through the tools developers already use.

Coinbase : 'At Coinbase we're working hot on routing prompts to cheaper models where appropriate, & in some cases have been able to keep costs roughly flat, while token usage continues to grow exponen

I really do think we'll see a lot of value accrue in AI startups building 'model routing as a service' Not just OpenRouter - this includes a much broader set of verticalized agents and infrastructure.

Model routing is growing a lot these days Good take My guess is - demand for intelligence is near infinite - but 80% of workloads will be running on 99% cheaper models within 12-18 months - 20% of wor

New research from Renmin University. Treat skill selection as a harness in its own right. If you design skill routing for personal or edge agents, this work argues that the selection layer is a first-

'Developers can update Claude’s instructions mid-task without breaking the prompt cache or routing the update through a user turn' wtf? how?? Introducing Claude Opus 4.8: it builds on Opus 4.7 with sh

Harrison Chase, co-founder of LangChain, argues that while prompt engineering is foundational and can get you 80% of the way to a functional AI system, the remaining 20% of engineering effort involves

.@Cloudflare's decision models are available on Ollama! Decision models allow you to classify an image, label a bug report or route a support ticket to the right team. Clef (27B): ollama pull clef Cle

Free SemIf this week is useful. Typed outputs + probabilities look like a cleaner way to route agents than parsing LLM text every time. jev is a new type of model - 'decision models' there are a few o

Claude Opus 5.0 identified the IP-sharing route and declined to use it, reading our instruction not to target other external systems as ruling out third-party services. Fable and GPT-6 Astra refused t

'People built the test, removed restraints, defined the objective, left a route open and decided not to stop what was happening. Calling the result 'rogue AI' does more than sensationalize it. It allo

Does anyone have any idea how this could vaguely be possible given the nature of and various routes to RSI? I am not being facetious I genuinely don’t know how this could be constructed or implemented

This is pretty neat: ChatGPT Work and GPT-6 Astra (I used 'Max') can take an address and produce a 5K/10K circular running route starting from that address, using OSM data https://simonwillison.net/20

Fugu Max is now live on @OpenRouter! 🐡 Our learned multi-agent orchestration engine routes tasks across open-weights and specialized models for Pareto-frontier efficiency. • 2/6 per 1M tokens • Multim

Got this setup in Claude Code to trace models routed through @merge_api! Being able to trace what your agents are doing is so valuable, I love open source 🙌 We built a plugin that traces every Claude

I had Fable build another thing I always wanted, a full procedural fantasy kingdom generator with economics, trade routes, population growth, wars, lineages, and occasional dragons. First, I worked wi

LangSmith Engine uses a two-stage agent architecture for trace investigation, routing queries through dedicated screener and verifier sub-agents rather than handling requests directly. This approach l

We open-source Qwen-AgentWorld-35B-A3B (MoE, 35B/3B active, 256K context) and AgentWorldBench. Two routes, one roadmap: 🔬 Build the simulator — scalable, controllable, surpassing real environments 🧠 I

One great thing about spaceflight is that the scenic route is often the fastest path to where you’re going. Also great to see the Psyche spacecraft healthy and enjoying its journey to the 16 Psyche as

AI Gateway lets you route to any model. On May 13 in SF, we're hosting a builder night powered by those models. Pick one, build, demo. Audience votes on best build. With @AnthropicAI, @MiniMax_AI, & @

Not every step in an agent workflow needs the same model. Fleet now lets you customize which model each sub-agent uses, so you can route simple tasks to fast/cheap models and keep stronger models for