Has anyone actually benchmarked where the 'big-model orchestrator + local-model worker' split breaks down?

I keep seeing the "use a big model via API as the architect, run local small/mid models as workers" pattern recommended for people with modest local hardware. I've been running it myself (orchestrator on a hosted model, local Qwen-class 27B workers doing scans/refactors/test runs), and it works - but I have a nagging feeling the win is smaller than the hype once you account for round-trip latency and the fact that the orchestrator still has to read everything the workers produce. What I'd actually like to see from this sub: has anyone measured, with real numbers, the point where the local worker becomes the bottleneck vs. where the orchestrator's reasoning is genuinely load-bearing? Specifically: - token/s on your local card when the worker is doing high-volume boilerplate vs. when it's doing judgment calls - whether the orchestrator-via-API + local-worker setup actually beats just running a bigger local model end-to-end (if your VRAM allows) - failure modes you hit that pure "all local" or "all API" didn't have Not looking for a recommendation - I want data/war stories. If you've A/B'd it, what changed your mind? submitted by /u/InterviewDesigner777 [link] [comments]

Source: r/LocalLLaMA | 2026-07-31