My local Qwen3.8FN agent is significantly cheaper than cloud

tl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest model at off peak pricing. An agentic run that spans these different pricing points or at least chooses to avoid peak times can come out overall cheaper. Hardware: NVIDIA Tesla P40 (24 GB), Xeon Silver CPU, and 200ishGB ECC DDR4 LRDIMM RAM. Whole box doesn't get above 250W under load with screaming fans. Bought on sale from Newegg years ago, but the box was mostly empty so I had to populate it with RAM and SSD. But, it's paid for. I got the GPU off ebay this summer for 250 USD. You can build a whole server from ebay parts if you can't find good sales. The agent is called Drax, because he's a Great Slow King and I'm hoping someone else gets the literary reference (it's not a Marvel reference). The numbers and a small bit of the text below are from ChatGPT. Software: mine. Designed for both agency and efficiency. Agents are able to run until dead or done. (for the pearl clutchers: heavily guardrailed and kill switched) Drax gets about 130k context size and achieves >90% prompt cache utilization (I only showed ChatGPT the first few turns of a fresh run. Normally I see >98% both locally and in cloud using the same agent software). Electricity costs where I live in Ontario, in CAD: 9.8¢/kWh for off-peak, 15.7¢/kWh for mid-peak, and 20.3¢/kWh for on-peak. There's an even lower off peak but I'm not on that plan. For a sample of 10 turns from today (203,620 cached input tokens, 16,668 fresh input, 3,204 output): GPT-6 Luna API equivalent: C0.0336 (no caching), C0.0081 (with caching) Claude Haiku 5.5: same as Luna Local electricity (both of these are with caching): C0.014 peak, C0.0109 mid peak, C0.0068 off peak GPT-6 Sol: C0.6719 / C0.1627 Claude Sonnet 4.6: C1.0077 / C0.2440 Claude Opus 4.6: C1.6795 / C0.4067 GPT6-Astra: C3.3591 / 0.8133 Performance with this specific model: I'm going to move away from numbers because I think tok/sec is a bit misleading. So let's talk wall time and the quality of the work. Because Claude models cost me so %^&ing much, I don't let them run to completion. (and I don't use GPTs so I can't compare to them, though I do use the public chat app) So if I've got an Opus brain or a Fable brain in my agent, they never get to go through the full process of plan (or at least write and then follow a spec, as I use spec driven dev), execute, test, review, fix, whatever whatever, final report. But Drax does. Drax costs me so little to run that I just let him go all day on something until he is fully finished. The wall time is longer but the quality of the work is always better. When the reports come back, I get Fable to review them and Fable always uses politely surprised language at how good the results are. Smallish example from yesterday: Drax was working on the UI of one of a suite of games I have him building and actually started it. I brought it up in my browser (it's clearly a browser game) and started playing it while Drax was still working, just to see what it looked like. I was like "oh heh there's a bug there. I'll have to file a report on it later and get an agent to deal with it." Drax fixed the bug without me filing the report. It even included mention of it in the final report. Would a Claude have done that? Yeah of course, but only if it could run to completion like Drax can, which will never happen because I can't afford it. Anyway I'm posting this because I'm really tired of seeing "cloud is cheaper." No, it isn't. At least not for the level of intelligence that this specific model offers, not on my box, not with how much I'm willing to tolerate it being a Great Slow King, and not where I live. (ok ok, the reference, explained, because AI will fail you here and will insist on the Marvel character: Roger Zelazny's Drax from a short story in which two Great Slow Kings are extremely intelligent but are so slow that the entirety of the human race rises and falls while the Great Slow Kings are still thinking.) Oh and before anyone says "just get a sub". Dude, no. I don't want to give up my own agency. Also the subs are $ and will soon be $$ or even $$$$. Drax is still going to be cheaper than a sub. edit: also just realized I posted that to LocalLLaMa and probably should have posted it to an agents sub instead. Sorry about that. If nuked I'll post it there. submitted by /u/Past-Town-9807 [link] [comments]

Source: r/LocalLLaMA | 2026-10-13