CompanyAnthropic7 recent entries17 Aug 2026Long Review: Qwen 3.8 27B is VERY good at tapping into it's real-world knowledge. It's 'overthinking' brings it to Sonnet level performance with the potential for Opus level results.Hi all! I finally just got around to testing out Qwen 3.8 27b. I'm using Unsloth's UD-Q8_K_XL quant as a sit-in replacement to Qwen 3.6 27b, same quant size. Wow -- this thing isn't messing around. I →17 Sept 2026Qwen Next 3.8 / Claude Opus level local model - What to buy in order to deploy?Hey Reddit. This is a post asking for advice / user experience. The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence,
CompanyOpenAI2 recent entries19 Aug 2026NVFP4 on VOLTA! Despite being built for Blackwell, I made four 2017 V100s run Qwen 3.8 NVFP4 natively and match my $6000 RTX 5090.Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It →13 Oct 2026My local Qwen3.8FN agent is significantly cheaper than cloudtl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest mo
CompanyGoogle3 recent entries18 Aug 2026Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all→4 Oct 2026Poor People Vulkan GPUs listHelp with this list. Give me your recommendation on 'not supported anymore' GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with so→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res
CompanyMeta8 recent entries16 Aug 2026I'm going crazy. Why does it use the CPU...I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload →18 Aug 2026Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all→4 Sept 2026Qwen3.8-Flash-Next: 256k context, 16tok/s on DDR4 and a Tesla T4I've got an refurb Dell R740 running Proxmox that I put a Tesla T4 in, mainly to run some CTC local transcription work, but thought it would be fun to try DS4 when it came out, and it was appalling at→6 Sept 202648 tg/s 440 prefill on my grandma's cluster (2xP40) (sort of)TL;DR: switching KV cache to f16 may give a boost in speed if using MTP and ngrams. I have a self-built 'AI mega-cluster' with 2x P40s on a cheap Chinese motherboard and a Xeon CPU (around $1,100 to b→11 Sept 2026Running Qwen3.8-27B-Q4 at max context on a 32 GB GPU while avoiding kvcache quantizationI can't stand kv cache quantization. Even at q8_0, I can feel the difference. But realistically, when running Qwen3.8-27B-UD-Q4_K_XL on my 32 GB GPU, I only have room for ~170k tokens (with mtp and mm→19 Sept 2026I turned an asymetric pair of Tesla V100s PCIe both (16 GB + 32 GB) into a surprisingly capable local LLM lab — 1.38k prompt tok/s, 40 decode tok/s with qwen3.8 27B Q6 and Q8...TL;DR: I run a mismatched Tesla V100-PCIE pair—one 16 GB card and one 32 GB card, 48 GB total—in a Proxmox/LXC-based local-inference lab. The practical winner so far is a recent CUDA build of llama.cp→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→13 Oct 2026Problems with Tesla V100 SXM2Hi guys! I got a problem with one of my V100 SXM2 Modules. In 'nvidia-smi' its shows with 'Pwr:Usage/Cap' as 'N/A / N/A'. When I try to use it i get the following error in dmesg: NVRM: Xid (PCI:0000:0
CompanyDeepSeek6 recent entries9 Aug 2026DeepSeek v4 Flash 0731 locally on CPUAfter seeing the benchmark results for the full release of DS v4 Flash 0731, I replaced my 2 x 16GB DDR4 ram sticks with 2 x 32GB DDR4 ram sticks to get a max supported of 128 GB RAM, in hope to be ab→16 Aug 2026I'm going crazy. Why does it use the CPU...I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload →17 Aug 2026Long Review: Qwen 3.8 27B is VERY good at tapping into it's real-world knowledge. It's 'overthinking' brings it to Sonnet level performance with the potential for Opus level results.Hi all! I finally just got around to testing out Qwen 3.8 27b. I'm using Unsloth's UD-Q8_K_XL quant as a sit-in replacement to Qwen 3.6 27b, same quant size. Wow -- this thing isn't messing around. I →18 Aug 2026Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all→19 Aug 2026NVFP4 on VOLTA! Despite being built for Blackwell, I made four 2017 V100s run Qwen 3.8 NVFP4 natively and match my $6000 RTX 5090.Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It →24 Aug 2026The journey of letting Qwen 3.6/3.8 autonomously coding a c compiler.Hi, Back in late march I begun playing around with Qwen 3.6 27b and found like everyone else that it's notoriously good at tool calls, where every model I tried before just derailed after a few turns
CompanyNVIDIA8 recent entries19 Sept 2026Modello senza limitiDa un po mi sono ritrovato in garage un hp z840 da 64gb ddr5 montate (ora) e 2 processori con una gpu un po scarsina una nvidia quadro k680 (da aggiornare) avrei bisogno di un consiglio: quale modello→19 Sept 2026I turned an asymetric pair of Tesla V100s PCIe both (16 GB + 32 GB) into a surprisingly capable local LLM lab — 1.38k prompt tok/s, 40 decode tok/s with qwen3.8 27B Q6 and Q8...TL;DR: I run a mismatched Tesla V100-PCIE pair—one 16 GB card and one 32 GB card, 48 GB total—in a Proxmox/LXC-based local-inference lab. The practical winner so far is a recent CUDA build of llama.cp→26 Sept 2026Are there still any hidden gem gpu's left?you read about gpu's like the Tesla P40, V100, AMD M150 etc that people pick up for cheap. probably many others as well. Most are either server gpu's being phased out or mining discards, right? The pr→4 Oct 2026Poor People Vulkan GPUs listHelp with this list. Give me your recommendation on 'not supported anymore' GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with so→4 Oct 2026A Strata fork for IBM AC922 running Qwen3.8-FN UD-Q4_K_XL is doing up to 7,357 tk/s prefill and 113 tk/s decodeI forked Strata and worked with Opus 5.5 with some heavy changes to it to make it work on an IBM AC922 I have access to. The IBM AC922 is a 2018 era beast with two POWER9 20 core SMT4 CPUs that are co→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→13 Oct 2026Problems with Tesla V100 SXM2Hi guys! I got a problem with one of my V100 SXM2 Modules. In 'nvidia-smi' its shows with 'Pwr:Usage/Cap' as 'N/A / N/A'. When I try to use it i get the following error in dmesg: NVRM: Xid (PCI:0000:0→13 Oct 2026My local Qwen3.8FN agent is significantly cheaper than cloudtl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest mo