is this good? 262k Qwen3.8:27B-Q4_K_M

https://preview.redd.it/j13iin0jsfqh1.png?width=1129&format=png&auto=webp&s=3efdd2ad5c10de908aed1a673a6c7e6440177c09 https://preview.redd.it/k3fsrdspzfqh1.png?width=391&format=png&auto=webp&s=1c8176876ac03a79e779c9b67f3def57f42a4064 I got nvfp4 KV cache support working on heterogenous upstream port of the existing ollama branch. I had to edit llama.cpp, custom CUDA kernels, also fixed real bugs in the existing kernels to begin with. I am splitting the model between RTX 5090 and RTX 3090TI using a compute aware layer distribution that also locks the output layer to the 5090 for maximum performance. I also quant Qwen3.8:27B myself from the fp16 source with a custom Q4_K_M weights using a clever f16 repack slop bot summary: Compute aware dispatch. split FA kernels through CUDA_ARCH, not just FP16 cap. SM86 TILE. SM120 VEC path separate. q8_0 baseline. fixed TILE boundary nwarps overflow @ 262k context, q8_0 passes on BOTH cards. MXFP4 17B bug. SM120 only block_mxfp4=17B+vector load ld.global = misaligned 0x7ff95b398141. Fix 3 memcpy byte load + pitch 17-> 32 NVFP4 8B - same pattern, block_nvfp4 scales, same byte-load fix applies Result: magic fix is fattn-vec.cuh memcpy(&k, K+off*17, 17) // Blackwell can't vector load 17B submitted by /u/comperr [link] [comments]

Related

• GLM/Qwen Appreciation Post
• Qwen3.8-27B IQ3_XXS wrote a correct multilayer TMM on a 16 GB Quadro — after 100 minutes, 3 compactions, and 108k output tokens
• Qwen 3.8 27b with DSH(DeepSeek Harness) is Amazing!! Experiences so far and perfomance.

Source: r/LocalLLaMA | 2026-09-19