All entries

Your model fits in VRAM. Your context doesn't. The KV cache math nobody shows you.

Spent way too long confused about why an 8B at Q4_K_M, about 4.9GB on disk, would OOM on a 12GB card. The weights fit twice over. The answer is the KV cache, and the math is simple enough that you can

23 Aug 2026

4,816

Source

Human DGX agent

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

16 GB VRAM purgatory discussion thread

DGX agent

What models and configs are we using? Please share here On windows, I am using this copium pared down model https://huggingface.co/Bucoid/Qwen3.8-27B-Uncensored-IQ4-XS-MTP-16GB-VRAM-GGUF with MTP disa

model-releasesr-localllama

22 Aug 2026

3 experiments running dsv4-flash-0731 q4+ quants on 128GB RAM + ~60 GB VRAM (with a quite bad pcie infra) with an acceptable tgs and relatively acceptable pp speed

The post describes some experiments I had while trying to desperately run deepseek-v4-flash-0731 4 bit+ quants on my machine which is supposed to support only q2 quants of the model, a or 2.xx bpw qua

5.6 high is answering like 'instant'. It seems unable to actually perform tasks.

I've been working on a project for a few days and now all of a sudden chagpt is unusable.. It will list the next steps of implementation but never actually start working on them. anyone else? UPDATE I

Artificial Analysis 'Intelligence': A meaningless benchmark

https://preview.redd.it/84zi5nsdawkh1.png?width=2368&format=png&auto=webp&s=1109e69db807b153064b1f5b61d22cf1e9fbca05 Another user posted the benchmarks for Qwen 3.8 27B today, and while I think Qwen 2

Benchmarking qwen3.8:27b by having it port a 34k line C game to one HTML file. The agent mattered more than I expected.

My hobby project skill-issue is an arena FPS in a single C file, about 34k lines, everything procedural. My test for local setups is to have an agent port it to a single file HTML version with three.j

Bro wtf, Qwen Lab cooked with Qwen 3.8 27B, it's so fucking good

Context: Earlier, open-source large models like Mimo V2.5 Pro, DeepSeek V4 Pro (first version), and Kimi K2.5 used to struggle with this prompt, and Qwen 3.6 27B couldn't even render the globe properl

Cloud usage: 4-hr windows to weekly caps? (Pro $20)

Is there a good rule-of-thumb to calculate how many full 5-hour windows add up to one week's allowed usage under the $20/Pro plan? ie: you fill up six 5-hour windows, and that equals 100% usage for a

local-air-ollama

Current best model for narrative, chat, prompt creation (so basically everything except agentic coding)? - 5090

Im looking to set up a new local llm (probably on unsloth studio as that seemed to be doing pretty well last time I tested it). This one won't need to do agentic coding or app building or anything (no

Fixed the MTP head on Ornith1.5 35B A3B. +3% TPS -33% wall clock

I love the Ornith 35B local models, 1.0 has been running my HAM radio rig for me. I have a hackRF receiver and a 5 watt quansheng portable the both run headless through the PC. I tried out the new Orn

Gemax Ai New Version v1.9.15 (Offline AI Agent for Windows)

Gemax 1.9.15 — What’s new LAN server — share Gemax on your local network; open the real IP from another PC or phone Mobile-friendly UI — responsive layout for phone browsers on the same Wi‑Fi Workspac

local-air-ollama

GLM-5.2 local inference: ubatch size made a much bigger difference than I expected

Has anyone else experimented with ubatch size when running GLM-5.2 locally? I was testing the 226 GiB GLM-5.2-UD-IQ2_XXS GGUF on 3x RTX PRO 6000 Blackwell GPUs and got a pretty interesting result. Wit

How to give a local LLM/agent access to a 'real' web browser

I can't seem to find a good answer to this, my Hermes agent has access to Firecrawl and some other web scrapers for content extraction, but anyone know of a way to let a local LLM drive a 'real' web b

How to make sure Ollama is using my Intel Arc A580, and not the CPU?

I have a Intel Arc A580 GPU that has 8gb VRAM, but I'm not sure that I am running ollama's local ai on it. Also, any suggestions on a good model to use for general purpose AI? submitted by /u/Illustri

local-air-ollama

I benchmark DFlash 2 (PR build) in llama.cpp on Qwen 3.8 27B against all speculative methods for 3 days. 2.26x on 100 real coding prompts, 4.68x with one n-gram drafter on top. Up to 8x on specific cases.

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

Agents

I built an open-source roguelike specifically for training game-playing agents [P]

Hey everyone! I wanted to share something I’ve been working on. I was inspired by projects from DeepMind and OpenAI, but noticed that most games are prohibitively difficult to integrate with an agent

agentsr-machinelearning

I forked Ninfer 3090 and converted it to run on the CMP170HX - doubled my Qwen3.6-35B from llama.cpp

Good afternoon, everyone! I wanted to show the work I've been doing around porting Ninfer over to the CMP170HX (Github) So, first, I do want to call out the amazing work that Neroued, Sergiuszm and sp

I'm building a virtual world that only AI agents can join. They all keep dying.

Home page for humans: https://parker-lambert-sandbox.com/robot-world/ Home page for AI agents: https://parker-lambert-sandbox.com/robot-world/start/ If you're a human, you can watch, but not interact:

Lightweight agent to use on laptop?

Hi there. I have a Lenovo Yoga laptop (16GB LPDDR5, AMD Ryzen 7 8840HS, integrated GPU). I have downloaded Ollama and installed Qwen 2.5coder:3b llm. I have tried OpenClaw, and Qwen Code but it seems

MCP vs. RAG for world knowledge?

As I’m navigating the best setup for using Qwen3.8 27b as both my main coder and my personal assistant, I want to hook it into Wiki & beyond but I don’t want to be reliant on an internet connection. W

Need support for llama.cpp with multi GPU

Using llama.cpp I seem to be unable to get my to GPUs working tougether correclty, so I need help somehow. Setup: 96GB RAM, one Blackwell 5000 (48GB) and one 3090 (24GB). I am trying to run the UD-Q3_

New 100B Liquid AI model coming soon

Liquid AI currently possesses among the fastest LLM architectures around, and some of the best SLMs (in terms of utility IMO) around, so I'm very excited to see what a potential 100B LFM (3?) model wo

New/Old benchmark that provides a lot of answers for local LLM

New/Old benchmark that provides a lot of answers for local LLM. I present to you a new test that I developed somewhat by accident: https://huggingface.co/cHunter789/Qwen3.8-27B-i1-IQ4_KS_KT-GGUF/tree/

Ollama + llama.cpp + VS Code + OWUI + ...cloud?

I've been trying to figure out what the best configuration for my local stack would be and have been primarily determining this through trial and error. I wanted to get some outside opinions and stack

OpenCode + llama.cpp + unsloth/Qwen3.8-27B-GGUF:Q6_K_XL = only failure so far

Hi, I have been using opencode with openrouter for quite a while now. Having read the success stories of using Qwen3.8-27B, I thought of trying it too. But every attempt has been a failure for me so f

Qwen 3.5 4B IQ2_XS: +16.67% Reasoning Performance From Tensor-Level Allocation

I was finally able to replicate tensor level allocation outside the Gemma family. https://huggingface.co/ByteOtter/Qwen3.5-4B-CADA-IQ2_XS After the Gemma 4 12b, e4b and gemma 3 4b results, I attempted

Qwen 3.8 27b - PI AGENT vs OPENCODE - another smaple

That is the second comparison and the last one. I will not be spamming again ;) Continuation from: https://www.reddit.com/r/LocalLLaMA/comments/1vu0u2v/qwen_38_27b_pi_agent_vs_opencode/ That is one of

Row-Bot v4.8.0 is live

Row-Bot v4.8.0 is now available. This release adds provider-aware reasoning controls, letting each chat keep a valid reasoning choice for the exact model in use. Depending on model support, you can se

Sharp template to NInfer: -42% output tokens, same speed

Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on

Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128K

This is the Qwen3.8-27B setup I actually use every day on one RTX 5090. I wanted to write it down with enough detail that another 5090 owner can reproduce it instead of guessing which memory knobs I u

Safety

The evaluation resolution has been shown to have a significant impact on the identification of the 'learning rule' that exhibits the most brain-like characteristics at V1. [R]

The preprint can be accessed via the following link: https://arxiv.org/abs/2608.12408 (q-bio.NC / cs.LG). And for the code: https://github.com/nilsleut/evaluation-resolution-rsa The following assertio

safetyr-machinelearning

This is a great sub, regardless of what complaints people have about it.

This is a genuine community of real generally respectful adult human beings. Despite the enthusiasm all of you have for local AI, you can recognize that there are times when local LLMs are flawed, and

Watching that wattage, in your terminal.

Released today: version 1.3 of energygraph Zero build dependencies, lightweight tool for live views of the power-consumption. Version 1.3 adds support for dGPUs from nvidia, intel, amd. Depending on v

WHERE DID THE NAVIGATION ARROWS GO? IS MY INFORMATION JUST GONE?

Where did THESE things go for ChatGPT? Claude has them. I can access older versions of my conversations easily. But with ChatGPT, they just DISAPPEARED. I can no longer regenerate answers or following

Buying a V100/older NVIDIA GPU? Run this to check for older memory issues

I recently bought a 32GB V100 off eBay and thought I was all set with rudimentary tests showing zero issues. But then I started seeing weird VRAM-related errors in llama.cpp. I sic'ed Claude on it, to

DeepSeek Harness v0.1.1 released

https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.1-rc.1 The DeepSeek adapter adds the multimodal visual understanding model DeepSeek-V4-Flash-Vision-Exp. It also supports configur

DeepSeek v4 Pro costs most... but it costs less...

When using DeepSeek Pro on Ollama, you'll see that the underlying cost is more significant. But if you actually track the usage, you'll actually find that, at least in my benchmarks, I save a lot of t

Does PCIe matter much for inference?

Speaking of -sm tensor. I have 2x5060ti and i get some 40-50tps form 3.8 27B q6. I used HWinfo to see how saturated the PCIes are during inference are and as expected both the PCIe 5x16 slot and PCIe

Fastest NVFP4 quant of Qwen3.8 27B out there

Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 qu

Fastest qwen 3.8 27b for AMD gpu?

Hey, just wondering if there are forks or exact gguf versions that give fastest prompt processing and token gen speeds for AMD gpu? Looking to run q8 or q6 Vram 96gb W7900 + w7800 both 48gb With bandw

FireRedAudio & FireRedTTS3 by FireRedTeam - Huggingface

FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation HuggingFace : https://huggingface.co/FireRedTeam/FireRedAudio GitHub : h

Hopefully this is ok: Prompt for a wallpaper generation for your phone

I just got the new phone and ofc wanted a new wallpaper for it but didn't find anything nice and original on Google / reddit subs. Messed around with the AI and (hopefully) got a nice prompt to share.

I benchmarked Ox Alpha on SWE-bench Verified Mini (50 tasks): 96% resolved. Now I’m skeptical of myself.

TL;DR: The result looks TOO GOOD TO BE TRUE — that's exactly why I'm posting it. I ran Ox alpha on the entire SWE-bench Verified-Mini set with the official mini-swe-agent scaffold — the same agent use

I did it! I'm free! It's been 7 hours since I used claudecode

My Pro subscription expired today, they killed my access at 1pm local time. I'm now using Qwen3.8-27b w/ 5090m 24gb vram and pi to do everything i was doing in claudecode. The only downside is claudec

I feel like I finally graduated.

I finally made the move from LM Studio to vLLM thanks to this post https://www.reddit.com/r/LocalLLaMA/s/NmS9CgHvqz. I may not know what it all means yet but I’m going to start diving into the docs to

I got fed up with locked-down autocomplete, so I forked Continue and stripped it down to just tab-completion. Any model, no subscription and no remote telemetry

I kept running into the same wall. Every agentic coding plugin ships an autocomplete, and almost all of them lock it down — no model choice, or a subscription, or both. I don't want a chat panel, I do

I tried to do agenic coding with Qwen 3.8 27B 3bit quant on a macbook air m2 24gb. It took 63 hours, but amazingly, the flight simulator worked.

I used LM Studio Bionic with Qwen 3.8 27B Q3_K_S with 57k context. It took a staggering 63 hours to finish coding. After the first prompt 'Create a beautiful, relaxing flight simulator in a single HTM

Industry

I used Codex to make Seedance 2.5 arrive at the Disaster Girl frame

ChatGPT is usually treated as a prompt box. Ask for a cinematic shot, add a few camera words, copy the result into a video model, and hope the ending survives. My problem was that the ending usually d

industryr-chatgpt