CompanyAnthropic8 recent entries10 Oct 2026Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernelI've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and cod→10 Oct 2026I tried three ways of getting Gemma and Qwen to update a notes file in OllamaI ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, o
CompanyOpenAI8 recent entries10 Oct 2026Benefits of using bigger models than Qwen 3.8 flash next?I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared →11 Oct 2026Now you can grow Bonsai on your potatoNo more excuses for GPU-poor folks not to start LLMing! Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) ~9.3x smaller than FP16 (ideal) | 98.2% of FP16 intell→12 Oct 2026I tested GPT-5.6 and GPT-6 with Medium → Instant → Medium. Both models failed on Instant, both succeeded on Medium. Here's what I found.I've been investigating some serious ChatGPT reliability issues over the past several days, and my latest tests revealed something particularly interesting: Both GPT-5.6 and GPT-6 showed problems unde→13 Oct 2026Running decision models: Picking quant, VRAM usageHi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom→13 Oct 2026Chatgpt Astra has been gutted. That's what the people living in the walls tell me.It's confirmed. OpenAI has been secretly replacing Astra with increasingly shit models since release. My evidence is overwhelming. Three Reddit comments, a benchmark that fluctuated 4%, and a guy who →13 Oct 20264x 6000 Pro for the companyHere is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo→14 Oct 2026I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile)I built BitRun because I wanted to run 1-bit ternary LLMs on ordinary laptops without melting my battery, spinning up a dedicated GPU, or wrestling with complex build environments. Right now, BitRun g→14 Oct 2026I built a cheap dual 3090 rig from used parts, but why isn't it much faster?Hey everyone, I recently put together a local LLM machine using mostly second hand hardware. I wanted to experiment with local models for app development without spending a fortune on a new workstatio
CompanyGoogle8 recent entries1 Oct 2026We benchmarked 18 RAG pipelines against an agent loop on Google's FRAMES. The best pipeline hit 78.9%. The agent loop hit 92.7%.Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb→4 Oct 2026Built a quick, sub-15ms Rust CLI/TUI to pack repos into prompts without burning 40k tokens on lockfiles and junkWhenever I feed codebases into local models (Qwen, DeepSeek R1) or API models, the biggest annoyance is prompt pollution: - Lockfiles (`Cargo.lock`, `package-lock.json`) burning 30,000+ tokens for zer→6 Oct 2026MiMo V2.6 Pro almost matched GPT-6 Astra at figuring out what a user actually wantsIntroducing InferBench: A benchmark testing how well frontier LLMs infer a user's priorities from their instructions. We tested 12 LLMs across 20 scenarios with 2.8k conversations to see which models →6 Oct 2026Aplomb 1: open-weights 5.3B decision model, 1M context, text/image/video/audio in one request, #1 among 4B models on the Decision IndexWe released Aplomb 1 today, a 5.3B decision model with open weights. It reads up to 1M tokens of text, JSON, images, video and audio in a single request, and on our API it makes a decision on a 1M-tok→7 Oct 2026A stock GLM-Edge-1.5B-Chat on a 4GB Galaxy A04e completed a real Amazon cart taskYesterday TechCrunch published a piece about a growing problem for AI agents: websites are starting to block them. Amazon blocking Meta's Muse is the obvious example. At almost exactly the same time, →8 Oct 2026Whatever happened to BABA is AI from 2024? [D]We test three state-of-the-art multi-modal large language models (GPT-4o, Gemini-1.5-Pro, Gemini-1.5-Flash) and find that they fail dramatically when generalization requires that the rules of the game→8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→10 Oct 2026Benefits of using bigger models than Qwen 3.8 flash next?I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared
CompanyMeta8 recent entries8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→9 Oct 2026My 'Anthropic/ChatGPT in a box' is... now open source and publicly hosted on Github.It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu→10 Oct 2026Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernelI've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and cod→10 Oct 2026Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAMRunning HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM. →10 Oct 2026Improve token per second without touching quantSpent the past month tweaking and experimenting with many different numbers to achieve 30tps. Hardware: -Rx6700xt 12gb vram (AMD) -2x16 ddr4 3200 ram -r5 5600x -llama.cpp vulkan sdk -window11 (no wsl →10 Oct 2026I tried three ways of getting Gemma and Qwen to update a notes file in OllamaI ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, o→10 Oct 2026Created laya : Now Introducing a new 800 Million Param physics-based typed decision model with 73k context and image supportAround 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the →13 Oct 2026Update: Converting dense models into Mixture-of-ExpertsThis is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also convertin
CompanyMistral8 recent entries6 Aug 2026nvidia/NVIDIA-Nemotron-Parse-2.0 · Hugging FaceNVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blu→13 Aug 2026How do you plan to run Qwen3.8-2.4T-A95B locally?To my fellow crazies, the few. Those who dared wrestle with llama-70b, mistral-large, goliath, mistral8x22B, DeepSeekV2/3, wept when llama4 behemoth was announced, picked yourself up and are now wrest→31 Aug 2026The Chrono Trigger plot challenge - Crono awakens in his modest bedroom of 2095...A hallucinated event, Crono awakens in his modest bedroom of 2095... I am using local models since 2025 January. My daily driver is qwen 3.6 35B A3B, which works pretty well for coding tasks, but I al→11 Sept 2026Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B modelsHey everyone, If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning e→12 Sept 2026For those of you forced to only use open models from Western labs in production, what are you deploying?First off, I know that GLM, Qwen, and DeepSeek absolutely dominate in terms of SOTA Open Source models, and that’s what I use in my personal projects and for school, however, I’m also responsible for →15 Sept 2026CrofAI 'cheapest inference provider in the world' gets exposed as an OpenRouter wrapper, routing requests to smaller, cheaper models at up to 20x markup. CrofAI responds to Wire Fraud allegations by denying everything, then backtracking, then 3 hours later wiping their entire online presenceDisclaimer: no AI was used whatsoever to write this post Cautionary tale about chasing cheap tokens. exposé: https://kendell.dev/blog/crofaifalse/ reaction by nahcrof, announcing the shutdown of the s→2 Oct 2026[16GB AMD] Smartest local model that also does not refuse? 7950X + RX 6950 XTI want the smartest local model I can run that also does not refuse. Not a tiny 'uncensored' 8B. Not a censored model with a jailbreak prompt. Tried dolphin-llama3:8b with a custom SYSTEM prompt. It c→12 Oct 2026QLoRA on ExLlamaV3 Quants - Train @ 2-8bpwEXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra
CompanyxAI8 recent entries29 Aug 2026Terminal Bench 4.0 just dropped, GLM-5.3 is at the same level as Fable 5, accounting for margin of errorAnnouncement: https://www.tbench.ai/news/terminal-bench-4-0 Leaderboard: https://www.tbench.ai/ Imo the best aspect in their announcement is their focus on rapidly iterating on TerminalBench to keep t→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→14 Sept 2026Animated transition from AA Intelligence Index v4.1 to v4.3I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison. All intelligence and price per task are sam→16 Sept 2026Can current MiniCPM5-2B/any best SOTA under 10B + modern harness beat pre-March 2025 frontier models like Grok 3/GPT-4o?Hey experts! I genuinely have this question and would love a general consensus from people actually using these models. LLMs have advanced a lot in benchmarks and in practical usage. Even GPT-4o and G→20 Sept 2026PromptDeck v1.1.0 – open-source desktop app to benchmark local AND cloud LLMs side-by-side (Ollama, LM Studio + OpenRouter, Groq, DeepSeek…)Hey r/LocalLLaMA! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists. What it is: an open-source desktop studio (Tauri + React) for pr→21 Sept 2026I just tried Grok 4.7 and it is...I never thought there would be a model worse than the Gemini series, but here we are. I remember Elon tweeting that this model would beat Astra & 5.1 across all benchmarks. submitted by /u/prvthvm [li→6 Oct 2026MiMo V2.6 Pro almost matched GPT-6 Astra at figuring out what a user actually wantsIntroducing InferBench: A benchmark testing how well frontier LLMs infer a user's priorities from their instructions. We tested 12 LLMs across 20 scenarios with 2.8k conversations to see which models →8 Oct 2026Last week some of South Korea's biggest banks were hit by a cyberattack. We now know the entire hack may have been done by a single person. He used a combined stack of an open-source AI penetration tool named ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code (CrowdStrike)CrowdStrike report: https://www.crowdstrike.com/en-us/blog/unknown-threat-actor-uses-artex-to-target-south-korean-finance/ From: Andrew Curran on 𝕏: https://x.com/AndrewCurran_/status/2108108695876092
CompanyDeepSeek8 recent entries7 Oct 2026Local Qwen 3.8 27B vs DeepSeek Flash API: Is local good enough?Running a model on your own machine used to be a privacy story with a quality tax. On this generation that trade has narrowed to where we can state it plainly: for daily work, the local model is good →8 Oct 2026Last week some of South Korea's biggest banks were hit by a cyberattack. We now know the entire hack may have been done by a single person. He used a combined stack of an open-source AI penetration tool named ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code (CrowdStrike)CrowdStrike report: https://www.crowdstrike.com/en-us/blog/unknown-threat-actor-uses-artex-to-target-south-korean-finance/ From: Andrew Curran on 𝕏: https://x.com/AndrewCurran_/status/2108108695876092→9 Oct 2026My 'Anthropic/ChatGPT in a box' is... now open source and publicly hosted on Github.It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu→9 Oct 2026Just another purely open-weight models benchmarkLive results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area→10 Oct 2026Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAMRunning HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM. →11 Oct 2026Qwen3.8 Flash Next fixed my GNOME extensionI love Dash2Dock Lite, but Icedman is always a week or two before updates. I randomly ran pacman for the first time in weeks, not realizing GNOME had a major update. I told it to fix it in DeepSeek Ha→13 Oct 20264x 6000 Pro for the companyHere is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo→14 Oct 2026The AI race: China's open-weight models trail US closed models by just 7 monthsSource: whatmodel.sh, an open source AI leaderboard I built (and launched here yesterday). Every point comes from published benchmark results: 22,113 of them across 987 benchmarks, from 1,011 sources,
CompanyNVIDIA8 recent entries9 Oct 2026Just another purely open-weight models benchmarkLive results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area→10 Oct 2026Strata with Qwen3.8 Flash Next UD-Q4_K_XLBeen seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted→11 Oct 2026vllm-ascend updates (GLM5.3-flash) on dual 310p Ascend cardsAlright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v→11 Oct 2026I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokensDISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and h→12 Oct 2026Lecture: Building a GPU Compiler From Scratch. Lowering PyTorch Graphs to CUDA via Multi-Level IRs (GPU_MODE video + article)https://preview.redd.it/3ekngxk224vh1.png?width=1080&format=png&auto=webp&s=9bfe5c92033e17d5b2fb4529f1320c99455a138b I built a GPU compiler and wrote an article about it, focusing on concepts shared b→12 Oct 2026A Record of the Process of Increasing MFU from 21% to 50% While Pre-training 21B MoE Using a Single GH200 Chip (Includes Code and Logs)I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have com→13 Oct 2026PSA for 3090 FE owners: Running LLMs overheats your VRAMhttps://preview.redd.it/p6i9qifyy8vh1.png?width=2069&format=png&auto=webp&s=2b1e3c7c4fc95f12f0f7d417a4d21410aebfbd35 The 3090 Founders edition has a reputation for hot VRAM. Since LLM token generation→14 Oct 2026I built a cheap dual 3090 rig from used parts, but why isn't it much faster?Hey everyone, I recently put together a local LLM machine using mostly second hand hardware. I wanted to experiment with local models for app development without spending a fortune on a new workstatio