CompanyAnthropic8 recent entries15 Aug 2026SOTA Apple Silicon Inference (August 15, 2026)This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, an→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa
CompanyOpenAI8 recent entries25 Jul 2026Old Coder Needs help with New AI Development and wants to get up to speed to understand it all.Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing→28 Jul 2026Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 IncidentThe first autonomous agent cyberattack is an unprecedented event that deserves unprecedented transparency. Today we're sharing everything we can: a full technical timeline, an interactive replay, and →18 Aug 2026tencent/UI-Mate-27B · Hugging FaceOverview UI-Mate-27B is an open-weight foundation GUI agent for long-horizon work across applications and operating systems. It observes live screenshots, reasons over the visible state, and produces →7 Sept 2026Cybersecurity is local AI model's killer use caseThis weekend I posted about the gap closing between frontier models and open source models. Well, now I'm coming with receipts. I've been running local + cloud models against real public github codeba→14 Sept 2026UkisAI Swift-Qwen3.8-27B / -58.3% thinking, x1.95 speed while keeping the accuracy of xhighHi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without 'attacking' the reasoning length directly then fixe→15 Sept 2026censorship has begun on HuggingFaceexample: https://huggingface.co/audnai/penclaw-GLM-5.3-abliterated-for-offensive-cyber. edit: good context for why this was done here https://www.reddit.com/r/LocalLLaMA/comments/1whbvpx/comment/pa16g→21 Sept 2026[Splash Engine] Qwen3.8-27B in native 8-bit at 37–55 tok/s on Apple Silicon: Extending Splash to Q8, 256k context scaling, and the 'Reasoning Cliff'https://preview.redd.it/nulsv53o8vqh1.png?width=4500&format=png&auto=webp&s=74765dbd409f4c221640f9f6000a685f6fdbb242 Spent weekend benchmarking the Splash engine (by Incoai) and extending its architec→10 Oct 2026microsoft doubles down on local ai with nvidia, but with a costhttps://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starti
CompanyGoogle7 recent entries22 Jul 2026Cactus Hybrid: We taught Gemma 4 to know when it's wrongHey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-t→26 Jul 202623 Gemma4-E4B models compared with abliterlitics: the most downloaded one is also the most brokenThis is our biggest comparison yet. We've taken 23 Gemma 4 E4B models from huggingface and ran them through the abliterlitics gauntlet. We also have a new abliterlitics discord, feel free to jump on a→3 Aug 2026Was the release of deepseek v4 flash planned to take spotlight against 5.6 luna?Id figured since they first emailed people about api price changes coming mid july then delayed the v4 flash release to late july, I wonder if they delayed it for the sake of stealing spotlight from o→18 Aug 2026Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)This post was made with AI. I tried to remove as much slop as possible and keep it straight to the point to save your time as I know how annoying AI slop posts can be, but I still wanted to retain all→10 Sept 2026Muse-glimmer-30b really punches above its weight(s) for creative writingAfter noticing that it is ranked among MUCH larger frontier models in the EQ-Bench Creative Writing benchmark and the Hemingway-bench, I decided to give it a try and was very impressed. I didn't do ve→23 Sept 2026Which small model works best with Ollama on a Raspberry Pi 5?I've been comparing and working on Gemma 4, Phi-4-Mini, and Qwen3-4B for local LLM use on an 8 GB Raspberry Pi 5. The results were indeed close, but each model has a different strength: Gemma 4 E2B: a→27 Sept 2026Swift 1.5 Qwen3.8 27b (A must-have for low thinking!)Just made this post for those who missed it : https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27b UkisAI released their updated Qwen 27B (tuned for token efficiency). I grabbed the IQ4_XS quant to tes
CompanyMeta8 recent entries27 Aug 2026friendly reminder you can legally torrent ai models.Repost because reddit keeps thinking this is piracy or illegal. It is neither. A lot of people are skeptical Nvidia will keep huggingface intact now that they will buy huggingface. There's a lot of do→28 Aug 2026I am Concerned if Nvidia Acquires Llama.CPP, Dev Team and HF, Anybody else?I dont know about others, but Nvidia seem to close the lid on older GPUs since they want to push their new technology. Llama and team has been the to go places for older GPUs like V100s. Knowing how N→29 Aug 2026Nemotron-3.5-Lightning at 11.77 GiB, a 16 GB option for a model that didn't have oneTL;DR: Every public low-bit GGUF of this model is secretly ~4.70 bpw. Shim the rows to 256 and it becomes a real 3.07 bpw / 11.77 GiB file that runs 262K context on 16GB. Needs patched llama.cpp — not→30 Aug 2026Uncensored Multi-Model Releases, LongCat-Flash-Lite-Sparse with MTPs and LSAs, Qwen3.8-27B with MTPs, Qwen3.5-122B-A10B with MTPs, Qwen3-Coder-Next and Laguna-S2.1 with Vision, All Available in GGUF Format! Bonus: Links to my llama.cpp Fork for LongCat-Flash-Lite Support and J-Wash Enhanced Fork!Been working really hard for the past month to bring to the community all these models, the hardest was for sure LongCat-Flash-Lite-Sparse who required TONS of work, first I needed to have Heretic sup→17 Sept 2026Update : Small model + EngramI posted something about a 9b model a few days ago. The real problem was 2 fold. Someone suggested the size was too big to prove it all out. It's a fair argument. I needed the model depth though. The →2 Oct 2026Adding memory to search instead of sampling in reward maximization tasks [R]I am one of the authors of FLEET - an algorithm that enhances Best-of-N generation by attributing external rewards to particular tokens and then uses MCTS to adjust logits during the next run. I find →3 Oct 2026Dual Radeon MI50 benchmarksStill don't have a good cooling solution, but here are few benchmarks. I lowered the power limit (TDP) to 145 watts each. I changed the firmware on one MI50 to activate the miniDP port. Did have to us→10 Oct 2026Created laya : Now Introducing a new 800 Million Param physics-based typed decision model with 73k context and image supportAround 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the
CompanyMistral1 recent entries2 Oct 2026[16GB AMD] Smartest local model that also does not refuse? 7950X + RX 6950 XTI want the smartest local model I can run that also does not refuse. Not a tiny 'uncensored' 8B. Not a censored model with a jailbreak prompt. Tried dolphin-llama3:8b with a custom SYSTEM prompt. It c
CompanyxAI2 recent entries25 Jul 2026Old Coder Needs help with New AI Development and wants to get up to speed to understand it all.Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa
CompanyDeepSeek8 recent entries24 Aug 2026Agnes-AI/Agnes-2.5-Pro-Alpha • HuggingFacehttps://preview.redd.it/usib0obv1blh1.png?width=1171&format=png&auto=webp&s=fa7b6e9155fcc461a2d397f9cf75aece4c72235c https://preview.redd.it/w453mok02blh1.png?width=2613&format=png&auto=webp&s=693182d→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→7 Sept 2026Cybersecurity is local AI model's killer use caseThis weekend I posted about the gap closing between frontier models and open source models. Well, now I'm coming with receipts. I've been running local + cloud models against real public github codeba→10 Sept 2026Deepseek V4.1 Flash is 748B, not 552BPeople keep on getting confused about this, so I looked at the safetensors on hf. To be precise, the main model about 551.566B parameters with 40 layers. The FFN experts total to 543.582B parameters, →17 Sept 2026Update : Small model + EngramI posted something about a 9b model a few days ago. The real problem was 2 fold. Someone suggested the size was too big to prove it all out. It's a fair argument. I needed the model depth though. The →21 Sept 2026[Splash Engine] Qwen3.8-27B in native 8-bit at 37–55 tok/s on Apple Silicon: Extending Splash to Q8, 256k context scaling, and the 'Reasoning Cliff'https://preview.redd.it/nulsv53o8vqh1.png?width=4500&format=png&auto=webp&s=74765dbd409f4c221640f9f6000a685f6fdbb242 Spent weekend benchmarking the Splash engine (by Incoai) and extending its architec→24 Sept 2026I built a desktop benchmark for local Ollama models — TTFT, tok/s, VRAM, with a HuggingFace GGUF catalogueBuilt this because I got tired of guessing whether a model would fit in my 6GB VRAM before downloading it, and because ollama.com's library doesn't let you sort by size. What it does: - Benchmarks Oll→9 Oct 2026[Paper] EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional MemoryConditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Be
CompanyNVIDIA8 recent entries31 Aug 2026snkii/Sori-1B: Audio-Grounded LM Trained From Scratch (No Text-Only Pretraining)Sori-1B is a 1B-parameter audio-language model built by a single SNU researcher whose core claim to fame is that its decoder is trained entirely from scratch on audio-paired text — no text-only pretra→3 Sept 2026'ModelScope' Is a Hugging Face Alternative now that Nvidias deal is a GoI liked the Nvidia that focused on just GPUs for gaming, not on the Nvidia of today which seem want power consolidation. Modelscope is another platform for those that simply want to know an alternativ→5 Sept 2026NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa→10 Sept 2026Muse-glimmer-30b really punches above its weight(s) for creative writingAfter noticing that it is ranked among MUCH larger frontier models in the EQ-Bench Creative Writing benchmark and the Hemingway-bench, I decided to give it a try and was very impressed. I didn't do ve→14 Sept 2026UkisAI Swift-Qwen3.8-27B / -58.3% thinking, x1.95 speed while keeping the accuracy of xhighHi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without 'attacking' the reasoning length directly then fixe→16 Sept 2026What's the current best LLM uncensoring method?With the recent Nvidia Huggingface acquisition and frontier AI labs screaming about safety and putting guardrails everywhere, I think it's important that we have local models that aren't affected by a→3 Oct 2026Dual Radeon MI50 benchmarksStill don't have a good cooling solution, but here are few benchmarks. I lowered the power limit (TDP) to 145 watts each. I changed the firmware on one MI50 to activate the miniDP port. Did have to us→10 Oct 2026microsoft doubles down on local ai with nvidia, but with a costhttps://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starti