CompanyAnthropic8 recent entries12 Oct 2026Qwen3.8-27B vs Frontier vs OthersHere're results from my final test of coding, coding (panda/numpy), data analysis, data-driven decision-making, and local RAG domains I interact with most often. This test, which is refined from a 500→12 Oct 2026llama.cpp optimizations for specific hardware and modelsI've made a few posts recently, just talking about the hardware I've got and the model I use. Pair of P100s, Unsloth Qwen3.6 35b A3B UD Q4 K XL. There's been a ton of talk of model and hardware specif→
CompanyOpenAI8 recent entries12 Oct 2026A 'free' multi-modal 'embedding' model (already) included with your LLM: vLLM plugin + upstream PRDisclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings →13 Oct 2026Running decision models: Picking quant, VRAM usageHi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom→13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you→13 Oct 2026'New AI fake text generator may be too dangerous to release, say creators' (2019 Guardian headline)I'm sure some here read this at the time and remember, but I recently found this hilarious Guardian article from 2019 through Wikipedia: https://www.theguardian.com/technology/2019/feb/14/elon-musk-ba→13 Oct 2026Lipwise - Smart dictation with processing built inLipwise is a [yet another] dictation app for Windows, macOS and Linux. You hold a hotkey, talk, and it types into whatever app has focus but with a twist. Since most of us that speak to our computers →13 Oct 2026How far are we from a local model being able to optimize it's own runtime stack?A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontie→13 Oct 2026Horsey challenge | Strata Fork | Qwen 3.8 FN NVFP4So I wanted to try the horsey challenge myself on the latest engine i been testing which is a strata fork by sergqwer. https://github.com/sergqwer/strata-nvfp4. I seen a post earlier and seen i believ→13 Oct 20264x 6000 Pro for the companyHere is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo
CompanyGoogle8 recent entries8 Oct 2026GPU - Vulkan llama.cpp benchmarks sorted by price to performanceThis table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res→9 Oct 2026tencent/Youtu-Parsing-Omni · Hugging FaceYoutu-Parsing-Omni is a compact (5B) omni-modal parsing model. Given a single input — a document page, a natural image, a chart / flowchart, a geometry figure, an audio clip or an audio-visual video —→9 Oct 2026DGX Monarch - Dual Spark Rendering w/ ComfyUIFor those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at →10 Oct 2026Open-source Mac app that runs EmbeddingGemma 2 locally to search your files by what’s in themDigUp is a free Mac app that runs Google DeepMind’s new EmbeddingGemma 2 locally over your own files. The model puts text, images, audio and video in one space, so you describe what you remember and l→10 Oct 2026Benefits of using bigger models than Qwen 3.8 flash next?I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared →12 Oct 2026What are you doing for a reliable, well designed local model based AI search?Ive been using OpenWebUI's inbuilt web search and I can't put my finger on it, but it's not quite as polished, reliable, seamless as using Claude or Gemini. And its often slow beyond a limit that make→12 Oct 2026Explore DNA with DeepMind’s AlphaGenome and in-browser MedGemma 1.5 4B via WebGPUHey everyone, Sharing an educational tool called AskCyph Health Explorer to explore how single-letter DNA mutations lead to disease, how they alter 3D anatomy, inheritance probabilities, and available→13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you
CompanyMeta8 recent entries12 Oct 2026Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included.Write-up, serve command and raw data: https://github.com/lukolszewski/vllm This shows vLLM fixes and setup that let you put PLE on SSD and save 108GB of system RAM, use int8 kv and squeeze more users →12 Oct 2026openbmb/MiniCPM-V-4.7-35B-A3Bgguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in →12 Oct 2026openbmb/MiniCPM-V-4.7-1BGguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-1B-gguf 'MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 1B variant. It builds on the family's single-image, multi-image,→13 Oct 2026Update: Converting dense models into Mixture-of-ExpertsThis is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also convertin→13 Oct 2026Quartermaster 1.7 update now lets you run llama.cpp's new decision models (/v1/systemone) with a playground tabQuartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop →13 Oct 2026Problems with Tesla V100 SXM2Hi guys! I got a problem with one of my V100 SXM2 Modules. In 'nvidia-smi' its shows with 'Pwr:Usage/Cap' as 'N/A / N/A'. When I try to use it i get the following error in dmesg: NVRM: Xid (PCI:0000:0→13 Oct 2026How far are we from a local model being able to optimize it's own runtime stack?A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontie→13 Oct 2026Horsey challenge | Strata Fork | Qwen 3.8 FN NVFP4So I wanted to try the horsey challenge myself on the latest engine i been testing which is a strata fork by sergqwer. https://github.com/sergqwer/strata-nvfp4. I seen a post earlier and seen i believ
CompanyMistral8 recent entries15 Sept 2026CrofAI 'cheapest inference provider in the world' gets exposed as an OpenRouter wrapper, routing requests to smaller, cheaper models at up to 20x markup. CrofAI responds to Wire Fraud allegations by denying everything, then backtracking, then 3 hours later wiping their entire online presenceDisclaimer: no AI was used whatsoever to write this post Cautionary tale about chasing cheap tokens. exposé: https://kendell.dev/blog/crofaifalse/ reaction by nahcrof, announcing the shutdown of the s→30 Sept 2026Follow-up: my native Rust + Vulkan Transformer training backend — 14 days later, now 14 parity-verified architectures and full PEFTFollow-up to my post from about two weeks ago. A lot has changed since then, so I wanted to post an update on where the backend is now. Where the green architectures stand When I posted last time, 7 a→2 Oct 2026Anyone wonder why americans lag so far behind in the open LLM market?I mean , DeepSeek, Kimi, GLM, MiniMax -- the list of Chinese LLM's is such a long freaking list. As American's -- why don't we feel .. a little funny .. about being so far behind? Chinese entrepreneur→2 Oct 2026[16GB AMD] Smartest local model that also does not refuse? 7950X + RX 6950 XTI want the smartest local model I can run that also does not refuse. Not a tiny 'uncensored' 8B. Not a censored model with a jailbreak prompt. Tried dolphin-llama3:8b with a custom SYSTEM prompt. It c→7 Oct 2026stuck on max.. no mimo2.6.. no chong?Never saw mimo2.6, now im doubting mistral 4 wont land either ... I feel trapped but im not sue its worth the $100 ... just few options anyone know any details behind the change? submitted by /u/stack→9 Oct 2026GLM 5.3 Flash opensource @ the top of Artificial Analysis Cyber Index over Claude.It seems Dario's 'too powerful for you users' strategy is paying off: we have two open models at the top of the leaderboard, surpassing every single model from Anthropic. Open source prevails. Even Mi→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM RankedIt's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit→12 Oct 2026QLoRA on ExLlamaV3 Quants - Train @ 2-8bpwEXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra
CompanyxAI8 recent entries20 Sept 2026PromptDeck v1.1.0 – open-source desktop app to benchmark local AND cloud LLMs side-by-side (Ollama, LM Studio + OpenRouter, Groq, DeepSeek…)Hey r/LocalLLaMA! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists. What it is: an open-source desktop studio (Tauri + React) for pr→21 Sept 2026I just tried Grok 4.7 and it is...I never thought there would be a model worse than the Gemini series, but here we are. I remember Elon tweeting that this model would beat Astra & 5.1 across all benchmarks. submitted by /u/prvthvm [li→25 Sept 2026Watched my LLM router burn $11 on a 3-word prompt. Here's the fix.I run a personal always-on agent fleet — eight seats claiming work in parallel from a shared task board, routing across Anthropic, OpenAI, xAI, Google, and a local Ollama floor. This week the spend fo→27 Sept 2026Nonobench v1.2: 43 LLMs on nonogram puzzles. Open-weight DeepSeek V4 Pro ties for 4th, and no open model solves the new 20×20 Hard modeFollow-up to my January post: https://www.reddit.com/r/LocalLLaMA/comments/1q4i19c/benchmarking_23_llms_on_nonogram_logic_puzzle/. That thread shaped v1.2: Reasoning effort is explicit per run Every p→29 Sept 2026Reflection 70B was released two years ago (September 2024)You may think that jev, OpenClaw or TurboQuant are super cool, but actually the coolest LLM invention happened two years ago As we all know, the best source of reliable information about LLMs is YouTu→6 Oct 2026MiMo V2.6 Pro almost matched GPT-6 Astra at figuring out what a user actually wantsIntroducing InferBench: A benchmark testing how well frontier LLMs infer a user's priorities from their instructions. We tested 12 LLMs across 20 scenarios with 2.8k conversations to see which models →8 Oct 2026Last week some of South Korea's biggest banks were hit by a cyberattack. We now know the entire hack may have been done by a single person. He used a combined stack of an open-source AI penetration tool named ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code (CrowdStrike)CrowdStrike report: https://www.crowdstrike.com/en-us/blog/unknown-threat-actor-uses-artex-to-target-south-korean-finance/ From: Andrew Curran on 𝕏: https://x.com/AndrewCurran_/status/2108108695876092→9 Oct 2026DGX Monarch - Dual Spark Rendering w/ ComfyUIFor those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at
CompanyDeepSeek8 recent entries10 Oct 2026Best small model + setup together with qwen3.8-flash-nexthello, I currently use swift qwen3.8 gsq rco in iq3 xxs in strata. The model is only used for hermes agent. I have 64GB of ram + 32GB R9700 where the model runs on. But I also have an rx7800xt with 16→11 Oct 2026[vllm] Is it possible to run Qwen3.8-27B-FP8 on a 5090?Based on my experiments and feedback from claude, it seems extremely unlikely and even if I get it working, not practical because of minimum context. But just asking here as a last effort. I'm able to→11 Oct 2026Qwen3.8 Flash Next fixed my GNOME extensionI love Dash2Dock Lite, but Icedman is always a week or two before updates. I randomly ran pacman for the first time in weeks, not realizing GNOME had a major update. I told it to fix it in DeepSeek Ha→11 Oct 2026Qwen 3.8 27B Q5 vs Qwen 3.8 Next Q3_S for document analysisI have an app that processes ~500 different documents every day. An agent analyzes them, sorts them, tags them, extracts various legal nuances, creates descriptions, etc. So far I've been using Qwen 3→11 Oct 2026OMG! If you have a Mac with 64GB, try Qwen3.8-Flash-Next-oQ4e-mtp with oMLX!I'm genuinely shocked! I was able to run Qwen3.8-Flash-Next-oQ4e-mtp on M3Max 64GB with oMLX! Even a couple of weeks ago, I wasn't able to get it to run. I just tried the latest commit for fun, and it→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM RankedIt's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit→12 Oct 2026Improve my vibe-slopped pipelinetl;dr: Are there other models I should try? Is there something better than Kimi K3 for a code review? Is GLM-5.3 better at finding/patching security vulnerabilities? Are skills like https://github.com→13 Oct 20264x 6000 Pro for the companyHere is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo
CompanyNVIDIA8 recent entries11 Oct 2026I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokensDISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and h→11 Oct 2026Every Model That Can Be Run On 10-16GB VRAM RankedIt's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit→12 Oct 2026Qwen3.8-Flash-Next on six 3090s with vLLM: 7 users at 262k, 6.5k t/s prefill, 57 t/s decode solo / 206 t/s aggregate with all 7 generating. Fixes for the upstream included.Write-up, serve command and raw data: https://github.com/lukolszewski/vllm This shows vLLM fixes and setup that let you put PLE on SSD and save 108GB of system RAM, use int8 kv and squeeze more users →12 Oct 2026Lecture: Building a GPU Compiler From Scratch. Lowering PyTorch Graphs to CUDA via Multi-Level IRs (GPU_MODE video + article)https://preview.redd.it/3ekngxk224vh1.png?width=1080&format=png&auto=webp&s=9bfe5c92033e17d5b2fb4529f1320c99455a138b I built a GPU compiler and wrote an article about it, focusing on concepts shared b→12 Oct 2026A Record of the Process of Increasing MFU from 21% to 50% While Pre-training 21B MoE Using a Single GH200 Chip (Includes Code and Logs)I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have com→13 Oct 2026Problems with Tesla V100 SXM2Hi guys! I got a problem with one of my V100 SXM2 Modules. In 'nvidia-smi' its shows with 'Pwr:Usage/Cap' as 'N/A / N/A'. When I try to use it i get the following error in dmesg: NVRM: Xid (PCI:0000:0→13 Oct 2026open-source desktop agent app running its agents on local ollama models: two agents on qwen3.5:9b and qwen3.8 on a 16 gb card, real recording, including the limits i hiti'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you→13 Oct 2026LiquidAI/LFM2.5-24B-A2BGguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai 'LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, p