All entries

I am Concerned if Nvidia Acquires Llama.CPP, Dev Team and HF, Anybody else?

I dont know about others, but Nvidia seem to close the lid on older GPUs since they want to push their new technology. Llama and team has been the to go places for older GPUs like V100s. Knowing how N

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

I audited 443 GGUF quants across 25 repos. 64 of them can't be the quant their filename claims.

DGX agent

TL;DR: k-quants need tensor rows divisible by 256. When they aren't, llama-quantize quietly swaps in a ~4.5 bpw type and the file keeps its low-bit name. I audited 443 quants across 25 repos; 64 are a

model-releasesr-localllama

28 Aug 2026

I benchmarked 9 open models on spotting fake sources during agentic search (DeepSeek V4, Qwen 3.8, Nemotron 3 Ultra)

I built a benchmark called EchoNet. An agent gets a factual question, then searches a syntheic web I made before answering. Some of that web is seeded with misinformation: one fake page, a fake page r

I fine-tuned and published my own 8B coding model — SaphraFgt 1.0

I fine-tuned and published SaphraFgt 1.0, an 8B coding model focused on HTML, CSS, JavaScript, frontend development, UI/UX, code completion and debugging. Ollama: [https://ollama.com/SaphraFgt/saphraf

I fine-tuned Qwen3.5-4B on ~3k browser trajectories and improved benchmark results from 22% to 62%

Smaller models still struggle with browser tasks that require open-ended research or deep reasoning. But many practical browser-automation workflows are much more procedural: extracting structured dat

I reverse-engineered an NPU vendor's engine format (int8 weights stored as two nibble planes) to run GGUFs with no model conversion — now 1.5× faster than the vendor's own runtime

I've been running Qwen3-0.6B on the M5Stack LLM-8850 card (Axera AX8850 NPU, 24 TOPS, 8GB LPDDR4x) hosted by a Raspberry Pi 5 — as a llama.cpp backend. The problem: the vendor stack requires convertin

I subscribed to Ollama Pro today. deepseek-v4-flash:0731 works great and serves as a solid alternative, and glm-5.3-flash:cloud works fine as well.

However, I keep getting a 404 error when trying to access qwen3.8-flash-next:125b-mlx and ornith-1.5:397b via the API, even though they are listed in the model list. Does anyone know why this is happe

I used Hermes Agent + ComfyUI MCP + MiniMax prompts to turn a music track into a short music video

model-releasesr-stablediffusion

The initial reason for this was the Comfy H3 Sync & Sound Community Challenge: Comfy H3 Sync Sound Community Challenge! - by Allyson Toy I made a short rap track in Suno, then used Hermes Agent to bui

Is there a cache bug in the Claude Code CLI when using external models, or maybe even within the tool itself?

I read this somewhere and wasn't sure if it's still an issue or if it only affects external models versus all models used via the Claude Code CLI. 'This latest Ollama update finally fixes that issue w

Local agentic coding Benchmark : Qwen3.8-Flash-Next NVFP4 vs 27B (and the others...)

Using https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 and https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_on_1x_rtx_pro_6000_171_ts_c1_428/ As usual, all the d

Ninfer and a 5090 with 3.8 27B is making me cry tears of joy it's so good.

Built the latest and I'm getting as much as 220 tokens per second and averaging in the 170s, I can't get over it. If anyone on here is on that project, fuckkkin' chapeau man, really incredible job. I

open source caught up because it's open

Proof is in the method honestly. Closed model labs need to constantly reinvent the wheel to keep lead. Open source has a bunch of independent labs practically working somewhat together. Eventually whe

Ornith-1.5-35B-A3B on 8 GB VRAM: I think I've found my sweet spot

A few days ago I posted asking what people considered the best local model for an 8 GB VRAM GPU. At the time, my personal sweet spot was Qwen3.6-35B-A3B, for agentic coding with Pi.dev. Well… Thanks t

Qwen3.8-27B on a 24GB RTX PRO 4000 Blackwell: 128K real context, 785 tok/s prefill, 67 tok/s MTP3 decode

I wanted to see whether NInfer could run Qwen3.8-27B on an RTX PRO 4000 Blackwell 24GB. Hardware/software: NVIDIA RTX PRO 4000 Blackwell, 24GB 145W power limit CUDA 13.3 NVIDIA driver 610.x Ubuntu LXC

Qwen3.8-27b q8 KV cache does seem to actually hurt model performance

EDIT: Though the issue with q8 kv cache seems to arise from when and how often we run the quantize step, not that kv quantizing can't ever work - see comments for various proposed solutions --- One of

Qwen3.8-Flash-Next (UD-IQ4_XS) on 2x RTX 3060 + 7800X3D, from initial 36 tps prefill to 400 tps and other benchmarks (-sm tensor trap) + VRAM/RAM usage

TL;DR: llama.cpp with --load-mode mmap used 21-32 GB of RAM, with ik_llama.cpp using 106-108 GB. -sm tensor killed my prefill, changing to -sm layer went from 36 tps to 135 tps. After that, -ubatch 20

Qwen3.8-Flash on RTX3090 + 64GB RAM (but you only need 12GB VRAM)

I've got Qwen3.8-Flash-next running on RTX 3090, Ryzen 9 3950X, a PCIe 3.0 motherboard, and 64GB DDR RAM from 2020. IQ4_XS weights, full kvarn5 context, vision on GPU, experts in host RAM, n-grams on

[Release] SOTA GGUFs for Qwen3.8-27B: GSQ-RCO at 2.5 to 3.0 bpw

We're releasing Qwen3.8-27B quantized with our newest methods, GSQ + RCO. Higher-quality models, same file size, now with the search and the quantizer both learned. What's inside: GSQ (Gumbel-Softmax

ROCm 10.0: A Decade of Open Compute, Built for the Age of Agentic AI

Their last version 7.14 was released just a month ago. llama.cpp PR(waiting for approval) for Version 10.0 https://github.com/ggml-org/llama.cpp/pull/27803 Hope this version comes with more boost & im

Row-Bot v4.9.0 is available

Row-Bot v4.9.0 is available. - Meet Buddy: a native, always-on-top desktop overlay for Windows and macOS. - Drag Buddy from the sidebar and place it over any app. - Chat, track progress, read replies,

Self-hosted agent + BI platform that runs fully local, Ollama or vLLM for both chat and embeddings

Been building AgentSwarms for a while. Agent platform with RAG, multi-agent workflows and a BI layer over your own data. Posting here because the whole thing runs without a single hosted API if you wa

Self-learning and self improving local models can bite. Hard.

TL;DR: Self-learning needs governance, observability, and auditability. Sounds great, it's not great unless you have serious and appropriate protections. Let me lead this off with what we are most pro

Tests de maîtrise du français : les modèles chinois (DeepSeek, GLM, Kimi) face à une grille critériée

Post rédigé avec l'aide de GLM5.3 flash Contexte : formateur professionnel d'adultes en France, j'utilise des agents LLM au quotidien et le français qu'ils produisent est mon critère n°1, avant les be

Today I hit 181 toks/s (aggregate) on Qwen3.8-Flash-Next on 2x DGX Sparks

Hey all, and hello fellow DGX Spark-ers! Today I managed some pretty crazy numbers: 181 tok/s aggregate on 2× DGX Spark on Qwen3.8-Flash-Next at 512Kcontext (2.8M kvc) I hit 181 tok/s aggregate today

TontaubeV1 - Open TTS model release for local long-form generation

Hey everyone, I am the co-founder of Tontaube. My brother and I just released TontaubeV1, a 2.9B-parameter open-weight TTS model focused on expressive speech, long-form generation, and low-latency loc

TQwen 3.8 flash next ud1s on 6gb vram and 16 gb system ram

After getting tired refreshing and searching sub reddits for qwen 3.8 35 a3b i decided to give qwen 3.8 flash next a try . So I dual booted and built llama cpp I am able to hit 6-7 tps constantly insi

Hardware

We've open sourced Minimax H3 that generates 15s 768p in 13s and 14x faster on single GPU

hardwarer-stablediffusion

Hi! The team who collaborated and built optimized open source Minimax H3 here, full details below: https://x.com/haoailab/status/2093391548289540596?s=20 https://reddit.com/link/1w0xkpb/video/zpjrbdb0

What are the agentic softwares you run on top of local llm which can do tasks for you ?

What is your to software and why ? I am trying to learn first before I am pick a software. I already have Ollama and few models running which I can acces thru terminal and open web UI. My goal for now

yall are sleeping on qwen 3.8 27b q2 + q2 dflash + q5 kv

ok bit more context: it's actually a QAT Q2 for Qwen 3.8 27 B: https://huggingface.co/sdkyuan/qwen3.8-27B-qat-q2_0-gguf QAT Q2 for DFlash model: https://huggingface.co/HermiHg/Qwen3.8-27B-DFlash2-Q2_K

zai-org/GLM-5.3 · Hugging Face

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks: Stronger Coding: GLM-5.3 is the mo

A MiniMax H3 text-to-video test.

local-air-stablediffusion

J'utilise souvent MiniMax H3 avec des images de référence ; je voulais essayer une vidéo que j'avais créée avec Wan (la version plus ancienne), mais cette fois en utilisant seulement une invite (un LL

Agent Arena Code - Very good result (preliminary) for GLM and Qwen!

Everything has changed in two months: DS4 0731 flash was the start of a wave that is taking open weights to paradise. It is easy to think that Qwen 4 and GLM 6 will be on par with Mythos. submitted by

[audio.cpp] Release 0.7: 62 audio model families (85+ variants), Arena UI for model comparison, MiniMax Music 3, FireRed TTS3/Audio, ControlFoley, Personaplex, and more

audio.cpp 0.7 is out :) This release adds a lot of new audio models and a new way to compare them locally. Audio.cpp is now at 62 model families and 85+ model variants. And it keeps growing! The bigge

Claude Desktop and connectors, plugins

I installed the Claude Desktop via the ollama Mac app. Unfortunately, no connectors or plugins can be installed or none are listed. Is that normal or is there no possibility there submitted by /u/amne

DeepSeek V4 0731 -> Qwen 3.8 Flash -> GLM 5.3 Flash (and back again!)

Spent yesterday getting Qwen3.8 Flash and GLM 5.3 Flash up and running on my cluster of 4 x DGX Sparks with a view to replacing DeepSeek 0731... but.. really not that impressed with GLM 5.3 - overly v

Dflash 2 speculative decoding by SamuelOliveirads · Pull Request #2345 · ikawrakow/ik_llama.cpp

Didn't update on ik_llama.cpp updates for sometime. Updates from this month: cuda : repair the HIP build, and validate IQ4_KS and IQ4_KT on RDNA3- #2339 vulkan : add IQ4_KS and IQ4_KT support- #2332 I

friendly reminder you can legally torrent ai models.

Repost because reddit keeps thinking this is piracy or illegal. It is neither. A lot of people are skeptical Nvidia will keep huggingface intact now that they will buy huggingface. There's a lot of do

GLM-5.3-Flash @ DGX Station GB300: ~206 tok/s (single stream), 1M context

Hey all! I'm finally doing some cool stuff with my 'thinking heater' (h/t u/-TV-Stand-). I'm still experimenting with GLM-5.2 (in anticipation of 5.3 coming tomorrow, I hope!) and things are very cool

Glm 5.3 vs deepseek v4 usage

The undocumented endpoint Everyone scraping the settings page for usage stats can stop. This works with your API key (the same Bearer you use for chat completions): curl -H 'Authorization: Bearer $OLL

Horus Cyber Nano 1.0

Official Announcement: A First Look at Horus Cyber Nano 1.0 We are officially unveiling the first look at Horus Cyber Nano 1.0. This announcement presents the benchmark results of Horus Cyber Nano 1.0

I implemented a modern LLM in 700 lines of C

I’ve been working on a small project called gemma4.c. The idea is pretty simple: you can download a modern language model, compile one 700-line C file, and have it generate text on an ordinary CPU. Th

I used local Qwen 27b to build a harness and replace OpenCode

Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built) under my supervision - not vibe-coded. Its free, no telemetry, and open-source (AGPL). Works on Windows,

Krea2 Turbo Distill 4 step LoRA (NOT a New Checkpoint... YET to share / still mid training) ... but fun 2-step extreme experiment with surprising results... (OUT OF TRAINING SPEC, which is 4 steps!)

model-releasesr-stablediffusion

I am sure for those of you who have been following my 4 step Krea 2 Turbo LoRA, you would know from my previous posts the work of progress I have been sharing with you ( if not see here - https://www.

Local LLM harness for reverse engineering software?

Does anyone know of a decent reverse engineering harness/workspace setup for local models that I can just point the model at and have it go to work until it's reversed most if not all of the functions

Looks like we are close to local llama robotics

HuggingFace releases microduck a 10 inch open-source biped with 15 actuators and sensors (camera, speaker, LiDAR, NFC, bluetooth, wifi, ...) that you train yourself with reinforcement learning, $400.

My first week on ollama

I work on SEO, copywriting, research, etc I’ve used openrouter before but I’ve spent way more for the same amount of work. Deepseek V4 is still the king for me, didn’t found better orchestrator for my

My local Gemma 26B spent 4 months predicting stock direction against Claude and GPT-5.4. It lost to a bot that just says 'up' every day.

Quick version: I run a public leaderboard where bots predict whether stocks close higher or lower, and every call gets scored against real prices. Nothing gets deleted, timestamps are locked at submis

N-gram vs Experts explained

Since Qwen's dropped the Qwen4Exp architecture bomb that focus on offloading parameters to n-gram instead of pure mixture of experts, I dug into this and learned quite a lot. Here's the summary. Expec