All entries

My RULE of Thumb of choosing a models

This is mostly for setting up for expectation, since personally without LLM i could take 3 days (15 hours of active programming) to debug or implement a feature, but with Qwen 27B (even before Qwen 3.

Need to decide: DGX spark vs framework desktop vs Mac mini/studio

4,783

Source

Human DGX agent

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

I’ve been running qwen on my personal Mac but I’m getting to the point where’d I’d like to have something always on, running various jobs, and some more ability to experiment and earn about fine tunin

model-releasesr-localllama

3 Sept 2026

New ollama plan usage

Bruh wtf, i switched to new method now im pissed, 20 dollars a month for 60 api cost a month ?? Open code provides this for 10 dollars a month , and if I use a beast like kimi k3 I can drown my months

local-air-ollama

Ollama Pro is 20, but the new '60 credits' are more like $7-8 elsewhere

Everyone is talking about how the new credit system gives significantly less usage than the old Ollama Pro limits, but there's another part of the pricing that seems a bit crazy to me. For example, I

Hardware

Open-sourced an experimental standalone DLSS 5 video player for neural rendering

hardwarer-stablediffusion

I’ve been experimenting with neural rendering outside a game engine and built a native Windows video player around it. It prepares a neural-rendered version of a video, caches it, and lets you switch

Openvurp: a wallet of agents that work together

I have a few AI subscriptions and I kept running into the same problem. I had powerful models available, but I struggled to use them in real life in an organized way. I looked at projects like OpenCla

Qwen 3.8 27B Vs. Qwen 3.6 27B on oMLX

Quality: 81.1 → 87.7 (+8%) Speed: 35 → 29 tok/s (−16%) Runtime: 8m51s → 44m39s (5x longer) Output tokens: 18K → 78K (🤯) Noticeably better quality, but you're paying for it with tokens and time. Full b

Qwen-3.8-Next-Flash Ngram Hot-Swappable Knowledge Injector for llama.cpp

Looking into the new Qwen architecture, I was curious if you could modify the Ngram PLE Table to make it work like a long-term knowledge database. It turns out that, with some limitations, you can. I

Qwen3.6 35b Q2_XXS: Being GPU poor in 2026 is not so bad

A potato can create a very cool RPG in 24 minutes Laptop I3 8gb RAM 0gb VRAM, Windows 11 llama-server.exe --host 0.0.0.0 --port 8080 -m modelsqwen3.6-35b-InstructQwen_Qwen3.6-35B-A3B-IQ2_XXS.gguf -c 8

Qwen3.8-Flash-Next MTP merged in ik_llama.cpp (integrated head or separate -md file)... 45 → 90 tok/s on a 5090 + 128GB, works down to a 12GB 4070

ik_llama.cpp merged qwen4exp MTP support yesterday (PR #2369, mine, reviewed and tested by four other people on their own hardware). It's on main now, no fork or patch needed. Posting since the last c

Qwen3.8-Flash-Next on 2x3090 + DDR4: 17 → 25-29 t/s decode with the expert cache PR

Sharing some numbers because most posts on this model are either using a single 3090 or unified systems from what I've seen. My current setup: 2x RTX 3090 (PCIe 3.0), dual Xeon E5-2696 v4, 188 GB usab

[RELEASE] - SupraGDN-5M - a tiny GatedDeltaNet model competing with other community while models trained on MUCH less data!

https://preview.redd.it/5rs3d7vtgbnh1.png?width=950&format=png&auto=webp&s=df828c1146bc6167bd09d3300018aba426fc1ab2 Hey, r/LocalLLaMA ! We are releasing SupraGDN-5M. It's a tiny GatedDeltaNet (GDN1) b

Repodify, a fully local & opensource podcast summarizer, or BYOK if don't have GPU.

Disclaimer: I'm the builder. --- Over the past 2 yrs, I was working on a SaaS ML project & got very interested in ML/DL/AI. As everybody else, there were some normal paths I took to build a solid unde

Testing Deepseek-V4-Flash-Vision-EXP on Dual RTX6000 build

Upgraded my work LLM server from DSV4-Flash-0731 to the new Vision-Exp model and did this little test for fun to see quality comparison. Test was performed in Openweb UI connected to my vLLM container

Testing orangesouth/MinimaxH3CinematicRealism and vpakarinen/better-human-motion-h3-lora 'dean winchester vs sub zero' pruned fp8 model no turbo lora 32steps

https://huggingface.co/orangesouth/MinimaxH3CinematicRealism/tree/main https://huggingface.co/vpakarinen/better-human-motion-h3-lora/tree/main prompt integrated_multimodal_description: dy, [Shot 1] Ph

Training-free hack to reduce MoE reasoning tokens by 8.5% without losing accuracy but increasing Active parameters per token in MOE (Qwen 35B A4B+)

Hey everyone, Wanted to share a short paper we just published exploring a simple but surprisingly effective optimization for sparse MoE reasoning models. The problem: MoE reasoning models (especially

We built an open-source, model-neutral agent harness and compared it with claude managed agents - for the same model, got same accuracy, upto 75% lower cost

We have been working on an open-source, model-neutral agent harness for general purpose agents called TrueForge, and wanted to understand how much the harness itself actually matters. So we ran 14 tas

We need a better taxonomy for what people are calling 'continual learning'

Continual learning isn't some fake term but a real goal and arguably one of the most important open ones in 2026. It roughly means: ‘building systems that keep acquiring useful knowledge or skills aft

What are your best sources to learn about LLM inference?

The job of doing inference is not the same for every model ou there, as it depends on the architecture and configurations and the available machines. Many times we would not have the right machines to

WVY is a handwritten language model. Every response was written by one person to demonstrate that the illusion of intelligence is not exclusive to parameter count.

***This screenshot is an app i made for creating a dataset from scratch, this is not a real chat with the model*** First of all i want to shout out everyone that actually tested our work .. we got 500

An endless AI TV channel on a single gaming GPU — MiniMax H3, generating faster than it plays

local-air-stablediffusion

There is a video stream running on my desktop right now. It has sound, it has never repeated itself, and it will not stop. I point VLC at a local URL and it plays. One RTX 5090 does all of it — no clo

Android Studios native Gemma 4 runs on llama.cpp

https://preview.redd.it/6e9xb57a42nh1.png?width=787&format=png&auto=webp&s=ffae7996bbf8ab00498cc62c733e7597dc550f24 I'm not sure how many people care about Android Studio, but I think it's cool that G

Confirmed bolting Q8 NGram into IQ4 Qwen no speed degradation

This came from another thread or comment. I forgot exactly where, but the basic idea was to replace the 51B N-gram layer in Qwen 3.8 Next with a much higher precision version. Someone running a 5090 r

Tutorials

Detailed explanation of how to create a text-to-image model from scratch.

tutorialsr-stablediffusion

Posting this here even if it's not a model you can use directly. It's about building a text-to-image model from scratch. The cookbook includes all the research material that you may or may be not inte

First local-LLM tuning attempt: Qwen3.8-27B true Q4_K_M at 13.2 tok/s near 50-61K context on RTX 5080 16GB

This was my first serious attempt at tuning a local LLM. I started because Qwen3.8-27B IQ3 was fast on my RTX 5080 but the coding quality disappointed me, and the Q4 profiles I tried in LM Studio were

Tutorials

Fizgig 5.2 - combining two Minimax training methods beats either alone

tutorialsr-stablediffusion

Two of the ways that exist (im sure there are more) to train a LoRA on H3 well are on two different trainers. Fizgig's is Optimised Likeness Learning: I've found the stable core of H3's identity lives

Hardware

I built a standalone DLSS 5 Neural Rendering video tool, no ReShade

hardwarer-stablediffusion

It supports images and full videos, native resolution NR or DLSS Super Resolution upscaling by scale factor / target resolution, GPU optical-flow motion vectors, scene cut handling, and all DLSS 5 NR

I'm free! No babysitting for me... GLM-5.2

I've said this in another post, but again started with Claude Code, loved OpenCode, but I felt chained and tied to them both for the longest time (especially CC, just the worst). I felt less productiv

LocalLLaMA is unironically one of the best places to go to get up to date AI news.

One of the other posts today by user u/Howard_banister confirmed what I've been seeing from the other AI subreddits as well. Most of these other subs are 90% trend hopping crypto-bros equivalent peopl

local-air-localllama

Looking for a small LLM for Linux command generation

I'm looking for a small LLM to act as a Linux command assistant. I will use llama.cpp. use case: - User asks in natural language. Model outputs only the shell command - Should work well with no reason

Mac Heads: Is there any point to MLX in September 2026?

This may be somewhat specific to Qwen3.8 27b and the Apple M5 series, perhaps, but enough of us are running this combo that it's worth tossing out there. GGUF models with MTP have been the fastest way

My Minimax H3 Workflow Benchmark Data -

Alright, I posted that I had my agent test a bunch of different workflows for over 12 hours and got the 'Bro just wasted 12 hours of credits'. It was obvious the proof should come from the visual data

Old Pro plan isn't the same anymore

I'm still with the old pro plan, and I've never (almost) reached my weekly limit so fast like today with that many requests and using OpenClaw for pretty basic stuff. There is some serious monkey busi

local-air-ollama

Opencode vs Deepseek harness: my experience with Qwen 3.8 27b

Anybody else struggling with deepseek after the initial prompt? Somehow it is getting mixed up very easily, even button functionality has been PITA when doing stuff. Never had these problems with Open

Perplexity open-sourced their Mac inference server for Qwen 3.6

https://preview.redd.it/6h4xc5o8l6nh1.png?width=1158&format=png&auto=webp&s=b65074b6baaa1faa2347e5259229c8ba803bcd4b Here is link to repo: https://github.com/perplexityai/pplx-garden/tree/main/lily It

PSA: If you’re on the old Ollama Pro plan, don’t switch - ~67% LESS usage

Ollama has just announced, based on 'feedback', that they have switched to a credit model. If you're still on the old 20 Pro compute plan, DON'T SWITCH. I decided to calculate it before I risked click

Pulled the trigger, RIP $6,279

(Paid 5,849 + tax, which came out to 6,279) TL;DR - Bought this 5090 prebuilt and I want to sanity check if I made the right decision and at the right time. Hey everyone. So I want to start off by say

Quad R9700 AI Pro with vLLM-Radiance easily reaching 17,6k PP

https://preview.redd.it/74bmvel9b5nh1.png?width=1602&format=png&auto=webp&s=0d0c1adaa016a486ffd97c4c466e980dc611b139 I've only recently started looking deeper into vLLM after running llama.cpp for a g

Qwen will be the king?

Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, w

Qwen3.8 Flash AP Quants

Quite surprised to be beating other high quality quants. It took a lot of benchmarking to get here and we are quite pleased with these, hope they are useful to the community. It required a modified wa

local-air-localllama

Qwen3.8-flash-next sees corruption everywhere

Hi, I've noticed that the model often sees 'garbled text' in its context. Sometimes it declare that the tools instructions are corrupted, sometimes it is the content of some .md files, ora other files

The 1967 Spider-Man TV Show intro, updated to live action with MiniMax H3

R2V Rendered at 0.9 MP (1280x736 then upscaled using RTX (Ultra) to 1920x1080. Edited and merged using OpenShot video editor. This was all run on my Windows 11 machine, RTX 4060 ti (16 GB) and 64 GB R

Value analysis of the old Ollama Pro sub model vs the new per-token model (from 50 days of real usage)

Why I have this data: one Pro account kept hitting limits, so I run three Pro subs through 9router, a self-hosted router that sits between my coding tools and ollama cloud. It spreads requests across

VoxGen, an AMD-optimized TTS inference engine for VoxCPM 2 models

Hi, everyone, I’ve just released VoxGen, a lightweight native inference engine for VoxCPM2, written in Rust and using Vulkan compute instead of Python/PyTorch/CUDA. Why VoxGen? The main reason I start

Which one will you choose and why, between R9700 32GB vs W7800 48GB?

I'm planning to upgrade my workstation (linux with 5700X/64GB DDR4) for local inference and pytorch training. I'm trying to decide between: 2× AMD Radeon AI PRO R9700 32GB 2× AMD Radeon PRO W7800 48GB

Will this server hardware work?

Years ago I built a server, shelved it after some time, but recently realized I have it sitting there. I'm looking to pop in a Tesla V100 32GB card (with proper cooling). But are the component too old

Your favorite fastest abliterated/safety removed 3.6 and 3.8 27b?

Not written by AI all mistakes mine. I saw people on the subreddit saying that 3.6 works better without thinking. It made me want to know for certain about which is better, 3.6 or 3.8 for low thinking

A very confusing report from Puget Systems