All entries

New with Local models, need some help

Good day to all. I'm fairly new running local models and would like to pick your brains on an issue I'm running into. I'll first tell you the issue, then give you my set up. I run into issues where th

Hardware

NInfer day0 support for Qwen3.8 27b: ~200 tok/s generation, with tons of engine improvments

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

Qwen3.8-27B is finally here, and NInfer already has Day-0 support! Weights: https://huggingface.co/neroued/Qwen3.8-27B-NInfer Just update to the latest source and give it a try. On a single RTX 5090,

hardwarer-localllama

14 Aug 2026

Poll results 6 months later: When will we have Opus level with 30b model? Optimists win!

Poll at beginning of the year: https://www.reddit.com/r/LocalLLaMA/comments/1qj935h/poll_when_will_we_have_a_30b_open_weight_model_as/ The least voted option, 6 months, wins in my opinion, with 18 vot

Problems with Ollama/Claude Code and Qwen3.8:27b-mlx

I've downloaded qwen3.8:27b-mlx through ollama and attempting to run it with Claude Code. On any command, I'm getting 'API Error: 500 system message must be at the beginning. This is a server-side iss

model-releasesr-ollama

Qwen 30b MoE - 30tps - 6GB vram - Done!

So, I have been dreaming of getting 17 tokens per second using my RTX 3050 6GB version on a decent context window for Hermes needed above 60k. The hope is that has was a 22GB of DDR 4, hoping they can

Qwen 3.8 27B - Aquarium Burst Sample Test

Tested this prompt on the full version (BF16). Though this was a single prompt, I executed using vscode GH copilot extension on agent (allow all) mode and let it do its thing. So there were 54 model t

Qwen Live EP2 - Agent First: Multimodal Gets to Work - Streamed live 11 hours ago

Possibly best way to spend current hour before grabbing 27B model(Instead of opening duplicate repeated 27B posts here). I tried to grab summary(thought of including in this thread) of this video usin

Qwen3.8-27B is identical to Qwen3.6-27B!

Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements! See the diff (0 changes) here! https://hfviewer.com/compare/qwen3.6

Qwen3.8-27B is now up to ~3× faster on Apple Silicon with mlx-dspark

mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter

Qwen3.8 Benchmarks Converted to Charts

https://preview.redd.it/uvdb7g4o7djh1.png?width=1600&format=png&auto=webp&s=1db6176144bcf8f40efb15cc04e8370ad52bfe2f https://preview.redd.it/rj4l66eq7djh1.png?width=1257&format=png&auto=webp&s=692b5f7

Raspberry pi 5 and ai hat+ 2 any good?

I am thinking about getting a raspberry 5 for a simple home web server and I came across the ai hat+ 2. However, most of the info I find on YouTube etc is influencers installing it. Is it an useful op

Request: More Transparency on Ollama Cloud Subscriptions

I've been an Ollama Cloud subscriber for ~6 months. I generally use the the latest GLM models available for coding as well as a personal instance of Open WebUI. I have tried to get answers directly vi

RetroCraft - Qwen 3.8 27B Q8, one shot with exact performance data on dual 3090s.

The demo video above was created with a single prompt using Qwen 3.8 27B Q8_K_XL. Prompt: create me a single page, html/js/css GeoCities, Laser Wave, 80's retro, minecraft clone named RetroCraft. Embe

Return of the local King! Qwen3.8-27B performance chart

Now we have it, a local Opus4.6 (Max) that runs on you local machine! The performance of Qwen3.8-27B blows Glimmer away, and a big leap from its own predecessor. Can't wait for GGUFs... Text: https://

Taking Qwen3.5-9B quants to SOTA. New lineup incoming :)

Hey Folks, Some of you saw my Muse-Glimmer-30B SOTA line this week. I recently went all in, got a few more techniques hooked up to my pipeline, and re-quantized the 3.5 9B. And oh boy, did it demolish

The difference between 'medium' and 'xhigh' reasoning effort for Qwen3.8-27B is actually insane.

I'm currently testing out Qwen3.8-27B using Unsloth's UD-Q4_K_XL running a freshly rebuilt llama.cpp. I have a 22GB RTX 2080TI on which I'm able to fit 100k context with q8_0 quantization, and using M

Web-search eating substantial quota

I prefer the model to have the internet as a source of truth and info and my coding work needs lot of web lookups. The middle bar section ( light blue) is the weekly quota taken by it. 120 web search

model-releasesr-ollama

What can I realistically do?

I am currently building up a local assistant profile on my MacBook Pro M2 with 32gbs. With Claude I am building out this Hermes agent to be my assistant. I am using Qwen3.6 A3B 4bit. We have Frankenst

model-releasesr-ollama

WinCore

Looking for testers and feedback for WinCore on Windows I’m working on WinCore, an open-source Python library focused on AI and machine-learning workflows on Windows, particularly around Python/PyTorc

A 124B emitted 15,128 tokens in a single response on one DGX Spark, decode went 35.62 → 35.68 tok/s across the whole thing

Throughput observation more than a demo. Box and recording are sudoingX's on X, shared with his okay; I work on Ling at inclusionAI. He handed the web UI on his llama-server a 33-token prompt — build

CYBER SLAYER — a 1995-style action trailer made with MiniMax H3, Wan 2.2 and ComfyUI

model-releasesr-stablediffusion

Hey all! This started as a way to mark my 15th anniversary working in video game cinematics. I thought it would be fun to make a completely ridiculous, fictionalized version of how I got into the indu

Deepseek Harness is Up!

DeepSeek Harness (dsh) is an open-source agent harness developed by DeepSeek AI. It uses an architecture where everything is a plugin, and is powered by Cordis, whose design is described in A Programm

Doom running on an LLM -- Hugging Face checkpoint included

There's no training anywhere in this. I ported Doom's actual rendering algorithm into transformer weights using a compiler I wrote (torchwright) -- every weight computed, none learned. The prompt carr

dots-studio/dots3-note-prev · Hugging Face

dots3-note preview is the first open-weight model in the dots3 family. It is a Mixture-of-Experts model with 280B total parameters, 16B activated parameters, and support for a context length of up to

DS4 cloud (30 min) vs Qwen3.6 36B (2 min) vs Muse Glimmer 30B (3 min) on Llama.cpp (RTX 5080)

Some people told me that the difference in richness and layout between Glimmer and Qwen wasn't clear to them. This example makes it super clear. I'm aware that comparing Glimmer 30B (a dense model) wi

EXPERIMENT: Qwen3.8-2.4T-A95B running locally on an RTX 5090 + RTX 5060 Ti at ~0.80 tok/s

I managed to get Qwen3.8-2.4T-A95B running locally with llama.cpp on mu PC just for fun, cause why not. I was using the Unsloth Qwen3.8-2.4T-A95B-UD-Q1_0 GGUF quantization. The full GGUF is about 397

Fixed Jinja chat template for Qwen 3.5, 3.6, and the new 3.8 release

Qwen just released their first 3.8 model. The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.

Gemma 4 12B Q3: +8.55% Coding Performance From Tensor-Level Quantization Allocation

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom ima

ggml-cpu/ops: vectorize flash-attention V-cache F16 to F32 conversion by jinzihao · Pull Request #26947 · ggml-org/llama.cpp

Overview ggml_cpu_fp16_to_fp32 leverages hardware F16C intrinsics (AVX-512, AVX2, etc.), faster than the software-only ggml_fp16_to_fp32_row, bringing 17-31% gain in prompt processing rate for a small

GitHub - deepseek-ai/deepseek-harness

🧩 DeepSeek Harness v0.1 is now available in Developer Preview! 🔹 We’re opening it up to developers building agent harnesses worldwide and open-sourcing the codebase in MIT license. 🔹 Powered by the Co

How do you plan to run Qwen3.8-2.4T-A95B locally?

To my fellow crazies, the few. Those who dared wrestle with llama-70b, mistral-large, goliath, mistral8x22B, DeepSeekV2/3, wept when llama4 behemoth was announced, picked yourself up and are now wrest

I asked DeepSeek-V4-Flash to work with Muse-Glimmer for Vision ability in PI agent and it produced this

Same old prompt, just appended a TIP in the end: 'Write a single HTML file with a full-page canvas and no libraries. Simulate a realistic side-view of a moving car as the main subject. Keep the car vi

Interesting uses for Muse Glimmer 30B?

Hey everyone, Non-native speaker, writing my post by hand, let me know if I make mistakes (can only learn from it!) Muse Glimmer 30B is so far quite nice, but I haven't found a clear-cut case yet what

Minimax-H3 can generate 42s videos natively on an RTX Pro 6000 in 80 minutes

model-releasesr-stablediffusion

The maximum frame count allowed by the native 'MiniMax H3 Reference to Video' node technically is 1008, even if that's way over the training range of the model, which is 324 frames. But why not try? S

Minimax Music 3 open weight release soon?

Diffusers has a PR with deets: https://github.com/huggingface/diffusers/pull/14456 Minimax is working on this repository right now and put up a bunch of samples: https://github.com/MiniMax-AI/music3-d

Nvidia doubles RTX PRO 6000 Blackwell's MSRP to a staggering 16,000 — 96GB card started pre-orders below 8,000 last year

Given Gavin Bakers recent interview where he says: > Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publ

Ollama Pro annual subscription feels borderline scam at this point: $200 paid, Kimi K3 paywalled, lower limits, and still no refund response

I’m genuinely pretty disappointed with how my Ollama Pro annual subscription turned out. For context, I’m a technology editor and developer, and I had actually used Ollama Pro before. My previous expe

Qwen3.8 2.4T UD-Q1_0 - 178 token generation - 11 min 38s - 0.25 tokens/sec

https://preview.redd.it/nqr9nj5028jh1.png?width=1516&format=png&auto=webp&s=47af40a0737429dc65f2e455b87b2db81725cb14 So I wanted to see... is it possible/viable to run this perhaps once in a while som

Safety

Reproducible canvas-aligned low-level patterns in somerandomllm-generated images and their possible relation to iterative editing artifacts [D]

I may have stumbled onto something interesting while trying to figure out a recurring artifact in ChatGPT image generation and editing (maybe applicable to other models as well?). It started with a ve

safetyr-machinelearning

Row-Bot v4.7.0 is live

This release brings smaller prompts, longer-running conversations, safer remote access, and more reliable model streaming. External tools and skills are now discovered only when relevant. Long chats g

SenseNova-Vision: a 7B open model that does segmentation, depth, detection, OCR, and 3D reconstruction with no task-specific heads

Stumbled across this new vision model, SenseNova-Vision. It's a 7B MoT model, Apache 2.0 license, which is cool. The main idea is it treats pretty much all computer vision stuff as just one generation

Safety

Stolen LLM Reasoning: How come OpenAI, Anthrophic, Google have the same vulnerabilities?

If you haven't checked the paper: https://arxiv.org/abs/2608.09867 TLDR: the authors show that you can swap out the 'encrypted' reasoning of the biggest model, like Opus, Sol, and put them into weaker

safetyr-localllama

Applications

Today I have unsubscribed from Suno thanks to Minimax Music.

applicationsr-stablediffusion

With all the Suno drama over their download limits and their heavy watermarking that could be used for future copystrikes if you stop paying them, this Minimax Music 3 couldn't be more on time. I trie

Trained a 1.5B to write shell commands so I'd stop googling tar flags. Runs on a laptop CPU in ~1 sec.

I've been googling 'tar extract gz' for about ten years. and I finally did something about it. It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-

Tutorials

UrgenT Help Detecting Performance Regressions Using Machine Learning and Hardware Counters [P]

tutorialsr-machinelearning

I’m working on performance regression detection using machine learning/anomaly detection. My setup is basically: Healthy runs are used to learn normal behaviour Regression runs are used to see whether

Agents

While waiting for the release of Qwen3.8-27B, let's try to guess what will happen

They highlighted 3 things on countdown page: VLM, Agentic Improvements, and Think mode. What improvements do you expect? Reply here! Personally, I want to meet a sage who has attained enlightenment. T

agentsr-localllama

Hardware

worldproof: diagnosing where world-model predictions break and a measurement of when pixel metrics stop being able to rank models at all [P]

hardwarer-machinelearning

I've been building an open-source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth an

According to AMD, Arm, and Microsoft, agentic AI could push CPU-to-GPU ratios from 1:4 to even1:1

In OCP APAC 2026, Tai AMD SVP of compute and enterprise AI said agents don't cut GPU demand but they just pile on a whole extra layer of orchestration, retrieval, and tool-calling work that runs on CP