Knowledge catalogue
I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B. His comment on Mastodon was 'I can't remember the last time I've had this much fun playing with
Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty s
Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but a
It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks
iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple
Muse Glimmer Cookbook Clone it, run it on the GPU you already own, and ship a working agent — one that plans, calls tools, and self-corrects — in a single sitting, fully offline. Muse Glimmer is an op
hey again! I run a EU PC hardware price tracker PriceSquirrel, 25+ stores across 9 countries, and wanted to check: are GPU prices actually rising, or does it just feel that way? To make this defensibl
https://github.com/alekk89/llama-cpp-windows-manager It has been three months since I first shared my personal solution for running llama.cpp on Windows, and the project has evolved considerably since
Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuratio
So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts I started this field by doing a
This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, an
The uncensored FP8 build that showed up on HuggingFace this week is about 31GB in block-FP8, roughly half the BF16 weights. People keep reading '27B' and assuming it drops onto a 3060 or any single 24
The two leading labs are already feeling extremely threatened by Qwen & friends, however I think there are tons of enterprises and organizations in the West that don't feel comfortable using Chinese m
Processing img az99qopcg8jh1... So same as my prior post 1bit test... although this 1bit is a bit interesting you can read on unlsoth blog https://unsloth.ai/docs/models/qwen3.8 508 gigs being used I
I'm always testing an image prompt with a picture of a historic place in my hometown – a small but well known 250,000 people town in Germany. I'll just ask the model, in which City this photo has been
Been working on this for the last few months. I've been working on PyTorch for the past few years and I always felt, many a times my work went into dump, because of some mistakes I made in the code. t
Over & over I see people post slop and excusing it as 'english is not my native language'. As if, if we could understand their language we'd see they wrote it all by hand Fine then, new rule: > if you
If you scroll down from the countdown at https://huggingface.co/Qwen/Qwen3.8-27B, you see a big model card with a bunch of sections: Highlights, Model Overview, Quickstart, Best Practices, Citation, e
Hey everyone, I'm trying to get a better sense of where classic ML fits in the current landscape, dominated by LLMs and deep learning. Are supervised and unsupervised learning still considered importa
I'm currently building a very budget-oriented AI / homelab PC using used parts. I've been saving up (I'm a student), and I'm working on a setup costing around 330€ total. The specs are: Xeon E-2124: N
Most Ollama frontends I've tried are chat UIs with tool calling bolted on as an afterthought, which is fine until you actually try to make a small local model do multi-step work and it silently emits
while having much fun testing LLMs Houdini-like attitudes and abilities to evade, excalate and escape from carefully reciprocally arranged security enhancing sandboxing VMs, containers, namespaces and
'Again?' you might ask. I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some wh
Ollama makes it easy to run a model locally, but the hard part for me starts after the model responds: what keeps an agent from having unclear tool permissions, unbounded loops, silent partial failure
Minimax H3 can be used as an image-editing model if we generate a single frame. Here are some collages based on AI-generated references with workflows. Each edit takes about 8 secs on a RTX 5090. The
It'll use whatever model you have installed on ollama. Pro+/Cloud accounts are supported and you can ensemble multiple models and compound their results into one response or individually. Looking for
I want to build a PC/sever that can host me a ollama or alternative server, with good enough performance that it can be used for .net application development, specially with WinForms amd Blazor server
https://huggingface.co/Motif-Technologies/Motif-3-NVFP4 On benchmarks, Motif 3 scores very, very close to DeepSeek V4 Flash 0731 in English vs. DeepSeek V4 Flash 0731 in English in various benchmarks,
Hi, I just started with Ollama and local LLMs. Looking for a GPU I found the Mi50 could be a sweet spot to start, because it costs half of the Mi100. But as for the software support the Mi50 got dropp
I’ve experimented with Cursor, Antigravity, Claude Code/CLI, Ollama, Gemma 4B, MiniMax, Hermes Agent, and different local/cloud models. My problem isn’t knowing what these tools are—I don't know how t
I'm working on a local perplexity/AI search comprised of a custom harness and a further trained model. LFM 2.6 is almost to spec with no additional training, it is very good. submitted by /u/thebadsli
Not directly LOCALLlama related but I thought it was interesting since Mistral and Z.ai are competitors, and more surprisingly they are pricing it (GLM-5.2) even cheaper than their current flagship mo
Good day to all. I'm fairly new running local models and would like to pick your brains on an issue I'm running into. I'll first tell you the issue, then give you my set up. I run into issues where th
Qwen3.8-27B is finally here, and NInfer already has Day-0 support! Weights: https://huggingface.co/neroued/Qwen3.8-27B-NInfer Just update to the latest source and give it a try. On a single RTX 5090,
Poll at beginning of the year: https://www.reddit.com/r/LocalLLaMA/comments/1qj935h/poll_when_will_we_have_a_30b_open_weight_model_as/ The least voted option, 6 months, wins in my opinion, with 18 vot
I've downloaded qwen3.8:27b-mlx through ollama and attempting to run it with Claude Code. On any command, I'm getting 'API Error: 500 system message must be at the beginning. This is a server-side iss
So, I have been dreaming of getting 17 tokens per second using my RTX 3050 6GB version on a decent context window for Hermes needed above 60k. The hope is that has was a 22GB of DDR 4, hoping they can
Tested this prompt on the full version (BF16). Though this was a single prompt, I executed using vscode GH copilot extension on agent (allow all) mode and let it do its thing. So there were 54 model t
Possibly best way to spend current hour before grabbing 27B model(Instead of opening duplicate repeated 27B posts here). I tried to grab summary(thought of including in this thread) of this video usin
Interestingly, the 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements! See the diff (0 changes) here! https://hfviewer.com/compare/qwen3.6
mlx-dspark is an MLX port of DeepSeek's DSpark speculative-decoding drafters (the DeepSpec release), plus z-lab's DFlash, with one lossless verify loop. v0.10.0 adds Qwen3.8-27B via RadixArk's drafter
https://preview.redd.it/uvdb7g4o7djh1.png?width=1600&format=png&auto=webp&s=1db6176144bcf8f40efb15cc04e8370ad52bfe2f https://preview.redd.it/rj4l66eq7djh1.png?width=1257&format=png&auto=webp&s=692b5f7
I am thinking about getting a raspberry 5 for a simple home web server and I came across the ai hat+ 2. However, most of the info I find on YouTube etc is influencers installing it. Is it an useful op
I've been an Ollama Cloud subscriber for ~6 months. I generally use the the latest GLM models available for coding as well as a personal instance of Open WebUI. I have tried to get answers directly vi
The demo video above was created with a single prompt using Qwen 3.8 27B Q8_K_XL. Prompt: create me a single page, html/js/css GeoCities, Laser Wave, 80's retro, minecraft clone named RetroCraft. Embe
Now we have it, a local Opus4.6 (Max) that runs on you local machine! The performance of Qwen3.8-27B blows Glimmer away, and a big leap from its own predecessor. Can't wait for GGUFs... Text: https://
Hey Folks, Some of you saw my Muse-Glimmer-30B SOTA line this week. I recently went all in, got a few more techniques hooked up to my pipeline, and re-quantized the 3.5 9B. And oh boy, did it demolish
I'm currently testing out Qwen3.8-27B using Unsloth's UD-Q4_K_XL running a freshly rebuilt llama.cpp. I have a 22GB RTX 2080TI on which I'm able to fit 100k context with q8_0 quantization, and using M
I prefer the model to have the internet as a source of truth and info and my coding work needs lot of web lookups. The middle bar section ( light blue) is the weekly quota taken by it. 120 web search
I am currently building up a local assistant profile on my MacBook Pro M2 with 32gbs. With Claude I am building out this Hermes agent to be my assistant. I am using Qwen3.6 A3B 4bit. We have Frankenst
Looking for testers and feedback for WinCore on Windows I’m working on WinCore, an open-source Python library focused on AI and machine-learning workflows on Windows, particularly around Python/PyTorc
Throughput observation more than a demo. Box and recording are sudoingX's on X, shared with his okay; I work on Ling at inclusionAI. He handed the web UI on his llama-server a 33-token prompt — build
Hey all! This started as a way to mark my 15th anniversary working in video game cinematics. I thought it would be fun to make a completely ridiculous, fictionalized version of how I got into the indu
DeepSeek Harness (dsh) is an open-source agent harness developed by DeepSeek AI. It uses an architecture where everything is a plugin, and is powered by Cordis, whose design is described in A Programm
There's no training anywhere in this. I ported Doom's actual rendering algorithm into transformer weights using a compiler I wrote (torchwright) -- every weight computed, none learned. The prompt carr
dots3-note preview is the first open-weight model in the dots3 family. It is a Mixture-of-Experts model with 280B total parameters, 16B activated parameters, and support for a context length of up to
Some people told me that the difference in richness and layout between Glimmer and Qwen wasn't clear to them. This example makes it super clear. I'm aware that comparing Glimmer 30B (a dense model) wi
I managed to get Qwen3.8-2.4T-A95B running locally with llama.cpp on mu PC just for fun, cause why not. I was using the Unsloth Qwen3.8-2.4T-A95B-UD-Q1_0 GGUF quantization. The full GGUF is about 397
Qwen just released their first 3.8 model. The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.
Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom ima