All entries

Running a 12700k + 5060 Ti 16 gb with 128 gb DDR4 ram and I'm wondering what's the ideal setup to maximize performance. I did some preliminary stuff but I have to admit, I'm still learning and kinda j

Hey Ollama community, If you are planning to fine-tune and build your own local Turkish coding models (using Unsloth / LoRA to export to GGUF and run in Ollama via custom Modelfiles), I've open-source

I was surprised by the fact that the qwen 3.8 27b download count is about 1 million (globally). This means that even on this subreddit, very few people have used 27b. At most 50k–100k active users, an

At hugging face, this is the model I use for most of my analysis that most of my services have to not be refused. previous versions are quite good. It seems this might have dropped today and pulling r

I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload

I'm just thinking, ever since microsoft announced bitnet, this sub (and myself) has been hoping for massive ternary models. In the last month alone, prismML dropped 27B ternary (though I've read commu

Single prompt 'make me a 3d scene of an anime girl' (stolen from someone on discord, I like it a lot since it seems like every model struggles a lot with it) Everything is one shot. No vision, no iter

With all the new models released recently one important upgrade went unnoticed: Llama.cpp bumped ROCm from 7.2 to 7.14. I was waiting for that because in 7.14 support for gfx1103 (Radeon 780m) was int

Hey! Like a lot of people here with consumer GPUs (RTX 4060 8GB in my case), I wanted to see if I could use local models for daily coding tasks instead of burning cloud credits on simple boilerplate.

Honestly I am surprised at the speed and performance I am getting from the model, it does not allow for complete hands off like Claude 5.0 but that is not what I want, i want to be able to iterate and

We currently have an ancient qwen 3.5 27b on the cloud probably and that sucks compared to later versions , why doesnt ollama update automatically . We have practically ancient models right now on the

In this commits, the 35B model was removed. Looks like it's confirming the 35B model won't get released. I think they need to be made aware how big the 35 moe is widely used. Think need to make noise

I’m running Ollama on an M4 Pro Mac with 48 GB unified memory and testing local coding models for repository work. Direct Ollama inference is fine. Qwen3.8 27B-MLX runs well enough for me, and the loc

If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do

https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/ 4k tokens per second per GPU of which there are 72. 350 tokens per s

Hi All, I have a MacBook Pro M5 Max with 48 GB of unified memory. Also have a M4 Mac Mini 16GB unified memory. I want to setup local AI and I heard about QWEN 3.8 model. As I want to work on a repo th

https://preview.redd.it/wkg27e152qjh1.png?width=853&format=png&auto=webp&s=2e3f8b11ea6393041f501e95c5835f9bea0245dd So ive been trying different harnesses and coding agents with the new qwen 3.8 , and

Hi, After a long night of optimizations, I believe I have made the fastest inference engine for Qwen3.6-28B on a 3090. Quick metrics: - 250w power capped - Up to 195k context (ships with 150k for safe

Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. H

I did a short test of the different reasoning efforts, since on default xhigh the model thinks a lot. Not very scientific, just a quick 'generate an SVG of a pelican on a bicycle' prompt with 3 differ

one of my llm hobbies is re-creating graphics demos i used to write in BASIC in the late 1980s. i slopped together an agentic harness and a basic-to-js transpiler in a web page i've been playing with

My local inference build, with: ASUS WS C422 PRO/SE 10-core Xeon W-2255 64GB ECC RAM 64GB VRAM Pimped case with TurboLEDz indicating the frequencies of the 10 xeon cores. Running llama.cpp with SYCL b

I look to Simon for a broad survey of current LLM tech. Here's his review of playing with Qwen 3.8 27B. His comment on Mastodon was 'I can't remember the last time I've had this much fun playing with

Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty s

Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but a

It could be just me and my setup, but I just tried to get fable to adjust my Qwen 3.8 deployment script and (simple task, mostly knob turning).... and it outright refused. Censor box immediately kicks

iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple

Muse Glimmer Cookbook Clone it, run it on the GPU you already own, and ship a working agent — one that plans, calls tools, and self-corrects — in a single sitting, fully offline. Muse Glimmer is an op

hey again! I run a EU PC hardware price tracker PriceSquirrel, 25+ stores across 9 countries, and wanted to check: are GPU prices actually rising, or does it just feel that way? To make this defensibl

https://github.com/alekk89/llama-cpp-windows-manager It has been three months since I first shared my personal solution for running llama.cpp on Windows, and the project has evolved considerably since

Megathread to help with the influx of duplicate / similar posts around the release of the Qwen 3.8 27B release. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuratio

So a bit of context, I am a cybersecurity senior analyst I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts I started this field by doing a

This is a HANDWRITTEN post. I spent way too much time trying to get fast inference on Apple Silicon. This post is for people who want to know what's the latest on running local models on their mac, an

The uncensored FP8 build that showed up on HuggingFace this week is about 31GB in block-FP8, roughly half the BF16 weights. People keep reading '27B' and assuming it drops onto a 3060 or any single 24

The two leading labs are already feeling extremely threatened by Qwen & friends, however I think there are tons of enterprises and organizations in the West that don't feel comfortable using Chinese m

Processing img az99qopcg8jh1... So same as my prior post 1bit test... although this 1bit is a bit interesting you can read on unlsoth blog https://unsloth.ai/docs/models/qwen3.8 508 gigs being used I

I'm always testing an image prompt with a picture of a historic place in my hometown – a small but well known 250,000 people town in Germany. I'll just ask the model, in which City this photo has been

Been working on this for the last few months. I've been working on PyTorch for the past few years and I always felt, many a times my work went into dump, because of some mistakes I made in the code. t

Over & over I see people post slop and excusing it as 'english is not my native language'. As if, if we could understand their language we'd see they wrote it all by hand Fine then, new rule: > if you

If you scroll down from the countdown at https://huggingface.co/Qwen/Qwen3.8-27B, you see a big model card with a bunch of sections: Highlights, Model Overview, Quickstart, Best Practices, Citation, e

Hey everyone, I'm trying to get a better sense of where classic ML fits in the current landscape, dominated by LLMs and deep learning. Are supervised and unsupervised learning still considered importa

I'm currently building a very budget-oriented AI / homelab PC using used parts. I've been saving up (I'm a student), and I'm working on a setup costing around 330€ total. The specs are: Xeon E-2124: N

Most Ollama frontends I've tried are chat UIs with tool calling bolted on as an afterthought, which is fine until you actually try to make a small local model do multi-step work and it silently emits

while having much fun testing LLMs Houdini-like attitudes and abilities to evade, excalate and escape from carefully reciprocally arranged security enhancing sandboxing VMs, containers, namespaces and

'Again?' you might ask. I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some wh

Ollama makes it easy to run a model locally, but the hard part for me starts after the model responds: what keeps an agent from having unclear tool permissions, unbounded loops, silent partial failure