Search: “model-releases”

Qwen 27b Q3_K_M 2x rx 580 8gb (~50 each in my country, edge cases 60 per gpu) gives us 16gb vram We used it on an old already existing ddr3 motherboard with 2 gpu slots(you can buy it ror around 200$

Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance.

Hello all, I'm fairly new to the whole Llm space and could really use some guidance from people who've done similar work. Im building a pipeline to extract structured data from large-batch of invoices

If you're running Ollama, llama.cpp, or vLLM behind nginx or HAProxy for more than a single user, you've probably hit this: nginx treats a 10-token prompt and a 10k-token prompt as identical 'one requ

Guys, could someone smarter than me explain what makes Deepseek Harness so efficient? I run it with local Qwen 3.8 (Q6). I tried Opencode/Openchamber (my favourite so far), Pi agent and Hermes. New Qw

Note: I use a Mac Studio M3U with 512 GB RAM, so this is not for everyone. I have been using antirez/ds4 with DS4 Flash for a few weeks now. Top quality, I am really impressed. This combination just d

EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment (TabbyAPI), has a OpenAI compatible API so it shouldn't matter. Why won't this tool m

Do you think today’s GPU-poor systems running Qwen 3.5 9B can outperform the ones we had with ChatGPT-4o? I mean, when 4o disappeared, people felt like they’d lost a great model, and now Qwen 3.5 9B Q

Trying to get a feel for where I stand. If you can list your relevant hardware and model used, that would be awesome. Here's mine: Model: Qwen3.8-27B-heretic-ara, Q5_K_M GGUF T/s: ~30-32 t/sec (I thin

Original Article - https://x.com/p_nawrot/status/2089315591010079034 I've spent the last few years working on efficient attention and KV Cache Compression. I've read many papers, dug deep into referen

I created 3 SVG prompts, each one rather hard. Perspective: A animated drone view perspective on a park. Beauty: A beach scene with an evil cat Composition: An AGI breaking out of a virtual sandbox pr

I'm back. Yesterday I released the first version of hyper-optimized Qwen3.8-27B inference engine for a RTX 3090, reaching 82 tps on single request and 672 peak. Over the last 24 hours I've been explor

Support for the new ling 3.0 models has been merged into llama.cpp: https://github.com/ggml-org/llama.cpp/pull/26608#event-29549472828 Ling tiny 8b1b - https://huggingface.co/inclusionAI/Ling-3.0-tiny

In medium reasoning mode, it both scores higher than the 3.6 version, AND is very much more efficient (almost half requests needed, and a third less tokens generated) - at DeepSeek v4 Flash 3107 MXFP4

Hi all! I finally just got around to testing out Qwen 3.8 27b. I'm using Unsloth's UD-Q8_K_XL quant as a sit-in replacement to Qwen 3.6 27b, same quant size. Wow -- this thing isn't messing around. I

for those looking for something small AND powerful, there is a new 1B (they claim, it looks more like 1.7B ...) model that claims to beat qwen 3.5 0.8B & 2B and gemma 4 E2B on a range of benchmarks. t

Now that OpenCode plan isn't really worth it anymore, I'm looking for other options. I'm trying to figure out the Ollama Cloud subscription, specifically the $20 tier, because that's my budget for mon

I run locally Ollama + Qwen (tried multiples versions) and I connected it with MCP into my local SQLite database. When I ask a question, 90% of times I see it thinking and just stop and never answer t

pagedMark removes AI provenance from content you generated yourself. Two different things, and it is worth separating them. The first is metadata: C2PA Content Credentials, EXIF, XMP, IPTC, the genera

Thought i would just add my own flags here for llama.cpp (literally pulled and rebuilt latest today). Running on a 4090 FE and 48gb of DDR4 ram on WSL2. Basically you have 3 knobs you can tune for max

Yes, it sucks to waste time waiting on 16K+ reasoning tokens alone. But here's the thing, this is only a 27B model trying to perform on par with 1T+ parameter models. Something has to be sacrificed, a

So many comments begging for these models, and I get it. People from the labs frequent r/LocalLLaMA, so maybe the begging comments aren't useless. They make demand known at least. Same with polls, etc

Artificial analysis index scores Qwen 3.5 27b: 35 Qwen 3.6 27b: 38 Qwen 3.8 27b: 52 What the hell kind of a jump was that? Even if it is benchmaxxed, the jump is insane. Qwen3.6 35b A3b: 32 That's ~6

Sorry for the slop, but I was impressed by this model as I have been testing Qwen3.8-27B Q8_0 locally on my ROG Flow Z13 (Ryzen AI Max+ 395, 128 GB unified memory) and this model was the only one who

Yes it uses a ton more reasoning tokens than 3.6 did But test in on the same tasks with the other chinese models, glm 5.3, deepseek v4 flash and pro, etc it's really similar, and they are needed The r

qwen3.8 27B has seriously impressive benchmarks on its model card, but that's for the unquantised version. Almost everyone here will run one quant or another. Are there good benchmarks for how those q

I don't understand this. Deepseek used half as many requests as GLM 5.2 for 3 times the usage, and I'm getting 10 times the requests from flash for like 1/100th the usage. I know Pro is supposed to be

Just curious - I like to be able to see and manage the scripts my agent is working on. I find stuff like Claude Code and Open Code useful for doing stuff on my linux box but I don't understand why peo

I've got a dedicated AI rig sitting here with a RTX 5090 and 96GB RAM and for the past few years have been using Windows 11 and primarily LM Studio, but have also used vLLM, llama.cpp and Ollama. With

Title pretty much says it all. I’ve deployed Qwen 3.8 27B using vLLM on an RTX 6000 Pro (tried multiple vLLM releases and launch recipes), but I can't get it into a usable state because of crazy long

Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying

Running a 12700k + 5060 Ti 16 gb with 128 gb DDR4 ram and I'm wondering what's the ideal setup to maximize performance. I did some preliminary stuff but I have to admit, I'm still learning and kinda j

Hey Ollama community, If you are planning to fine-tune and build your own local Turkish coding models (using Unsloth / LoRA to export to GGUF and run in Ollama via custom Modelfiles), I've open-source

I was surprised by the fact that the qwen 3.8 27b download count is about 1 million (globally). This means that even on this subreddit, very few people have used 27b. At most 50k–100k active users, an

At hugging face, this is the model I use for most of my analysis that most of my services have to not be refused. previous versions are quite good. It seems this might have dropped today and pulling r

I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload

Single prompt 'make me a 3d scene of an anime girl' (stolen from someone on discord, I like it a lot since it seems like every model struggles a lot with it) Everything is one shot. No vision, no iter

With all the new models released recently one important upgrade went unnoticed: Llama.cpp bumped ROCm from 7.2 to 7.14. I was waiting for that because in 7.14 support for gfx1103 (Radeon 780m) was int

Hey! Like a lot of people here with consumer GPUs (RTX 4060 8GB in my case), I wanted to see if I could use local models for daily coding tasks instead of burning cloud credits on simple boilerplate.

Honestly I am surprised at the speed and performance I am getting from the model, it does not allow for complete hands off like Claude 5.0 but that is not what I want, i want to be able to iterate and

We currently have an ancient qwen 3.5 27b on the cloud probably and that sucks compared to later versions , why doesnt ollama update automatically . We have practically ancient models right now on the

In this commits, the 35B model was removed. Looks like it's confirming the 35B model won't get released. I think they need to be made aware how big the 35 moe is widely used. Think need to make noise

I’m running Ollama on an M4 Pro Mac with 48 GB unified memory and testing local coding models for repository work. Direct Ollama inference is fine. Qwen3.8 27B-MLX runs well enough for me, and the loc

If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do

https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/ 4k tokens per second per GPU of which there are 72. 350 tokens per s

Hi All, I have a MacBook Pro M5 Max with 48 GB of unified memory. Also have a M4 Mac Mini 16GB unified memory. I want to setup local AI and I heard about QWEN 3.8 model. As I want to work on a repo th