Search: “model-releases”

Current smallest usable coding model

I've been seeing a lot of news about the latest gemma 4 and qwen 3.6 being really good and the current go-to models but those are out of reach for my GPU at the moment. With 4GB VRAM and 40 GB RAM, I

For ollama cloud $20 plan what models are you guys using

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

I have been trying to do GLM 5.2 as plan / K2.7 as execute, but i hit my usage so fast it's not viable. It's hitting limits much faster than claude code / codex $20 plan. Using in opencode. What are y

model-releasesr-ollama

27 Jul 2026

I ran the 35B agentic comparison someone asked for (stock vs Ornith vs KAT-Coder, 120 runs)

Someone in the comments of my 27B post-train bakeoff asked for the 35B version, so I ran it. Same setup as last time: fresh Coder workspaces on my k8s cluster, each driving my own agent (Hermes) headl

Kat Coder 2.5 is insane. Especially considering I ran it at Q4_K_M

I tested Kat Coder 2.5 with this prompt: Create a spaceship game inspired by Star Fox using vanilla Three.js and HTML. It should have at least five levels, keyboard and mouse controls, enemies, and a

Kimi K3 weights drop today. We're deploying on A100s, H200s and B300s this week and the A100 math is already rough

tldr; we are going to host K3 on A100s (yes, thats correct, we'll try to see if it holds up), H200s & B300s - expect results for A100s & H200s this week while we setup the B300 cluster this weekend &

Ling-3.0-flash weights: SGLang says day-0, vLLM says when they land, llama.cpp closed the 2.6 request as not_planned

Some Ling-3.0-flash threads here last week ended on the same two questions with no real answer, so I went through the repos. State as of writing, with links so you can check instead of taking my word

Nifer is insane. 700t/s with Qwen 3.6 35B (no thinking). Purpose build for RTX5090. Full 250k context too.

I just managed to get it running on windows and this thing is fucking insane. I get around 550-720t/s depending on task at hand. Previously to get to such numbers i would have to do batching and agent

Qwen3.6-27B speculative decoding gets better on heavier quants

I finished the speed leg of my spec-decode benchmarking for Qwen3.6-27B, main algorithms across quants. Overall: the heavier the quant, the more spec-decode buys you (10 of 10 speculative configs rank

Small context windows + knowledge graphs: the serialization format alone doubled my multi-hop accuracy (benchmarked 10 formats)

Running local models means every token counts — an 8K or 16K window fills up fast when you're stuffing graph context into prompts for RAG. I benchmarked 10 graph serialization formats (JSON, GraphML,

We could really use Qwen3.8 in 27B, 35B, 122B and 397B sizes

Instead of 2T+ models, continuing to release highly capable small to medium size LLMs would really help to keep this community vibrant. Hardly anyone can even dream of running the recent 1.5-2T+ beast

You can now fine-tune my 3.96M-parameter TTS on your own voice or language

When I released Inflect v2 last week, I thought most people would ask whether a TTS model this small actually sounded decent. Instead, I kept getting two questions: “Can I train it on my own voice?” “

100B Models on Cheap Hardware: how realistic and limitations

There is a lot of buzz around running 100B parameter models on cheap local hardware using ternary (1.58-bit) quantization like Microsoft's BitNet architecture. The theoretical hardware shortcuts are i

16 bit better than lower quants for Qwen3.6-27B

I am writing a fairly complex C++ windows MFC application. I have a few 3090s and can run F16 Qwen3.6-27B with 256K context and MTP. The quality of code is exceptional with this quant vs its lower qua

23 Gemma4-E4B models compared with abliterlitics: the most downloaded one is also the most broken

This is our biggest comparison yet. We've taken 23 Gemma 4 E4B models from huggingface and ran them through the abliterlitics gauntlet. We also have a new abliterlitics discord, feel free to jump on a

90 agentic bakeoff runs: ThinkingCap vs Fable Fusion vs stock Qwen3.6-27B

Last week someone here said ThinkingCap and Fable Fusion 'really do beat the OG' for agentic work, so I ran it: 6 self-grading tasks, 5 reps, 3 models, 90 isolated runs. Tooling, since that's half the

BeeLlama.cpp v0.4.1: KVarN, KV precision tail, q2_0-q3_1 KV cache, improved support. KLD benchmarks: tail 1024 makes kvarn5 and q6_0 match q8_0, for much less VRAM

TL;DR llama.cpp fork with more KV cache quantization features, with all claims supported by benchmarks: KVarN, KV cache precision tail, additional types of standard KV cache (q2_0-q3_1, q6_0, q6_1), a

GLM 5.2 and ik_llama.ccp

Running GLM-5.2 (the new glm-dsa arch), Unsloth UD-Q4_K_XL, on a 4-socket Xeon E7-8880 v4 box with 1TB RAM and a single RTX 3060 12GB. ik_llama.cpp, experts on CPU (--cpu-moe), 24 attention layers on

Harness showdown: Claude Code vs OpenCode vs Pi with DeepSeek V4 Flash

I ran DeepSeek V4 Flash through Claude Code, OpenCode and Pi on my own benchmark, and the quality came out basically the same across all three while the time and tokens spent was wildly different. Cla

Has anyone compared pre-training, SFT/LoRA and reinforcement post-training on Qwen3.6-27B?

Qwen3.6-27B: SFT vs continued pre-training vs RL? I’m interested in adapting Qwen3.6-27B, but I’m increasingly unsure whether conventional SFT/LoRA is the best route if the goal is to add a capability

I want to use AI coding agents for machine learning projects [D]

model-releasesr-machinelearning

I'm a software engineer who mainly builds softwaes/applications, and I'm starting to work on machine learning projects. Since ML workloads often require GPUs, I know services like Google Colab and Kag

Local-first LLM pipeline tracer — @trace on any function, dashboard at localhost. Feedback welcome.

Hey r/LocalLLaMA — maintainer here, obviously biased. OpenSmith is an open-source Python tracing tool for LLM pipelines. The idea: drop u/trace on any function, run opensmith ui, get a full local dash

Need help with setup

I am setting up codex+ollama+qwen3.6:27b for my hobby coding project on a windows pc with rtx5090 - earlier i tried to setup vllm in Ubuntu container but couldn’t get that to work - now using ollama,

Open-weight 4B models approach o3-level medical question answering in Swedish [P]

model-releasesr-machinelearning

I have been running some experiments with smaller open-weight LLMs on multiple-choice questions of Swedish medical licensing exams. On a dataset called MedQA-SWE, GPT-4 scored 84% accuracy in 2024 and

Simple desktop GUI for multiple local TTS models (Tkinter)

https://preview.redd.it/qoogv5m1skfh1.png?width=688&format=png&auto=webp&s=3f06fb28ceec3cd3ab95bcebd0c71374332aac9b Built a simple desktop GUI (Tkinter) that supports multiple local TTS engines (curre

We compared different LLMs on IMO 2026 [R]

model-releasesr-machinelearning

There are a few reasons why problems from International Mathematical Olympiad function as a good benchmark for LLMs: - The problems are new, not included in the training data of any model - Hard math

Will small model intelligence be limited by parameter count?

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like par

4x 3090, 96gb vram what Model to drive Hermes?

3 year lurker, now i finally got my server up and running. dont know which model to choose. llama.cpp or vllm, what makes more sense? mainly single user with maybe 2-3 more additional users in family,

5700, 48GB RAM and a 3090 24Gb. Best OS and framework/model?

Hi everyone, I have a system which I have been using for gaming, R7 5700X, 48GB DDR4, RTX3090 24GB. But I want to use it for Local AI to reduce my reliance on cloud AI providers (mainly usage limits -

Benchmarks: TensorSharp vs. llama.cpp

Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like

Best C++ Local Model? (July 24th 2026 Edition :-P)

I apologize that this question has been asked in various flavors over time, but I couldn't find anything in the posts before that matches the options I have. I have a PC and a mac, both available in m

CachyLLama: llama.cpp fork with persistent SSD-backed KV caching for local agent workflows

If you run local agentic coding harnesses (Aider, Claude Code, etc.), prompt evaluation usually eats up most of your execution time. Every turn re-evaluates thousands of identical prefix tokens_system

Deepseek V4 flash - Hy3 or is Qwen3.6 27B still the most solid for agentic/coding?

I understand that the laguna model is either still buggy or potentially benchmaxxed. So I’d like to know for people who really tested, are DS flash or Hy3 really better in your usecase? submitted by /

DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report)

Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal is to reduce KV-c

Getting a second GPU in addition to my RTX3090

Hello, I've been learning how to use local LLMs for a year or so on my workstation, using a RTX3090. Current setup : - i5 12400 - 64gb RAM - RTX 3090 - OS : Fedora KDE workstation I'm using LMStudio t

Help me complete my AI collection

I’m building the ultimate AI tool vault, but every great collection has a few missing pieces. Note: I will react to every comment AI's currently installed: Qwen3.5-0.8B-UD-Q4_K_XL.gguf(classification)

I released Inflect v2: two ultra-tiny complete TTS models under 4M and 10M parameters

I’ve spent the past month trying to find the point where an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful. Today I’m releasing Inflect v2, with two

Im back from gemini. GPT is astronomically better again.

like 8 months ago I was tinkering with both and gemini was so much better i went with that. I had a project at work that I needed an AI to sift through a manual and schematic for and no matter what, g

model-releasesr-chatgpt

Is it worth getting 128GB MacBook Pro? Will it ever be comparable to today’s frontier models for coding?

I am a long time iOS app developer. In the last year I have been using Cursor+Claude/others to assist with app development. I am concerned that the current low pricing will disappear eventually. I am

Is this real ? Qwen3.6:27b with 128k context fit in 24Gb VRAM ?

https://preview.redd.it/yw41s1jikefh1.png?width=1942&format=png&auto=webp&s=3a180ae6443c1db9f7b0ce621533a4b2aa553921 Hi, I've been running Ollama on my Unraid server since the llama2 era. I use to be

Kimi Linear 48B A3B?

Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared to Qwen 3.6 35B. after some testing it seems capable of producin

Launching ComfyUI with a Blank Canvas (StabilityMatrix)

model-releasesr-stablediffusion

Hey everyone, I’m posting this question in the subreddit because I haven’t been able to figure it out with the help of AI. I’ve asked ChatGPT and Gemini, but their answers are all over the place. So,

Llama.cpp now has full MCP support!

After a long and grueling effort spearheaded by ngxson, llama.cpp now fully supports MCP for all protocols. Over-the-web HTTP servers were already supported in the client (since they don't require any

MI50 power curve tests

tests done power limiting the GPU on LACT - real power usage varies wildy at 20W it ranges from 25W to 56W same behavior happens on every setting prompt for the test runs: https://github.com/lukesdevl

Mobile Offline LLMs: What do you use them for?

I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to

Ollama Cloud Quota Benchmark

Recently I bought an Ollama Cloud sub and accidently spent my whole 5h quota upon using DeepSeek V4 Pro... but why? isnt it supposed to be a cheap model? Youd think there would be a correlation betwee

Ollama Qwen3.6:35b randomly stops outputting tokens

RTX 4070, 32gb system ram, Linux. NVIDIA-SMI 610.43.03, KMD Version: 610.43.03, CUDA UMD Version: 13.3 Systemd service modifications: [Service] Environment='OLLAMA_HOST=0.0.0.0:11434' Environment='OLL

[audio.cpp] Release 0.4: Higgs Audio v3 TTS 4B (10x real time)+ Fish Audio S2 Pro in C++/GGML, full GGUF loading, Q8 speed and VRAM gains

audio.cpp again :) Release 0.4 is out. The headline this time is new high-quality TTS coverage plus GGUF becoming a first-class across the project. What’s new: Added Higgs Audio v3 TTS 4B, Fish Audio

24 Jul 2026

CachyLLama’s: llama.cpp fork with persistent KV cache that makes long local-agent sessions much less painful

I’m not affiliated with this project, but I’ve been running it recently and I’m surprised it hasn’t received more attention here: https://github.com/fewtarius/CachyLLama CachyLLama is a fork of llama.