Search: “model-releases”

Experimenting with hypersurface-constrained dynamic weight updating [P]

Hey everyone! I want to share the results of a small language model architecture experiment I’ve been running as a side project. The ultimate goal is to test an approach to reduce the number of the mo

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Finally got qwen 3.8 q4 k_M 27b runnin slick 40tkps load at 240k inf 4qnl for stable agentic coding on a 7900xtx. I have been finally able to give hermes a task and come back to results. Built a panel to manage my inference servers from hermes.

DGX agent

Screenshot of heavy load. I did offload vision to a spare 2060 instead of local ram. The control panel was generated with qwen3.8-27b davidau q4k_m max mtp. Made the panel from scratch over a week, ba

model-releasesr-localllama

19 Sept 2026

I enjoyed the daily HF papers today

Top 3 papers on HF Daily Paper are all unusually delightful and interesting reads for anyone on the leading edge of local LLMs, agent harness optimization, etc, felt like sharing. DeepSeek-V4.1-Flash:

I tested Qwen3.8 27B IQ3_XXS (10.18GiB) vs Bonsai Ternary PQ2 (6.42GiB)

I did a small test of the new hyped quantisation of Qwen3.8 vs the biggest quant which fits into my limited 16GB VRAM with decent context. The results are interesting. Of course, the smaller file give

I turned an asymetric pair of Tesla V100s PCIe both (16 GB + 32 GB) into a surprisingly capable local LLM lab — 1.38k prompt tok/s, 40 decode tok/s with qwen3.8 27B Q6 and Q8...

TL;DR: I run a mismatched Tesla V100-PCIE pair—one 16 GB card and one 32 GB card, 48 GB total—in a Proxmox/LXC-based local-inference lab. The practical winner so far is a recent CUDA build of llama.cp

Improved TPS of Gemma 4 31B : the journey and also creating custom patches with VLLM fork

I improved the TPS of Gemma 4 31B. Improving TPS and performing optimisations requires understanding of the model architecture, and I had to fork VLLM and apply my own patch to break into making a con

is this good? 262k Qwen3.8:27B-Q4_K_M

https://preview.redd.it/j13iin0jsfqh1.png?width=1129&format=png&auto=webp&s=3efdd2ad5c10de908aed1a673a6c7e6440177c09 https://preview.redd.it/k3fsrdspzfqh1.png?width=391&format=png&auto=webp&s=1c817687

M1 Max 32GB, trying to run Qwen 3.8 27B at decent speeds and context

I would like to stay at q4, and serve the model to various coding harnesses. None of the models or tools I have tried will run without running out of memory. I am hoping for some advice. oMLX? MTPLX?

ProgramAsWeights: compile English function descriptions into neural programs that run locally [R]

Given the recent interest in tools like Jev, I wanted to share ProgramAsWeights (PAW), an open-source research project I'm working on at the University of Waterloo. You describe a text function in Eng

Qwen 3.8 27B Running LIVE on a RTX 5090 to solve an Open Math Problem - Covering Design C(25,15,5)

This post is a follow up to this other post where I let Qwen 3.8 run for 63 hours autonomously to try to solve the Riemann hypothesis: https://www.reddit.com/r/LocalLLaMA/comments/1wi9fau/qwen_38_27b_

Qwen 3.8 27B running on a single RTX 5090 researches and creates a full animation using only code.

The goal of this post is 2 fold. First, I wanted to show what a local model like Qwen 3.8 27B running on a single RTX 5090 GPU in a harness like Row-Bot can do. Second, I wanted to create a full featu

Qwen-3.8-Flash-Next on 1x RTX 5090: TG=50 t/s, PP=2300 t/s - with FreeToken

I was somewhat disappointed with the performance of Qwen 3.8 Flash Next on my single RTX 5090 using llama.cpp. One issue is that llama.cpp still has no Expert Caching implemented for MoE models. There

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro

Meet Inco Splash, open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents. Req

Reproduce it, or it doesn't count: why training-side decontamination can't be verified, and what an evaluation-side rule looks like [D]

Since OpenAI retired SWE-bench Verified in February (every frontier model tested could reproduce reference fixes for some tasks; underspecified tests rewarded knowing the intended fix), I've been tryi

Steer LLMs and Agents at the Token Level: An interactive tool for token visualization & control, model inspection and data annotation.

onPanda is designed for geeks, power users, curious minds, and engineers. Its UI is built for deep exploration and efficient data annotation. - The core loop is simple: hover over a token → click an a

Training a Neural Network on AMD MI50s Using Vulkan: Proof of ConceptOr: Why I Stopped Listening and Just Did It

Note: This writeup was put together with the help of AI. So I've got dual AMD MI50 32GB cards. If you know these cards, you know the story - AMD dropped official ROCm support for gfx906 after ROCm 5.7

Tuning Qwen 3.8 27B and OMP as a coding agent on 2× 3090s

Oh My Pi + vLLM on two 3090s. Average wait per turn went from 28s to 7s, mostly from changing omp settings: explicit effort level on every role (unset ones defaulted to xhigh) thinking_token_budget of

768gb vram for less than the price of one RTX 6000

I have always posted about budget builds on here, and often asked how we are going to run the next big models. Often Plenty of downvotes too or folks telling me that it's not running if I'm getting 5t

Ctx size in Ollama Vs llama.cpp

So I know that Ollama is built on top of llama.cpp, I decided to run a model on both and applied settings on llama.cpp like flash attention and q8/q4 kv cache. On Ollama (using the Python package) I w

Data-Free Flow Self-Distillation for Few-Step Video Generation — open-sourced a validated implementation on MiniMax-H3 [P]

Hi there, I'm Yuesong, a researcher at Video Rebirth. We open-sourced HyperFlow, an 8-step LoRA for MiniMax-H3. The core idea: data-free flow self-distillation. Diffusers' default sampler does 49 mode

DeepSWE-mini a 16 instance subset of DeepSWE that replicates the rankings of the leaderboard

I just released a new dataset https://huggingface.co/datasets/LocalLLaMA/deepswe-mini it is meant to be more efficent when testing and benchmarking local models on DeepSwe https://deepswe.datacurve.ai

dual 7900 xtx - some guy made a pretty optimized fork of lamacpp optimized for this setup Qwen 3.8 Q8 at 82 tokens / seconds decode

Note not my work, but something i found and wanted to share so hopefully more people can push this along even further. https://github.com/nasone32/llama.cpp-RDNA3-7900xtx-opt I basically run 2 x 7900

Flyweight: open-source C++/CUDA engine for running MoE models bigger than your VRAM on one GPU + system RAM. First PyPI release, looking for contributors.

Been building this for a few months, mostly for myself, and it just got a proper release so figured I'd post it. It's a native GGUF inference runtime with OpenAI/Anthropic-compatible APIs and a chat U

I stopped using the smartest AI models. Programming got faster and cheaper.

Vacations give you time to think and explore. I came across a few things and made a few discoveries of my own. It turns out that, as a programmer, I no longer need the smartest and most expensive AI m

Is HF starting to move against abliterated models?

Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastr

Made the horizontal open-source model for Jev with RLCD, and it surpasses all the Jev benchmarks. HF space, benchmark, model, repo

Thanks for the exceptional support (https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_literally_built_the_jev_architecture_one_year/) and for the dozens of requests to make a generic model, run b

MiniMax Code is now open source. Maybe M3.1 is the next thing to watch.

MiniMax has released the source code for MiniMax Code’s terminal agent: https://github.com/MiniMax-AI/minimax-code https://preview.redd.it/st9izmm8zaqh1.png?width=3016&format=png&auto=webp&s=5853bc316

Multi-hour llama.cpp optimization experiments on Qwen MoE models, patches, benchmarks, and reproduction guides

I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Ne

My long ChatGPT threads quietly forget the beginning, so I keep decisions somewhere else

Been working on this backend migration for like two weeks and my chat with ChatGPT is one of those endless scrolls. Last Tuesday it confidently told me we'd 'decided' on the wrong database. We had lit

model-releasesr-chatgpt

NGL, I’m hyped to see if Qwen3.8 27b can make me a sandwich. Instant buy for me.

Saw this little dude in a Forbes article (https://www.forbes.com/sites/johnkoetsier/2026/08/18/american-humanoid-robot-launches-for-just-1688-delivery-this-fall-made-in-san-francisco/) This is definit

Ollama vs. LM Studio vs. ?

Hi everyone, I’ve been experimenting with local LLMs on my 48 GB RAM MacBook Pro and I’m still trying to figure out what the best setup is. Ollama feels very fast, which I really like. The problem for

Prism-ML Bonsai 2 Joins Our Qwen3.8 Quantization Comparison

Hey r/LocalLLaMA, Prism-LM recently released its Bonsai 2 QAT models based on Qwen3.8, and they quickly gained traction. In our evaluation, the models strike a strong balance between throughput and qu

Question: UkisAI Swift Ternary Bonsai 2 27B?

Hey community, Jovan from UkisAI here, We're the team behind Swift Qwen3.8 27B, the Qwen model with token usage and overthinking error improvements Our estimate is that we can make a great improvement

Qwen3.8-Flash-Next (95.5 GiB) on a 64GB Mac at ~27 tok/s, checkpoint + fork

I've been running Qwen3.8-Flash-Next as my main local coding model from past few weeks. The file is 95.5 GiB and my Mac has 64 GB. It works because the routed experts stay on SSD and only get read whe

Ternary Bonsai is a headless chicken

Ternary Bonsai 2 27B Tried it out and was excited. Gave it a prompt 'create a 3d little planet globe where I (player can walk around) and it has all these biomes to explore, the globe doesn't have to

Testing same face with AI models

model-releasesr-stablediffusion

Tried the same detailed face prompt on a pile of models and in my Opinion KREA beats paid models Wanted something more real than the usual AI “pretty girl.” Same fictional character, same prompt, grid

We benchmarked 24 LLMs against human writers on 475 creative writing prompts

We just released the first version of our Creative Writing benchmark, comparing 24 LLMs against human writers across 475 writing prompts. Creative writing is subjective, so the rankings aren't meant t

What studies isolate back-and-forth LLM interaction from one-way sharing and self-refinement [D]

I'm trying to find out whether back-and-forth between two different LLMs improves task success beyond strong alternatives under a controlled resource budget — and I'd rather hear that it's already set

AndroidLife: Can an AI agent survive a day in the life of a real user? Qwen3.8-27b run: 56.7% SR

I let AI run my phone 60 real tasks, back to back, on the OnePlus I use every day - Best text model still failed 43% of them - Peak chip temp 98.2 C - 69% of the battery gone The benchmark is AndroidL

Are there any model architecture innovations/breakthroughs that we're looking forward to in the next wave of open model drops?

Basically the title. Training models take time and if a model was already in the process of pre-training/RL then it can't be re-cooked with any new architectural improvement that has been discovered i

Beginner confused about Ollama vs LM Studio vs llama.cpp vs vLLM vs Unsloth — can someone explain?

Hey everyone! I'm a beginner getting into local LLMs, and so far I'm familiar with Ollama. I've used it to download and run models locally, but I'm starting to see a lot of other tools being mentioned

Cactus Needle 3: A Sliceable 8-29MB Automation Foundation Model That Matches DeepSeek v4 Flash

Hey all, Henry from Cactus Compute here, I kinda wanted to share our latest model and get feedback from the family :) Needle 3 is a small foundation model for automation: you give it the functions you

Call for compute - help optimize Qwen inference speed on local hardware

Yoyo I renamed https://github.com/syv-ai/qwen38-27b-rtx3090 to https://github.com/syv-ai/HyperQwen/ to focus more on Qwen models and 'local' hardware in general, than just a single card and a single m

Claude Code + Ollama : impossible de faire fonctionner correctement les tools avec les modèles locaux

Salut, je cherche de l'aide pour un projet d'IA locale. J'ai un Dell OptiPlex 7070 Micro sous ubuntu server avec un i5-9500T, 32 Go de RAM et aucun GPU (j'en ajouterai un à l'avenir). Je sais que la m

deepseek-v4.1-flash vs glm-5.3-flash (old subscription)

I think these models are fairly comparable in performance but which uses more compute in the old subscription model? The old subscription model is at least still working in compute usage instead of co

First M5 Ultra benchmarks

just saw some benchmarks on the omlx website for the m5 ultra (don’t know how official they are but they seem reasonable): Link For Qwen 3.8 27B q4 it gets 50 tok/s th and 1800 tok/s pp at8k context a

IFM/K2-Horizon-7B-Uno · Hugging Face - 5200tps with no quality loss

IFM released K2-Horizon-7B , diffusion augmented LLM at upto 5200 tps, with claimed lossless speedup. causal LLM architecture and adds a plug-and-play diffusion adapter alongside the autoregressive we

Intel releases OpenVINO 2026.4

UPDATE https://github.com/ggml-org/llama.cpp/pull/29009 MERGED More Gen AI coverage and frameworks integrations to minimize code changes New models supported: On CPU: Gemma-3n On CPU, GPU: Kokoro-82M,