Search: “model-releases”

Need maybe say 'Use llama.cpp'

So I tried that miracle engine everyone is talking about. Asked the IQ3_S model to express its opinion on a post from this sub to measure the tps on a long-ish generation: Can you help with the follow

Nonobench: an open benchmark of 49 LLMs on nonogram puzzles, public and open source [P]

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

Nonobench measures how well LLMs solve nonograms (picross). Each model gets the row and column clues once and returns the full grid. No tools, one attempt per puzzle. Method: - Standard mode: 30 puzzl

model-releasesr-machinelearning

4 Oct 2026

Poor People Vulkan GPUs list

Help with this list. Give me your recommendation on 'not supported anymore' GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with so

poorman inference engine for 16GB GPU and 35B moe Qwen 3.6for coding

I forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B (~A4B) with Unsloth quants. On a (vant.ai) rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 to

Qwen3.5 arch implementation in FPGA fabric for 9B/27B INT4 models on relatively cheap eBay mining hardware

I've been wanting to test out LLM inference on FPGAs for a while now, but didn't have a big reason too since there were no frontier class models at 9B-27B scale (3.6 27B was great still but it didn't

The curse of 64GB system RAM

Not a bot. Not a Strata shill. Just sharing my experience. So, I have an R9700 in my machine, plus an RTX 5060 Ti, and 64GB DDR5 system RAM. Overall, not a bad setup. Anyway, I mainly run a daily driv

Update #4: Post training yandex/AliceAI-80B-A3B [instruct!] from scratch

Last update for those following: https://www.reddit.com/r/LocalLLaMA/comments/1wvyc3e/update_3_post_training_yandexaliceai80ba3b/ Project in a sentence: An instruct finetune of ALiceAI-80B-A3B-Base ca

ai slop game based on batlle video in real time with only open model

Check out Chimera Arena (https://chimeraarena.com/static/showcase/index.html?lang=en)! Here’s my slightly scrappy AI-generated teaser for Chimera Arena, my card game combining AI-generated video battl

Anyone sitting on a lot of slow system memory and a modest GPU.. try Strata + Qwen3.8 Next.

IQ3_XXS weights are just under 80GB and my slowww DDR4+7900XTX is stabilizing around 45-70/s (sometimes higher while coding depending on mtp). Looking online I'm seeing similar results for users with

Anyworld, a self-hosted multiplayer text RPG where a local LLM is the Dungeon Master

Hey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its brows

Dual Radeon MI50 benchmarks

Still don't have a good cooling solution, but here are few benchmarks. I lowered the power limit (TDP) to 145 watts each. I changed the firmware on one MI50 to activate the miniDP port. Did have to us

I built a code knowledge graph tool that's actually MIT licensed (fully local, no cloud)

So this is maybe a niche problem, but at my job I work on a huge Python codebase and every time I change some shared function I'm basically playing roulette. grep tells who mentions it in the code bas

I built Ninfer 4080 for 16GB class GPUs

Hi everyone, TL/DR I created NInfer 4080 to run ISTA-DASLab-Qwen-3.8-27B-GSQ at 100k context on an RTX 4080 16GB GPU using way more of the hardware capabilities (max overall: 2720 tok/s prefill, 262 t

Jev: Not Frontier, But Still Worth Your Attention [R]

model-releasesr-machinelearning

'TypeSafe AI sells Jev as a frontier-class reasoner that cannot hallucinate, built by the co-inventor of ChatGPT - fast, and almost free. We ran it live on 16,379 benchmark requests, measured its late

Krea2 Turbo Distill 2 step LoRA - new checkpoint released (chk41320)

Krea 2 Turbo — 2-Step Distillation LoRA (work in progress) Previous posts/releases - here, here and here. 🧪 A fast-preview adapter, from a project still in training. When the subject is close and fill

LiquidAI/LFM2.5-Encoder 250M/350M

LFM2.5-Encoder-350M is a multilingual bidirectional encoder built on the LFM2 architecture — a larger encoder for maximum downstream quality. It is a masked language model with full bidirectional atte

PotionUI is a free, open-source studio for generating images, video, music, and 3D on your own hardware with diffusion models - version 0.0.14

Reupload: I'm adding this again, since previous post had a totally stupid title which made people assume this app is some kind of cloud/proprietary models UI - this is totally not the case here. In ve

PSA: Krea 2 variance is easy in ComfyUI (no LoRA-s or custom nodes needed)

There are quite often posts here complaining that Krea 2 is too predictable / different seeds give almost the same output / you need to type in extremely long prompts to get good images / there is no

Qwen Image 2.1 fix v2.0 (updated due to feedback), plus a new sampler

This is actually two models: Qwen Image 2.1 Fix v2.0 and Qwen Image 2.1 Fix Opinionated v1.0. The former surgically targets Qwen's weird noise and messy details with almost no effect on composition, a

Qwen3.8-Flash-Next 177B running at 11–15 tok/s on a single RTX 5070 12GB + 32GB RAM DDR4

Benchmarking an LLM here with a NVIDIA RTX 5070 12 GB VRAM here I had been working on a llama.cpp based expert streaming setup for Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) on Windows. Benchmark is about 1

Resistance - Workflow

Workflow & Hardware The workflow was originally based on Pixaroma’s ComfyUI workflow. I made some modifications and added Sol Attn optimization and Comfy Kitchen Sink, including the very useful Model

Running Qwen3.8 Flash Next 176B on a 16GB RTX 3080 Laptop + 32GB RAM + SSD

I wanted to see how far I could push a fairly ordinary laptop with a huge MoE model. Turns out, Qwen3.8 Flash Next 176B can run on: RTX 3080 Laptop — 16GB VRAM 32GB system RAM SSD No 128GB/256GB RAM w

The fastest interference engine for RTX5090 and Qwen3.8 27B. Twice as fast as ninfer. 500+ t/s single coding, 2000+t/s up to 12 agents at the same time with 800k context. Smart VRAM-RAM-DISC Cache management, Loop Guard, Nice UI etc.

Hi guys, I am pretty happy to announce MegaCapybara. Purpose build engine for RTX5090 that is focused on Qwen3.8 27B (more will come later). GITHUB (Engine) HUGGINGFACE (weights) Why ? 1. It beats Nin

The Rise of Overfit Inference Engines

There seems to be a whole category of extremely narrow inference runtimes appearing: Strata, ninfer, DwarfStar, Splash, llamAmpere, gufo, etc. They deliberately give up the thing llama.cpp/vLLM are gr

Two local Qwen ( 3.8 27b unsloth Q6 and Qwen flash next strata coder ) models vs Claude Opus 4.6 on the same 3 coding tasks. One of them tied it. Not here to start a fight, just sharing numbers

Innanzitutto, due cose per evitare fraintendimenti. Non sto cercando di sostenere che un modello o un'azienda siano migliori. Non ho alcun interesse personale in nessuno di essi. Volevo solo verificar

Unitree just dropped UnifoLM-WLA-1.0 — a single 6B model that does 64 whole-body + tabletop tasks on a real humanoid

https://unigen-x.github.io/unifolm-wla.github.io/ Unitree Robotics released UnifoLM-WLA-1.0, their new general-purpose humanoid foundation model. Key points: • 6B parameters • Trained on ~2,500 hours

Why Doesn’t ChatGPT Warn You Before Wasting 30 Minutes of Compute?

I've been using GPT-6 Astra to create 70+ page documents from 100,000+ characters of source material, and it's jump in processing is great. Here's the weird part, when I give it a massive task it work

model-releasesr-chatgpt

Why is ChatGPT so… friendly??

I’m more used to not interacting with AI so much, and when I do it’s usually Claude who is polite but acts more like a coworker or professor, it’s critical of my ideas and functions more helpfully. Re

model-releasesr-chatgpt

[16GB AMD] Smartest local model that also does not refuse? 7950X + RX 6950 XT

I want the smartest local model I can run that also does not refuse. Not a tiny 'uncensored' 8B. Not a censored model with a jailbreak prompt. Tried dolphin-llama3:8b with a custom SYSTEM prompt. It c

4 mins plus for a response to simple 'Reply with: Ok'

What modle works on a 16 GB laptop i5 intel I have an old machine lying around and tried running a few models and non seem to be good, the response time is 4 mins and above Models tested: Google Gemma

Adding memory to search instead of sampling in reward maximization tasks [R]

model-releasesr-machinelearning

I am one of the authors of FLEET - an algorithm that enhances Best-of-N generation by attributing external rewards to particular tokens and then uses MCTS to adjust logits during the next run. I find

Anyone wonder why americans lag so far behind in the open LLM market?

I mean , DeepSeek, Kimi, GLM, MiniMax -- the list of Chinese LLM's is such a long freaking list. As American's -- why don't we feel .. a little funny .. about being so far behind? Chinese entrepreneur

best <40B alternatives to Qwen/Deepseek for (1) Coding (2) Long document QA test

Due to some reasons, Qwen/Deepseek Chinese models are NOT allowed in the my workplace. So, what local models, do you think, is the best alternatives to Qwen/Deepseek for (1) Coding (2) Long document Q

CalDec v1 - Fully Open Decision Model for Personal Assistants

Somebody just released a fully open-source, open-weights decision model that beats Jev! Just kidding, it's me and I this is my first time releasing a public model, recipe and dataset so I am looking f

I benchmarked Qwen 3.8 27B on my M4 Max — here's what I learned about local models, context, and coding performance

I've been experimenting with running larger models locally with Ollama, mainly because I'm interested in using them for agentic coding. I kept seeing benchmarks expressed as something like: 40 tokens/

I made my iPhone a second GPU for my 24 GB MacBook: Qwen 3.8 27B prefills 29–44% faster & my holds part of the CTX window.

**DISCLAIMER** THE PREFILLING TPS SHOWN ON THE PHONE IS COMPUTED ONLY FOR THE LAYERS IT HOLDS. ALREADY FIXING IT TO SHOW END-TO-END PREFILL RATE. NUMBERS BELOW ARE ACCURATE FOR E2E PREFILL RATE. Every

I made my own cybersecurity benchmark and ran Qwen3.8 27B, here's how a local model actually does at hacking

Hey local AI community, I've been working on this for a while and finally feel ok sharing it. It's a cyber benchmark where the model gets a shell in an isolated docker box and has to find the exact fl

I spent 3 weeks testing local Qwen3.8 on the new low-latency SGLang/vLLM recipes: DFlash2 2.8x. Builds: RadixArk + Inferact 27B NVFP4, 27B BF16, orcarouter 27B Uncensored, Flash-Next NVFP4

Hey guys, Last time I tested Qwen3.8-Flash-Next on its own. This time I put three Qwen3.8 checkpoints through the same 10 tests on the same RTX PRO 6000: RadixArk/Qwen3.8-27B-NVFP4 (dense) orcarouter/

Inference Engineering for Dummies

Hi all! I am a former SWE who has recently transitioned into inference engineering. I launched a side hustle a few months back and I've just taken it full time due to excessive demand. Its been such a

llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp

Now you can jev without jev https://huggingface.co/ggml-org/OpenJev-GGUF https://huggingface.co/ggml-org/lev-GGUF https://huggingface.co/ggml-org/Julia-1-GGUF https://huggingface.co/ggml-org/Laya-GGUF

microsoft/FrogNano-4B-2609 · Hugging Face

An agentic model from Microsoft for the GPU poor https://huggingface.co/bartowski/FrogNano-4B-2609-GGUF FrogNano is derived from Qwen/Qwen3.5-4B, a general-purpose post-trained model designed for lang

Model weight inferencing

I have 4050 6gb gpu, 24 gb ram which model should i choose to run i need speed. i try qwen 3.8 27b and feel too slow tried from onslot studio. I have heard of weight inferencing does it helpful what s

New benchmark on LMs fixing bugs before users run into them

Hi! This is a new benchmark that I created together with other researchers at Meta, Stanford, Harvard, UW. Most benchmarks these days seem to test models to just fix a bug that I as a user already enc

Peacebell - a from-scratch small language model

I've been using my free time during weeknights and weekends for the last 11 months working on and refining a small domain-specific language model. It specializes on information about World War II. The

Qwen3.8-27B-Humanlike-Chat 2.0: texts like a human, now with tool calls and better instruction following

Last month I posted a Qwen3.8-27B LoRA that makes it talk like a person instead of an assistant. It got a lot more attention than I expected: 700+ upvotes, 248 comments and 44k downloads since. I read

Row-Bot 5.0 is available: a new React interface, persistent agent goals, and a workspace for every conversation

Row-Bot 5.0 is a major rebuild of the local-first AI assistant. The NiceGUI interface is gone, replaced by one React app across desktop, browsers, phones and tablets. The biggest change is how work is

Self-hosting AI does not save money, and I do it anyway

Hi folks, I'm a long-time lurker and big fan of this subreddit and a massive self-hosting fan (also outside of AI). I doubt many people will disagree with me here because I see the same arguments bein

Should we plead opensource labs to still produce great non thinking (instruct) models?

The results are in, no thinking / instruct mode for new models degrade performance more than on old models such as 3.6 vs 3.8, where 3.6 takes the lead on several coding benches in instruct mode. I wo