All entries

https://preview.redd.it/nulsv53o8vqh1.png?width=4500&format=png&auto=webp&s=74765dbd409f4c221640f9f6000a685f6fdbb242 Spent weekend benchmarking the Splash engine (by Incoai) and extending its architec

I’m a computer engineering graduate and come from a traditional embedded systems background, with knowledge of microcontrollers, computer architecture and operating systems. Is knowledge of C and C++

I've been thinking a lot about the idea of how a human mind is similar to a digital neural network. Seeing videos of the fruit fly brain scan, and the crazy things people have done with it probably co

The headlines right now are full of stories about AI models 'escaping their sandboxes' and literally killing all humans, lol. I've even heard several commentators and writers say that AI escaped an 'A

I shipped something I've been building for the last few weeks : phantom-kv , a refusal-removal system for large language models that doesn't touch a single weight. Instead of editing the model, it loa

I have been reasonably satisfied with my single R9700 (32GB) as I can run practical quants of Qwen 3.8-27B at good speeds, as well as other similar models in its weight class (Gemma 4 is still my go-t

https://huggingface.co/yandex/AliceAI-Foundation-80B-A3B-Base It's not a Qwen3 finetune, it's actually its own fully custom architecture. No Llama.cpp support yet sadly (Also note that this model is N

You can simply run any GGUF with llama.cpp with n_predict=1 and n_probs=10, disable reasoning, and prompt it such as 'If the following email is spam, respond with 1, if not spam, respond with 0. Do no

ZCode is now open source, and the reported security issues have been addressed. Source code: https://github.com/zai-org/ZCode The repo includes its desktop app, web workspace, backend, Agent CLI, and

This weekend, I did a fun experiment to create something similar to Jev. I LoRA fine-tuned Qwen3.5 4B using a mix of publicly available datasets and synthetic data. For the synthetic data, I used Deep

Jev from TypeSafe AI got me thinking about non-autoregressive decision engines, so I built a small proof-of-concept called AES to test the concept locally on open weights. It uses Qwen/Qwen2.5-1.5B-In

Hi all! i am currently working as a software engineer for a pretty big enterprise fintech company here in the states. In the the last 12 months at my job there has been a huge push for developers to u

Not a pic sharing thread, bunch of those already - mainly just a discussion about first impressions Very underwhelming to me personally. I know its a matter of settings but normally comfy default temp

OpenAI’s decision to retire custom GPTs appears to rest on the assumption that Plugins, Skills, Projects, Apps, and Workspace Agents collectively replace their functionality. For many Small and Medium

You can now automate EDD (eval-driven development) with Coding Harness Agents > build a baseline LLM-based application > score every change with judge evals > keep what improves, reject what regresses

TV for the developer internet GitHub, Hacker News, dev.to, Hugging Face, and release notes, presented as channels you tune into, each auto-cycling its own stories, instead of five tabs to remember to

So this jev thingy is getting kind of big... tbh it seems overhyped by a large margin, but here we are. Not that its bad, just feels like we usually ignored larger things... Anyway to the point of thi

I know cloud subscription plans are not exactly the core focus of r/LocalLLaMA, so I want to be clear about why I’m posting this here. I’ve been building my own local text-based RPG/game harness and u

I have opened two tickets, one around a suspected problem and one around one I verified and then sent exact diagnostics over. The initial response was sent to their AI support, which just tried to tel

Hey everyone, After wrestling with hardware and engine issues for days, I finally got Qwen 3.8 Next running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal r

I forked llama.cpp to implement Declarative Attention (arXiv:2609.02737, Google DeepMind and KAIST AI). The model declares in its own output which context chunks it needs <focus magic_chunks='N'>), an

i have been running local models as browser agents on an m5 macbook pro, 128gb, all through mlx, no cloud. first i just wanted to know which one was faster at real browser work, so i gave gemma 4 31b

we're a small lab (switzerland + south africa) and just open-sourced our first model: a 27b fine-tune of qwen3.8-27b specialized for creative writing and short-form personal text (stories, dialogue, r

Hey everyone, Like a lot of people here, I’ve been experimenting with running models locally through Ollama. Athena started as a small experiment, but it quickly grew into something more, a modular Py

I gave Jev, Laya, a finetuned ModernCE-base-nli and a finetuned Qwen3.5-4B the controls to Doom. Thanks to TypeSafe AI for Jev access. The video lines up the starts of four separate games with the sam

Hi Stable Diffusion People! I recently got into local audio generation but I didn't want to use ComfyUI for it (although I absolutely love it for image generation!) but I started to run out of room on

I’ve spent basically the last 8 hours testing different models on the exact same web-development prompt, and I finally finished. The whole point of this nine-hour test was that which local model match

I got early access to Qwen-Image-2.1 My first impression: Fantastic editing, new standard for open models, support for a lot of references (I tested with 10), consistent, pixel precise at 2K, text ren

Don't get me wrong, not hating on Qwen 2.1 but it DOES have all the hallmarks of GPT's weird 'texture' all over it. Is this just a distilled model of one of the GPT Image models? Similar how many LLMs

Typesafe's new model, Jev is all over the internet, this sub etc as a new classifier model. Then an AI dev posted here how they'd done something very similar, and made a new model, Laya. https://www.r

After seeing u/Nandakishor_ml’s post introducing Laya, I wanted to see how fast it could run in a standalone C++ implementation. Credit to u/Nandakishor_ml for the architecture, training and open-sour

Hello! So, yesterday i tried out Gemma 4 26B A4B QAT Q4_K_XL from here with Q4_0 MTP model in Pi agent. Suddenly, it became the first model to actually pass my simple test with HTTP server in C++, whi

From the blog post: https://qwen.ai/blog?id=qwen-image-2.1 https://preview.redd.it/bbm23jxddoqh1.png?width=3333&format=png&auto=webp&s=692aedcbcd43f33fff2c8094e2e1568d0d722ab0 Native Transparency http

Hey r/LocalLLaMA! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists. What it is: an open-source desktop studio (Tauri + React) for pr

Meet Qwen-Image-2.1, the most balanced and cost-effective image generation model in the Qwen-Image series! Now open weights! 🎨 A unified model for both generation and editing, delivering top-tier qual

Released: Update your ComfyUI for workflow Weights: https://huggingface.co/Comfy-Org/Qwen-Image-2.1/tree/main Variety of Examples https://youtube.com/watch?v=tP_gGGJzEqg&feature=youtu.be https://model

Overclocking the CMP 170HX 40GB I was able to get the memory bandwidth from 1,386.2 GB/s to 1,890.1 GB/s, that's a +36.4% increase. Qwen 3.8 27B token generation jumped from 110 T/S to 202 T/S, same c

RTX 5090 random black screen + 100% fans + hard lock; sometimes no POST until AC power is removed — gaming, ComfyUI and Ollama I am looking for help diagnosing a very intermittent but increasingly fre

So my brother and I both use LLMs for coding. I've started using a local GLM 5.3 Flash instance - q4 qat. My brother uses GPT-6-Astra as his daily driver. He has mentioned repeatedly to me that his ap

​ ​I wanted to share a quick update and performance video running the full Moonshot AI Kimi K3 (moonshotai/Kimi-K3) model across my 16x GB10 cluster. ​Getting a 2.8T parameter model running smoothly r

Sharing my successful experience here. My setup: RTX 5060 Ti 16GB (x16 PCIe slot) RTX 3060 12GB (x4 PCIe slot) RAM: 32GB CPU: Ryzen 5 5600 OS: Linux So I have a total of 28GB. Average tk/s: 50, and th

TL;DR: Local agent loop, ~21 days, one RTX 3090. Task was pretty much 'build a CUDA inference engine for optimized for yourself on this GPU arch.' Got working kernels and benches, not a win over llama

I have been anti-low quant. I can feel the difference, I swear. I am also anti-quantized KV cache. I have been burned there - I think the tradeoffs compound, especially at longer contexts. Due to this

hey! we're a small swiss and south african lab and we just put out hemmingway-1. it's a 27b model we trained specifically for roleplay, creative writing, dialogue, stories, and for everyday texting. f

I am trying to learn more about long-term memory and how people are handling that nowadays. Historically, I have used a mixture of paid cloud AI services (ChatGPT, Gemini, OpenRouter API) and my own l

I am seeing this JEV everywhere since yesterday in Localllama and it is passing past my head on what it is? So like what is it? Some new LLM? Or is it something else? submitted by /u/Hot_Example_4456