All entries

Here is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo

Follow-up to last week's post about streaming MoE experts from disk in a browser tab. That ran on hand-written engines for two models. This week the same disk tier sits under llama.cpp's own WebGPU ba

I've been reading about all these modders using AI to reverse-engineer games and software. I've been trying out photocraft and gridcraft and have just watched a GTA5 mod featuring Sonic. It's pretty d

Dynamic vision, in action Basalt is an inference engine for 3.8 Flash Next, forked from Strata and heavily tuned for Blackwell. It supports vision, and for this to work you need to have the image enco

My project is optimising qwen flash next on Mac Mini m5 64gb through SSD streaming. I recently asked LocalLLama for guidance on how to optimise token output. Today a popular method to do this is a loo

I set up a dedicated workstation for local AI experimentation with various models and tools. But every single one of them wants all the available VRAM. To fix this, I built a request dispatching layer

I built a desktop app harness that keeps local meeting notes and can use them to autocomplete my typing as well as complement my dictation with additional context via reading my screen or using the pr

Hi Friends, I am a solo developer. Ever since I had Qwen 3.8 running on my RTX4090, I had an itch to use it on my phone for everything. That gave me the idea to build SteerClaw Mobile - a complete age

Sadly, all the tests were done using NanoGen, but maybe someone can try the idea on Krea 2 or Qwen-Image 2.1. Does our method work beyond ImageNet generation? Yes. We test it on NanoGen, a unified dif

Quick note, not my character, original came from schmede_AI on youtube. But yes my question is do any local models currently have the ability to match and generate additional images of a character fro

pixel-curator — curation tooling for similarity training (identity, subject, and style models) Repo: https://github.com/snoozey-mf-bnui/pixel-curator What it is pixel-curator is a local-first CLI that

A few days ago I posted “Gausify Orbit - I am building a web application to transform images into Gaussian Splats”, and it was removed because the project wasn’t open source yet. It’s now released und

So I wanted to try the horsey challenge myself on the latest engine i been testing which is a strata fork by sergqwer. https://github.com/sergqwer/strata-nvfp4. I seen a post earlier and seen i believ

A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontie

Hello everyone! My current setup has an RX 7800 XT 16GB GPU + 32GB RAM. I'm currently running Qwen3.8 27B at 4bit with max context 120k at around 40 t/s but the quality isn't that great, and I'd like

'A 4-step distilled LoRA for Qwen-Image-2.1, supporting text-to-image generation and image editing. Training was performed entirely on 16 Huawei Ascend 910B NPUs, using DMD in LightX2V.' I think it's

Lipwise is a [yet another] dictation app for Windows, macOS and Linux. You hold a hotkey, talk, and it types into whatever app has focus but with a twist. Since most of us that speak to our computers

Gguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai 'LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, p

I’m planning to get a Mac mini mainly to run local LLMs and AI agents 24/7. I plan to run models up to 27B parameters with Q4 quantization, and maybe not for coding, maybe just for everday task and mo

One of the frustrating things about local AI generation on Mac is getting different models and tools to work together smoothly and fast. Valtz is a new native macOS app powered by Vpipe, a high-perfor

I've been using this for a few days and I was surprised to see nobody had posted it here. It's a vae trained to remove the checkerboard artifacts that qwen image 2.1 produces. It works very well. You

tl;dr: running Qwen3.8FN Q4 locally is cheaper for me than every cloud model except Luna and the newer Haiku at mid-peak and peak electricity pricing where I live, and becomes the absolute cheapest mo

It just popped on my feed. I watched the teaser in the Netflix app and a few things jumped out The constant use of anthromorphism using terms like altruism and sacrifice when in engineering literally

I'm sure some here read this at the time and remember, but I recently found this hilarious Guardian article from 2019 through Wikipedia: https://www.theguardian.com/technology/2019/feb/14/elon-musk-ba

So I’ve only been messing around with Krea2 in ComfyUI for a little while, and I ran into the classic problem: I make an image I really like, then a week later I want the exact same settings back and

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

Hi guys! I got a problem with one of my V100 SXM2 Modules. In 'nvidia-smi' its shows with 'Pwr:Usage/Cap' as 'N/A / N/A'. When I try to use it i get the following error in dmesg: NVRM: Xid (PCI:0000:0

https://preview.redd.it/p6i9qifyy8vh1.png?width=2069&format=png&auto=webp&s=2b1e3c7c4fc95f12f0f7d417a4d21410aebfbd35 The 3090 Founders edition has a reputation for hot VRAM. Since LLM token generation

Quartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones. I had some weird looping issue in their chat, and some

Hi, I see that vllm released d3 (a family of decision models) and llama.cpp already has support for them. I want to try them out. Are decision models sensitive to quantization? I know that for autocom

I’ve been building SonicBoom, a native tensor runtime written in C++/CUDA with a C ABI and no Python anywhere in the stack. I just tagged v0.0.1, and the first target is local LLM inference on consume

This is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also convertin

New version of my MiniMax H3 timeline workflow released Some very important bug fixes included, and a lot of improvements. The YouTube video above includes a lot of tips for using the workflow in addi

Hi all, I'm the developer of VigenFlow, a local image generation and editing server that runs diffusion models on the AMD Ryzen AI NPU (XDNA 2) + CPU. If you have a Ryzen AI laptop or mini PC without

I came across this blog post from OrcaRouter: https://www.orcarouter.ai/blog/qwen-3-8-next-vs-qwen-3-8 Not only is the blog post nonsensical in some parts, it's also completely without any value. It b

Disclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings

I’ve wanted this for a long time: a Krea-2 model that gives you a magazine page in one shot. Flash-lit gravure sets, film grain, multi-photo layouts on white paper, big headlines, covers and back cove

I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have com

This was fun. The base images are made with Kroma, the videos with Minimax h3 and the text-to-speech with Microsoft Vibevoice. The text message screen recording thing is made by an LLM agent (GLM 5.3)

Tutorial Link : https://www.youtube.com/watch?v=DJcgVASlg38 Fully free with resources on GitHub : https://github.com/FurkanGozukara/Generative-AI-Tools-And-Tecniques-2026-2027-Fall submitted by /u/CeF

I designed a node to align any pair of images trying to correct the pixel drift on Qwen Image 2.1 https://github.com/zuanfilm/Qalign workflow This node don't need any model or text encoder Qalign is a

We're launching Precision Numerics by releasing Boole-1.7B, a 1-bit version of Qwen3-1.7B. 247 MB packed weights: 13.9× smaller than BF16. 85.9% of the BF16 model’s average score across our 10-task ev

Write-up: https://zhebrak.io/posts/brokenml/ Repo: https://github.com/zhebrak/brokenml I built a benchmark to determine the access level agents need to successfully fix broken ML workloads (PyTorch ov

With the vLLM Semantic Router team we released Decision 3.0 (d3), our new SOTA decision model family ranging from 0.6B to 27B, Apache-2.0, all reading text, images and video. How it works - Built on Q

LTX-2.5 makes a video with one mixed soundtrack. Our LoRA also outputs one track per captioned sound, in the same run. Prompt 'a jeep, its engine growling' and 'a taxi, its horn honking' and you get t