Knowledge catalogue
A few days ago I released Jeff-Qwen3.5-0.8B, a small 'System 1' model that picks between options you define and returns a calibrated probability for each, in one forward pass. Speed was great on my M4
Anyone else experiencing this issue this week in particular? I am getting WAY less requests this week than usual, no matter what model I am on. I actually had to discontinue things I was doing because
pi-llama-skip-reasoning is an extension for the Pi.dev harness that forces a local llama.cpp model to stop reasoning and answer / act immediately. When you are deep into the ctx session and ask 27B a
If you're using MTP with Qwen Flash Next and llama.cpp, you can switch to: quants: https://huggingface.co/ggml-org/Qwen3.8-Flash-Next-GGUF PR: https://github.com/ggml-org/llama.cpp/pull/29761 Please n
I put an old RTX 3090 to work as a local coding agent with Qwen3.8-27B, llama.cpp, and OpenCode. Here are the setup and results, including what failed. This is a summary of my own blog post, linked be
now you can use MTP with Qwen Flash Next, time to switch from Qwen 3.8 27B? (merged after 17h of development) quants: https://huggingface.co/ggml-org/Qwen3.8-Flash-Next-GGUF link to the previous discu
RuntimeAI's September 2026 AI Security Report covered 126 incidents across 38 named organizations — 22 critical, 102 high severity. 53 of those incidents had AI either as the attack tool or the target
Hello all, I wanted to share a little project I vibe-coded for myself that you may find useful. As many people here like to suggest, I wanted to give my small local LLMs access to information to impro
vLLM's tensor parallel requires that several of the model architecture numbers be evenly divisible by the number of GPUs selected for tensor parallel. This usually means that you can only use a number
Last update: https://www.reddit.com/r/LocalLLaMA/comments/1wu9ksu/update_yandexaliceai_80ba3b_fine_tune_progress/ - basically, an instruct fine tune on the base model using a synthetic distilled data
Hybrid search, reranking, query decomposition, and query expansion are often treated as must-haves for good RAG. We wanted to see how much each actually helped, so we tested them. Same model, same emb
Hello guys. Recently, there has been a boom in game decomps and recomps thanks to AI. If you look at the r/decomps and r/recomps subreddits, you can see it. They mostly seem to be using Claude or Code
Hello guys! I have been experimenting with qwen 3.8 for a long time and I hadn't been able to get reasonable speed. I am on a 5060Ti 16 GB, and although this gpu can game, I am aware that AI demands m
40 year tech gap? No problem! The Tandy runs DeskMind, a native DOS program. It talks over WiFi (a PicoMEM 2 card with mTCP) to a small Python server on my PC. That server drives Qwen3.8-27B (NInfer o
'AREX-2 is a 27B-parameter long-horizon agent model from the Beijing Academy of Artificial Intelligence (BAAI). It learns to improve a solution over multiple test-time rounds: propose, measure, reflec
Building a local voice chat pipeline: Struggling with fast, natural PT-BR conversation models I'm currently trying to build a local AI voice chat system that can use tools via an LLM. I'm testing tw
Our agent workflow is currently executing a group of 10 vllm api calls within a asyncio.gather we made them share the same prompt until the end where the queries/instruction prompts differ. These call
Tasks where an MLLM doesn't write text but only makes a decision, which started with Jev, seem to be getting popular. (This is sometimes called 'typed decision'.) I thought it would be fun to use insi
https://preview.redd.it/xs01ds930nsh1.png?width=4800&format=png&auto=webp&s=939762bcf3fbd93e137d4f97e927dda38aaf30fd Back in July I shared LokalBot here. It's a free, open-source Mac app that records
Follow-up to my post from about two weeks ago. A lot has changed since then, so I wanted to post an update on where the backend is now. Where the green architectures stand When I posted last time, 7 a
Running things locally already means you care more about control and reproducibility than most. Worth extending that to your prompts too. There's a live workshop on Oct 3 covering exactly that. Serj S
I've been closely following the rise of the Strata inference engine and as someone with 28GB VRAM and 32GB RAM I'm itching to buy 32GB more RAM just to use Flash Next. But of course before I make such
I run local models through Ollama and kept hitting the same quiet problems, so I wrote a scanner for them. llm-doctor checks your model stores (Ollama, LM Studio, Hugging Face, MLX) for duplicate weig
I built Image Studio, a local web app for Qwen-Image 2.1 on Apple Silicon. I had the model running on my Mac, but the only way to use it was the ComfyUI graph, and chaining edits meant wiring loaders,
There's so many new techniques like RL, RL LoRA, QLoRA, CPT LoRA. I believe i would have a usecase for them, but i don't know where to learn, youtube is filled with bad quality tutorials if i just sea
Running Qwen flash next of even Qwen 27b dense, I can do any,burning sticking to flash due to its speed, and the kwh cost is at 0.25€ where I live in, ranging from 0.11€ to 0.35€, so I used chatgpt to
Image-to-3D models redraw your picture, and fine detail - text, logos, faces - comes back garbled or scuffed. My local pipeline that fixes it. (repo link at the end of the post) The latest update adds
Hi again! This is a update to my previous post MageTrail V0.25 where I continued this tech demo toy model thingy called MageTrail, a Danbooru/E621 proof of concept Full-Finetune of Microsoft's MageFlo
Hola a todos! Tengo un MacBook Pro m3 con 16gb de RAM. No me siento cómodo con los modelos que he probado y me gustaría saber qué modelo os sentís cómodo y para que lo usáis Muchas gracias! submitted
I'm part of the Lokutor team that built this. Model: NVIDIA Conformer-CTC Small (13M params, int8). It runs on an ESP32-S3 with 8 MB PSRAM, no GPU or NPU. LibriSpeech WER is 3.7 / 8.2, versus 6.3 / 15
I've been struggling with the ollama update process. I have both nvidia and amd GPUs in my system. I have two instances of ollama running on different ports for this to work around issues. AMD 7800xt
I have a long list of my own domain specific eval questions that I run to validate which models I can rely on: coding, coding (numpy/pandas), data analytics decision making, local RAG, and voice assis
I just bought the GPUs after deliberating and debating for over two years. With costs not coming down any time soon and me just wanting to get this massive todo-box ticked, I decided to just YOLO it;
I can't seem to find what's wrong. I'm using Pi for my llama-server and sometimes it just stops processing for some reason and stuck after tool call. Logs seems to think that it's finished its job whi
I'm tired of watching my compute budget and topping off my Ollama account. It feels like I can use a mix with more flash models on Openrouter (things not all available at Ollama) and control my spend
I'm far from being an expert, but for me Qwen is not a Turbo Model so it cannot be as good as Krea 2 Turbo. Also their dataset seems to be not good, it's hard to make doodles, cartoon styles, and its
I think most people are sleeping on this inference engine. I tried multiple llama.cpp forks and none of them comes close to the inference speed of Strata. Initial version had some bugs with kv cache,
Ollama’s new limits and max plans are worrying me, so I’ve been going harder on my harness. (Thanks to everyone who’s signed up for the waitlist so far.) Wanted to share how my v1.10.0 will dramatical
Wulver v0.5 is a full fine-tune of Krea 2 Raw (the whole 12.8B DiT) for anime and anthro characters, including scenes where several characters interact. We made it at Vaelico, an independent studio. C
Hey guys - half a tutorial/guide, and half an I built this, so I went for resources. This is something that I created for myself recently when I couldn't find any good existing solution. I wrote this
Generated with H3: Mock H3 version of the ending to Wargames(83) as a test of a custom node I've been working on w/ both Claude(Opus/Fable) and OAI's Sol/Astra. One click generation using only 4 movie
We had a Dell B300 in the lab for a few weeks and used it to create two fine tunes of Qwen Flash Next. Victoria (coding and agents) Qwen3.8-Flash-Next cut down by 44% using a paper / technique called
A LoRA designed to reduce the typical smooth/plastic AI look and bring natural texture and realism to your images. What's new in V5 (KL9_V5_Pro): Summary: This version gives the best natural texture s
Hi folks, I've spent the last couple of months experimenting with Strix Halo and previously I released a proof of concept I called llama-halo-hybrid. I've continued updating it and it now performs ver
loss curve (taken from the last micro of every step, to explain the variation) some help from gemini 3.8 flash high About 40% of the way done with the initial fine tune. The loss is so spiky because I
Apple Foundation Model 3 through Private Cloud Compute is directly available in macOS 27.2 through familiar AI clients. For Mac users running local models such as Qwen 3.x or Gemma 4, AFM 3 PCC is a c
The collection includes kernels for more than 200 common ML operations, all of which can run entirely locally in your browser on WebGPU. We're also working to upstream these optimizations to Transform
Processor Intel(R) Core(TM) Ultra 5 226V (2.10 GHz) Installed RAM 16.0 GB (15.7 GB usable) Graphics card Intel(R) Arc(TM) 130V GPU (8GB) (128 MB) Storage 273 GB of 954 GB used System type 64-bit opera
Hi, I’m using Qwen3.8 27B in my vulnerability scanner, it does scan JS files, tries to generate exploits etc. I’m using uncensored version, the model is like 21GB. Running it on Mac M4 Pro 64GB ram, i
I wanted a fun way to visualize and stress-test a multi-agent ecosystem, so I pointed it at my favorite MMO (a vanilla-era private server emulator). Live view: https://design.kristiantalley.com/projec
I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy
kinda realized i type things into AI chats that i'd never say to a friend. money problems, work stuff, health worries, random late-night thoughts lol then i found out all of it is stored on their serv
We did a study: does post training actually make LLMs funnier? We used open models that publish every stage of post training, so we could compare a base model with the future models it became: Tulu 3
Here's a metric dashboard giving an idea of the last few days. Been testing with a variety of different agentic coding use-cases, mostly using a pi harness. qwen3.8-flash-next has seriously exceeded m
Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module us
Here Are My 7 Basic Prompts After 4 Years of Using ChatGPT I’ve been using ChatGPT since 2022, and over the years, through my own experiences and research, I’ve learned quite a lot about how it works.
Using only information you actually have from my conversation history, memory, files, or other context available to you, build a practical “operating manual” for me. Do not flatter me, diagnose me, or
If I were given one final message, and the premise were that I genuinely understood what was happening—that this was the last thing I would ever say, with no possibility of another response—I don’t th
First 3 disposition results. Hi there! My name is Jerry and I recently built Enclosure, e deterministic environment for testing LLMs. It's a simulated office enfironment https://github.com/openconstru
ninfer, dwarfstar, Splash, llamAmpere, gufo, etc. We've all seen them popping up, great tok/s, people loving them. Forks of llama.cpp or another engine, or made from scratch. For better or worse, the