Search: “routing”

I’m building DesktopLab, an open-source local-first control plane for development agents. I recorded the setup boundary that most agent demos skip: DesktopLab detects the host, proposes the supported

First of all, obviously I took some help from AI to type this post and this is the topic that enabled me to accomplish all that: https://old.reddit.com/r/LocalLLaMA/comments/1veow4b/deepseek_v4flash_2

Ran DeepSeek V4-Flash-0731 — the full official checkpoint, not a re-quant — on commodity used hardware. Sharing because I couldn't find anyone else publishing Ampere results for this engine. Why bothe

Hey fellow llamas. we have something new for Strix Halo owners we thought would be useful to share. i'll keep it short: We were able to fit DeepSeek V4 Flash plus its speculative draft on a single Ryz

You patch security holes by intentionally finding them. If the models refuse to do it, how can companies protect themselves against rogue AIs, whether they are Chinese or OpenAI/Anthropic themselves?

Since a lot more people are trying to build their own multi-GPU machines, I thought I should help to prevent a common mistake people make with building multi-GPU machines. Which is using an Intel cons

audio.cpp again :) Release 0.4 is out. The headline this time is new high-quality TTS coverage plus GGUF becoming a first-class across the project. What’s new: Added Higgs Audio v3 TTS 4B, Fish Audio

Hey r/LocalLLaMA — solo researcher here. I built a new attention architecture and want independent testers. Wave Field LLM replaces O(N²) dot-product attention with FFT wave convolution on a field. Tr

OpenStudio is a hybrid AI router that combines local model inference with cloud-based model access through OpenRouter, a unified API providing access to hundreds of AI models through a single endpoint

This r/ChatGPT thread addresses user-reported issues with ChatGPT's 'Thinking' mode not functioning as expected, a common frustration among subscribers. Users have noted problems such as the thinking

I was unable to retrieve the specific Reddit thread content from that URL through my search. Reddit threads often require direct access to load user-generated content, and the search did not return...

I know I am probably missing the point of this benchmark, but I wanted to see if the model is capable of generating something better or is that the best it can do. So I gave it some encouragement. I f

This continues the small-model browser experiments from the previous posts. Two standard, off-the-shelf models ran locally on the same Samsung Galaxy S21: Gemma 4 E2B, Q3_K_M as the reasoner Qwen3.5-0

pixel-curator — curation tooling for similarity training (identity, subject, and style models) Repo: https://github.com/snoozey-mf-bnui/pixel-curator What it is pixel-curator is a local-first CLI that

Gguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai 'LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, p

New version of my MiniMax H3 timeline workflow released Some very important bug fixes included, and a lot of improvements. The YouTube video above includes a lot of tips for using the workflow in addi

EXL3-QLoRA I've built this over the last few months and I think it's stable enough to share now. A fork of ExLlamaV3, this is a native ExLlamaV3 training implementation with plain PyTorch, without tra

I've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called rdna35-perf on top of halo-box/strix-llama.cpp, a llama.cpp fork a

If you build on open-weight reasoning models, you have probably wanted a 'how sure is the model about this answer?' signal that does not cost 5x sampling, a judge model, or a labeled dataset. Hence, w

This will be my third update on the bc-250 cluster and for my first forray into local ai I have been having a blast. This has turned from a weekend experiment into something I think I will use in the

Alright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v

It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu

Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Be

(Don't judge by the screenshot, the cache is cold. It hits 24+ tok/s with a warm cache!) About two months ago, I made a post here asking whether predicting which MoE experts would be used on the next

SlopSoup TV is a live, never-ending pixel-art TV network. Every script, character, voice, camera cut and schedule decision is made by models. It's made to be very absurd, dark and strange. Hardware: o

Post title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris

https://preview.redd.it/pr5bl6rcezth1.png?width=834&format=png&auto=webp&s=dca7440c56e169b53a586182368829f848805c9a They released FastH3 V2 three weeks ago: https://www.reddit.com/r/StableDiffusion/co

Maybe I'm mistaken, but wasn't one of the big pros of Ollama Cloud inference that it was all hosted by Ollama? If that's the case, why duplicate the Deepseek on-/off- hour premiums? Surely they do not

I wanted to build an offline bird/plant identifier that runs on a laptop with Ollama. Before building any UI, I tested it on 6 photos: Common Myna → gemma4:e2b said 'Jackdaw' (confidence: high). gemma

Disclosure: I made this. Sharing it here because it is fully local and small, and I want feedback from people who run models on their own machines. What it is: a 395M decision model (ModernBERT-large

How is JEV any different from using an embeddings model? I really will appreciate if someone can explain this to me - because I have yet to see the difference. I'll even give you my JEV server for fre

Meeting notes are one of the last parts of my workflow that still depend heavily on cloud tools. I've used Fathom and lately Bluedot. Bluedot works well for me because there's no meeting bot and I get

Hey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its brows

Something keeps happening in frontier AI, and the explanation we are given is always one size larger than the event. An OpenAI research model, running in a supposedly offline training environment, hit

Generated with H3: Mock H3 version of the ending to Wargames(83) as a test of a custom node I've been working on w/ both Claude(Opus/Fable) and OAI's Sol/Astra. One click generation using only 4 movie

Apple Foundation Model 3 through Private Cloud Compute is directly available in macOS 27.2 through familiar AI clients. For Mac users running local models such as Qwen 3.x or Gemma 4, AFM 3 PCC is a c

Wanted to try Opus 5.5 because as a plus sub with OAI, Luna just couldn't deliver. Opus is a true beast. It does consume tokens on a medium level yes, but the way it works is just refreshing. But... o

Hi! I'm the author of MoEspresso, which is my way of putting my own ideas about inference engines to the test. A lot of the fun has been trying different design choices, measuring what happens, and fi

Disclosure: I’m one of the authors of VeriLoop E2, a 27B model post-trained from Qwen3.8-27B. The weights are Apache-2.0; the harness we evaluate it in is not open source (details at the bottom). This

I've been experimenting with ways to run MoE models that don't fit comfortably in RAM, and I ended up making Overspill, a disk tier for FreeToken. The basic idea came from looking at how Colibri handl

I set out to test portable Engrams and accidentally ended up testing compiled external memory instead. Am I onto something useful or reinventing a known idea? I've been building a small open research

We hosted Kev 4B (Jared Palmer's Apache-2.0 fine-tune of Qwen3.5-4B) and ran it side by side with Jev on the same endpoint to see how it compares. We built a fresh set of 362 items published after bot

I've been building a small decision model for agent loops: gates, routers, 'should I ask the user or just act' checks. It's out now as Mica v0.1 4B (Apache-2.0). What it does You give it a state, a qu

I’m the developer of TensorSharp. Inspired by vLLM PR #57250, I added and optimized a native /v1/systemone endpoint for Jev-style structured decisions using DiffusionGemma GGUF. This uses DiffusionGem

Saw this today and found it very intriguing. Lots of interesting design choices here, and it's cool to see someone doing something different. Here's a few highlights: Looped transformer: dynamic recur

Small, free finding. I use local Qwen models for typed decisions: a state plus a question with fixed allowed answers, and I read the probability of each answer from the logits of one forward pass inst