Knowledge catalogue
https://preview.redd.it/7sknv6omnguh1.png?width=1023&format=png&auto=webp&s=4aef1dc464d008b00f104519eb5399ba6132ad69 Want to try? Just go here and click 'New seed' a lot. You will get different storie
For those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at
I posted on here a while back asking for advice on upgrading to an Epyc system. Thanks to all who commented. I’ve taken the plunge and ordered: - Epyc 7443 CPU (24 cores, 48 threads, 128 lanes) - H12S
Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-
It seems Dario's 'too powerful for you users' strategy is paying off: we have two open models at the top of the leaderboard, surpassing every single model from Anthropic. Open source prevails. Even Mi
Disclosure: I work at H2O.ai. We released H2O-Lightning-4B, an open-weight (Apache-2.0) model for the 'decisions API' style of inference that Jev made popular: you send a state plus typed questions (p
Last month I created a portable memory format as a .txt file as a small project. I named it as: Memory Journal, in a shape of a skill card. It's essentially a large prompt wrapped in a text file. Init
Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area
In the dawn of the LLM craze, when wild Llama2's roamed free and 4k tokens was considered a fairly long context for local models, I forked an interactive fiction and MUD library called Tale to experim
It's been a long time coming, and I should have done this in the first place. But LumaBrowser is now open source. Since I first released it a year ago, it's had several thousand installs and an absolu
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Be
Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures
Referring to specifically Unsloth's UD_Q4_K_XL quant because that's going to be a question, and is relevant regardless. It's old now. It's not great at coding medium sized or even small-ish projects.
(Don't judge by the screenshot, the cache is cold. It hits 24+ tok/s with a warm cache!) About two months ago, I made a post here asking whether predicting which MoE experts would be used on the next
Qwen-Image-2.1-Turbo, create and edit images in just 8 denoising steps! Open weights now available! Built on Qwen-Image-2.1, Turbo is an accelerated checkpoint on the same 7B visual generation archite
LemonSeed Studio is an iPad editor/IDE with on-device inference on an AMD GPU in a Thunderbolt enclosure. It embeds the unmodified upstream Linux amdgpu + amdkfd driver (mac_linuxgpu) as a PCIDriverKi
I wanted the uncensored Qwen3.8-27B (llmfan46's Heretic build, MTP head preserved) on my RTX 4080 with MTP on (because I've been testing TONS of configs, and realized that would be the one) And none o
Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is mode
I have rtx 4090 and 64 gb of ddr4 ram. This is enough to fit smaller quants of qfn, but I found them not great and switched back to qwen3.8. However, there are option that bother me a lot, buy huawei
HauhauCS ships their uncensored Qwen3.8-27B as GGUF only. NInfer, a C++/CUDA wanted its own format. Now the same model that ran at 91.6 tok/s / 131K under llama.cpp does: 262K context (the model's ful
SlopSoup TV is a live, never-ending pixel-art TV network. Every script, character, voice, camera cut and schedule decision is made by models. It's made to be very absurd, dark and strange. Hardware: o
All anyone ever talks in this sub is about x new model having x benchmark scores and how well it can code. Or some new tech to increase t/s for qwen. So I wanted to go back to the roots of this sub an
A bit of backstory : i got lucky in August and was able to grab a CMP 170 HX 8GB for around 500 on alibaba. I tried later to get a second one for around 1k but my order got cancelled and prices went t
Youtu-Parsing-Omni is a compact (5B) omni-modal parsing model. Given a single input — a document page, a natural image, a chart / flowchart, a geometry figure, an audio clip or an audio-visual video —
I tested Mellum2.1-12B-A2.5B (Q8) locally using Pi and llama-server. All five tests were one-shot. Results were pretty mixed: - Pelican SVG, Browser OS, Minecraft: Poor results. - Bouncing Hexagon: Ph
Preamble and disclaimer, Sample size: 8, at any size of form this research is just screwing around being writing it down. This was just a little,(well... big), curiosity test I wanted to run to see wh
Post title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris
Hi all, a bunch of performance improvements have been landed in audio.cpp. The biggest highlight is Higgs Audio TTS, which now runs with around 6 GB VRAM, a 48% reduction in peak memory usage compared
There are way too many forks to keep track of, so I've gotten lost. As far as I can tell, Radiance VLLM is best for models that fit in GPUs while some form of llama.cpp is probably best for MOE RAM-sp
Opus5.5, Sol6.1, Fable, and Astra have all proven they can and the scene has exploded this past week. Part of that is from the tools and feedback loops maturing though. Are any open weight models (at
came across a pretty interesting technical breakdown of chatgpt's newly launched 'intelligent ui' feature, and thought this subreddit might find it interesting. for anyone unfamiliar with the concept,
TL;DR: For my use case, mradermacher/Signal-3.8-27B-Terse-Coder.i1-Q4_K_M proved out. Ymmv based on your domain-specific tests. Time taken to solve problems compared to frontier models is massive; esp
Currently having high success with this little niche finetune i found sitting in the corner of huggingface If you want to try it out here is a smaller quantisation iq3_s works really well in my codeba
This table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res
With latest Halogen version update (0.17.2), decode is consistently at ~45 tps even at high context with Qwen 3.8 Flash Next on a 128GB Strix Halo. This is some great work u/peonist-ai. Have been pump
I've been optimizing long-running agents: rewriting their prompts, tools, skills and subagents, and keeping the changes that score better. I've been working on some version of this problem for over a
A week in Beijing and Shanghai with the people building AI in China some quotes from the article: ByteDance/Doubao is the only major lab keeping its flagship closed, because it is the only one that al
TornadoVM: The Java to CUDA engine: https://github.com/beehive-lab/TornadoVM jitLLM: The inference engine: https://github.com/beehive-lab/jitllm Deep dive talk: https://www.youtube.com/watch?v=HO5CpET
The other week I posted about Jev vs. Kev compared and since then, OpenAI released the decisions endpoint, Cloudflare released Clef and many here asked about Laya as well. This time we compared six po
CrowdStrike report: https://www.crowdstrike.com/en-us/blog/unknown-threat-actor-uses-artex-to-target-south-korean-finance/ From: Andrew Curran on 𝕏: https://x.com/AndrewCurran_/status/2108108695876092
tldr: Open sourced my prompt to game suite: https://github.com/ndamiano/ai-agent-test, it's MIT licensed, and this runs well on my 5090 with 64gb ram, but any model that can handle tool calls can mana
Having a lot of fun experimenting with three.js. Still very far from even a demo but its tons of fun. After this project I'll want to try a godot workflow. 3.8FN truly feels like Claude 4.6 at home. U
I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-
recently saw a bunch of open decision models pop out of nowhere in the last two weeks (laya, liquid's d1, cloudflare's clef-flash, interfaze's lev), so I wanted to see how far apart they actually are
Hi all, having searched google and asked several AIs without success, maybe s.o. can help. I have 2 x dgx spark in a cluster. deepseek-flash is running successful. Now I want to test qwen3.8-flash-nex
Hey everyone, Jovan from UkisAI (Swift Qwen) here! For those who don't know us, UkisAI is a small lab making tiny frontier LLMs, tools and datasets (+doing it open-source!). I'm one of the guys runnin
I've been tinkering with GLM-5.3-Flash on two 64GB CMP 170HX cards for a while, and the setup is finally stable enough that I figured I'd share it. I also compared it against the Qwen3.8-Flash-Next se
Yesterday TechCrunch published a piece about a growing problem for AI agents: websites are starting to block them. Amazon blocking Meta's Muse is the obvious example. At almost exactly the same time,
https://preview.redd.it/owhrvbbiu2uh1.png?width=4096&format=png&auto=webp&s=55d816753e7adee0e1391c9f581052b4bde44da7 d1-omni-600M d1-omni-600M is a 600M parameter decision model built on LFM2.5-Encode
Upgrade Question If we say the 'budget max' is 1700-1800, and this could include upgrading to a Taichi motherboard: With a new motherboard, no need to bifurcate 1. Would you add a second 5060Ti (16GB)
PrismML's Ternary Bonsai 2 27B fits a 24 GB card or Mac, but decodes at ~30 tok/s on an L4 and ~21 on an M4 Pro. z-lab's DFlash 2 drafter was trained on bf16 Qwen3.8-27B, so it guesses worse on the te
EmbeddingGemma 2 came out this week. It maps images and text into one 768-dim space, so you can search photos by describing them. I ported its text and vision towers to ruNNtime, a WebGPU inference li
Running a model on your own machine used to be a privacy story with a quality tax. On this generation that trade has narrowed to where we can state it plainly: for daily work, the local model is good
Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and vir
Introducing Open SLM Evaluations! Open SLM Evaluations is a dataset containing benchmarks for 106 SLMs(150M parameters and below). It has benchmark scores for each model, links, parameter counts, and
LIBERO is a robot manipulation benchmark: 130 tasks across five suites (spatial, object, goal, scene10, scene90), each with human demos and a language goal. It is the standard testbed for language-con
We spent the last weeks building a studio around YuE2, the open song model by m-a-p, and today it reaches its first release candidate. Write the style and the lyrics, and it composes, sings and render
My current setup: - single 3090 running turboderp/Qwen3.8-27B-exl3:SC_5.00bpw_H6_V6 - ~150k context, ~60t/s, unknown prefill because I didn't benchmark it (but it is ok) - Intel 12400 CPU with 32GB DD
This will be a potentially unpopular post - but it's the truth and grounded - so let's get to it. Hopefully you are familiar with my previous Tested in Coding series: https://www.reddit.com/r/LocalLLa