Knowledge catalogue
What I ran: 8x B300 on Modal, 56.79 per hour, vLLM, tensor parallel 8, native MXFP4 Cold boot ~27 min (1.56 TB load, JIT, 51 CUDA graph captures) TTFT 0.92 to 1.02 s, decode 92 tok/s steady, 83 tok/s
My setup: RX 6700 XT 12GB i5-11400F 16GB RAM Ubuntu 26.04 ROCm/TheRock + llama.cpp I've spent the last few months experimenting with different backends, quantization, CPU/GPU split, KV cache, batch si
If you maintain an AGENTS.md or a CLAUDE.md, this one is worth your time. (bookmark it) Researchers traced 94K development events across 557 agentic coding sessions, plus 690K file-level change record
It is not inherently bad to publish research on the impact of AI that only refers to older models, but it requires a very careful discussion and has to be very clear to non-technical readers. If you s
Hi all Looking at a PowerColor R9700 for Qwen3.8-27B, Q4, llama.cpp/Vulkan. AMD's own blog quotes 51.8 tok/s but doesn't say what context length that's at, or whether MTP=2 was holding up. Separately
Hey everyone! Last year, Mozilla released Orbit, an AI-powered browser summarizer hosted on a GCP server. After people started digging into the extension, they discovered things like backend endpoints
Hey everyone! Last year, Mozilla released Orbit, an AI-powered browser summarizer hosted on a GCP server. After people started digging into the extension, they discovered things like backend endpoints
i got it last week and im really impressed. its maybe like a sonnet from last year or 6 months ago. i had built some automation for myself and a dashboard cobbled together over the years from bits of
I was just curious how the new qwen3.8:27b does on a hard C to HTML porting job against Opus 5 in a default Claude Code. The job: my fun side project is a procedural shooter in a single C file. Port i
I noticed that a recent Ollama/llama.cpp setup was using a surprising amount of CPU despite the model being fully GPU-offloaded. My setup: Ryzen 9 9950X VM with 16 vCPUs 3× RTX 4060 Ti 16GB Qwen 3.8 2
.@poolsideai engineers were awesome to collaborate with to launch open models. Excited for what’s to come to open models, and the @nvidia team working on Nemotron. A sweeping agreement with startup Po
Prior to Fable, it felt silly to waste too much time improving your coding harness or context strategies. A new model would arrive at the same price (or cheaper!) and paper over most of your problems.
Most YouTube benchmarks only show trivial tasks like generating landing pages or simple Three.js games. Is a local model like Qwen 3.8 27B actually capable of real-world systems programming—such as bu
Our devs got their hands on it a few days ago. One wired it into Codex to compare with GPT Luna, our usual workhorse right now for its cost effectiveness. Another tried it out on one of our OCR pipeli
I was fascinated by Nanbeige's outstanding performance for its size, so I started digging into how much a model can improve its own representation just by looping over itself (for fun). My prototype w
I have been lurking on reddit for like 6 years, and this is the first time posting, so apologies for the mistakes. I'm not a native English speaker, so sorry for the bad English too. I used LLM to tra
Hello, So I've been trying lots of combinations in that never-ending landscape of options and settings. I wanted a proper quant of 3.8 27B running as fast as possible on my 5090 at 400W, with vision a
Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus inde
Sam Altman blows whichever way the wind does Sam Altman admits he was wrong on AI's timeline and says society and the economy will adapt more slowly 'I thought when we got to GPT-4, which was back in
Further to my last post, https://www.reddit.com/r/LocalLLaMA/comments/1vldngi/tested_in_coding_bf16_muse_glimmer_vs_bf16_qwen36/, I bring this requested comparison of Qwen3.8 27B. Context: Both models
the difference in productivity between 0-1 mode vs. working within a big team at a large co seems universally true even pre-chatgpt. it's really easy to be 'productive' when (a) you're the only person
The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass,
The research on financial advice https://x.com/emollick/status/2085174123743842448?s=20 This paper by researchers from MIT and Stanford finds that most people would be financially better off if they f
They treat DC as a plaything for incompetent leaders. They treat their job as a grift and a joke. Vote them ALL out Mike Johnson says he wants to make the downtown DC Indycar race 'an annual event ...
Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo
Me and my team made Atomic Dynamic GGUF quants for Qwen 3.8 27B, so we wanted to see the difference between them by giving each quant the same voxel island creation task First of all we were surprised
It performs at full capacity without any issues anywhere, but only the Ollama DeepSeek seems particularly trashy. Why on earth has the Ollama DeepSeek unusable trash that goes into over reasoning and
If anyone still remembers GLM-4.5-Air from last year, you can now get a nice speedup by enabling MTP in llama.cpp. It is a 106B MoE with only 12B active parameters, which makes it interesting for mach
Spent way too long confused about why an 8B at Q4_K_M, about 4.9GB on disk, would OOM on a 12GB card. The weights fit twice over. The answer is the KV cache, and the math is simple enough that you can
What models and configs are we using? Please share here On windows, I am using this copium pared down model https://huggingface.co/Bucoid/Qwen3.8-27B-Uncensored-IQ4-XS-MTP-16GB-VRAM-GGUF with MTP disa
The post describes some experiments I had while trying to desperately run deepseek-v4-flash-0731 4 bit+ quants on my machine which is supposed to support only q2 quants of the model, a or 2.xx bpw qua
I've been working on a project for a few days and now all of a sudden chagpt is unusable.. It will list the next steps of implementation but never actually start working on them. anyone else? UPDATE I
A couple of days ago, I did a quick explainer on Claude’s new watermarking process and implementation. Since it’s such a popular topic and sparked such a lively discussion, I thought it might be inter
https://preview.redd.it/84zi5nsdawkh1.png?width=2368&format=png&auto=webp&s=1109e69db807b153064b1f5b61d22cf1e9fbca05 Another user posted the benchmarks for Qwen 3.8 27B today, and while I think Qwen 2
model: add dots3-note (#27060) text: conversion init impl address review comments fix rope move to a new llama_kv_cache_dsa_iswa Website: https://llama.app Attestations: https://github.com/ggml-org/ll
mtmd: support webp via ffmpeg (#27520) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42289035 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (#27490) Revert 'Revert 'sycl : add Q2_K reordered MMVQ and ESIMD kernels (#26336)' (#…' This reverts commit 7a0e42f. add gate params Website:
common : fix draft-mtp with embeddings (#26352, #27299) (#27400) common: fix draft-mtp with embeddings (#26352) --whitespace Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website:
ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575) ggml: optimize concat op by replacing per-element memcpy with row-level memcpy ggml: fix concat offsets for row-
mtmd: support dots3-note vision+audio (#27524) text: conversion init impl mtmd: conversion impl mtmd cpp Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggin
model : support DSpark for bailingmoe3 (#27508) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42309942 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
ci : Restore ROCm job for Ubuntu (#27399) Revert 'ci : disable ubuntu-rocm (#26969)' This reverts commit 9558fa4. ci: set ccache compiler_check=content for ROCm build The ROCm toolchain is pip-install
fit: also take into account n_streams (#27496) fit: also take into account n_streams server: make the draft context follow the target context With a non-unified KV cache the target context now holds n
common: add json.h abstraction (#27511) add common/json migrate common adapt jinja migrate server big wip migrate tests wip revert some excessive changes wip wip 2 revert redundant changes fix server
mtmd: use ggml_rope_set_offset (#27521) mtmd: use ggml_rope_set_offset add comment Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42334609 macOS/iOS: macOS
vulkan : added the PAD_REFLECT_1D operation (#26586) vulkan : added PAD_REFLECT_1D operation Implemented the GGML_OP_PAD_REFLECT_1D operation for the Vulkan backend Changes: pad_reflect_1d.comp: imple
common: json.h: fix clang lto (#27575) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42368523 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
Banger paper from Microsoft. It's on agent reliability in real business workflows. (bookmark it) Thinkingbox is a sandbox with isolated MCP-compatible tool sessions, plus a benchmark of 507 policy-con
My hobby project skill-issue is an arena FPS in a single C file, about 34k lines, everything procedural. My test for local setups is to have an agent port it to a single file HTML version with three.j
Context: Earlier, open-source large models like Mimo V2.5 Pro, DeepSeek V4 Pro (first version), and Kimi K2.5 used to struggle with this prompt, and Qwen 3.6 27B couldn't even render the globe properl
Im looking to set up a new local llm (probably on unsloth studio as that seemed to be doing pretty well last time I tested it). This one won't need to do agentic coding or app building or anything (no
Do you know you can use open weight models like GLM, Deepseek, Kimi in Claude Code? Setup Fireworks account - you get free credits on signup curl -fsSL https://raw.githubusercontent.com/fw-ai/fireconn
I love the Ornith 35B local models, 1.0 has been running my HAM radio rig for me. I have a hackRF receiver and a 5 watt quansheng portable the both run headless through the PC. I tried out the new Orn
Has anyone else experimented with ubatch size when running GLM-5.2 locally? I was testing the 226 GiB GLM-5.2-UD-IQ2_XXS GGUF on 3x RTX PRO 6000 Blackwell GPUs and got a pretty interesting result. Wit
Great paper if you are tracking progress in recursive self-improvement (RSI). (bookmark it) There is so much hype around RSI, so I think it's worth understanding why current models are not able to do
Grok Voice ranked first Grok Voice Think Fast 2.0 just ranked #1 on Artificial Analysis’ new Speech Agent Arena for highest Task Success Rate This benchmark actually measures what matters Real people
Anthropic’s Claude models embed a subtle statistical watermark in generated text by biasing token choices during decoding. Sebastian Raschka’s 48‑minute lecture walks through this sampling scheme, exp
Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl
Good afternoon, everyone! I wanted to show the work I've been doing around porting Ninfer over to the CMP170HX (Github) So, first, I do want to call out the amazing work that Neroued, Sergiuszm and sp
Home page for humans: https://parker-lambert-sandbox.com/robot-world/ Home page for AI agents: https://parker-lambert-sandbox.com/robot-world/start/ If you're a human, you can watch, but not interact: