AI Wiki — Knowledge Base

R9V: A designer set of kernels I've been working on for R9700s/RDNA4. Qwen3.8-Flash-Next Unsloth IQ4_XS (w/ TP on 2 R9700s, MTP, SSD n-gram, 128k ctx, vision): TG256 of 78 tok/s (~3x increase), PP8192 of 1510 tok/s (~30x increase).

TL;DR: Ninfer/DS4 but for RDNA4 Highly custom kernels built for RDNA4, applied to vLLM-Radiance to greatly improve Qwen3.8 Flash Next speeds. This is mostly for dual R9700s with preferably 48GB of RAM

model-releasesr-localllama

31 Aug 2026

Search: “routing”

Jev-style structured decisions with DiffusionGemma GGUF in TensorSharp — benchmarks vs. LocalJev

mini-AGI: Continual-learning dynamically looped transformer with evolutionary grown (on a laptop)

Putting the question before the context took my local Qwen from 89% to 100% on a decision benchmark, and from ~400 ms to ~80 ms

Training a Neural Network on AMD MI50s Using Vulkan: Proof of ConceptOr: Why I Stopped Listening and Just Did It

Made the horizontal open-source model for Jev with RLCD, and it surpasses all the Jev benchmarks. HF space, benchmark, model, repo

Multi-hour llama.cpp optimization experiments on Qwen MoE models, patches, benchmarks, and reproduction guides

XingChen-AGI/Xing4.0-29B-A4B MoE

Write up on building and improving reliability of agentic harnesses + a big dataset from my work.

XeBoostLM: Native C++ Local LLMs for Intel NPUs & iGPUs

Where are you running DeepSeek V4.1 Flash reliably? I can't get it to finish a single 2-hour benchmark run

Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poors

college student replicates claude cowork for open source models

Run Claude Code on Ollama models with per-model context scaling and cost tracking

How many agents can 2×4090 actually run at once? Three weeks of llama.cpp concurrency data — soft cap 5 @ 64k, hard cap 9, and why.

I built Infercat: Share your local AI with friends over an encrypted p2p tunnel. Connect with an invite code from anywhere.

Is anyone working on wave-superposition-based pattern recognition instead of neural-network weights? [R]

exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!

I've found myself using Local LLM's like 3D printers.

NInfer fork: 555k context@fp4 for 5090 with YARN, reliable kv host cacheing, monitoring, jinja, opened model support

NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090

Increasing active parameters per token in MOE (Qwen 35B A4B+) reduce reasoning token by 8.5% - and you don't need to train or finetune!

Qwen3.8-Flash-Next MTP merged in ik_llama.cpp (integrated head or separate -md file)... 45 → 90 tok/s on a 5090 + 128GB, works down to a 12GB 4070

Training-free hack to reduce MoE reasoning tokens by 8.5% without losing accuracy but increasing Active parameters per token in MOE (Qwen 35B A4B+)

R9V: A designer set of kernels I've been working on for R9700s/RDNA4. Qwen3.8-Flash-Next Unsloth IQ4_XS (w/ TP on 2 R9700s, MTP, SSD n-gram, 128k ctx, vision): TG256 of *78 tok/s* (~3x increase), PP8192 of *1510 tok/s* (~30x increase).

llama.cpp Open PRs list - CPU/RAM/Disk/Hybrid Related - Better for CPU-only & Hybrid inference

Today I hit 181 toks/s (aggregate) on Qwen3.8-Flash-Next on 2x DGX Sparks

2x Strix Halo speed-up with an R9700

Lemonade end-of-summer project update, now serving 15 engines!

Open Source Kernel in Qwen3.6-35B-A3B for AMD MI350X: 78,498 output tok/s on 8 GPUs

Improving an ollama model with rag and raft

MCP vs. RAG for world knowledge?

Row-Bot v4.8.0 is live

AirLLM - Recent Updates - with Qwen3.8-27B, Kimi-K3 too

How much of the weight-space perception gap is actually symmetry? Evidence from ~1.8M fitted SIRENs [R]

Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.

Built a token-aware gateway/load balancer for local LLM stacks — because nginx has no idea what a token costs

How to make any Sparse Attention / KV Compression look good? [D] [R]

Built a Mac app that actually uses Ollama for work, not just chat — with a calibration step for models with flaky tool calling

What unique, custom QOL upgrades have you given your local agents?

Planning/RL for a stochastic single-player merge puzzle: afterstates, previewed chance events, and long-horizon throughput [D]

Comparing how Cline, Kilo, and Qwen Code handle long-task context/state (and why context loops keep happening)

300b on 32gb MoE-streaming findings + optimisations

Serving Deepseek v4 Flash 0731 on 2x DGX Spark — 5-7 GB OS headroom, what would you do to lower VRAM usage and increase OS available RAM?

DeepSeek V4 Flash 0731 (Q4) now reaches 1,328 tok/s prefill and ~29 tok/s decode on one RTX PRO 6000

I got tired of ad-filled mobile wrappers for Ollama, so I built PocketLLM Lite an open-source, offline Android client (Local GGUF, SKILL.md plugins, local RAG)

Expert-only IQ3 requant of DeepSeek-V4-Flash-0731: better KLD than UD-IQ3_S, 1.4x decode on a CPU-spill rig

I pushed Kimi K3 onto one CPU with 8 GB of RAM

[Release] WinterMix — Qwen3.5-122B-A10B in native MLX: an 82 GiB build that beats 94–95 GiB quants, plus a 68 GiB build for agent swarms

R9V: A designer set of kernels I've been working on for R9700s/RDNA4. Qwen3.8-Flash-Next Unsloth IQ4_XS (w/ TP on 2 R9700s, MTP, SSD n-gram, 128k ctx, vision): TG256 of 78 tok/s (~3x increase), PP8192 of 1510 tok/s (~30x increase).