AI Wiki — Knowledge Base

The fastest interference engine for RTX5090 and Qwen3.8 27B. Twice as fast as ninfer. 500+ t/s single coding, 2000+t/s up to 12 agents at the same time with 800k context. Smart VRAM-RAM-DISC Cache management, Loop Guard, Nice UI etc.

Hi guys, I am pretty happy to announce MegaCapybara. Purpose build engine for RTX5090 that is focused on Qwen3.8 27B (more will come later). GITHUB (Engine) HUGGINGFACE (weights) Why ? 1. It beats Nin

model-releasesr-localllama

3 Oct 2026

Search: “context”

I benchmarked Qwen 3.8 27B on my M4 Max — here's what I learned about local models, context, and coding performance

Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)

Is llama.cpp meant to be slow at long context, even when you aren't using that context?

Single RTX 5090: Qwen3.8-27B NVFP4 at a real 262K context in vLLM — 77 tok/s short-context, 64.7 tok/s at 128K

Y'all this is a sexy paper; context language models

3 days benchmarking most llama.cpp flags on my weird 40gb vram laptop + tb4 egpu setup. Got +70% generation, +40% prefill, 60k more context, and filed a bug in llama around MTP. What I learned.

[Splash Engine] Qwen3.8-27B in native 8-bit at 37–55 tok/s on Apple Silicon: Extending Splash to Q8, 256k context scaling, and the 'Reasoning Cliff'

1M context with 17 GB model in 24 GB VRAM: 'for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text'

[Benchmark] Optimal DFlash2 quants for speed and context size, 5090 RTX, llama.cpp, Qwen 3.8 27B Dynamic3 Unsloth. Comparison with MTP

Running Qwen3.8-27B-Q4 at max context on a 32 GB GPU while avoiding kvcache quantization

Validate your local LLM advertised KV cache against real pressure; see exactly how old contexts get evicted from cache

:3 Love how it understands context without context.

Mreaia v1.0.0: Universal Zero-Token Engine & Hardware-Locked Context Distillation for LLMs (>99.9% Reduction)

Running Next Flash IQ3_XXS at ~70 tok/s with 100k context or 2 instances of Qwen 3.6 35B A3B IQ4 at ~145 tok/s with 256k all on $500 of ex mining BC-250 boards

Running an LLM-driven town with 800+ persistent agents: concurrency, context caching, and inference costs

2x CMP 170HX 64GB: GLM-5.3-Flash at 384K context / ~90 tok/s (EXL3, HBM-first setup) + Qwen3.8 comparison

Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAM

The fastest interference engine for RTX5090 and Qwen3.8 27B. Twice as fast as ninfer. 500+ t/s single coding, 2000+t/s up to 12 agents at the same time with 800k context. Smart VRAM-RAM-DISC Cache management, Loop Guard, Nice UI etc.

Qwen3.8-27B: >70 tok/s (>160 tok/s concurrent), 10k tok/s prefill, full context on 2x3090 (or and 48GB or larger on ampere or higher), vanilla vllm

Qwen3.8-27B uncensored Q6_K at 156K context on one RTX 5090, 140-190 tok/s with DFlash2

What pi.dev plugin do you suggest for context, compaction and memory management of local models?

Qwen 3.8 27B at 50 tok/s with 100k Context on a 16GB GPU! (beellama.cpp)

Qwen3.8-27B on a 24GB RTX PRO 4000 Blackwell: 128K real context, 785 tok/s prefill, 67 tok/s MTP3 decode

After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)

club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B

Qwen 3.8 27B Q4 with 100K context on a 16 GB RX 7800 XT guide

Qwen3.8-Flash-Next on MLX-serve, 1m context is released!

Your model fits in VRAM. Your context doesn't. The KV cache math nobody shows you.

Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant.

KIV: 1M token context window on a RTX 4070 (12GB VRAM), no retraining, drop-in HuggingFace cache replacement - Works with any model that uses DynamicCache [P]

Qwen3.8-Flash-Next in llama.cpp from CPU-only to 96GB VRAM: 8.5 to 109 tok/s, max context and parameters test. My findings on RTX 6000 PRO.

MTP doubled my generation, but at 64k context it cuts total latency by only ~3%. Measured the prefill wall on a 3090.

I made a open source CLI agent for 8k tokrn context windows - v0.3- improves ollama compatibility and speed by up to 2 times

hands on workshop: context engineering for multi agent systems [D]

Qwen3.8-Flash-Next at 1M context on Strix Halo: 38 tok/s decode, 18 min prefill (halogen 0.12.0)

An open-source context layer for building AI on top of company data

how to create .md files and set context window more than 64k for ollama and claude running locally.

I ran Qwen 3.8 Flash-Next on my AMD 7900 XTX at 500k context. All local and what a shift it has been.

Is anyone running Qwen3.8 Flash Next with a 1M context?

Qwen3.8-Flash-Next (5.05bpw + ngram at bf16) exl3 on one r9700: 863 t/s prefill and 35 t/s decode at 230k context (256k max), is that ok or am i missing something?

How are you guys thinking about context now, and building around it?

LayerStoRm open-source expert streaming: 1M context GLM-5.3-Flash [UD-Q4_K_XL] at 24.5 tok/s @8k on just 2× RTX 5090 + 2× RTX 5080 (186 GiB MoE on 96 GB VRAM)

Running local models for coding — what's your actual context strategy for large codebases?

What setup do you use, which harness? and how do you customize it to run qwen 3.8 flash next swift with a high context length and a reasonable amount of tok/sec?

Qwen Flash Next @ 137 tok/s & 3,497 tok/s Prefill w/ 512k context on a 5090, 192gb ram, Windows Build, comparing Strata and Infernix

Learning to Learn a Language: in-context learning of natural language from a synthetic non-linguistic prior [R]

Run Claude Code on Ollama models with per-model context scaling and cost tracking

Qwen3.8-Flash-Next on 2x3090: 9–12% faster decode at ~119k context, with a completed quality screen