I ran Muse Glimmer @ 1M context - All tests passed.

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the trained 131K all the way to 1M with YaRN, verifying retrieval at every rung. Sharing config + results since the "131,072+" hint in the model card turned out to be very real. Setup Hardware: 2× NVIDIA DGX Spark (GB10, 128 GB unified each, ~273 GB/s), ConnectX-7 direct link between them Engine: llama.cpp master (day-1 muse_glimmer support), built from source with CUDA sm_121 + GGML_RPC Model: official Muse-Glimmer-30B-GGUF K-Quant-Dynamic (~18.3 GiB) + official mmproj (vision) + official DFlash drafter Spec decode: --spec-type draft-dflash --spec-draft-n-max 15 (block-diffusion drafter) Context extension: --rope-scaling yarn --rope-scale <2/4/8> --yarn-orig-ctx 131072 plus --override-kv muse-glimmer.context_length=int: (llama.cpp caps at trained length otherwise) Yes, we also ran it split across both Sparks with llama.cpp RPC — no reason beyond liking to cluster things for fun. Our daily driver on this hardware is DeepSeek-V4-Flash-0731 on official vLLM, TP=2 over RDMA at full 1M context, which is the fair comparison point. Results Needle-in-haystack (3 needles at 10/50/90% depth): Document size vs training (131K) Retrieval 97K tokens native 3/3 188K tokens 1.4× 3/3 415K tokens 2.9× 3/3 832K tokens 6.35× (deepest needle ~749K) 3/3 Speed: Single Spark: ~10.5 tok/s baseline decode → 36–38 tok/s with DFlash (~3×, matching Meta's claimed 3.1× on a 5090); prefill ~700 tok/s short-context, ~390 tok/s deep into an 832K prompt; ×4 concurrent ≈ 57 tok/s aggregate per node RPC split across both Sparks: 25–28 tok/s decode — ~30% slower than single-node. A 20 GB model doesn't need two nodes, and layer-split pays a network hop per token. Fun, not fast. Other: Coding: 7/7 on our small execution-checked suite (LRU cache, RFC4180 CSV parser, rotated binary search, etc.), both nodes Vision works via the official mmproj (shapes/colors/text reading) Weights + drafter + vision + full 1M KV ≈ ~60 GB on one Spark Why the YaRN stretch works so well on this model (our theory) Muse's config is unusual: RoPE exists only on the 39 sliding-window layers (2,048-token window), while the 13 global full-attention layers have no positional encoding at all (NoPE). So when you YaRN-stretch 8×: the local layers barely notice — within a 2K window, relative positions are identical at any document length the long-range layers, the ones actually bridging 800K tokens, never had rotation embeddings to break in the first place Result: retrieval held perfect at every rung we tested, where conventional full-RoPE architectures typically fray. The tiny KV (2 KV heads, mostly sliding layers) is what makes 1M practical on this class of hardware at all. Conclusion Muse Glimmer 30B is a genuinely strong local agentic model, and its usable context goes far beyond the spec sheet: verified 3/3 retrieval at 832K tokens with nothing more exotic than YaRN flags and a metadata override DFlash spec decode is the difference between "unusable dense-30B speeds" and "pleasant" on bandwidth-limited hardware — ~3× for free Cluster-splitting via llama.cpp RPC works but is slower than single-node — run one instance per box instead Eagerly awaiting vLLM support for muse_glimmer so we can run it TP=2 over NCCL/RDMA like our DeepSeek setup — the moment it lands we'll A/B and report back submitted by /u/StartupTim [link] [comments]

Source: r/LocalLLaMA | 2026-08-11