Model Releases
on DGX Spark: llama serve -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M -hfd ggml-org/Qwen3.8-27B-GGUF:Q4_0 --spec-default --spec-type draft-…
Georgi Gerganov showcased running a LLaMA server on an NVIDIA DGX‑Spark, serving the Qwen 3.8‑27B model in GGUF format (`ggml-org/Qwen3.8-27B-GGUF`). The command demonstrates two quantization options—
Georgi Gerganov showcased running a LLaMA server on an NVIDIA DGX‑Spark, serving the Qwen 3.8‑27B model in GGUF format (ggml-org/Qwen3.8-27B-GGUF). The command demonstrates two quantization options—Q4_K_M and Q4_0—and specifies a draft specification type (--spec-type draft-mtp) along with runtime arguments (e.g., 8 24 345 18K). This illustrates how to launch the model using the minimal “llama serve” invocation.
Related
- llama-server -hf ggml-org/Qwen3.6-27B-GGUF --spec-default
- Pushed: DFlash implementation for llama-cpp. buun-llama-cpp/llama-server -m Qwen3.6-27B.gguf -md dflash-draft-q4_k_m.gguf --spec-type dflash
- Running Qwen3.5-397B-A17B (4bit quants, 177 GB) on two DGX Sparks using llama.cpp with RPC and RDMA:
- llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the …
Source: Georgi Gerganov (X) | 2026-08-14