Model Releases
Does MTP head get loaded in VRAM by default?
I ran into a doubt when using the following command. It seems that the System RAM usage keeps increasing even though there is >10GB of space left in VRAM while using the MTP mode. Does the MTP head lo
I ran into a doubt when using the following command. It seems that the System RAM usage keeps increasing even though there is >10GB of space left in VRAM while using the MTP mode. Does the MTP head load separately from the main model? Do I need to set the device here as well? /mnt/ml/llama.cpp/llama.cpp-cuda-13.2-20260723/build/bin/llama-server -dio --no-warmup --jinja --swa-full --no-mmap -m /mnt/ml/Models/lm-studio-models/CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF/Nvidia-Qwen3.6-27B-NVFP4-Q8.gguf -ngl 999 -c 262144 -b 2048 -ub 512 -fa on --device CUDA0 -np 4 --kv-unified --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75 --host 0.0.0.0 --port 8021 --slot-save-path /home/linuxadmin/.config/myapp/checkpoints --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --chat-template-kwargs {preserve_thinking: true} submitted by /u/xornullvoid [link] [comments]
Related
Source: r/LocalLLaMA | 2026-07-30