Quartermaster 1.7 update now lets you run llama.cpp's new decision models (/v1/systemone) with a playground tab

Quartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop a decision GGUF (Kev, OpenJev, Laya, ...) in your models folder and it gets its own config entry, sized for VRAM like everything else. Playground tab. Write a prompt, ask one question, pick the answer shape, hit Run. Each option gets a probability bar. Useful for quick testing Model browser. A Decision tab finds them on Hugging Face. Same API, proxied. POST /v1/systemone goes through Quartermaster's scheduler and API keys, so it swaps in next to your chat model like any other request. Needs a llama.cpp new enough to have /v1/systemone (upstream b11361+; Quartermaster will warn about older builds). Project info: https://quartermaster-labs.github.io/Quartermaster/ Release: https://github.com/Quartermaster-Labs/Quartermaster/releases/tag/v1.7.0 Happy to hear what people are using these for. submitted by /u/OneMoreName1 [link] [comments]

Related

• Aplomb 1: open-weights 5.3B decision model, 1M context, text/image/video/audio in one request, #1 among 4B models on the Decision Index
• Running decision models: Picking quant, VRAM usage
• llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp

Source: r/LocalLLaMA | 2026-10-13