MindControl - llama.cpp fork to guide the reasoning process via injection during sampling

The primary driver of this project is that I'd become frustrated with the reasoning behavior of smaller local models such as Qwen3.6-27B (i believe particularly at lower temperatures, and where system prompts are highly specific), their reasoning process is highly unreliable and often tends to spiral into neverending "But, wait" loops or, occasionally, complete garbage. The core principle is simple - when the sampler sees an opening tag, it kicks off the thought process with a self-aware statement to nudge the model to behave properly - ie. "I have a thinking budget of tokens, my thought process should remain concise" - this is then prefilled, and sampling continues from there. Once reaching another threshold of, say, 70% of the thinking budget, it again interjects with a statement bringing attention back to the budget - "I've reached 70% of my reasoning budget, let me start working towards a conclusion" When the actual budget limit is hit - it gets given some grace period during which the sampler waits for a good time to cut the thought process off - usally a newline. At that point it'll inject something like "I've reached the end of my thinking budget, now i will provide the user an answer" In my testing so far, this technique has proved noticeably effective at guiding the thought process. Next steps would probably be to generalise the concept and develop something like a "reasoning grammar" or template-based approach - which could enforce different reasoning approaches based on the task at hand. The repo is public, linked below - there is also a pre-built docker image for AMD64 + CUDA I'd be curious to see if this type of enhancement is useful for anyone other than myself lol github.com/laurencehardman/llama-mindcontrol submitted by /u/hellajacked [link] [comments]

Source: r/LocalLLaMA | 2026-07-22

Related