Model Releases
Sharp template to NInfer: -42% output tokens, same speed
Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on
Sharp v22.1 is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. NInfer is a hyper-tailored inference engine that only runs certain Qwen models on 5090. NInfer doesn't support changing Jinja templates, so I overlaid the behavior in C++ instead in a fork: ninfer-sharp --chat-style sharp-v22.1 appends Sharp's terseness instruction to the system prompt --reasoning-effort with 7 levels, none = thinking off Official model artifact untouched (NInfer verifies templates by hash) In my tests, the output remained byte-identical to the actual Sharp Jinja template except the engine's internal thinking markers, which it depends on for parsing. Tool-call format is still NInfer's own, not Sharp's. Numbers (5090, Qwen3.8 27B, xhigh (default)): default sharp completion tokens baseline -42.2% wall time baseline -22.6% decode speed baseline same submitted by /u/xrailgun [link] [comments]
Related
- I pushed Qwen3.8-27B limits again... Dflash2 - 134 tps on a RTX 3090
- OpenCode overrides the samplers for Qwen models to the wrong values
- I pushed Qwen3.8-27B to 124 tps on a single request on a RTX 3090
- I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090
Source: r/LocalLLaMA | 2026-08-22