Model Releases
Fixed/improved Jinja chat template for Qwen 3.8
'Again?' you might ask. I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some wh
"Again?" you might ask. I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some while also introducing new issues. Now you have a new chat template that you can download. One issue among others with yesterday's template is the deviation from the original prompt format. The model was most likely only trained on the exact original format. Modifying how the input is rendered to the model can degrade output quality, not so much that it becomes obvious in manual testing, but enough to reduce benchmark scores a bit (example: Qwen 3.6 underperforms when the system prompt is changed) - it just gives you less output quality in a subtle way. The previous modification deviated in some places, including the (from a caching perspective) desirable empty think fix. I chose to stay on the safe side here. () *Full disclosure: GPT 5.6 Sol xHigh did the analysis and comparison. I then reasoned at 0.5 tokens per second for quite a bit what to change and how. Sol applied the changes and drafted the table below. Claude Opus 5 xHigh verified all changes and claims programmatically via Python test scripts using jinja2. It found and creatively solved one sub-issue in the extra tool calling support. It also updated the table after stating that Sol was "too kind" in the evaluation of the templates. I then wrote this posting and edited the table further - at way less than 1 token per second due to not offloading to GPU. Area Original template Other modified This modified Invalid reasoning effort Raises, but only while thinking is enabled Silently converts invalid values to xhigh Always raises high effort alias Unsupported (raises) Treated as xhigh Documented alias for xhigh <:think_on:> / <:think_off:> (Thanks, markdown parser!) Not supported Scanned from system, developer, and user content; has precedence and removal bugs Not supported Historical reasoning fields reasoning_content only, and only when it is a string reasoning_content, thinking, and heuristic parsing from content reasoning_content and thinking; no heuristic parsing Non-string reasoning field Silently dropped (empty think block) Coerced to a Python repr ({'a': 1}) Coerced to a Python repr — unchanged, still a latent flaw Literal in an answer Not interpreted by the template Could be mistaken for a reasoning delimiter Never parsed as reasoning Empty historical think blocks Preserved consistently Removed when reasoning is empty Preserved consistently Raw reasoning in assistant content Not normalized Heuristically normalized Must be normalized by the adapter Tool-call instructions Original trained wording Rewritten to mandate formatting Original wording, byte-identical Multiple tool-call separation Original single-newline behavior Added additional separation in places Original behavior, byte-identical XML mapping arguments Supported Supported Supported, byte-identical XML Boolean/null arguments JSON true/false/null Regressed to True/False/None JSON true/false/null JSON-string arguments (OpenAI wire shape) Crashes at :items Emits raw JSON with no wrapper Renders correctly: auto selects the JSON tool-call form, arguments passed through byte-exact; "{}" accepted JSON tool-call mode Unsupported (XML only) Supported Supported; auto-selected only when XML is impossible Tool format validation N/A Unknown values silently select XML Three-state auto/xml/json; unknown raises; explicit xml with string args raises and names the remedy Tool error detection & message injection None Guesses from tool-response text. Susceptible to false-positives None, there is no reliable way Missing real user query Raises Uses an arbitrary history-length fallback Raises Developer messages Unsupported (raises) Converted to system anywhere Leading system/developer messages are merged Later system messages Raises Silently accepted Raises Unknown message roles Raises Converted to user text Raises, naming the role Malformed multimodal list items Silently corrupts: a string containing "image" becomes a vision token; non-string text coerced Non-mapping items silently stringified; non-string text coerced Explicit validation and errors for both Image/video ambiguity Not checked Not checked Ambiguous items raise Leading tool message Emits a tool turn with no <:im_start:>user Opens a user turn correctly Opens a user turn correctly Artifact integrity (special tokens, whitespace) Intact; 0 literal-text tokens Intact; 0 literal-text tokens Intact; 0 literal-text tokens (Jinja lexer) Byte-equality with the trained original Reference Differs by design (instructions, separators, arg types) 23/23 semantically-equivalent cases byte-identical Prefix-cache stability Preserves reasoning by default Can alter empty-think history and system prompt Deterministic reconstruction; auto flips the system prefix once if a single history mixes mapping and string args Notes: Default thinking is "xhigh" (= "high"). Set it to "medium" or "low" if too slow. Medium should probably be the default as it's the native model reasoning, but I left it at xhigh to not surprise anyone who does not read this. My general approach was to support extra functionality and compatibility introduced by the previous modification as long as it was reasonably safe. Note that the compatibility is mostly for outdated and not well-behaved harnesses. Especially not using the correct tool call format is technically supported now but will likely degrade result quality a bit. The template triggers a single prompt-reprocess if it detects that the harness uses the unsuitable format. I replaced | by : in the table above, as edits otherwise broke my table. submitted by /u/Chromix_ [link] [comments]
Related
Source: r/LocalLLaMA | 2026-08-14