Model Releases
b11103
spec : support DFlash for HunyuanOCR (#28890) model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (re
spec : support DFlash for HunyuanOCR (#28890) model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (res->t_layer_inp[il]) - the draft model reads those tensors to build its cross-context. Qwen3 and the other DFlash-capable targets register them, but the Hunyuan graphs do not, so serving a DFlash draft against a HunyuanOCR target aborts during the first graph build: GGML_ASSERT(t_layer_inp[il] != nullptr && "layer input tensor is null") Register the tensor at the top of the layer loop, mirroring qwen3. The layer input is the residual stream entering layer il, i.e. the output of layer il-1, which is what the draft's target_layers metadata refers to (the converter writes target_layer_ids+1). hunyuan-dense.cpp reuses this graph, so it is covered as well; hunyuan-moe has a separate graph and is untouched. The vector is only read when a speculative implementation enables those layer ids, so there is no behaviour change without a draft model. Tested with tencent/HunyuanOCR 1.5 and its DFlash draft: image requests now run, draft acceptance is ~0.5 and the OCR output is byte-identical to the non-speculative run. Co-authored-by: wendadawen wendadawen@qq.com convert : fix DFlash draft conversion against HunYuan targets Converting a DFlash draft with a HunYuan target failed in two ways. DFlashModel.set_vocab() reuses the target class' vocab handling by calling it unbound with the draft instance, but HunYuanModel.set_vocab() called self._fix_special_tokens(), a method that only exists on HunYuanModel, so the conversion always aborted with AttributeError: 'DFlashModel' object has no attribute '_fix_special_tokens' Make the vocab helpers module-level functions taking the model explicitly, so they do not depend on the instance being a HunYuanModel. They have no other callers, so the two id lookups are folded into _fix_special_tokens(). The delegated call runs with self.dir_model pointed at the target but keeps the draft's self.hparams, so config lookups inside the target's vocab code (the pad_token_id < 0 guard, eod_token_id) read the draft's config instead of the target's. That aborts on targets with pad_token_id = -1 (e.g. the HunyuanOCR v1.0 checkpoint) and otherwise writes special token ids that disagree with the target. Add _vocab_hparams(): it returns the target's config (with text_config merged to the root, as TextModel does) when the model is a draft converted with --target-model-dir, and the model's own hparams otherwise, so a normal conversion is unaffected. Tested: converting tencent/HunyuanOCR/dflash succeeds with both the 1.5 and the v1.0 target; converting the base model without --target-model-dir produces a byte-identical GGUF to before. Co-authored-by: wendadawen wendadawen@qq.com convert : fix DFlash draft vocab against HunYuan targets Switch hparams to the target config for the duration of the borrowed set_vocab(), matching the existing dir_model swap, instead of teaching HunYuanModel::set_vocab about draft models. convert : fix HunYuan special token ids for DFlash drafts convert : use load_hparams for HunYuan special token ids Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49255677 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Android: Android arm64 (CPU) Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI
Related
Source: llama.cpp Releases | 2026-09-22