Model Releases
LoRA over GGUF: Train DeepSeek-V4-Flash in 90G VRAM
https://github.com/woct0rdho/transformers5-qwen3.5-recipe An update on my progress with low-VRAM LoRA training over GGUF base model: Now we can train DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, with
https://github.com/woct0rdho/transformers5-qwen3.5-recipe An update on my progress with low-VRAM LoRA training over GGUF base model: Now we can train DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, with no CPU offloading. On Strix Halo it runs at 19 s/it. All the WTF parts - sliding attention, CSA, HCA, mHC now have vibe-coded Triton kernels, and they're faster than any other implementations I've seen. Besides training, I also hope the work on integrating GGUF into the PyTorch ecosystem can make non-training model surgeries more accessible, such as Heretic. Abliteration with mHC is still an unsolved task, see https://huggingface.co/blog/RadicalNotionAI/mhc-ablation-challenges submitted by /u/woct0rdho [link] [comments]
Source: r/LocalLLaMA | 2026-07-28