Local Ai
Training Krea2 LoRA on 16GB VRAM without block swapping
Hey everyone, I wanted to share my recent experience training a Krea2 LoRA on single 16GB VRAM without block swapping. I was really impressed by Krea2's output, but the model itself is massive. With o
Hey everyone, I wanted to share my recent experience training a Krea2 LoRA on single 16GB VRAM without block swapping. I was really impressed by Krea2's output, but the model itself is massive. With only 16GB VRAM, I had to use the int4 convrot quantized version just to fit the checkpoint without block swapping (I really hates block swapping for speed degradation). I don't know if there are some existing tools to train a LoRA on top of such quantized models, but I couldn't find it works with my favorite framework musubi. So I decided to modify it. Initial tests were done on a RTX 4080 Super, and the results weren't bad, about 13s/it. Then I moved the training to my RTX 5060 Ti and got ~18s/it, unbearably slow. I tried mixing int4 convrot + nvfp4 (since full nvfp4 exceeded 16GB), but the speed improvement was minimal. Then I discovered that the latest ComfyUI with cu130 gave a significant boost in inference, about 2s/it (without cfg, 1k res). It applied to training of course, about 9s/it. Further using Flash Attention + torch.compile brought it to 5–7s/it, depending on the dataset. That is the fastest speed i can get. I can't directly compare these results to native bf16 LoRA training since I don't have a card with enough memory, but I did notice a V-shaped loss curve during training in some datasets. Not sure if that's related to my datasets or quantization (which I suspect as a QAT method). Anyway, I've open-sourced code here if anyone want to try: https://github.com/IceSandwich/musubi-tuner/tree/support_int4convrot_krea2 As i notice, RAM usage increases about 2G and VRAM usage stays 12-13G during training. Here are my settings related to VRAM usage: - Training resolution: 1024 - Batch size: 1 - Optimizer: AdamW8bit - Gradient accumulation steps: 1 comparative methods 1.6h for 1200 steps submitted by /u/Corgice [link] [comments]
Related
- My LTX 2.3 LoRA Training Journey: Fighting for VRAM even with a 5090
- What am I doing wrong in my LoRA training?
- 3 hours of lora training completely wasted on Runpod. Any alternatives?
- [[workflow-custom-node-release-i-vibe-coded-my-way-into-gettin|[Workflow + Custom Node Release] I vibe coded my way into getting an existing ltx ic-lora model to spit out 16bit raw ARRI alexa output, from any mp4 footage of any size, using any rtx graphic cards agnostic of its VRAM.]]
Source: r/StableDiffusion | 2026-10-19