Local Ai
How sparse is too sparse for H3?
So me and various other people have implemented their own Sparse Attention nodes and you can see many people argue about what % you should actually run these nodes on to maintain prompt adherence etc.
So me and various other people have implemented their own Sparse Attention nodes and you can see many people argue about what % you should actually run these nodes on to maintain prompt adherence etc. So to help come to the bottom of this, I extensively tested various settings. https://huggingface.co/datasets/Zironic/h3-attention-breakpoint-10s My main conclusion is that videos only truly visually break below 10% retained attention but various noticable semantic changes can still happen all the way up to 50%. However just increasing the attention at the early steps get you most of that semantic consistency back. I got the best result when tapering the attention down in a ramp which is why I've now implemented that as Denser Early ramp in my own node. Sidenote: That's not even the SLA version of the lora that's trained on 15% attention. It's really surprisingly viable to use the LTX turbo 8 step lora and sparse attention at the same time. submitted by /u/Zironic [link] [comments]
Related
Source: r/StableDiffusion | 2026-08-28