MiniMax-H3: ~38 GB less VRAM with Runtime LoRA Bypass — DoRA Dynamic LoRA Loader v1.0.39

GitHub: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader Release v1.0.39: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader/releases/tag/v1.0.39 Also available through ComfyUI Manager as: ComfyUI-DoRA-Dynamic-LoRA-Loader Runtime LoRA bypass for MiniMax-H3 and other large models v1.0.39 adds an optional: Runtime bypass LoRA (low VRAM) mode for supported standard LoRAs. The main reason for adding it is the amount of VRAM that normal ComfyUI LoRA materialization can consume on very large models such as MiniMax-H3. My measurements were done with: MiniMax-H3 Ref2VA, pruned BF16 model running in HIGH_VRAM mode. With a normal materialized LoRA, the tested H3 LoRA targeted 208 model weights. Tracing the individual patches showed that ComfyUI retained one additional BF16-sized live allocation for each affected weight. Total additional live allocation: ~38,220 MiB / ~37.3 GiB This was actual live PyTorch allocation, not merely CUDA reserved/cache memory. The LoRA file itself obviously isn't anywhere near 38 GB. The cost comes from materializing patched copies of the enormous H3 base weights it affects. What runtime bypass changes The normal materialized path effectively does: W' = W + ΔW output = W'x Runtime bypass leaves the original model weight untouched and evaluates the low-rank LoRA contribution during the forward pass: output = Wx + ΔWx For a supported standard additive LoRA, these represent the same intended transformation. The important difference is that runtime bypass does not need a complete materialized copy of all affected base weights. That is where the large VRAM saving comes from. Actual MiniMax-H3 Ref2VA results With runtime bypass enabled on the same MiniMax-H3 Ref2VA pruned BF16 workflow, the ~38 GB patched-weight duplication disappeared. After ComfyUI had settled into its normal HIGH_VRAM residency state, repeated runs looked roughly like: ~60 GB settled ↓ ~72–73 GB during generation ↓ ~60 GB settled again Changing the LoRA strength and generating again returned to essentially the same settled allocation instead of continuously stacking more VRAM. The Turbo LoRA I tested also remained clearly effective with runtime bypass enabled. So in this particular HIGH_VRAM H3 setup, this isn't a 1–2 GB optimization. It removes a model-sized duplication worth roughly 38 GB of live VRAM. Important: the ~38 GB figure is specifically a HIGH_VRAM result This does not mean every MiniMax-H3 user will suddenly free 38 GB. The amount saved depends heavily on ComfyUI's memory mode and how much of the LoRA-targeted model is resident at once. In HIGH_VRAM, ComfyUI can keep the full model and the materialized patched weights GPU-resident, which exposes the worst-case duplication very clearly. In NORMAL_VRAM and LOW_VRAM, ComfyUI already uses partial loading/offloading. That means those users generally won't have the entire ~38 GB patched copy resident on the GPU at once. So the expected behavior is more like: HIGH_VRAM: potentially enormous persistent VRAM savings NORMAL_VRAM with most/all of the model resident: still potentially substantial NORMAL_VRAM with partial offloading: lower persistent VRAM saving LOW_VRAM: likely much smaller steady-state GPU saving, because ComfyUI is already aggressively loading/offloading pieces of the model Runtime bypass can still be useful outside HIGH_VRAM because it avoids materializing supported LoRA patches and can reduce patch/repatch memory pressure and temporary merge overhead. The maximum saving therefore scales with how much LoRA-targeted base-weight material ComfyUI would otherwise have materialized at the same time. This uses ComfyUI's existing bypass architecture The underlying concept already exists in ComfyUI. ComfyUI currently exposes separate experimental nodes: Load LoRA (Bypass) (For debugging) Load LoRA (Bypass, Model Only) (for debugging) They are marked experimental and normally hidden from node search unless experimental nodes are enabled. The regular Load LoRA node does not currently expose this as an option. v1.0.39 integrates runtime application directly into the DoRA Power LoRA Loader behind a simple toggle: Runtime bypass LoRA (low VRAM) It is OFF by default, so existing workflows retain their current behavior unless explicitly enabled. Does bypass change the LoRA effect? For supported standard LoRAs, it is not intended to weaken or approximate the LoRA. Conceptually: (W + ΔW)x and: Wx + ΔWx are mathematically equivalent. There can still be small floating-point differences because the operations are executed separately and can follow a different dtype/order path. The tradeoff is that runtime bypass evaluates the low-rank LoRA operations during the forward pass instead of paying the merge/materialization cost once up front. So the tradeoff is essentially: much lower LoRA materialization memory usage vs. some additional low-rank compute during inference For a model the size of H3, that tradeoff can be very favorable. DoRA itself is NOT runtime bypassed Despite this being the DoRA Dynamic LoRA Loader, the new runtime path currently applies only to supported standard LoRAs. ComfyUI's current bypass implementation does not reproduce DoRA's magnitude normalization/rescaling semantics. The loader therefore deliberately refuses unsupported cases rather than silently changing the mathematics. Runtime bypass currently rejects cases such as: DoRA / dora_scale Diffusers/PEFT lora_magnitude_vector non-LoRA adapter types reshape metadata sliced / offset / transformed adapter targets incompatible patch-strength semantics For an actual DoRA, leave runtime bypass disabled and the loader uses the existing materialized DoRA path. Multiple LoRAs and strength changes Runtime bypass supports stacked compatible standard LoRAs. The implementation handles multiple adapters on the same module and removes runtime hooks in reverse order so repeated injection/ejection does not restore stale module state. Changing the LoRA strength updates the runtime adapter multiplier rather than rematerializing the entire affected base-weight set. Unsupported non-adapter patches retain normal ComfyUI materialized behavior. Existing loader features remain unchanged The loader still supports: standard LoRAs and DoRAs multiple LoRAs in one Power-LoRA-style node per-LoRA strengths auto-strength redistribution Flux / Flux2 compatibility Diffusers / PEFT DoRA compatibility OneTrainer compatibility Z-Image Turbo / Lumina2 compatibility exact Q/K/V LoRA fusion sliced/offset DoRA handling fp32 DoRA normalization/intermediates DoRA direction-matrix fixes State Manager integration Runtime bypass is an additional opt-in path, not a replacement for the existing loader behavior. v1.0.39 also adds proper compatibility testing The release now has automated package/frontend validation and runtime-bypass tests against pinned ComfyUI versions: v0.29.2 v0.30.2 v0.31.1 Coverage includes: standard adapter capture without weight materialization stacked-LoRA additive equivalence repeated hook injection/ejection cache invalidation DoRA rejection reshape/offset rejection non-adapter fallback behavior cloned ModelPatcher handling fail-closed unsupported loader output For MiniMax-H3 users in particular, especially HIGH_VRAM users loading standard LoRAs, the runtime bypass option is the main reason to update. submitted by /u/marres [link] [comments]

Source: r/StableDiffusion | 2026-08-11