Hardware
DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
arXiv:2607.22614v1 Announce Type: new Abstract: RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from sever
arXiv:2607.22614v1 Announce Type: new Abstract: RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long-tail rollout latency. We present DynaResize, a runtime GPU reallocation system that dynamically switches GPUs between Rollout and Training to balance stage execution times without changing RL semantics. DynaResize decomposes resizing into fine-grained operations and removes non-startup-critical work from the critical path through communicator reuse, bounded state staging, and hysteresis-based resizing. Experimental results show that DynaResize can improve end-to-end throughput by 66.5% and reduce total execution time by 33% over the optimal static configuration, while hiding 27% of role-switching overhead.
Related
- Accelerating Disaggregated RL for Visual Generative LLMs with Diffusion-Based Parallelism and Trainer-Assisted Generation
- TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training
- Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads
- TrainMover: An Interruption-Resilient Runtime for ML Training
Source: arXiv cs.AI | 2026-07-28