Hardware
Restore LLM Inference Capacity in Seconds with Shadow Engine Recovery in NVIDIA Dynamo
Shadow Engine Recovery in NVIDIA Dynamo keeps a fully-initialized standby LLM engine on the same GPU, using the GPU Memory Service (GMS) to persist model weights and share them across engines without
Shadow Engine Recovery in NVIDIA Dynamo keeps a fully-initialized standby LLM engine on the same GPU, using the GPU Memory Service (GMS) to persist model weights and share them across engines without duplication. When the active engine process fails, the shadow engine takes over within seconds—reducing failover time from roughly 283 s to 7.3 s in GLM‑5.2 on B200 nodes—and thus improves tail‑latency, decode throughput and SLA compliance while keeping HBM overhead minimal.
Related
- NVIDIA Dynamo Snapshot: Fast Startup for Inference Workloads on Kubernetes
- ModelExpress: Distributing Model Artifacts at the Speed of Light
Source: NVIDIA Developer | 2026-08-25