Model Releases
ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding
ReplaySSM stores an SSM state as a high‑precision checkpoint plus a buffer of recent inputs, cutting memory traffic by ~50 % and accelerating SSM decoding kernels on NVIDIA Blackwell Ultra B300 GPUs b
ReplaySSM stores an SSM state as a high‑precision checkpoint plus a buffer of recent inputs, cutting memory traffic by ~50 % and accelerating SSM decoding kernels on NVIDIA Blackwell Ultra B300 GPUs by 1.46×–1.80× at batch size 256. The approach also improves speculative decoding throughput by 1.85×–2.10× on 120‑B models by appending draft inputs to the buffer instead of snapshotting state, while a high‑precision checkpoint with low‑precision reads keeps most evaluated models within ≈ 2 % FP32 accuracy on GPQA‑Diamond. ReplaySSM is available as open source and can be integrated into hybrid‑model inference pipelines such as TensorRT‑LLM for NVIDIA Nemotron and other SSM‑based architectures.
Related
- Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer
- Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference
Source: NVIDIA Developer | 2026-10-12