Hardware
Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton
Generative recommender systems model recommendation as high‑cardinality event streams, turning user actions, context, and candidate items into tokens for sequence modeling over user behavior. NVIDIA’s
Generative recommender systems model recommendation as high‑cardinality event streams, turning user actions, context, and candidate items into tokens for sequence modeling over user behavior. NVIDIA’s recsys‑examples supply an end‑to‑end HSTU inference workflow that compiles ranking models with PyTorch AOTI into native C++ artifacts while FlexKV caching reuses attention states; deploying the eight‑layer model on an RTX PRO 6000 GPU yielded up to 5.93× lower latency at batch size 8 with a 100 % cache hit compared to a non‑cached configuration. The NVIDIA recsys‑examples guide, Dynamo‑Triton documentation, and benchmark reports provide step‑by‑step deployment instructions, production serving details, and latency scaling insights.
Related
- Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton
- How Generative Recommenders Are Redefining RecSys at Scale
- Deploy an Open Model from Checkpoint to Inference in Two Commands with NVIDIA TensorRT Model Connect
Source: NVIDIA Developer | 2026-09-30