Hardware
How Generative Recommenders Are Redefining RecSys at Scale
Generative recommenders (GRs) replace traditional embedding‑based methods with transformer‑style sequence models like HSTU and Semantic IDs, addressing scalability, cold‑start, and long‑tail issues in
Generative recommenders (GRs) replace traditional embedding‑based methods with transformer‑style sequence models like HSTU and Semantic IDs, addressing scalability, cold‑start, and long‑tail issues in large catalogs. NVIDIA’s recsys‑examples repository offers production‑ready GR components—DynamicEmb hash‑table embeddings, a KV cache, fused CUDA kernels, and seamless integration with Megatron‑Core/TorchRec—optimized for NVIDIA GPUs. The nv‑embedding‑cache supplies a hierarchical, sharded embedding layer replacement that supports concurrent lookup/eviction, enabling low‑latency inference and high throughput on production‑scale recommender workloads.
Related
- Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference
- Mesh-Attention: A New Communication-Efficient Distributed Attention with Improved Data Locality
- BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
Source: NVIDIA Developer | 2026-08-20