Engrams Embedding Entendre: Codesign for Efficient DRAM/SSD Offloading

Engram Embedding Entendre is a model‑architecture optimization that extends standard token embeddings with learned multi‑token lookups, enabling recurring local patterns to be retrieved directly from embedding tables rather than recomputed via attention and feed‑forward layers. This design codesigns parameter offloading so that each token accesses only a few embedding rows whose addresses depend on token IDs; the runtime can prefetch these rows from host DRAM (or SSD) while earlier layers compute, keeping large HBM payloads for weights and KV cache free and supporting larger batches or concurrent sessions. Experimental results with DeepSeek‑V4.1‑Flash show that memory‑mapped embedding tables offloaded to SSD require roughly 189 GiB of storage with only modest performance impact, illustrating how Engram can mitigate rising HBM constraints as highlighted by NVIDIA’s recent chip roadmap reductions.

Related

• Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
• REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling
• WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
• Multi-Probe Zero Collision Hash (MPZCH): Mitigating Embedding Collisions and Enhancing Model Freshness in Large-Scale Recommenders

Source: SemiAnalysis | 2026-09-18