KIV: 1M token context window on a RTX 4070 (12GB VRAM), no retraining, drop-in HuggingFace cache replacement - Works with any model that uses DynamicCache [P]
DGX agentKIV is a project shared on r/MachineLearning presenting a drop-in replacement for HuggingFace's `DynamicCache` that enables up to 1 million token context windows on consumer hardware with only 12GB of