Research
DimGrow: Memory-Efficient Field-level Embedding Dimension Search
arXiv:2505.12683v2 Announce Type: replace Abstract: Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such
arXiv:2505.12683v2 Announce Type: replace Abstract: Key feature fields need bigger embedding dimensionality, others need smaller. This demands automated dimension allocation. Existing approaches, such as pruning or Neural Architecture Search (NAS), require training a memory-intensive SuperNet that enumerates all possible dimension combinations, which is infeasible for large feature spaces. We propose DimGrow, a lightweight approach that eliminates the SuperNet requirement. Starting training model from one dimension per feature field, DimGrow can progressively expand/shrink dimensions via importance scoring. Dimensions grow only when their importance consistently exceed a threshold, ensuring memory efficiency. Experiments on three recommendation datasets verify the effectiveness of DimGrow while it reduces training memory compared to SuperNet-based methods.
Related
- Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation
- Evaluating the trustworthiness of the Frechet Inception Distance with stochastic embedding representations
- SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models
Source: arXiv cs.LG | 2026-09-30