LESS: Lightweight Evolutionary Supernet Search in Minutes

arXiv:2610.01468v1 Announce Type: cross Abstract: Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves (93.189pm0.467%) CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately (1/24) of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by (0.577) percentage points while changing best-visited accuracy by only (0.054) points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with (69.615pm1.139%) and (43.720pm1.697%) accuracy. Applied without tuning to the larger DARTS space, LESS achieves (96.95pm0.14%) on CIFAR-10 and (82.43pm0.80%) on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.

Source: arXiv cs.LG | 2026-10-02