Adaptive Partitioning Schemes for Optimistic Optimization

arXiv:2610.04039v1 Announce Type: cross Abstract: Applications such as engineering design often require us to optimize a black-box function, i.e., a system whose inner processing is not analytically known and whose gradients are not available. Practitioners often have a fixed budget for the number of function evaluations and the performance of an optimization algorithm is measured by its simple regret. In this paper, we study the class of "Optimistic Optimization" algorithms for black-box optimization that use a partitioning scheme for the domain. We develop algorithms that learn a good partitioning scheme and use flexible surrogate models such as neural networks in the optimization procedure. For multi-index functions on an m-dimensional subspace within d dimensions, our algorithm attains ilde{O}(n^{-eta / d}) regret, where eta = 1 + frac{d-m}{2m-1}, as opposed to ilde{O}(n^{-1/d}) for SequOOL, a state-of-the-art optimistic optimization algorithm. We use our approach to improve the quality of Activation-aware Weight Quantization (AWQ) of the OPT-1.3B model, achieving sim10% improvement in performance relative to the best possible unquantized model.

Related

• An Open-Source Training Dataset for Foundation Models for Black-box Optimization
• Inducing Permutation Invariant Priors in Bayesian Optimization for Carbon Capture and Storage Applications
• Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
• Variance-Optimal Control Variates for Learning with Black-box Feedback

Source: arXiv cs.LG | 2026-10-06