Constrained Hyperparameter Optimization for Streaming Data

arXiv:2608.24712v1 Announce Type: cross Abstract: Optimization of hyperparameters is a critical factor to obtain optimal model performance. While existing research has predominantly concentrated on batch-learning scenarios, addressing the complexities inherent in data streams presents a challenge. The deployment of sophisticated methodologies to manage data streams becomes highly important. Consequently, the capacity for self-adjusting hyperparameters during on-line learning phases emerges as a goal. Many hyperparameters exhibit constraints and are confined within bounded search spaces, rendering specific solutions unacceptable upon applying optimization operators. To solve this issue, employing boundary constraint- handling techniques becomes imperative to rectify invalid solutions. This paper presents strategies for effectively managing boundary constraints within constrained numerical optimization problems. Recent methodologies, including heuristic and evolutionary-based optimization, employ a "boundary" strategy, wherein values that surpass boundary thresholds for a given hyperparameter are realigned to the respective limits. Our study introduces four strategies to navigate boundary constraints in online optimization algorithms. Through empirical investigations conducted on established datasets, we demonstrate that adopting boundary strategies outperforms the "boundary" strategy.

Related

• AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
• CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG
• DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning
• Beyond What to Select: A Plug-and-play Oscillatory Data-Volume Scheduling for Efficient Model Training
• Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

Source: arXiv cs.AI | 2026-08-26