Frontier LLMs are effective batch optimizers: Assessing reasoning models in continuous and discrete settings

arXiv:2609.03177v1 Announce Type: new Abstract: Frontier large language models (LLMs) have become attractive priors for optimization due to their large-scale pretraining that enables them to navigate a variety of optimization settings. However, the effectiveness of modern reasoning LLMs in batch optimization settings remains underexplored. Here we investigate the performance of the current generation of frontier LLMs as batch optimizers in both continuous and discrete settings. We find that while LLMs are competitive zero-shot batch optimizers for numerical test functions, their performance is brittle compared to classical non-LLM optimization approaches. However, LLM priors are significantly better in semantically rich settings, indicating that their batch optimization behavior is highly effective when navigating and reasoning over the discrete spaces most similar in structure to their pretraining data.

Related

• Evolutionary Task Discovery: Advancing Reasoning Frontiers via Skill Composition and Complexity Scaling
• LEPO: nderline{L}atent Rnderline{e}asoning nderline{P}olicy nderline{O}ptimization for Large Language~Models
• When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model
• Deriving Scaling Laws for OpenEuroLLM Models: Learning Rate, Batch Size and Loss

Source: arXiv cs.LG | 2026-09-04