I agree with what this AI paper suggests. Self-improving agents should evolve their benchmarks too. (bookmark it) Self-improving agents are …

I agree with what this AI paper suggests. Self-improving agents should evolve their benchmarks too. (bookmark it) Self-improving agents are one of the most important directions in AI right now, and most of them optimize against a fixed benchmark. This paper argues the benchmark should evolve alongside the agent. The setup is a self-evolving Lean proof agent. A small trusted runtime wraps a fully mutable workspace of workflow, prompts, and tools, so the agent can rewrite how it decomposes proofs, uses compiler feedback, and repairs failures. What makes it different is coevolution. Between generations the champion revises the active task distribution through a mastery-throttled curriculum that adds harder proof obligations only after the current level is mastered. All of it stays inside a Lean-grounded loop, so a success counts only when the behavior yields verified proofs under a trusted snapshot. Over 15 generations the best coevolving agent reaches 45.1% held-out solve rate on miniF2F, versus 12.7% for the seed and 32.0% for the best fixed-benchmark agent. Grounding every reward in a formal verifier is how you get self-modification without reward hacking, and coevolving the curriculum pushes the ceiling higher. Paper: https://arxiv.org/abs/2607.17352 Learn to build effective AI agents in our academy: https://academy.dair.ai/

Source: DAIR.AI (X) | 2026-07-22

Related