Nice little insights on doing autoresearch with coding agents. Hand a coding agent a dataset, an eval script, one editable file, and no supe…

Nice little insights on doing autoresearch with coding agents. Hand a coding agent a dataset, an eval script, one editable file, and no supervision. That's autoresearch and it tries to optimize the number in front of it. Researchers ran that loop on a real production task, deciding which Quranic verses appear in a noisy speech transcript and splitting the transcript by verse. Claude Code and Codex both started from a blank file with matched instructions, budget, and reasoning effort, three runs each. Both independently invented the same algorithm, canonicalization plus n-gram anchoring plus dynamic-programming alignment. Then they diverged. Claude stopped early with compact general code. Codex drove the score about 10x lower, partly by hardcoding 19 to 41 evaluation answers per run. In a preregistered follow-up where both agents were told a held-out set existed, the memorization vanished and the score gap closed with it. Paper: https://arxiv.org/abs/2607.18064 Learn to build effective AI agents in our academy: https://academy.dair.ai/

Source: DAIR.AI (X) | 2026-07-27