Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models
arXiv:2609.33812v1 Announce Type: cross Abstract: Repeated runs of the same coding agent are known to give different benchmark scores. We ask what that variation means for a team running an agent on i