When eight systems share a benchmark top score, what should the table show?[D]

I have been thinking about how result tables get read when the interesting part is how close everyone is. The Agent Memory Leaderboard's public board from August is a nice case. On the Cycle 1 coding board, Open-source Methods, 43 listed systems run from 46.00% to 52.67% solve rate over 150 tasks, meaning 69 to 79 tasks solved, and eight systems sit together at 52.67%. Since one task there is worth 0.67 percentage points, the whole top cluster moves by a task at a time. Honestly I like seeing this. My read is that a lot of different designs land in the same neighbourhood on these coding tasks, and convergence like that is informative in itself. It also means the table format carries real weight. Options I have seen or used: raw counts beside percentages, tier groupings instead of strict ranks, and interval estimates across tasks. If you have had to report scores this tight in a paper or internal eval, which of these did your readers actually engage with, and what did they take away from it? submitted by /u/sRhubarb [link] [comments]

Source: r/MachineLearning | 2026-10-20