Skill libraries are shipping in agent harnesses on the assumption that writing skills down compounds. A new benchmark tests that directly. C…

Skill libraries are shipping in agent harnesses on the assumption that writing skills down compounds. A new benchmark tests that directly. ContinualSkillBench covers five domains, each with 100 interconnected subtasks ordered by increasing difficulty and built with deliberate opportunities for cross-task skill reuse. Sequential execution generally improves performance, with gains varying substantially across models and domains. On average, maintaining an explicit skill library performs comparably to plain in-context learning. Much of the improvement comes from adapting to prior context and feedback rather than from reusable skill abstraction. Explicit skills still pay off selectively, on tasks that need reusable procedures or precise outputs. There is a useful diagnostic buried in the results. Less capable models accumulate larger, more fragmented collections of task-specific skills, which is what failed abstraction looks like from the outside. Current in-context skill evolution supports continual adaptation. Consolidating experience into transferable skills is still open. Paper: https://arxiv.org/abs/2608.03874 Track more trending AI papers in our academy: https://academy.dair.ai/

Source: DAIR.AI (X) | 2026-08-05

Related