Research
Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans
arXiv:2609.08003v1 Announce Type: new Abstract: Behavioral foundation models have been proposed as stand-ins for human participants across settings, but it is unclear whether theories discovered on th
arXiv:2609.08003v1 Announce Type: new Abstract: Behavioral foundation models have been proposed as stand-ins for human participants across settings, but it is unclear whether theories discovered on them generalize to humans or merely characterize the simulator. We ran the Automated Cognitive Scientist (extsc{AutoCog}), a closed-loop discovery system in which LLM agents design theory-discriminating experiments, collect responses, arbitrate between competing theories, and synthesize successors, entirely on behavior simulated by Centaur, a foundation model of human behavior. In a multi-attribute decision-making setting, the theories extsc{AutoCog} found on Centaur generalized to human data: they outperformed canonical theories on ten held-out experiments and were rivaled only by theories found by running the same loop on people. We argue that this succeeds despite the simulator's inevitable imperfections because a discovery loop that arbitrates between competing theories demands less of its simulator than estimation does. The simulator only needs to capture the regularities that distinguish the theories, and not necessarily reproduce behavior precisely. Imperfect simulators can therefore widen the search over theories, with human data then testing whether the surfaced theories generalize.
Related
- Evaluating LLMs as Human Surrogates in Controlled Experiments
- Naturalistic Computational Cognitive Science: Towards generalizable models and theories that capture the full range of natural behavior
- How Do Large Language Models Judge Social Attraction? Evidence from Theory-Grounded Persona Ratings Across Multiple LLMs and Humans
Source: arXiv cs.AI | 2026-09-10