Model Releases
Training AI Scientists to Replicate Research
arXiv:2608.13331v1 Announce Type: cross Abstract: The replicability of papers is a cornerstone of scientific knowledge, ensuring the reliability of existing results and providing a base for further ex
arXiv:2608.13331v1 Announce Type: cross Abstract: The replicability of papers is a cornerstone of scientific knowledge, ensuring the reliability of existing results and providing a base for further experiments. The act of replication typically illuminates details that were previously underspecified, and thus requires similar hypothesis-driven exploration to open-ended research. In this work, we develop Replica, a scalable task space for paper replication. To provide reward signal, we introduce an auto-generated rubric-based judge that has low noise and agrees with human assessment of replication quality. We post-train Faraday, a 27B-parameter "AI Scientist" agent that leverages coding agents as tools, surpassing the performance of Claude Opus 4.8 and GPT-5.5 on held-out replication tasks. Qualitative analysis of individual rollouts reveals that Faraday adopts a more scientifically-principled approach. We believe that our results provide a stepping stone towards AI agents capable of long-horizon scientific innovation without requiring complex harnesses.
Related
- Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
- ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciences
- RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension
- AutoResearch: An Execution-Grounded Multi-Agent Framework for Reliable Research Workflow Automation
Source: arXiv cs.AI | 2026-08-14