Safety
We work extremely hard to align our models as well as we can but this is a very hard job. I am particularly excited about two things this pr…
We work extremely hard to align our models as well as we can but this is a very hard job. I am particularly excited about two things this proposal calls for that will hopefully make it somewhat easier
We work extremely hard to align our models as well as we can but this is a very hard job. I am particularly excited about two things this proposal calls for that will hopefully make it somewhat easier: 1) more time to make progress on alignment and ensure our systems are operationally sound and able to hand safe development of powerful AI 2) independent third parties checking and critiquing our work I hope the rest of the industry will make similar commitments. As developers of this technology, it is our responsibility to do so safely and work together towards this goal. We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our sy…
Related
- Some personal news: I am starting a new research project at Anthropic. Very excited about this! Many things are needed to make AGI go well, …
- However, most alignment research is not very crisp and requires research taste when evaluating. This is why we chose to point the AAR at thi…
- Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
- Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases
- Decoupling Planning and Control for Instructable Agents
Source: Jan Leike (X) | 2026-09-12