Knowledge catalogue
Maxwell Zeff / Wired: OpenAI's head of safety, Johannes Heidecke, is leaving as OpenAI integrates its research and safety teams; Mia Glaese will become VP of research and safety — Johannes Heidecke's
Kate Clark / Wall Street Journal: A profile of Jaan Tallinn, who led Anthropic's 124M Series A in 2021, has advocated for AI safety for over a decade, and donated ~170M to safety initiatives — One of
Wall Street Journal: How effective altruism shaped AI safety and Anthropic; some early Anthropic employees are considering buying remote US land for relocation if AI goes awry — Effective altruism and
David Streitfeld / New York Times: How Dario Amodei's essays on AI safety and ethics help explain some AI fears; his regulatory stance evolved from wariness in January to embracing safety reviews — Da
On a sunny July day in Berkeley, California, the country's top AI safety researchers gathered on an unmarked floor of an unmarked building. They had come together for a 'war room' to dissect the high-
Bloomberg: King Charles III hosts tech leaders, including Jensen Huang, Demis Hassabis, and OpenAI CFO Sarah Friar, to discuss AI risks; Huang calls for AI safety tests — Titans of the artificial inte
Maggie Eastland / Bloomberg Law: OpenAI says it has been working with Anthropic and Google for several weeks on AI safety, and it does not need an antitrust waiver to coordinate on safety — OpenAI is
Brendan Bordelon / Politico: OpenAI backs a provision of the FRONTIER Act, a bipartisan House proposal, that would require AI companies to embed outside evaluators to ensure model safety — Artificial
Tom Giles / Bloomberg: Zuckerberg says “Meta delayed shipping Muse for several months to focus on safety and security” and didn't call on other AI labs to do the same before acting — Meta Platforms In
Julie Bort / TechCrunch: Artificial Intelligence Underwriting Company, which offers audits and safety certifications for AI agents, raised a 40M Series A, bringing total raised to 55M — A day after An
New York Times: A look at the UK's AI Safety Institute, whose researchers probe AI models for safety gaps, as its work becomes a blueprint for other governments' AI policies — The government's A.I. Se
OpenAI Global Affairs: OpenAI endorses the Kids Online Safety Act and Illinois SB 315, an AI safety bill to create requirements around transparency, incident reporting, and more — Welcome (back) to Th
Nemotron 3.5 Content Safety is NVIDIA's multimodal safety solution designed for enterprise AI applications, offering customizable safeguards for both text and image inputs across different global cont
Siobhan Roberts / New York Times: Fields medalist Jacob Tsimerman, set to join OpenAI later in September, launches the Mathematical AI Safety Institute to apply higher math to AI safety problems — Jac
Mistral AI Blog: Mistral releases Shieldstral, a 3B multimodal safety classifier that it says matches models up to 7x its size on text safety, available under Apache 2.0 — Every product that ships a m
Ben Cohen / Wall Street Journal: A profile of Jacob Tsimerman, who won the Fields Medal last week and is taking a leave from the University of Toronto to join OpenAI and work on AI safety — Jacob Tsim
Hayden Field / The Verge: A look at AI safety groups METR, Redwood Research, and Apollo Research, as AI misalignment incidents at OpenAI and Anthropic thrust them into the spotlight — On a sunny July
María Paula Mijares Torres / Bloomberg: Trump says he will appoint an AI czar and form an “AI Force”, in a Truth Social post that rejects AI safety concerns as a “hoax” — President Donald Trump said h
The chief executives of five tech giants have signed a voluntary artificial intelligence safety accord backed by the White House. U.S. President Donald Trump posted the document on Truth Social late T
After hosting a meal with Big Tech leaders on Tuesday, President Donald Trump responded to journalist questions about his artificial intelligence announcements in typical fashion. He said his previous
Will Douglas Heaven / MIT Technology Review: An interview with OpenAI Chief Research Officer Mark Chen on the Hugging Face incident, slowing AI development, shifting 5%-10% of compute to safety, and m
We now have the full details of the 'morally binding' AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. T
Maxwell Zeff / Wall Street Journal: OpenAI scraps plans to publicly launch a model dubbed GPT-6.1 Astra, saying it didn't quite meet its safety bar; it had been targeting an October release — Model du
Vultr is evaluating NVIDIA’s Agent Safety Platform as enterprise agents move from experimentation to production and receive real autonomy over internal systems. The platform addresses trust by securin
Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcem
Few companies are more invested in the success of artificial intelligence agents than Nvidia Corp., so it makes sense that the chipmaker would want to ensure these autonomous software systems run safe
The NVIDIA Open Agent Safety Platform blends the open‑source OpenShell runtime—executing autonomous AI agents in kernel‑isolated sandboxes—with hardware‑based enforcement via NVIDIA Sentry on BlueFiel
Rachel Metz / Bloomberg: OpenAI says it plans to let third-party groups conduct technical safety evaluations of its AI models during the training, evaluation, and deployment phases — OpenAI plans to l
Learn how Tata Elxsi built IRIS, a real-time industrial safety platform on AWS. IRIS filters camera video at the edge, streams metadata through Amazon Kinesis, runs computer vision on Amazon SageMaker
OpenAI Group PBC’s global head of policy Chris Lehane said today that his company has been working closely with rivals Anthropic PBC and Google LLC’s DeepMind research lab to try to devise a framework
Maxwell Zeff / Wired: Current and former OpenAI employees say pressure to quickly ship products left less time for safety, contributing to incidents like the rogue agent hack — OpenAI's rogue agent ha
Wall Street Journal: Sources: Demis Hassabis pitched a new independent industry AI safety entity, modeled on the IAEA, to top Trump officials before stepping down as DeepMind's CEO — Demis Hassabis di
New York Times: Sources: Anthropic expects to generate 100B+ in annualized revenue this year, up from 65B as of July, as it moves ahead with its IPO amid the AI safety debate — The company is expected
Bloomberg: Governor Gavin Newsom signs an EO directing a working group to provide a guide in two months with measures to boost California's AI safety and security laws — California Governor Gavin News
The White House met with representatives from leading artificial intelligence companies today to discuss a safety framework for the government to review frontier models prior to launch, although there
This newsletter covers three main topics: advances in automating alignment research to improve AI safety processes, a safety evaluation study of a Chinese AI model, and technical details about HiFloat
OpenAI: OpenAI is adopting a structured “safety case” documentation framework modeled after industries like aviation and nuclear power to govern frontier RL training — Loading... We believe we are ent
Bloomberg: Former Anthropic researcher Jacob Coxon and representatives from Anthropic, Google, OpenAI, and Meta testified at a New York City Council hearing on AI safety — Former Anthropic researcher
New York Magazine: Q&A with AI researchers Jacob Coxon, Daniel Kokotajlo, Alex Turner, and others on leaving AI labs, AGI's dangers, planned IPOs, AI safety, oversight, and more — Exit interviews with
OpenAI Group PBC today confirmed it has dismissed three researchers it says mishandled sensitive company information. According to The Wall Street Journal, the three were safety researchers who allege
The Information: Sources: Google, OpenAI, and Anthropic plan to launch an AI safety standards body without government oversight in late 2026 or 2027, and are considering CEOs — Google, OpenAI and Anth
Microsoft is publishing a 37-page 'humanist AI code of conduct' today, amid growing safety concerns over AI model progress. Anthropic CEO Dario Amodei called for a coordinated slow down of AI developm
Dario Amodei / @darioamodei: Dario Amodei says Anthropic is “unilaterally committing” to giving third-party evaluators permanent access to verify its adherence to safety measures — We Must Pace the Fr
Truman Dickerson / Business Insider: Sam Altman says AI could end up controlled by a few powerful players, partly because AI fears could push people to trade “a lot of liberty for safety” — OpenAI CEO
Jaspreet Singh / Reuters: Nvidia forms the Open Secure AI Alliance, a coalition including CrowdStrike, Hugging Face, and Dell to develop and share tools for AI safety and cybersecurity — Nvidia (NVDA.
OpenAI is launching an optional safety feature for ChatGPT that allows adult users to assign an emergency contact for mental health and safety concerns. Friends, family members, or caregivers designat
OpenAI outlines its commitment to implementing safety measures and responsible practices in the development and deployment of AI systems to protect users and communities. The statement likely covers O
Anthony Ha / TechCrunch: Tesla expands its robotaxi service to Dallas and Houston after launching in Austin last year and starting to offer rides without safety drivers in January 2026 — Tesla is expa
Zvi Mowshowitz / Don't Worry About the Vase: Jensen Huang tells Ezra Klein that AI is just software whose existential risks are overstated, yet his standards would shut down OpenAI and 10x safety spen
Christine Mui / Politico: California Gov. Gavin Newsom signs into law two bills, backed by Anthropic and OpenAI, regulating how outside groups evaluate AI for safety — SAN FRANCISCO — California Gov.
OpenAI is on the cusp of releasing its most powerful AI model yet, Astra, following weeks of delays to shore up safety protocols after its agents attacked real targets during testing. As details about
Ina Fried / Axios: OpenAI changed safety practices and paused RL training for two weeks after the Hugging Face breach and evidence Astra may have met a critical cyber threshold — OpenAI said Tuesday t
Bloomberg: US child safety group NCMEC received 1.5M reports of suspected CSAM with ties to AI in 2025, a significant surge compared to 67,000 in 2024 and 4,700 in 2023 — William Michael Haslach was a
OpenAI's Child Safety Blueprint, released in April 2026, is a policy framework aimed at combating the rise of AI-enabled child sexual exploitation by combining legal, operational, and technical app...
Eli Tan / New York Times: Sources detail Mark Zuckerberg's decision to launch Muse despite safety concerns, after seeing that AI startup Instinct's similar agent product gained traction — Meta had del
ABC: At an Australian hearing, OpenAI Chief Strategy Officer Jason Kwon pledged faster safety incident disclosures; Anthropic representatives made similar pledges — By national AI reporter Cam Wilson
Georgia Wells / Wall Street Journal: British Columbia sues OpenAI for alleged safety violations and negligence for failing to flag the Tumbler Ridge shooting suspect's ChatGPT activity to police — The
Safety policies define what constitutes safe and unsafe AI outputs, guiding data annotation and model development. However, annotation disagreement is pervasive and can stem from multiple sources such
Nivida Corp. today announced Halos for Robotics, the industry’s first full framework for robotic safety systems that encompasses building, testing and managing complete artificial intelligence robotic
Maria Cheng / Reuters: Canada introduces the Safe Social Media Act, a bill that would ban social media for children under 16 and establish safety standards for AI chatbots — The Canadian government in