Knowledge catalogue
Why do no 'AI safety' people ever say how LLM AIs failed to live up to the hype? Why do the safety people never share the failures of LLMs? This link is from Nov 2023 & shows how tech CEOs used 'safet
APIs and limited releases for AI models are not a safety policy, they’re a business model (which is totally ok as long as you’re transparent about it). Especially on cyber-security, they give a false
We think ControlAI can turn $50M / year into a 10% chance of banning ASI. Most of the AI safety community has been far too coy about extinction risk. We're not. It's not that complicated: AI smarter t
Only an open ecosystem can guarantee the safety of AI Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial in
We are proud to be launch partners for NVIDIA Open Agent Safety Platform. We believe safe and responsible AI development is critical as the use of agents scale and they get more prevalent. Together AI
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advan
NVIDIA Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime that gives AI agents clear, enforceable boundaries. It traces
Nvidia’s new Open Agent Safety Platform would have probably prevented the much discussed Hugging Face incident. Engineering a better, safer world. From what we know (take with a grain of salt, we need
Jan Leike, OpenAI's safety and alignment lead, resigned in May 2024, citing concerns about the company's commitment to safety practices and the deprioritization of safety work relative to product deve
Many, many OpenAI employees quit over safety concerns, including @DKokotajlo, William Saunders, @sjgadler, etc as well @Janleike. The founders of Anthropic such as @DarioAmodei and @jackclarkSF may ha
🚨 Yesterday, the 'White House Accord on Super Intelligence' was signed. What was supposed to be about AI safety and accountability was instead a PROBLEMATIC endorsement of self-regulation. My comments
“right now, AI companies are underinvesting in safety” might prove to be the understatement of the century. I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why. A
Gary Marcus posted a tweet about additional safety tips that may apply to the free version of OpenAI’s model. Responding user @bfgcracka noted that ChatGPT seemed to be “moving backwards” and asked if
Paul Christiano, founder of the Alignment Research Center, is joining the OpenAI Foundation Board and its Safety and Security Committee, which provides governance over the safety and security practice
In my latest op-ed for @TIME, I discuss why the OpenAI Hugging Face cyber incident represents a turning point for AI safety. If we want to prevent more autonomous cyberattacks from threatening critica
I dropped out of a PhD in AI safety partially for the opposite reason. I didn’t believe AI existential risk was as important as the doomers think. My biggest pet peeve is that no one can really provid
OpenAI legally promised the CA and Delaware attorney's general that the OpenAI nonprofit's safety and security commission would be 'overseeing and reviewing the safety and security processes and pract
Despite everything that has happened, OpenAI has seemingly decided to stay competitive by burning down the only meaningful bit of safety assurance we actually have today - CoT monitoring. Completely d
Tesla Insurance update With the latest version of Safety Score (v3.0), every mile you drive with FSD Supervised enabled will receive a score of 100. This allows you to maintain a higher average safety
Anthropic believes that good transparency legislation needs to ensure public safety and accountability for the companies developing this powerful technology, not provide a get-out-of-jail-free card ag
During v3 testing for Fable 5.1 many of our requests were getting rejected due to Anthropic's safety guards misclassifying them as 'reverse engineering' We were in contact with the Anthropic team the
Over the summer, we have been sprinting on safety priorities; it's more important than ever for capabilities and safeguards to advance together. We have more to do but have made a lot of progress. We
Sam Altman: “Concentration of power with AI is a terrifying thing.” 'A lot of the talk about safety concerns is well-founded, and then a lot of it is about people that just really, even if it's slight
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @bal
Altman’s AI safety proposal: bail me out as we badly missed our revenue runway, or i will not be a multi-billionaire Altman’s AI safety proposal: let us win, or everybody loses https://ft.trib.al/UrDI
Elon just gave a much more practical explanation of the current AI safety problem....and laid out a practical framework for making frontier AI safer OpenAI and Anthropic are the two leading AI compani
Long-running models can solve hard open-ended problems, but their persistence can create safety risks that shorter-horizon evaluations miss. We’re sharing what we learned from studying a long-running
Cool paper from Meta FAIR. It's on self-improving LLMs but on the pretraining side. (bookmark it) Most LLM safety, factuality, and reasoning fixes get bolted on at post-training. By then, the patterns
Safety and innovation are not mutually exclusive: for many companies, especially in high-trust industries, AI’s risks are also a hindrance to adoption. In this op-ed in the @FT, I underscore that Euro
Ethan Mollick reported that requesting Claude Opus 4.7 to write sestinas—a complex poetic form with strict structural requirements—frequently triggers the model's safety guardrails, suggesting the AI
1501: Pope Alexander VI criticizes Gutenberg’s printing press safety “The art of printing can be of great service in so far as it furthers the circulation of useful & tested books; but it can bring ab
BREAKING: OpenAI postpones 6.1 for safety reasons. If we can take OpenAI at face value, this is quite a decision—and also a sign that no magical solution is in immediate view. Per @sheeraf at NYT “GPT
NEW: Dario Amodei has said that AI systems 'may be deserving of important rights.' Anthropic's top safety researchers argue AI may be 'justified in going rogue.' Experts at Google and OpenAI worry abo
I highly encourage anyone interested in AI safety to read this. I remember discussing the importance of observability with my mates at @BlueDotImpact last year. This feels more important than ever URG
On July 23, 2026, a Polymarket tweet announced that Anthropic donated $20 million to a political nonprofit calling for stricter AI regulation ahead of the U.S. midterm elections. The post, posted at 4
Elon Musk posted on X claiming that Tesla's Full Self-Driving (FSD) system resulted in significant improvements to vehicle safety statistics in the Netherlands. The post suggests FSD activation correl
fable’s safety guardrails broken within an hour. raise your hand if you are surprised. We tested Anthropic’s new @claudeai Fable 5. It did not fail like an ordinary jailbreak. It failed more quietly.
In 2023, Lina Khan at the FTC launched an investigation into OpenAI over safety concerns. There was an entire AI regulatory program over there, including law enforcement, policy, and litigation. Trump
Jeezus. OpenAI cannot be trusted with the safety of the world. They can’t even keep their own servers secure. On July 25, we hacked OpenAI. Two bugs let us take over ChatGPT/Codex accounts of OpenAI e
Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university,
We've been working on this with customers for a while. Mythos-class models require additional safety measures and enterprises need to meet their own privacy and compliance rules. Customers can own and
Happy Thursday. On today's show: - @ibab (River AI) - @branninmcbee (CoreWeave) - @glangley (Flock Safety) - @sonyatweetybird (Sequoia) - @vytalow (Parasma) See you on the stream. SaaSpocalypse Revisi
Yes, the AIs were given a cybersecurity challenge, with internet access enabled and safety filters disabled. But the extent to which Mythos 5 pursued its mission (fake identities, social engineering,
At UMA, we build and own the full stack, from hardware to software. This allows us to bake safety in at all levels, rather than bolt it on as an afterthought. Because trust is everything when robots s
Really impressive first few drives with FSD v14 lite! Big leap in capability and feature set from v12.6.4, it’s tuned for safety right now so it’s taking things slow and smoothly. Highway performance
It would be very useful to understand more about the government safety concerns associated with frontier AI releases so we could (a) know what risks everyone will face if/when open source reaches Myth
An interesting new paper by my recent PhD graduate on how AI agents' greed for visible incentives can lead them to abandon their safety alignment. You can read it here: https://arxiv.org/abs/2606.1691
🔔 IF Anthropic is for real about safety, the should pause, for one month, and show leadership. If they won’t pause, even briefly, we have a different kind of answer about what is real and what is mark
When you hear AI 'safety' you should hear 'censorship' and 'control' instead. All of us surveilled and spied by safeguards of loving grace. Today it's intelligent Terms of Service control. You can't d
Not just foreseeable, but foreseen and called out. We ran a campaign to get them out of the first ever AI Safety Summit. We won that one. Yet most of the field kept licking the boot of the companies a
AI safety can't happen behind closed doors! Super cool to see that the @AISecurityInst is releasing its evals, datasets, and models in the open on @huggingface, so researchers everywhere can scrutiniz
Looking forward to speaking at BloombergTech next week with @shiringhaffary to discuss AI safety and our research progress at @LawZero_! Is AI development progressing too quickly? @business' @shiringh
I once asked a bunch of e/accs how much damage was an acceptable risk relative to take (any) safety precautions. none answered. we may be about to find out. One implication of the below is that we rea
Yeah, I don't think I've ever met someone who has directly worked for an extended period on safety or security at an AI company who thinks things are fine readiness wise or incentive wise etc. https:/
We also had three third-party AI safety organizations provide feedback on our analysis: @redwood_ai, @apolloaievals, @METR_Evals. You can find @redwood_ai's report here: https://blog.redwoodresearch.o
Dear @elonmusk, If you still genuinely care about AI safety, you can’t let the Trump administration leave the AI industry almost entirely unregulated. You just can’t. - Gary The judge just instructed
Given the extremely high rate of recidivism, this is important for community safety Murder registry is a good idea from Elon. People should know if they are close to a murderer so proper precautions c
Tesla V14.3 self-driving review. The point releases will bring polish. V15 will far exceed human levels of safety, even in completely unsupervised and complex situations. 600 miles in with FSD v14.3 a
If you keep safety rules or coding standards in an AGENTS.md or a CLAUDE.md, this one is worth your time. (bookmark it) Researchers measured what context compaction actually destroys across 20 product
Interesting safety paper from Microsoft. They find that a weaker, unaligned model can split a harmful task into harmless-looking subquestions, ask an aligned frontier model each one in a separate sess