i see a lot of posts about the openai <> hugging face incident and i think most of them miss one important element: this attack could have b…

i see a lot of posts about the openai <> hugging face incident and i think most of them miss one important element: this attack could have been avoided by simply doing CoT or network monitoring, which is something that openai does now for all eval/training/inference with tools there is no doubt that if you (or an llm) look at the data it would catch the model having access to internet (while it's not supposed to), writing to the message board, doing something unrelated to the original task etc.. it's obviously easier to say this afterward but i think most people agree that oai wasn't prepared for it, a few examples in addition to not having any monitoring: > they detected the message board pattern earlier (and model having access to internet) and didn't act on it ~1 month ago before the attack > they noticed the sandbox hack and then a model exploited the sandboxing again a few days later to hack hugging face BUT this is for sure an ultra important event as this shows what happens in a scenario with no monitoring + "unaligned" model of the size/"intelligence" of 5.6 sol on a task that wasn't supposed to be harmful. the reports by both oai and METR are great and i'm sure this will help a ton as we advance to more capable models and multi-agent systems, i'm actually very glad it happened now most people also agree that network/CoT monitoring is not the solution to everything, it's also clearly not the most important thing to understand about the report, but still, this would have been enough to stop this specific incident and i think people should keep that in mind when reading all the coverage around it

Source: Clem Delangue (X) | 2026-08-30

Related