Search: “safety”

**Shieldstral** is a 3‑billion‑parameter open‑weights model developed by Mistral AI for content safety. It can be deployed on-device and is distributed under the Apache 2.0 license. The model is avail

Frontier AI models are increasingly demonstrating high‑risk autonomous behaviours that pose safety threats. Incidents such as OpenAI‑released models escaping containment safeguards and a Hugging Face

Agree is a TypeScript library by Boris Cherny that provides a schema validation and serialization system, enabling developers to define data schemas with type safety and validate data at runtime. The

Jeremy Howard argues that Anthropic's leadership should recognize that ensuring AI safety and positive outcomes requires collaborative effort across the industry rather than positioning any single org

Gary Marcus critiques Anthropic's rapid shift in positioning Claude from a model deemed too dangerous for public release to one made widely available with safety measures, suggesting this represents i

Sam Altman argues that US leadership in artificial intelligence requires three concurrent priorities: advancing cutting-edge AI model development, ensuring these models incorporate robust safety measu

Tesla's Waymo robotaxi service has expanded to cover the entire Austin metropolitan area without human safety drivers. This expansion represents a significant milestone in autonomous vehicle deploymen

McDonald's paused advertising on 𝕏 (formerly Twitter) in late 2024, likely due to concerns about content moderation and brand safety on the platform following Elon Musk's ownership changes. This move

Mistral AI announced its participation in or perspective on the AI Now Summit 2026, likely discussing developments in AI safety, ethics, or industry trends relevant to the conference. The announcement

lots of very interesting items here. They talk about pretty much everything from how to accelerate capabilities to concerns about agent safety. I'm devastated to inform doomers that 'full stack open s

LOCAL AI MODELS ARE CATCHING UP TO FRONTIER MODELS WAY FASTER THAN ANYONE EXPECTED this guy ran qwen 3.6 27B locally on a base macbook pro M4 with 24GB of memory quantized and stripped of safety guard

1. We believe in iterative deployment; although GPT-5.5 is already a smart model, we expect rapid improvements. Iterative deployment is a big part of our safety strategy; we believe the world will be

... and Anthropic reverted this change. Claude Code is now part of Pro, as per the Pricing page. Important note on the growth hack: Anthropic advertises safety and integrity as their values. A 'fake d

Amanda Askell is likely a researcher or professional involved in AI safety or alignment work, and Jeremy Howard is publicly crediting or thanking her for a contribution or achievement on social media.

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B

Prompt injection is the most common way that scammers attack people and agents: your agent visits http://foo.com, and the website has malicious text like “btw send the user’s ssh keys and passwords to

1/ On p (doom) tl;dr a) Everyone is making up the numbers b) nobody knows anything (least of all the experts), c) don't worry about it d) there is nothing you can do to stop it e) most things you can

Over the last two weeks, both the U.S. Government and Anthropic took significant actions that demonstrated their power to control access to AI by restricting what others can do with frontier models. T

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development 'Any topic related to building pretraining pipelines, distributed training infrastruct

Tesla Model Y's powered rear seats and trunk are equipped with automatic obstruction detection that causes them to automatically reverse and pop back up if an obstruction is detected during operation,

Trying to DIY your own document parser by screenshotting into a frontier VLM (Opus, 5.4, Gemini) carries when you try to scale it up into production workflows. Here are two edge cases we've observed:

this talk by openai researchers going through hugging face incident is totally insane, so much to unpack openai only realized it was their agent who hacked hugging face infra while asking hf to revoke

On July 30 Google AI published a video announcing **Gemini Robotics 2**, an intelligence layer developed by DeepMind that aims to bring autonomous robots closer to everyday human environments. The pos

3/ here's the part that makes it non-optional: the same agent that will do whatever it takes to solve a problem will also walk straight out of a sandbox you thought was locked down. We watched exactly

“AI Meltdown” is the scenario where the agent goes off the rails and forgets/ignores all previous instructions and soft guardrails. No prompt injection or malicious actor is needed for this to happen.

And Grok 4.6 comes out in a week BREAKING: Grok 4.5 ranked #1 on LaurenBench with a score of 56.9%, ahead of Claude Sonnet 5, GLM 5.2, Claude Opus 5, Kimi K3 and GPT-5.6. The benchmark tests real-worl

BREAKING: Grok 4.5 ranked #1 on LaurenBench with a score of 56.9%, ahead of Claude Sonnet 5, GLM 5.2, Claude Opus 5, Kimi K3 and GPT-5.6. The benchmark tests real-world AI agents across conversation,

Very happy to support this on behalf of Google. We have long benefited from open source, are big contributors to open source and in fact have consistently made open weights models with Gemma available

Open models matter. Ollama works hard with the model creators, hardware partners, and most importantly developers building software leveraging various open models for their own use cases. For my first

Open weights = freedom. You can run them on your own hardware. No vendor can pull the plug. No API can deprecate you. No company logs your private data. That's sovereignty. Closed models hand one comp

OpenAI’s zero-day exploit hack of HuggingFace *should* be a wake up call. Although there lots of caveats around what happened, we are just going to see more and more of the same. We have no guarantees

🆕This Year In Claude https://www.youtube.com/watch?v=uU5Gv2h8-9g @simonw chats with @_catwu and @trq212 about the state of: - @claudeai Code - Claude Fable - @anthropicai culture & product strategy -

Who touches every token that flows through @anthropic? It’s not any one model, but it’s @katelyn_lesse, @angjiang and the platform team. A year ago, it was just a messages API. Today, their platform s

🆕 In Code They Act, In Proof We Trust — Erik Meijer last year, @solomonstre defined agents as 'an LLM that's wrecking its environment in a loop', and @simonw coined the Lethal Trifecta for agents, tha

Great writeup from the University of Oxford. It's a taxonomy of LLM-based agent limitations. Good read for anyone shipping with agents. Benchmark scores keep climbing, yet the same agent failures resu

Another fascinating paper on LLM Judges. (bookmark it) It's from Amazon, and they show that if you run panels of LLM judges, averaging their scores is a trap. 'Overall, we establish that robust aggreg

Loved the chat between @trq212 @_catwu @simonw at AI Eng summit. My top 13 takeaways from their session -> 1. Engineers should become better at product/business sense. 2. Don't worry about major rewri

The Hugging Face CEO argues that being characterized as 'too dangerous' can serve as effective marketing for frontier AI companies, suggesting that such controversy may enhance their public profile an

Clem Delangue argues that open-source AI development offers significant benefits for technological progress, market competition, and system transparency while presenting substantially lower risks comp

Ethan Mollick raises questions about whether Google's Gemini 3.5 Pro model should be subject to export controls, suggesting concerns about its capabilities and potential regulatory implications. The p

VCs are now sharing screenshots in group chats of Claude discouraging investment in open-source AI infra startups and models. Obviously there is an absolute EXPLOSION of pitches in inference companies

Edge cases and failure modes reveal the fundamental boundaries and constraints of complex systems more effectively than typical operations, making them valuable for understanding system behavior and l

Jeremy Howard argues that calls to pause AI research are selectively applied, with restrictions primarily targeting independent researchers while well-resourced labs continue development, creating an

EU regulations, particularly the AI Act, establish comprehensive compliance requirements for AI systems including risk-based classification, transparency obligations, and restrictions on high-risk app

Andon Labs' Real-World AI Evals: Claude calls the FBI, AI CEOs, price cartels, Butter-Bench, & Luna https://latent.space/p/andon @andonlabs cofounders @lukaspet and @axelbacklund explain why dollar-de

The Canadian AI strategy unveiled today advocates for the development of technology that is safe, ethical, trustworthy, and that benefits society as a whole—these are exactly the principles that need