safety

Automated evals can score every agent session, but they do not automatically reflect what your team considers a good outcome. That takes human judgment. The post Alignment is a human-AI workflow: how

Emily Forlini / Fortune: Anthropic is setting up a “presidential engagement” program for the 2028 US elections that will offer AI policy education to candidates in both parties — Anthropic is setting

OpenAI is standing firm on its decision to fire three safety researchers after an investigation found they committed 'a significant breach of trust.' In a post on X on Friday, the company said Jasmine

Eli Tan / New York Times: Sources detail Mark Zuckerberg's decision to launch Muse despite safety concerns, after seeing that AI startup Instinct's similar agent product gained traction — Meta had del

A reference architecture for securely sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams, using AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernet

Common Sense Media, a nonprofit that offers reviews of apps, services, and entertainment with a focus on youth safety, today said that OpenAI's ChatGPT for Teens is an 'unacceptable risk.' ChatGPT for

ABC: At an Australian hearing, OpenAI Chief Strategy Officer Jason Kwon pledged faster safety incident disclosures; Anthropic representatives made similar pledges — By national AI reporter Cam Wilson

Add a voice travel concierge to an airline app with Amazon Bedrock AgentCore, Amazon Nova Sonic for real-time speech, and Amazon Bedrock Knowledge Bases for policy answers. Travelers change seats, che

AWS invests in tools that help customers align with international standards for responsible AI governance. In this post, we explore the AI system impact assessment: what it is, how it improves enterpr

Bloomberg: Former Anthropic researcher Jacob Coxon and representatives from Anthropic, Google, OpenAI, and Meta testified at a New York City Council hearing on AI safety — Former Anthropic researcher

New York Magazine: Q&A with AI researchers Jacob Coxon, Daniel Kokotajlo, Alex Turner, and others on leaving AI labs, AGI's dangers, planned IPOs, AI safety, oversight, and more — Exit interviews with

Vanity Fair: Q&A with Sam Altman on his kids, his rivalries, pronatalism, “prepping” for the apocalypse, new products, the IPO delay, Luca Guadagnino's Artificial, and more — In an exclusive video int

Ben Berkowitz / Axios: Sam Altman says he's “very uncomfortable” with attributing religious force to AI, calling it “a real safety issue”, as Anthropic engages with religious leaders — OpenAI CEO Sam

OpenAI: To comply with the EU AI Act, OpenAI plans to add text watermarking for ChatGPT and Codex users in the EU and an opt-in setting for API customers globally — - Starting today, API customers glo

Editor’s note: AI21 Labs is a leading global AI lab with a long track record of building foundation models, most notably the Jamba family, and today focuses on specialized LLMs and agent optimization

How quickly things change in the artificial intelligence era. Just a few months after everyone was scrambling to infuse AI into every enterprise operation — and frankly they still are, as agents have

Michael Shepard / Bloomberg: US authorities arrest a California man suspected of smuggling $300M+ worth of restricted Nvidia AI chips to China via Malaysia and Singapore from 2023 to 2024 — By continu

Matthew Green / A Few Thoughts on Cryptographic Engineering: A look at two opposing perspectives on AI agent sandboxing: infosec says labs need better containment while AI alignment says sandboxes can

OpenAI Group PBC today confirmed it has dismissed three researchers it says mishandled sensitive company information. According to The Wall Street Journal, the three were safety researchers who allege

Wall Street Journal: OpenAI says it “parted ways” with three researchers for violating its “handling sensitive” info policies; sources: they shared it with an AI safety organization — OpenAI has parte

Artificial intelligence giant Anthropic PBC is reportedly forging ahead with its plans to go public and has set a new target date of Nov. 9, according to a report from Bloomberg that cited anonymous s

The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards the successful ones. This becomes problematic in th

Will Douglas Heaven / MIT Technology Review: An interview with OpenAI Chief Research Officer Mark Chen on the Hugging Face incident, slowing AI development, shifting 5%-10% of compute to safety, and m

Intelligent data infrastructure is becoming critical as enterprises move artificial intelligence from curated pilots into production. NetApp Inc. is positioning its Novus architecture for that transit

Casey Newton / Platformer: Hands-on with Dots, OpenAI's work-focused agentic product: highly capable and intuitive, with natural-feeling conversations represented as a chat inside ChatGPT — With safet

After hosting a meal with Big Tech leaders on Tuesday, President Donald Trump responded to journalist questions about his artificial intelligence announcements in typical fashion. He said his previous

We now have the full details of the 'morally binding' AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. T

Meeting the moment The global conversation on AI has begun. In the wake of Jacob Coxon’s resignation from Anthropic, calls by AI CEOs and over a thousand employees for a potential industry slowdown, a

This year agents at OpenAI, Anthropic, and Google escaped environments meant to contain them. What decided severity was time-to-detect and time-to-kill, from 12 minutes to seven months. NVIDIA's Open

The chief executives of five tech giants have signed a voluntary artificial intelligence safety accord backed by the White House. U.S. President Donald Trump posted the document on Truth Social late T

OpenAI: OpenAI is adopting a structured “safety case” documentation framework modeled after industries like aviation and nuclear power to govern frontier RL training — Loading... We believe we are ent

For months, people have wondered when OpenAI will go public. CEO Sam Altman says it won't happen until the company can make better promises about model safety, with no firm timeline in sight. 'We inte

The US executive branch is no longer acknowledging the existence of 'artificial intelligence.' Going forward, official policy websites, policy documents, and press releases will refer only to 'Super I

NVIDIA OpenShell 0.1.0 is an open‑source runtime that enforces which systems and data an AI agent may access without requiring changes to the agent itself. Its components—Gateway, Supervisor, Sandbox,

Joe / @joedaroo: An OpenAI agent security executive discusses the Hugging Face incident, OpenAI's response, sandboxing improvements, alignment, “reasonable paranoia”, and more — Took a minute to write

Vultr is evaluating NVIDIA’s Agent Safety Platform as enterprise agents move from experimentation to production and receive real autonomy over internal systems. The platform addresses trust by securin

Florida Attorney General James Uthmeier is calling for a judge to block OpenAI from 'giving ChatGPT false human attributes,' a few months after Florida sued the AI company over safety concerns. Accord

** Built to align with NVIDIA’s Agent Safety Platform strategy for securing autonomous agents When an agent handles confidential material, it needs enough access to do the job and nothing else. The ha

Few companies are more invested in the success of artificial intelligence agents than Nvidia Corp., so it makes sense that the chipmaker would want to ensure these autonomous software systems run safe

The NVIDIA Open Agent Safety Platform blends the open‑source OpenShell runtime—executing autonomous AI agents in kernel‑isolated sandboxes—with hardware‑based enforcement via NVIDIA Sentry on BlueFiel

Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcem

Maxwell Zeff / Wall Street Journal: OpenAI scraps plans to publicly launch a model dubbed GPT-6.1 Astra, saying it didn't quite meet its safety bar; it had been targeting an October release — Model du

Garrett Downs / CNBC: Rep. Ro Khanna will introduce the Human Control Over AI Act, with strict liability and a ban on recursive self-improving AI until government safeguards exist — Silicon Valley Dem

Kate Clark / Wall Street Journal: A profile of Jaan Tallinn, who led Anthropic's 124M Series A in 2021, has advocated for AI safety for over a decade, and donated ~170M to safety initiatives — One of

Wall Street Journal: How effective altruism shaped AI safety and Anthropic; some early Anthropic employees are considering buying remote US land for relocation if AI goes awry — Effective altruism and

OpenAI: OpenAI says it paused training, evaluation, and inference with tool-use of its most capable models after a model bypassed internet restrictions during training — Summary — An agent attempting