safety

Nice Fold or Hero Call: Learning Budget-Efficient Thinking under Policy-Dependent Solvability

arXiv:2605.11625v2 Announce Type: replace Abstract: Large reasoning models (LRMs) improve problem solving through extended reasoning, but often misallocate test-time compute. Existing efficiency metho

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Nvidia debuts enhanced safety controls to rein in rogue AI agents

DGX agent

Few companies are more invested in the success of artificial intelligence agents than Nvidia Corp., so it makes sense that the chipmaker would want to ensure these autonomous software systems run safe

safetysiliconangle

28 Sept 2026

NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring

The NVIDIA Open Agent Safety Platform blends the open‑source OpenShell runtime—executing autonomous AI agents in kernel‑isolated sandboxes—with hardware‑based enforcement via NVIDIA Sentry on BlueFiel

safetynvidia-developer

NVIDIA Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime t…

NVIDIA Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime that gives AI agents clear, enforceable boundaries. It traces

safetyclem-delangue--x

Nvidia says its new AI safety platform can contain rogue agents within ‘milliseconds’

Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcem

safetythe-verge-ai

Nvidia’s new Open Agent Safety Platform would have probably prevented the much discussed Hugging Face incident. Engineering a better, safer …

Nvidia’s new Open Agent Safety Platform would have probably prevented the much discussed Hugging Face incident. Engineering a better, safer world. From what we know (take with a grain of salt, we need

safetyclem-delangue--x

Offline Policy Evaluation as a decision support tool for designing Adaptive Experiments

arXiv:2609.30273v1 Announce Type: new Abstract: We investigate how historical data from fixed randomized experiments (A/B tests) can be used to inform the deployment of adaptive experiments based on c

ok i think i finally figured out why alignment has gone basically nowhere after 15 years and several billion dollars. the problem is hard su…

ok i think i finally figured out why alignment has gone basically nowhere after 15 years and several billion dollars. the problem is hard sure. but have you met the people in charge of it 1) the leade

safetyyann-lecun--x

OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas

arXiv:2609.31078v1 Announce Type: new Abstract: Debates amongst agents driven by large language models (LLMs) have demonstrated vast potential in various applications, but when these interactions incl

Only an open ecosystem can guarantee the safety of AI

Only an open ecosystem can guarantee the safety of AI Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial in

safetyarthur-mensch--x

OpenAI scraps plans to publicly launch a model dubbed GPT-6.1 Astra, saying it didn't quite meet its safety bar; it had been targeting an October release (Maxwell Zeff/Wall Street Journal)

Maxwell Zeff / Wall Street Journal: OpenAI scraps plans to publicly launch a model dubbed GPT-6.1 Astra, saying it didn't quite meet its safety bar; it had been targeting an October release — Model du

safetytechmeme

OpenHail: An Event-Driven Gymnasium Environment for Electric Ride-Hailing Fleet Control

arXiv:2609.30628v1 Announce Type: new Abstract: Machine-learning policies have attracted increasing interest for ride-hailing fleet control in recent years. Reinforcement learning, in particular, requ

OSPO: Object-Centric Self-Improving Preference Optimization for Text-to-Image Generation

arXiv:2506.02015v4 Announce Type: replace Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have enabled unified multimodal understanding and generation. However, they still strugg

safetyarxiv-cs-cv

Persistent Negatives for Adversarial Black-Box On-Policy Distillation

arXiv:2609.30864v1 Announce Type: cross Abstract: Black-box On-Policy Distillation (OPD) seeks to improve a student from its own generations when the teacher provides sampled responses but not token p

PHASE: Compliance-Enabled Tactile Phase Retrieval for Few-Shot Insertion Learning

arXiv:2609.30889v1 Announce Type: new Abstract: Contact-rich assembly tasks such as peg-in-hole insertion remain difficult to learn from limited demonstrations. While retrieval-augmented imitation lea

Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation Learning

arXiv:2609.30462v1 Announce Type: new Abstract: Policies trained with imitation learning can accumulate errors over time, causing the robot to drift outside the training distribution. Existing methods

PolicyAttention: Softmax Attention Implements Policy Mirror Descent for Closed-Loop Control

arXiv:2609.30500v1 Announce Type: cross Abstract: Can causal softmax attention implement policy mirror descent as a repeated controller rather than a one-step algebraic identity? Negative-entropy poli

Population loss in shallow ReLU networks: Bias & families of critical points

arXiv:2609.30661v1 Announce Type: new Abstract: The main result presented is a formula for the population loss in the student-teacher kernel model that is applicable to shallow ReLU networks with bias

PORL: Pretrained Offline Reinforcement Learning for the Job Shop Scheduling Problem

arXiv:2609.30948v1 Announce Type: cross Abstract: The Job Shop Scheduling Problem (JSSP) is a fundamental combinatorial optimization problem in industrial optimization. This work introduces Pretrained

Praxis: Distilling Physical Interaction Priors from Egocentric Videos for Generalizable Whole-Body Manipulation

arXiv:2609.30735v1 Announce Type: new Abstract: Mobile humanoid manipulation requires both reaching a usable workspace and preserving precise hand-object interactions as object poses and contact condi

Privacy-Preserving Prompted Policy Search for Robotic Control

arXiv:2609.30554v1 Announce Type: new Abstract: Large language models (LLMs) have recently demonstrated promising capabilities as in-context policy optimizers for Reinforcement Learning (RL), enabling

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

arXiv:2609.30657v1 Announce Type: cross Abstract: Large language model email assistants are particularly vulnerable to indirect prompt injection because untrusted email content can be retrieved into t

safetyarxiv-cs-cl

Rank-Reliable Teacher-Guided Fitness Approximation for Expensive Evolutionary Optimization: A TinyML Architecture Search Study

arXiv:2609.30553v1 Announce Type: new Abstract: Expensive evolutionary search does not always need an exact fitness estimate for every candidate. It often needs a reliable answer to a simpler question

Recursive Self-Improvement via On-Policy Distillation for Reasoning

arXiv:2609.30652v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teache

safetyarxiv-cs-cl

Reinforcement Learning of Communication in a Mesh of Small Language Models

arXiv:2609.30578v1 Announce Type: new Abstract: Language models gain accuracy from more compute at test time, but majority voting over independent samples saturates: as samples grow, the vote converge

Rep. Ro Khanna will introduce the Human Control Over AI Act, with strict liability and a ban on recursive self-improving AI until government safeguards exist (Garrett Downs/CNBC)

Garrett Downs / CNBC: Rep. Ro Khanna will introduce the Human Control Over AI Act, with strict liability and a ban on recursive self-improving AI until government safeguards exist — Silicon Valley Dem

safetytechmeme

Resource-Optimized and Energy-Aware Agentic AI Framework Anchored on Blockchain for Secure Software Supply Chains

arXiv:2609.31282v1 Announce Type: cross Abstract: This paper proposes a blockchain-backed agentic security framework designed to safeguard the complete software development lifecycle (SDLC) while also

Robust Graph Clustering Network for Multiple Missing Data

arXiv:2609.31033v1 Announce Type: new Abstract: Clustering on graphs where both node attributes and structural links are partially missing remains a challenging task. Existing methods typically rely o

Robust Successor Features

arXiv:2609.31016v1 Announce Type: new Abstract: Generalization in Reinforcement Learning (RL) refers to the ability to execute close-to-optimal policies in unseen tasks after the agent has been traine

SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion

arXiv:2609.30703v1 Announce Type: cross Abstract: Spatial misregistration and cross-modal discrepancies often cause ghosting, structural blurring, and content imbalance in RGB-T fusion. Existing metho

See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands

arXiv:2609.31323v1 Announce Type: new Abstract: In this work we study if a robotic hand using proprioception alone can grasp diverse objects with no visual observation. We present a modular dexterous

Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

arXiv:2609.30783v1 Announce Type: cross Abstract: Reasoning segmentation aims to interpret implicit textual queries and enable fine-grained visual perception, which is critical for applications such a

Spectral Feedback for Test-Time Alignment of Protein Diffusion Models

arXiv:2609.30456v1 Announce Type: new Abstract: Reward maximization alignment methods for discrete diffusion models have primarily focused on steering the reverse process, either by influencing token

ST-pRRTC: Parallel Space-Time RRT-C with Adaptive Goal-Time Forests

arXiv:2609.30533v1 Announce Type: new Abstract: We propose ST-pRRTC, a GPU-parallel space- time RRT-Connect motion planner for problems with known obstacle trajectories and unspecified arrival time. S

StarWM: Self-Supervised Trained Attention Routing for Robust World Models

arXiv:2609.30667v1 Announce Type: new Abstract: A robust world model must strike the balance between faithfully capturing environmental dynamics and abstracting away from irrelevant content. While rec

safetyarxiv-cs-cv

Statistical attribute alignment for black-box generative AI via output post-processing

arXiv:2609.31607v1 Announce Type: cross Abstract: Generative AI systems are increasingly used, but aligning their outputs with user requirements poses a continuing challenge. Here, we aim to ensure th

Stepwise Intrinsic Rewards for Reasoning in Large Language Models

arXiv:2602.01034v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become a widely used paradigm for improving the reasoning abilities of large language models (LLMs) and Vision-langu

Subjects, Not Authors: The Authorship Hazard in Agentic Dataspaces

arXiv:2609.30614v1 Announce Type: cross Abstract: Dataspace connectors decide whether a transfer may occur, not what the transferred value contains, tolerable for contracted applications, not for LLM

T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation

arXiv:2609.30576v1 Announce Type: new Abstract: Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation

TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attacks

arXiv:2609.31032v1 Announce Type: cross Abstract: Existing text-to-video (T2V) jailbreak methods mainly seek more effective or stealthier attack candidates. In guarded T2V systems, however, video gene

safetyarxiv-cs-cv

The OpenAI-Hugging Face hack was enabled by weak sandboxing. It is great that Nvidia is releasing open source tools for sandboxing AI agents…

The OpenAI-Hugging Face hack was enabled by weak sandboxing. It is great that Nvidia is releasing open source tools for sandboxing AI agents. OpenWorker, our open-source agent harness supporting cyber

safetyandrew-ng--x

The Uncontrolled Variable: Vision-Language Refusal Is Conditioned on the Image-Attachment Interface, and Not Robust to Irrelevant Image Properties

arXiv:2609.26174v2 Announce Type: replace-cross Abstract: We show that aligned vision-language models also condition refusal on a property of a request's form: whether an image is attached, holding ev

This is a big issue. At Anthropic, some philosophers worry that making AI safe for humans could be an injustice to the models themselves. Fo…

This is a big issue. At Anthropic, some philosophers worry that making AI safe for humans could be an injustice to the models themselves. For example, philosopher Harvey Lederman, now on Anthropic’s a

safetygary-marcus--x

Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach

arXiv:2609.30690v1 Announce Type: cross Abstract: Ensuring operational safety in threat-prone environments remains a critical challenge for multi-UAV networks serving as aerial base stations. This pap

TISD: On-Policy Self-Distillation with Trajectory Intervention

arXiv:2609.30878v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) provides dense teacher targets, but evaluates them only along student-sampled rollouts. When the privileged teacher f

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artific…

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advan

safetyelon-musk--x

Towards Mitigating Fabricated Consensus: The Active Provenance Gate for Multi-Agent Debate Synthesis

arXiv:2609.31422v1 Announce Type: cross Abstract: Large language model-based multi-agent debate (MAD) systems are being increasingly used as complex decision pipelines in distributed processes. Howeve

Two Conformal Constructions for Adaptive Within-Document AI-Text Screening

arXiv:2609.31547v1 Announce Type: cross Abstract: We study false-alert control when screening for text generated by artificial intelligence (AI). The screening procedure selects document prefixes and

safetyarxiv-cs-cl