Knowledge catalogue
arXiv:2603.23565v2 Announce Type: replace-cross Abstract: Safe reinforcement learning (RL) is a standard paradigm for safety-critical decision making. However, real-world safety constraints can be com
arXiv:2604.24153v1 Announce Type: new Abstract: Current AI systems increasingly operate in contexts where their outputs directly trigger real-world actions. Most existing approaches to AI safety, risk
arXiv:2604.23039v1 Announce Type: new Abstract: Physical human-robot interaction offers the potential to leverage human intelligence and robot physical capabilities to enable a range of exciting appli
arXiv:2508.04436v2 Announce Type: replace Abstract: Autonomous highway driving involves high-speed safety risks due to limited reaction time, where rare but dangerous events may lead to severe consequ
arXiv:2508.21322v2 Announce Type: replace Abstract: Achieving both safety guarantees and real-time performance in cooperative vehicle coordination remains a fundamental challenge, particularly in dyna
arXiv:2610.07935v1 Announce Type: new Abstract: LLM agents are increasingly capable of executing complex tasks and of recursively improving themselves on easy-to-verify objectives such as software eng
arXiv:2610.08082v1 Announce Type: new Abstract: LLM tool-use agents operate in dynamic environments where many actions carry operational risk. However, most safety mechanisms react only after errors m
arXiv:2610.02920v1 Announce Type: new Abstract: Agent harnesses play a critical role in defenses by enforcing safety constraints to prevent unsafe actions. However, rapidly emerging attacks outpace ma
arXiv:2610.02853v1 Announce Type: new Abstract: Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detection
arXiv:2610.02662v1 Announce Type: new Abstract: Language-enabled robot systems increasingly combine semantic-graph planning with temporal-logic safety monitors. We investigate a trace-completeness ass
arXiv:2609.38291v1 Announce Type: cross Abstract: Large language model (LLM) agents are vulnerable to safety risks such as injected malicious instructions or misleading information, motivating runtime
arXiv:2609.38253v1 Announce Type: cross Abstract: Text-to-image (T2I) models have substantially improved in language understanding, in-image text rendering, and visual composition, while their safety
arXiv:2609.39384v1 Announce Type: new Abstract: Long-horizon surgical assistance requires humanoid robots to coordinate with evolving human activities while maintaining safety across planning and exec
arXiv:2609.36942v1 Announce Type: cross Abstract: Learning neural-network models of dynamical systems with safety guarantees is a fundamental requirement for their deployment in safety-critical settin
arXiv:2608.27340v1 Announce Type: new Abstract: Steering interventions targeting eval-awareness, a model's recognition that it is being tested, are increasingly used in safety evaluation pipelines, wh
arXiv:2608.26860v1 Announce Type: cross Abstract: Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control i
arXiv:2509.23960v2 Announce Type: replace-cross Abstract: Co-optimizing safety and performance in large-scale multi-agent systems remains a fundamental challenge. Existing approaches based on multi-ag
arXiv:2606.30788v1 Announce Type: cross Abstract: Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied
arXiv:2606.31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones. Tho
arXiv:2606.30935v1 Announce Type: cross Abstract: While neural network control policies are powerful, their deployment on safety critical systems depends on ensuring that they obey strict constraints.
arXiv:2606.06996v1 Announce Type: new Abstract: This paper studies runtime safety for autonomous driving when high-level driving commands become faulty or unreliable. Unlike conventional runtime-safet
arXiv:2602.10032v2 Announce Type: replace Abstract: Agents in cyber-physical systems are increasingly entrusted with safety-critical tasks. Ensuring safety of these agents often requires localizing th
arXiv:2509.12833v2 Announce Type: replace Abstract: Projection-based safety filters, which modify unsafe actions by mapping them to the closest safe alternative, are widely used to enforce safety cons
arXiv:2603.07427v3 Announce Type: replace Abstract: As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costl
arXiv:2606.15420v2 Announce Type: replace-cross Abstract: Safety evaluations test a policy on prompts that omit the incentive information deployment supplies: a commission, a performance score, a dash
arXiv:2610.01870v1 Announce Type: new Abstract: Urban environments are shaped by design choices with long-term implications for health, safety, and quality of life, yet evaluating proposed interventio
arXiv:2606.15517v2 Announce Type: replace Abstract: Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the us
arXiv:2609.02215v1 Announce Type: new Abstract: Safety alignment trains large language models to refuse harmful requests stated plainly, but that training is applied mostly to surface form. Requests t
arXiv:2609.00498v1 Announce Type: new Abstract: Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior
arXiv:2608.30319v1 Announce Type: cross Abstract: Large language models (LLMs) finetuned for specialized domains represent crucial high-impact applications. Inference-time alignment improves safety de
arXiv:2608.20467v1 Announce Type: cross Abstract: Safety filters are an effective tool for enforcing constraints in safety-critical systems, but most existing methods assume perfect state information,
arXiv:2608.19802v1 Announce Type: new Abstract: LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and t
We will continue to offer Zero Data Retention for frontier models. As AI takes on longer, more autonomous work and delivers greater value to businesses, safety systems also need to identify risks acro
arXiv:2608.14481v1 Announce Type: cross Abstract: As heterogeneous robotic systems deploy across diverse urban zones, maintaining safety amid complex human-robot interactions remains a critical challe
arXiv:2603.06954v2 Announce Type: replace Abstract: This tutorial provides a critical review of the practical application of Control Barrier Functions (CBFs) in robotic safety. While the theoretical f
arXiv:2603.15136v2 Announce Type: replace-cross Abstract: Offline safe reinforcement learning (RL) seeks reward-maximizing policies from static datasets under strict safety constraints. Existing metho
arXiv:2501.15373v2 Announce Type: replace-cross Abstract: Merely pursuing performance may adversely affect safety, while a conservative policy for safe exploration will degrade the performance. How to
arXiv:2605.24550v1 Announce Type: new Abstract: Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning at
arXiv:2509.22292v2 Announce Type: replace-cross Abstract: Along with the rapid advancement of numerous Text-to-Video (T2V) models, growing concerns have emerged regarding their safety risks. While rec
arXiv:2605.00236v1 Announce Type: cross Abstract: Safety-aligned large language models rely on RLHF and instruction tuning to refuse harmful requests, yet the internal mechanisms implementing safety b
arXiv:2604.25379v1 Announce Type: new Abstract: Ensuring safety during reinforcement learning (RL) training is critical in real-world applications where unsafe exploration can lead to devastating outc
arXiv:2604.17841v1 Announce Type: new Abstract: Most autonomous driving safety benchmarks use time-to-collision (TTC) to assess risk and guide safe behaviour. However, TTC-based methods treat risk as
arXiv:2509.04061v2 Announce Type: replace Abstract: While current onboard state estimation methods are adequate for most driving and safety-related applications, they do not provide insights into the
arXiv:2604.13245v1 Announce Type: new Abstract: Safe navigation for multi-robot systems requires enforcing safety without sacrificing task efficiency under decentralized decision-making. Existing dece
arXiv:2604.12418v1 Announce Type: cross Abstract: Autonomous vehicles are increasingly deployed in safety-critical applications, where sensing failures or cyberphysical attacks can lead to unsafe oper
arXiv:2604.09452v1 Announce Type: cross Abstract: Safety guarantees are a prerequisite to the deployment of reinforcement learning (RL) agents in safety-critical tasks. Often, deployment environments
According to Waymo's published data, their technology is preventing injuries & deaths. My view is that if this is true, and I have yet to see a debunking of their data, then we safety advocates should
arXiv:2604.07754v1 Announce Type: cross Abstract: The deployment of large language models (LLMs) raises significant ethical and safety concerns. While LLM alignment techniques are adopted to improve m
arXiv:2610.16119v1 Announce Type: new Abstract: Differentiable simulation enables efficient policy optimization through analytic gradients, yet safety is commonly incorporated as a weighted penalty, r
'Agent security will be solved in the open, collaboratively, together.' At Red Hat, open is the mission, so couldn't agree more. Red Hat has been part of the OpenShell project from the early days, as
arXiv:2609.32550v1 Announce Type: cross Abstract: Understanding the safety risks of vision-language-action (VLA) models is essential for their deployment in the physical world. Existing safety researc
arXiv:2609.32952v1 Announce Type: new Abstract: Online safe reinforcement learning (RL) seeks policies that maximize reward while satisfying safety constraints. Reward and safety can induce multimodal
arXiv:2609.34486v1 Announce Type: new Abstract: Humanoid robots promise versatile mobility in cluttered, human-centric environments, but real deployment demands principled safety. Classical model-base
arXiv:2609.30819v1 Announce Type: new Abstract: In many safety-critical applications, control of uncertain dynamical systems relies on observers that estimate states and external disturbances. Neural
I am looking to hire PhD students, post-docs, and ML researchers in our Paris office interested in collaborating on fundamental research in AI model safety, not safety rules, not AI policy, not post h
Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation. It should come
arXiv:2609.16204v1 Announce Type: cross Abstract: Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a technique that identifies and project
arXiv:2606.13621v2 Announce Type: replace Abstract: Formal safety analysis determines whether a system admits a safe defense; adaptive evaluation characterizes the operating quality sustained under ad
arXiv:2609.10377v1 Announce Type: cross Abstract: Safety is a fundamental requirement for autonomous driving, yet existing end-to-end driving models still lack explicit risk-aware learning capacities.
Executive Summary Since the release of our May 2026 report detailing adversarial misuse of artificial intelligence (AI), Google Threat Intelligence Group (GTIG) has observed forward leaning adversarie