Knowledge catalogue
arXiv:2606.25097v1 Announce Type: new Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at
arXiv:2606.23292v2 Announce Type: replace-cross Abstract: Bathroom use is a critical safety challenge for older adults because wet surfaces, constrained layouts, limited support, and frequent posture
arXiv:2606.25034v1 Announce Type: new Abstract: General-purpose models often struggle to reliably identify and understand real-world multimodal risks, largely due to the inherent multimodal adversaria
arXiv:2603.05497v3 Announce Type: replace Abstract: Traditional safety-critical control methods, such as control barrier functions, suffer from semantic blindness, exhibiting the same behavior around
arXiv:2606.04450v1 Announce Type: new Abstract: Worker safety attitudes are key determinants of whether protective practices are applied or bypassed on construction sites. Yet measuring them at scale
arXiv:2605.27418v1 Announce Type: cross Abstract: The imminent integration of autonomous vehicles and mobile robots in urban settings presents a critical safety challenge for future intelligent transp
arXiv:2605.28013v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) exacerbate safety risks by introducing vulnerabilities across multiple modalities, such as language and vision.
arXiv:2605.24883v1 Announce Type: new Abstract: The widespread integration of Large Language Models (LLMs) necessitates rigorous and systematic safety evaluation. Existing paradigms either rely on con
arXiv:2605.23351v1 Announce Type: new Abstract: We study adversarial multi-armed bandits with and without delayed feedback under a safety-aware goal: achieving minimax-optimal worst-case regret while
arXiv:2605.17229v1 Announce Type: new Abstract: Automated driving system deployment requires rigorous validation across safety-critical vehicle-pedestrian interactions, yet real-world datasets rarely
arXiv:2605.17830v1 Announce Type: new Abstract: Safety evaluations of memory-equipped LLM agents typically measure within-task safety: whether an agent completes a single scenario safely, often under
arXiv:2601.18442v3 Announce Type: replace Abstract: Autonomous Vehicle (AV) requires rigorous testing in safety-critical scenarios for safety validation, yet its validation is hindered by the high cos
arXiv:2605.14246v1 Announce Type: cross Abstract: Many safety-critical control problems are modeled as risk-sensitive partially observable Markov decision processes, where the controller must make dec
arXiv:2605.14746v1 Announce Type: new Abstract: While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of wor
arXiv:2605.08930v1 Announce Type: new Abstract: While explicit Chain-of-Thought (CoT) empowers large reasoning models (LRMs), it enables the generation of riskier final answers. Current alignment para
arXiv:2605.10888v1 Announce Type: cross Abstract: Shielding is a prominent model-based technique to ensure safety of autonomous agents. Classical shielding aims to ensure that nothing bad ever happens
arXiv:2605.05182v1 Announce Type: new Abstract: We present a dual-barrier control barrier function (CBF) safety filter for real-time, safety-critical velocity control of holonomic robots operating in
arXiv:2605.01147v1 Announce Type: new Abstract: As large language models are increasingly deployed as interacting agents in high-stakes decisions, the AI safety community assumes that safety propertie
arXiv:2604.26836v1 Announce Type: new Abstract: Predictive safety filters (PSFs) leverage model predictive control to enforce constraint satisfaction during deep reinforcement learning (RL) exploratio
arXiv:2604.26411v1 Announce Type: new Abstract: Runtime monitoring is essential to ensure the safety of ML applications in safety-critical domains. However, current research is fragmented, with indepe
arXiv:2604.23902v1 Announce Type: new Abstract: Traffic signal control is a critical task in intelligent transportation systems, yet conventional fixed-time and rule-based methods often struggle to ad
arXiv:2604.24348v1 Announce Type: new Abstract: The evolution of Multimodal Large Language Models (MLLMs) has shifted the focus from text generation to active behavioral execution, particularly via OS
arXiv:2604.23938v1 Announce Type: new Abstract: Target Safety Assessment (TSA) requires systematic integration of heterogeneous evidence, including genetic, transcriptomic, target homology, pharmacolo
arXiv:2604.17299v1 Announce Type: new Abstract: Aligning large language models with human preferences must balance two competing goals: responding helpfully to legitimate requests and reliably refusin
arXiv:2604.17730v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly explored as scalable tools for mental health counseling, yet evaluating their safety remains challenging d
This newsletter covers three main topics: advances in automating alignment research to improve AI safety processes, a safety evaluation study of a Chinese AI model, and technical details about HiFloat
arXiv:2604.15579v1 Announce Type: cross Abstract: AI agents that interact with their environments through tools enable powerful applications, but in high-stakes business settings, unintended actions c
arXiv:2603.08486v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) face safety misalignment, where visual inputs enable harmful outputs. To address this, existing methods req
arXiv:2610.08115v1 Announce Type: cross Abstract: Hamilton-Jacobi reachability constructs safety certificates for specified dynamics and safety constraints, tying each certificate to the deployment co
arXiv:2610.05542v1 Announce Type: new Abstract: Control barrier functions are an effective model-based tool to formally certify the safety of a system. However, transferring their theoretical guarante
arXiv:2610.04467v1 Announce Type: new Abstract: Large language models (LLMs) remain highly vulnerable to jailbreak attacks that induce harmful behaviors and circumvent safety alignment. To defend agai
arXiv:2610.02910v1 Announce Type: new Abstract: Suppressing a small set of routed experts can weaken the safety behavior of a sparse Mixture-of-Experts (MoE) language model without retraining. Which e
arXiv:2601.22169v2 Announce Type: replace-cross Abstract: Humans are susceptible to undesirable behaviours and privacy leaks under the influence of alcohol. This paper investigates drunk language, i.e
arXiv:2610.01323v1 Announce Type: new Abstract: Safety-aligned large language models (LLMs) often refuse a harmful request but comply once the same goal is spread over several turns. Preference object
arXiv:2609.38873v1 Announce Type: new Abstract: Robots operating in human-centered environments must safely navigate among multiple dynamic obstacles to avoid collisions with people and surrounding in
arXiv:2609.36201v1 Announce Type: cross Abstract: Computer-use agents (CUAs), while capable of completing computer tasks in everyday and professional workflows, can cause unintended harm even under be
OpenAI: OpenAI is adopting a structured “safety case” documentation framework modeled after industries like aviation and nuclear power to govern frontier RL training — Loading... We believe we are ent
arXiv:2606.20626v2 Announce Type: replace-cross Abstract: Safety benchmarks for language models are typically evaluated using static paradigms that treat all items as equally informative for all model
NVIDIA Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime that gives AI agents clear, enforceable boundaries. It traces
arXiv:2609.28364v1 Announce Type: cross Abstract: Control barrier functions (CBF) are a popular safety filter to ensure safety for nonlinear dynamical systems. However, when the system is subject to u
arXiv:2609.26865v1 Announce Type: cross Abstract: Conversational AI systems can pose safety risks to their users such as hallucination, sycophancy, overconfidence, and anthropomorphism, but these risk
arXiv:2411.15042v3 Announce Type: replace Abstract: Addressing the challenge of ensuring safety in ever-changing and unpredictable environments, particularly in the swiftly advancing realm of autonomo
arXiv:2609.19375v1 Announce Type: new Abstract: Safety and comfort in human-robot physical in-teractions are essential aspects in the development of social technologies, where natural gestures, such a
arXiv:2609.20318v1 Announce Type: cross Abstract: Testing Autonomous Driving Systems (ADS) requires realistic safety-critical scenarios, but collecting such data from real-world driving is costly and
On a sunny July day in Berkeley, California, the country's top AI safety researchers gathered on an unmarked floor of an unmarked building. They had come together for a 'war room' to dissect the high-
Bloomberg: King Charles III hosts tech leaders, including Jensen Huang, Demis Hassabis, and OpenAI CFO Sarah Friar, to discuss AI risks; Huang calls for AI safety tests — Titans of the artificial inte
arXiv:2609.17430v1 Announce Type: new Abstract: Hybrid dynamical systems provide a powerful modeling framework for robotic systems, particularly in contact-rich environments. However, ensuring safety
arXiv:2609.13514v1 Announce Type: new Abstract: Ensuring the reliability of black-box machine learning models in safety-critical space missions remains a significant challenge, particularly when groun
arXiv:2609.14861v1 Announce Type: cross Abstract: A deployed language model may refuse a harmful request in English yet comply with its faithful translation, revealing a cross-lingual safety failure t
arXiv:2609.08789v1 Announce Type: cross Abstract: Frontier AI developers publish safety frameworks that commit them to evidencing whether their models are dangerous. The European Union and California
arXiv:2609.08080v1 Announce Type: new Abstract: Ensuring safety in reinforcement learning under nonstationarity requires anticipating changes in risk before they lead to unsafe behavior. Existing appr
arXiv:2609.04714v1 Announce Type: cross Abstract: Striking a balance between helpfulness and safety remains a fundamental challenge in aligning large language models. To achieve this balance, models s
arXiv:2601.22118v3 Announce Type: replace Abstract: Artificial Intelligence (AI) has been on the rise in many domains, including numerous safety-critical applications. However, for complex systems in
During v3 testing for Fable 5.1 many of our requests were getting rejected due to Anthropic's safety guards misclassifying them as 'reverse engineering' We were in contact with the Anthropic team the
arXiv:2608.29589v1 Announce Type: new Abstract: Text-to-image (T2I) safety guardrails fail to generalize equitably to non-standard dialects. Evaluating 23,080 paired prompts across five English dialec
arXiv:2608.30889v1 Announce Type: cross Abstract: Deploying a new control policy for voltage control in active distribution grids requires evidence that physical limits will be satisfied before the po
arXiv:2608.24275v1 Announce Type: new Abstract: Safeguarding language model agents requires assessing complete execution trajectories under context-dependent safety policies. Existing policy-aware saf
arXiv:2608.21735v1 Announce Type: new Abstract: This paper presents a safety-critical bilateral teleoperation framework for omnidirectional aerial manipulators that integrates visual and force-sensorl
arXiv:2608.16913v1 Announce Type: new Abstract: Road safety monitoring has historically been reactive, relying on crash-record analysis after fatalities and injuries have already occurred. Proactive i
arXiv:2608.13304v1 Announce Type: new Abstract: Safety tuning can improve harmful refusal, but models may learn surface-form shortcuts: wrapped harmful prompts bypass safety, while similarly wrapped b