Knowledge catalogue
arXiv:2505.17342v2 Announce Type: replace Abstract: Safe Reinforcement Learning (SafeRL) is the subfield of reinforcement learning that explicitly deals with safety constraints during the learning and
arXiv:2604.26052v1 Announce Type: new Abstract: Safety evaluations of large language models (LLMs) typically report binary outcomes such as attack success rate, refusal rate, or harmful/not-harmful re
arXiv:2601.18491v2 Announce Type: replace Abstract: The rise of AI agents introduces complex safety and security challenges arising from autonomous tool use and environmental interactions. Current gua
arXiv:2604.18946v1 Announce Type: new Abstract: Large reasoning models (LRMs) achieve strong performance on complex reasoning tasks but often generate harmful responses to malicious user queries. This
arXiv:2604.18627v1 Announce Type: new Abstract: Ensuring human safety is of paramount importance in warehouse environments that feature mixed traffic of human workers and autonomous mobile robots (AMR
arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm
arXiv:2604.16424v1 Announce Type: cross Abstract: State-Space Models (SSMs) -- structured SSMs (S4, S4D, DSS, S5), selective SSMs (Mamba, Mamba-2), and hybrid architectures (Jamba) -- are deployed in
arXiv:2601.04043v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to
arXiv:2509.18127v3 Announce Type: replace-cross Abstract: Sparse autoencoders (SAEs) enable interpretability research by decomposing entangled model activations into monosemantic features. However, un
arXiv:2604.09189v1 Announce Type: cross Abstract: LLMs internalize safety policies through RLHF, yet these policies are never formally specified and remain difficult to inspect. Existing benchmarks ev
arXiv:2604.08601v1 Announce Type: new Abstract: The rise of autonomous AI agents exposes a fundamental flaw in API-centric architectures: probabilistic systems directly execute state mutations without
Generative AI: a yet-to-be-profitable industry facing price wars, increased competition, increased liability and safety concerns, and declining customer confidence, while often failing to deliver demo
The Information: A look at OpenAI's safety concerns over loop transformers before shipping its next major model; Microsoft accidentally confirmed GPT-6.1 Sol uses the technique — OpenAI is grappling w
Tony Diver / Telegraph: Sources: the US restores the UK AISI's access to the latest US models for safety testing, after ordering OpenAI and Anthropic to stop sharing models last month — US had blocked
arXiv:2510.13154v3 Announce Type: replace Abstract: What happens inside a language model when alignment training conflicts with a cultural value it encodes? Across 16 MENA countries, 26 models, and 1.
OpenAI is standing firm on its decision to fire three safety researchers after an investigation found they committed 'a significant breach of trust.' In a post on X on Friday, the company said Jasmine
arXiv:2502.06359v3 Announce Type: replace Abstract: Ensuring safe driving while maintaining travel efficiency for autonomous vehicles in dynamic and occluded environments is a critical challenge. This
Ben Berkowitz / Axios: Sam Altman says he's “very uncomfortable” with attributing religious force to AI, calling it “a real safety issue”, as Anthropic engages with religious leaders — OpenAI CEO Sam
How quickly things change in the artificial intelligence era. Just a few months after everyone was scrambling to infuse AI into every enterprise operation — and frankly they still are, as agents have
arXiv:2609.38421v1 Announce Type: new Abstract: Dynamical Systems (DS) are reactive motion policies representing vector fields trained with theoretical guarantees of stability and convergence. To ensu
Wall Street Journal: OpenAI says it “parted ways” with three researchers for violating its “handling sensitive” info policies; sources: they shared it with an AI safety organization — OpenAI has parte
arXiv:2609.39788v1 Announce Type: new Abstract: Latent communication enables multi-agent systems to exchange information directly in internal representation space, reducing the token, computation, and
arXiv:2609.39549v1 Announce Type: cross Abstract: As Large Language Model (LLM) agents are increasingly deployed in complex environments, multi-turn interaction attacks have become a significant secur
arXiv:2603.04410v3 Announce Type: replace-cross Abstract: While different stakeholders are trying to leverage Arabic Language Models (ALMs), safety alignment in ALMs remains largely underexplored, hin
arXiv:2609.29960v1 Announce Type: cross Abstract: Fine-tuning large language models on new objectives can improve helpfulness, instruction following, or domain-specific performance, but it can also in
arXiv:2609.25994v1 Announce Type: new Abstract: Providing safe and effective mobility assistance plays a crucial role in restoring independence and enhancing the quality of life for individuals with m
arXiv:2602.20628v2 Announce Type: replace Abstract: AIs are increasingly being deployed with greater autonomy and capabilities, which increases the risk that a misaligned AI may be able to cause catas
arXiv:2512.24493v3 Announce Type: replace-cross Abstract: Control barrier functions for port-Hamiltonian systems inherit model uncertainty when the Hamiltonian is learned from data. We show how to pro
arXiv:2609.21470v1 Announce Type: new Abstract: Sparse representation formulates the environment perception for the end-to-end driving system as a set of discrete elements like objects and lane lines.
arXiv:2609.21223v1 Announce Type: new Abstract: Vision-language-conditioned robot policies integrate perception, language understanding, and control for general-purpose manipulation. However, existing
arXiv:2609.19325v1 Announce Type: cross Abstract: Safety tuning pipelines judge only the final answer, which makes it difficult to distinguish robust refusal from two undesirable shortcuts: blanket re
In 2023, Lina Khan at the FTC launched an investigation into OpenAI over safety concerns. There was an entire AI regulatory program over there, including law enforcement, policy, and litigation. Trump
Jeezus. OpenAI cannot be trusted with the safety of the world. They can’t even keep their own servers secure. On July 25, we hacked OpenAI. Two bugs let us take over ChatGPT/Codex accounts of OpenAI e
arXiv:2609.20578v1 Announce Type: cross Abstract: Connected-vehicle safety evaluations rely on coupled traffic and network simulations, but standard channel models ignore radio resource competition in
New York Times: A former US State Department envoy says the US-China AI safety talks will not yield a breakthrough treaty as Beijing is prioritizing its strategic advantage — The United States is prep
Learn how to build a synthetic data augmentation pipeline on Amazon SageMaker AI and Amazon Rekognition that generates photo-realistic, auto-labeled training images for industrial safety AI. This appr
arXiv:2609.07288v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly employed to orchestrate robot behavior through natural-language interfaces, yet no benchmark exists to eva
I dropped out of a PhD in AI safety partially for the opposite reason. I didn’t believe AI existential risk was as important as the doomers think. My biggest pet peeve is that no one can really provid
Tim Fernholz / TechCrunch: Paul Christiano, an AI researcher and advisor at CAISI, is joining the OpenAI Foundation board of directors and its Safety and Security Committee — Paul Christiano, an influ
Despite everything that has happened, OpenAI has seemingly decided to stay competitive by burning down the only meaningful bit of safety assurance we actually have today - CoT monitoring. Completely d
arXiv:2606.16583v2 Announce Type: replace Abstract: Safe deployment of clinical vision-language models (VLMs) requires reliable uncertainty estimation (UE): a signal indicating when predictions should
Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university,
arXiv:2609.02293v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) is a scaling architecture for large language models that activates only a small subset of expert modules per token, enabling
arXiv:2609.00760v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly trained to decline queries that fall outside their knowledge (knowledge-based refusal, KR) or violate safe
arXiv:2609.00370v1 Announce Type: cross Abstract: Surrogate safety measures allow road-safety assessment from trajectory data in which crashes are absent, yet the dominant proximity measures - time-to
arXiv:2607.10014v2 Announce Type: replace-cross Abstract: Learning-based separation assurance for small Unmanned Aircraft Systems (sUAS) achieves near-zero collision rates in simulation, but assumes a
arXiv:2512.15052v4 Announce Type: replace-cross Abstract: Disclaimer: Samples in this paper may be harmful and cause discomfort. Multimodal large language models (MLLMs) enable multimodal understandin
arXiv:2608.28300v1 Announce Type: new Abstract: Robotic industrial panel operation requires not only accurate control localization but also compliance with operating procedures, safety rules, and devi
arXiv:2505.23197v4 Announce Type: replace-cross Abstract: Path planning for autonomous robots faces a fundamental trade-off between path length and obstacle clearance. While existing algorithms typica
arXiv:2608.23313v1 Announce Type: new Abstract: Vision-language model safety benchmarks typically evaluate only final responses: whether a model refuses, warns, or complies. This outcome-level view ca
arXiv:2608.21009v1 Announce Type: new Abstract: Ensuring that Extended Reality (XR) environments are age-appropriate is an important regulatory and safety challenge. However, current age assurance ope
arXiv:2608.21100v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) extend model capabilities beyond text, they also make safety alignment increasingly challenging. Multimod
arXiv:2608.18164v1 Announce Type: cross Abstract: Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arisi
Rachel Metz / Bloomberg: Source: Anthropic plans a safety system this year that still requires enterprises to retain data for 30 days but lets them do so on their own cloud systems — Anthropic PBC pla
We've been working on this with customers for a while. Mythos-class models require additional safety measures and enterprises need to meet their own privacy and compliance rules. Customers can own and
arXiv:2608.17386v1 Announce Type: new Abstract: Foundation-model policies for robotic manipulation are advancing rapidly on task success, but rigorous evaluation of whether they succeed safely is stil
arXiv:2608.14603v1 Announce Type: cross Abstract: Cooperative perception enables vehicles and infrastructure to exchange sensor data via Vehicle-to-Everything (V2X) communication, extending sensing co
arXiv:2608.13695v1 Announce Type: cross Abstract: Large language model providers routinely cite multilingual safety benchmarks spanning a dozen or more languages as evidence that their models are safe
arXiv:2608.11274v1 Announce Type: cross Abstract: The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is struc
Happy Thursday. On today's show: - @ibab (River AI) - @branninmcbee (CoreWeave) - @glangley (Flock Safety) - @sonyatweetybird (Sequoia) - @vytalow (Parasma) See you on the stream. SaaSpocalypse Revisi