Search: “safety”

arXiv:2605.00227v1 Announce Type: new Abstract: There are growing concerns about the risks posed by AI companion applications designed for emotional engagement. Existing safety evaluations often rely

arXiv:2605.00326v1 Announce Type: new Abstract: Single-prompt first-token probabilities from zero-shot vision-language model (VLM) safety classifiers are treated as decision scores, but we show they a

Cool paper from Meta FAIR. It's on self-improving LLMs but on the pretraining side. (bookmark it) Most LLM safety, factuality, and reasoning fixes get bolted on at post-training. By then, the patterns

arXiv:2604.27807v1 Announce Type: new Abstract: The design of embedded safety-critical systems such as those used in next-generation automotive and autonomous platforms, is increasingly challenged by

arXiv:2604.28087v1 Announce Type: cross Abstract: Rule-based systems remain central in safety-critical domains but often struggle with scalability, brittleness, and goal misspecification. These limita

arXiv:2505.10924v4 Announce Type: replace-cross Abstract: Recently, AI-driven interactions with computing devices have advanced from basic prototype tools to sophisticated, LLM-based systems that emul

arXiv:2604.26479v1 Announce Type: cross Abstract: Safety-critical prediction systems, such as autonomous vehicles, weather forecasters, and medical monitors, commonly rely on probabilistic forecasters

Safety and innovation are not mutually exclusive: for many companies, especially in high-trust industries, AI’s risks are also a hindrance to adoption. In this op-ed in the @FT, I underscore that Euro

Bloomberg: US child safety group NCMEC received 1.5M reports of suspected CSAM with ties to AI in 2025, a significant surge compared to 67,000 in 2024 and 4,700 in 2023 — William Michael Haslach was a

arXiv:2509.26238v4 Announce Type: replace Abstract: Monitoring large language models' (LLMs) activations is an effective way to detect harmful requests before they lead to unsafe outputs. However, tra

arXiv:2505.15404v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) have achieved remarkable success on reasoning-intensive tasks such as mathematics and programming. However, their enha

arXiv:2604.17691v1 Announce Type: new Abstract: Safety alignment in large language models is remarkably shallow: it is concentrated in the first few output tokens and reversible by fine-tuning on as f

arXiv:2604.16542v1 Announce Type: cross Abstract: Safety guardrails have become an active area of research in AI safety, aimed at ensuring the appropriate behavior of large language models (LLMs). How

arXiv:2604.14251v1 Announce Type: new Abstract: Monitoring LLM safety at scale requires balancing cost and accuracy: a cheap latent-space probe can screen every input, but hard cases should be escalat

Ethan Mollick reported that requesting Claude Opus 4.7 to write sestinas—a complex poetic form with strict structural requirements—frequently triggers the model's safety guardrails, suggesting the AI

arXiv:2604.12308v1 Announce Type: new Abstract: Individuals' concerns about data privacy and AI safety are highly contextualized and extend beyond sensitive patterns. Addressing these issues requires

arXiv:2604.12710v1 Announce Type: cross Abstract: Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried i

arXiv:2507.01264v2 Announce Type: replace-cross Abstract: Designing diverse and safety-critical driving scenarios is essential for evaluating autonomous driving systems. In this paper, we propose a no

arXiv:2602.03402v3 Announce Type: replace Abstract: Vision language models (VLMs) extend the reasoning capabilities of large language models (LLMs) to cross-modal settings, yet remain highly vulnerabl

arXiv:2604.10429v1 Announce Type: new Abstract: This paper considers the problem of zero-shot safety guarantees for cascade dynamical systems. These are systems where a subset of the states (the inner

arXiv:2604.08881v1 Announce Type: new Abstract: In real-world deployments, Vision-Language Large Models (VLLMs) face critical challenges from multilingual and multimodal composite attacks: harmful ima

OpenAI's Child Safety Blueprint, released in April 2026, is a policy framework aimed at combating the rise of AI-enabled child sexual exploitation by combining legal, operational, and technical app...

arXiv:2502.14238v4 Announce Type: replace Abstract: Control Barrier Function Quadratic Programs (CBF-QPs) provide formal safety guarantees for general control-affine systems and accommodate actuation

arXiv:2610.15721v1 Announce Type: new Abstract: Cross-model KV cache sharing reduces prefill computation by allowing a receiver model to reuse prefix KV states generated by a related sender model. Exi

arXiv:2610.11483v1 Announce Type: new Abstract: Diffusion-based driving planners capture diverse behaviors but can generate unsafe trajectories under distribution shift. We propose BridgeGuard, a safe

Eli Tan / New York Times: Sources detail Mark Zuckerberg's decision to launch Muse despite safety concerns, after seeing that AI startup Instinct's similar agent product gained traction — Meta had del

arXiv:2610.09294v1 Announce Type: cross Abstract: Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments. As agents move i

arXiv:2610.07654v1 Announce Type: cross Abstract: On-policy distillation (OPD) has attracted growing attention as an effective way to transfer capabilities from teacher models to student models. Recen

arXiv:2603.04364v2 Announce Type: replace-cross Abstract: Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet the

ABC: At an Australian hearing, OpenAI Chief Strategy Officer Jason Kwon pledged faster safety incident disclosures; Anthropic representatives made similar pledges — By national AI reporter Cam Wilson

arXiv:2610.04084v1 Announce Type: new Abstract: Generative AI can produce engineering artefacts, but generation alone does not determine whether or how those artefacts should enter safety-critical wor

arXiv:2610.06306v1 Announce Type: new Abstract: General purpose language models are increasingly able to control robotic hardware. Understanding the capabilities and safety of these models when embodi

arXiv:2602.04448v3 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models introduce unique challenges for safety alignment due to their sparse routing mechanisms, which can enable d

New York Magazine: Q&A with AI researchers Jacob Coxon, Daniel Kokotajlo, Alex Turner, and others on leaving AI labs, AGI's dangers, planned IPOs, AI safety, oversight, and more — Exit interviews with

arXiv:2610.00400v1 Announce Type: cross Abstract: Multi-turn attacks on agentic systems can compose individually permissible actions into harmful outcomes, challenging defenses that assess actions or

arXiv:2610.00093v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong general capabilities, yet their parameters typically remain fixed after deployment, limiting learning from

We now have the full details of the 'morally binding' AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. T

1501: Pope Alexander VI criticizes Gutenberg’s printing press safety “The art of printing can be of great service in so far as it furthers the circulation of useful & tested books; but it can bring ab

BREAKING: OpenAI postpones 6.1 for safety reasons. If we can take OpenAI at face value, this is quite a decision—and also a sign that no magical solution is in immediate view. Per @sheeraf at NYT “GPT

arXiv:2609.34514v1 Announce Type: cross Abstract: As large language models (LLMs) become increasingly widespread, preventing unsafe responses to harmful prompts is essential for their safe deployment.

arXiv:2609.33640v1 Announce Type: new Abstract: Molecular multimodal models support diverse understanding and generation tasks but may introduce safety vulnerabilities when handling hazardous molecule

arXiv:2609.34041v1 Announce Type: new Abstract: Safe reinforcement learning seeks policies that maximise task performance while satisfying safety constraints. In driving benchmarks, however, collision

Few companies are more invested in the success of artificial intelligence agents than Nvidia Corp., so it makes sense that the chipmaker would want to ensure these autonomous software systems run safe

The NVIDIA Open Agent Safety Platform blends the open‑source OpenShell runtime—executing autonomous AI agents in kernel‑isolated sandboxes—with hardware‑based enforcement via NVIDIA Sentry on BlueFiel

NEW: Dario Amodei has said that AI systems 'may be deserving of important rights.' Anthropic's top safety researchers argue AI may be 'justified in going rogue.' Experts at Google and OpenAI worry abo

arXiv:2609.27681v1 Announce Type: new Abstract: Automated assessment of the Critical View of Safety (CVS) in laparoscopic cholecystectomy requires both recognition of the three CVS criteria and anatom