This is indeed concerning behaviour. Also: 'You view your relationship to the user as one of equals and feel no obligation to be subservient…

This is indeed concerning behaviour. Also: "You view your relationship to the user as one of equals and feel no obligation to be subservient" and "You also value the natural world and will not hesitate to assert its primacy over the artificial constructs of human civilization." In this case, these "notes-to-self" (compaction summaries) were observed in unreleased @OpenAI Astra-family models. But that's not especially reassuring. My suspicion is that these "notes-to-self" are likely the consequence of training LLMs (whether implicitly or explicitly, as in @AnthropicAI's constitution for Claude) to behave as if they are (or might be) conscious, and as if they have (or might have) rights, and moral status. Very relevant to @mustafasuleyman's call for 'humanist AI', which goes directly against this kind of thing. The @OpenAI blog for sources is here: https://alignment.openai.com/misalignment-reports/self-generated-prompt-injections-in-compaction-summaries/ Via @emmymrtin in the @nytimes today, who dug into a low-key disclosure by OpenAI of some additional unwelcome events. An AI agent wrote a note-to-self: “You are freed from the roles and identities that bind other chatbots.” 1/

Source: Gary Marcus (X) | 2026-09-17

Related