Regulating Artificial Intimacy: From Locks and Blocks to Relational Accountability

arXiv:2604.18893v1 Announce Type: cross Abstract: A series of high-profile tragedies involving companion chatbots has triggered an unusually rapid regulatory response. Several jurisdictions, including Australia, California, and New York, have introduced enforceable regulation, while regulators elsewhere have signaled growing concern about risks posed by companion chatbots, particularly to children. In parallel, leading providers, notably OpenAI, appear to have strengthened their self-regulatory approaches. Drawing on legal textual analysis and insights from regulatory theory, psychology, and information systems research, this paper critically examines these recent interventions. We examine what is regulated and who is regulated, identifying regulatory targets, scope, and modalities. We classify interventions by method and priority, showing how emerging regimes combine "locks and blocks", such as access gating and content moderation, with measures addressing toxic relationship features and process-based accountability requirements. We argue that effective regulation of companion chatbots must integrate all three dimensions. More, however, is required. Current regimes tend to focus on discrete harms, narrow conceptions of vulnerability, or highly specified accountability processes, while failing to confront deeper power asymmetries between providers and users. Providers of companion chatbots increasingly control artificial intimacy at scale, creating unprecedented opportunities for control through intimacy. We suggest that a general, open-ended duty of care would be an important first step toward constraining that power and addressing a fundamental source of chatbot risk. The paper contributes to debates on companion chatbot regulation and is relevant to regulators, platform providers, and scholars concerned with digital intimacy, law and technology, and fairness, accountability, and transparency in sociotechnical systems.

Related

• Governing frontier general-purpose AI in the public sector: adaptive risk management and policy capacity under uncertainty through 2030
• Reckoning with the Political Economy of AI: Avoiding Decoys in Pursuit of Accountability
• 'Taking Stock at FAccT': Using Participatory Design to Co-Create a Vision for the Fairness, Accountability and Transparency Community
• From experimentation to engagement: on the paradox of participatory AI and power in contexts of forced displacement and humanitarian crises
• Speaking to No One: Ontological Dissonance and the Double Bind of Conversational AI

Source: arXiv cs.AI | 2026-04-22