SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design

arXiv:2609.09764v1 Announce Type: new Abstract: Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue. As language models become autonomous collaborators, it is central to building effective and trustworthy human-AI interaction. Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards, producing short-sighted policies that struggle to manage goal-relationship tensions across multi-turn interactions. We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges. First, we apply multi-turn reinforcement learning using PPO that propagates delayed outcome rewards back to each turn, enabling long-horizon planning. Second, we design six process reward dimensions capturing the goal-relationship trade-off, including goal advancement, relational attunement, contextual coherence, etc. A reward model dynamically generates fine-grained scoring criteria for each dimension, while a stage-aware weight schedule prioritizes relationship-building in early turns, goal advancement mid-way, and balanced closure late. Across multiple social-dialogue benchmarks, SocialRL improves Goal Achievement by an average of 9.2 percentage points over the corresponding Base models. These results demonstrate the effectiveness of SocialRL across synthetic and real social scenes, as well as standard and challenging social scenarios.

Related

• Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learning
• MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation
• Learning from Self-Debate: Preparing Reasoning Models for Multi-Agent Debate
• Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context

Source: arXiv cs.CL | 2026-09-10