MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents

arXiv:2610.12757v1 Announce Type: new Abstract: Task-oriented conversational agents remain fragile under realistic conditions, especially when users exhibit non-cooperative behavior. Existing function-calling benchmarks underrepresent adversarial conversation trajectories, thereby limiting the training resources available for developing robust agents. We present MADS (Multi-Agent Dialog Simulation), a synthetic data generation pipeline that produces structured, diverse dialogue trajectories. We inject adversarial conversation, producing validated conversation trajectories that capture realistic, unhappy paths. Using MADS, we construct MADS-Bench: a benchmark spanning six domains, 55 personas, and 1095 conversations, with approximately 66% of conversations containing failure-prone conversation scenarios. We evaluate the robustness of the trained models using a two-stage training approach that combines supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR).

Related

• ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation
• ProACT: Towards Breakdown-Aware Proactive Agent in Multi-User Collaboration
• Beyond Borrowed Histories: Person-Aligned User Simulation for Interactive Role-Playing Evaluation
• Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation

Source: arXiv cs.CL | 2026-10-12