Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix

arXiv:2606.09884v1 Announce Type: cross Abstract: We study two reproducible failure modes of deep multi-agent reinforcement learning in continuous-time pricing markets: (i) tacit cartel formation between competing DDPG agents, and (ii) actor--critic instability at high event rates. We instantiate both inside a single CT-MARL benchmark (Poisson-clocked price updates, observation latency elta, interior-optimum logit demand), show that synchronous DDPG agents reliably trigger Failure Mode 1 with collusion index Delta = 0.69 pm 0.11, and quantify a partial microstructure fix: asynchrony alone cuts collusion by 48% and adding latency drives it to a minimum of Delta = 0.28. The fix has clearly documented costs: it is partial (Delta remains supra-Bertrand), it is non-monotone in elta, and it does not survive Failure Mode 2, which emerges as DDPG critic divergence at lambda = 5 and corrupts the phase-diagram cell at (lambda{=}5, elta{=}1). We accompany the scalar collusion index with trajectory-level trace diagnostics that expose the within-episode signalling collapse and the post-shock non-recovery.

Source: arXiv cs.AI | 2026-06-10