TwinLoop: Simulation-in-the-Loop Digital Twins for Online Multi-Agent Reinforcement Learning

arXiv:2604.06610v1 Announce Type: cross Abstract: Decentralised online learning enables runtime adaptation in cyber-physical multi-agent systems, but when operating conditions change, learned policies often require substantial trial-and-error interaction before recovering performance. To address this, we propose TwinLoop, a simulation-in-the-loop digital twin framework for online multi-agent reinforcement learning. When a context shift occurs, the digital twin is triggered to reconstruct the current system state, initialise from the latest agent policies, and perform accelerated policy improvement with simulation what-if analysis before synchronising updated parameters back to the agents in the physical system. We evaluate TwinLoop in a vehicular edge computing task-offloading scenario with changing workload and infrastructure conditions. The results suggest that digital twins can improve post-shift adaptation efficiency and reduce reliance on costly online trial-and-error.

Related

• KD-MARL: Resource-Aware Knowledge Distillation in Multi-Agent Reinforcement Learning
• Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning
• Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems
• Karma Mechanisms for Decentralised, Cooperative Multi Agent Path Finding
• Before Humans Join the Team: Diagnosing Coordination Failures in Healthcare Robot Team Simulation

Source: arXiv cs.AI | 2026-04-10