Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination

arXiv:2607.12763v1 Announce Type: cross Abstract: Federated Reinforcement Learning (FedRL) enables coordination of distributed energy resources without sharing raw local data, but standard aggregation methods such as FedAvg do not account for system-level constraints, often leading to unsafe global behavior. In this work, we study constraint-aware aggregation for federated reinforcement learning in distributed energy coordination. We propose aggregation rules that incorporate both local performance and estimated constraint violation into the server-side update. Among these, a simple penalty-based rule, w_i propto R_i - alpha V_i, consistently provides the most reliable trade-off between reward and safety, without requiring dual optimization or modifications to local training. extcolor{black}{We evaluate our approach on DairyGridEnv, a benchmark modeling multiple farms coordinating battery storage under stochastic demand and a shared grid capacity constraint, and further assess robustness using real load-driven demand profiles from Finland and the German FIELD dataset. Across multiple seeds, penalty-based aggregation substantially reduces violations while improving reward relative to FedAvg in both synthetic and real load-driven settings.} A combined reward-violation scheme exposes a tunable trade-off via lambda, but is less stable. These results demonstrate that lightweight aggregation strategies can substantially improve empirical safety in federated reinforcement learning while preserving standard communication protocols.

Related

• Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity
• Safe-RULE: Safe Reinforcement UnLEarning
• Learning-Zone Energy: Online Data Selection for Efficient RL Post-Training
• Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning

Source: arXiv cs.AI | 2026-07-15