基于高维协同软演员-评论家的多智能体自动发电控制
柳丹 , 任建宇 , 席磊 , 刘治洪 , 全悦 , 施宇
南方电网技术 ›› 2025, Vol. 19 ›› Issue (4) : 93 -106.
基于高维协同软演员-评论家的多智能体自动发电控制
Multi-Agent Automatic Power Generation Control Based on High Dimensional Collaborative Soft Actor-Critic
随着新能源渗透率不断提高,风光等新能源出力的强随机性导致电网频率不稳定及控制性能变差。为此,从自动发电控制角度探索一种多智能体强化学习方法,即高维协同软演员-评论家算法。所提算法通过在最大熵框架下鼓励智能体进行随机探索,以解决Q学习及其衍生算法无法快速更新Q表以适应环境变化的问题。同时利用高斯分布策略生成连续动作值,使算法可在高维连续状态空间中寻找协同最优解,以解决传统强化学习高维“状态-动作”的维数灾问题,从而来有效应对新能源出力强随机性所导致的频率不稳定及控制性能变差的问题。通过在改进的IEEE标准两区域负荷频率控制模型及华中三区域负荷频率控制模型上的仿真实验,验证了该算法的有效性,且相较于其他算法,具有更优的控制性能及频率稳定性。
As renewable energy penetration continues to increase, the strong randomness of wind and solar renewable energy output leads to instability of grid frequency and deterioration of control performance. To address this, a multi-agent reinforcement learning approach is explored from the perspective of automatic generation control, that is the high dimensional cooperative soft actor-critic algorithm. The proposed algorithm encourages agents to engage in random exploration within a maximum entropy framework to address the inability of Q-learning and its derivative algorithm to rapidly update the Q-table in response to environmental changes. It also utilizes a Gaussian distribution strategy to generate continuous action values, enabling the algorithm to find collaborative optimal solutions in high dimensional continuous state spaces, thus solving the traditional reinforcement learning curse of dimensionality of high dimensionality "state-action". Frequency instability and declining control performance caused by the strong randomness of renewable energy outputs are effectively addressed. Through simulation experiments on an improved IEEE standard two-area load frequency control model and the Central China three-area load frequency control model, the effectiveness of the algorithm is validated. And the proposed algorithm has superior control performance and frequency stability compared to other algorithms.
自动发电控制 / 高斯分布 / 多智能体 / 软演员-评论家 / 高维协同
automatic power generation control / Gaussian distribution / multi-agent / soft actor-critic / high dimensional collaboration
/
| 〈 |
|
〉 |