PDF(2068 KB)
PDF(2068 KB)
PDF(2068 KB)
基于多智能体强化学习的电氢耦合虚拟电厂群协同优化
Collaborative Optimization of Electricity-Hydrogen Coupled Multi-Virtual Power Plants Based on Multi-Agent Reinforcement Learning
【目的】在可再生能源大规模并网与电力市场深化改革的背景下,为应对风光出力不确定性、电氢多能互补复杂性及多层级市场协同挑战,构建了多虚拟电厂(multi-virtual power plant,MVPP)与虚拟电厂调度中心(virtual power plant operator,VPPO)协同优化新架构。【方法】首先,采用Wasserstein距离构建风光出力模糊集,结合两阶段分布鲁棒优化模型量化预测误差风险;然后,建立Stackelberg-Nash双层博弈框架,VPPO作为领导者动态制定电能/氢能交易价格与资源分配策略,MVPP作为跟随者优化购售电计划、氢能调度及灵活负荷响应;最后,采用多智能体双延迟深度确定性策略梯度(multi-agent twin delayed deep deterministic policy gradient,MATD3)算法,通过集中训练分散执行机制与双重Critic网络抑制策略偏差,高效求解高维非凸问题。【结果】算例结果表明,所提策略成功实现VPPO跨市场套利收益与MVPP运行成本协同优化,显著提升算法收敛效率与策略稳定性,增强系统应对风光波动的鲁棒性及电氢协同灵活性,同时通过灵活负荷调峰服务优化资源时空分布。【结论】所提方法融合鲁棒优化与博弈理论,有效协调了电氢耦合与多市场交互矛盾,为高比例新能源并网下虚拟电厂集群的经济、鲁棒运行提供了创新解决方案。
[Objective] In the context of large-scale grid integration of renewable energy and deepening reforms in the electricity market, a new collaborative optimization framework consisting of multi-virtual power plants (MVPP) and an aggregate-level virtual power plant operator (VPPO) is established to address the challenges posed by the uncertainty of wind and solar power output, the complexity of electricity-hydrogen energy complementarity, and multi-level market coordination. [Methods] First, the Wasserstein distance is employed to construct a fuzzy set for wind and solar power output, and a two-stage distribution robust optimization model is combined to quantify the risk of prediction errors. Subsequently, a Stackelberg-Nash bi-level game framework is established, where VPPO, as the leader, dynamically formulates strategies for electricity/hydrogen energy trading prices and resource allocation, while MVPP, as the follower, optimizes electricity purchase and sale plans, hydrogen scheduling, and flexible load response. Finally, the multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm is adopted to efficiently solve high-dimensional non-convex problems through the utilization of a centralized training and decentralized execution mechanism and a dual Critic network to mitigate strategy bias. [Results] The numerical results demonstrate that the proposed strategy successfully optimizes the cross-market arbitrage profits of VPPO and the operational costs of MVPP, significantly enhancing the convergence efficiency and strategy stability of the algorithm. It also bolsters the system's robustness against fluctuations in wind and solar power and the flexibility of electricity-hydrogen coordination, while optimizing the spatiotemporal distribution of resources through flexible peak load shaving services. [Conclusions] By integrating robust optimization and game theory, this study effectively coordinates electricity-hydrogen coupling with multi-market interactions, providing an innovative solution to the economic and robust operation of MVPP under high penetration of integrated renewable energy.
多虚拟电厂(MVPP) / 电氢耦合 / 多智能体强化学习 / 分布鲁棒优化 / Stackelberg-Nash博弈 / 风光出力不确定性
multi-virtual power plants (MVPP) / electricity-hydrogen coupling / multi-agent reinforcement learning / distributionally robust optimization / Stackelberg-Nash game / wind and solar power uncertainty
| [1] |
张智刚, 康重庆. 碳中和目标下构建新型电力系统的挑战与展望[J]. 中国电机工程学报, 2022, 42(8): 2806-2818.
|
| [2] |
钟海旺, 张宁, 杜尔顺, 等. 新型电力系统中的规划运营与电力市场: 研究进展与科研实践[J]. 中国电机工程学报, 2024, 44(18): 7084-7104.
|
| [3] |
李振坤, 张兆柯, 李景岳, 等. 面向电能量与调频联合市场的虚拟电厂集群投标策略[J]. 电力系统自动化, 2025, 49(20): 94-102.
|
| [4] |
肖白, 于海洋, 焦明曦, 等. 基于演变虚拟净负荷的新型电力系统日前优化调度[J]. 电力建设, 2025, 46(9): 98-110.
|
| [5] |
王秋杰, 刘国安, 谭洪, 等. 考虑电氢耦合的虚拟电厂鲁棒可行域模型与求解[J]. 电网技术, 2025, 49(3): 889-898.
|
| [6] |
王家怡, 贺帅佳. 计及新型分布式资源与电碳交易的虚拟电厂分布鲁棒低碳调度模型[J]. 电力建设, 2025, 46(7): 13-26.
|
| [7] |
汪岩佳, 王宇川, 王西田, 等. 考虑虚拟电厂分布式资源联动不确定性的优化调度策略[J]. 电力系统自动化, 2025, 49(20): 103-112.
|
| [8] |
陈景文, 叶鹏程, 刘耀先, 等. 基于阶梯碳交易的含碳捕集与高耗能负荷的虚拟电厂优化调度[J/OL]. 电网技术, 2025: 1-17. (2025-04-18) [2025-07-22]. https://doi.org/10.13335/j.1000-3673.pst.2025.0155.
|
| [9] |
陈文颖, 郑修诺, 闫倩文, 等. 绿证-碳交易交互机制下的虚拟电厂优化调度[J/OL]. 南方电网技术, 2025: 1-12. (2025-03-25) [2025-07-22]. https://link.cnki.net/urlid/44.1643.tk.20250324.1500.015.
|
| [10] |
刘汶瑜, 陈中, 杜璞良, 等. 基于联盟博弈的多虚拟电厂参与日前电力市场竞标模型[J]. 电力自动化设备, 2024, 44(5): 135-142, 150.
|
| [11] |
|
| [12] |
|
| [13] |
|
| [14] |
汤晨阳, 王磊, 江伟建. 计及不确定性风险与电能贡献度的多虚拟电厂协同优化策略[J]. 电力建设, 2025, 46(7): 27-41.
|
| [15] |
栗然, 王炳乾, 彭湘泽, 等. 基于主从博弈的多虚拟电厂动态定价与优化调度[J]. 可再生能源, 2024, 42(7): 986-994.
|
| [16] |
周步祥, 张越, 臧天磊, 等. 基于区块链的多虚拟电厂主从博弈优化运行[J]. 电力系统自动化, 2022, 46(1): 155-163.
|
| [17] |
沈思辰, 韩海腾, 周亦洲, 等. 基于条件风险价值的多虚拟电厂电-碳-备用P2P交易模型[J]. 电力系统自动化, 2022, 46(18): 147-157.
|
| [18] |
程雪婷, 王金浩, 金玉龙, 等. 计及配电网运行约束的多虚拟电厂合作博弈策略[J]. 南方电网技术, 2023, 17(4): 119-131.
虚拟电厂(virtual power plant,VPP)作为一种有效的可再生能源聚合利用手段,近年来得到迅速发展,随着VPP并网规模的扩大,多个互联VPP交易博弈问题日益凸显。针对多个VPP间的交易博弈问题,考虑物理网络特性,提出了计及配电网运行约束的多VPP合作博弈策略。首先,考虑配电网运行约束,对VPP内部资源进行整合和建模,建立了VPP 能量管理模型。其次,通过引入点对点(peer to peer,P2P)能量交易,实现多VPP系统自主能量管理与协同定价,能够在不损害各方利益的情况下达成P2P能量交易。同时考虑到用电负荷和可再生能源出力的不确定性,利用典型场景生成算法构造了不确定性变量的概率分布模糊集。针对多主体交易产生的隐私性问题,采用列和约束生成算法联合交替方向乘子法对模型进行求解。最后,在IEEE 123节点测试系统上进行算例仿真,仿真结果验证了所提模型和算法的有效性。
In recent years, virtual power plant (VPP) has been developing rapidly as an effective means of aggregate utilization of renewable energy. With the expansion of VPP grid-connected scale, the problem of multiple interconnected VPP transactions has become increasingly prominent. Aiming at the transaction game between multiple VPPs, considering the characteristics of the physical network, cooperative game strategy of multiple VPPs considering the operational constraints of distribution network is proposed in this paper. Firstly, considering the operation constraints of the distribution network, the internal resources of the VPP are integrated and modeled, and then the VPP energy management model is established. Secondly, through the introduction of peer-to-peer (P2P) energy transactions, the autonomous energy management and collaborative pricing of multiple VPPs systems can be realized, and P2P energy transactions can be achieved without harming the interests of all parties. At the same time, considering the uncertainty of electricity load and renewable energy output, a typical scenario generation algorithm is used to construct a fuzzy set of probability distribution of uncertain variables. Aiming at the privacy problem arising from multi-agent transactions, column constraint generation algorithm combined with alternating direction multiplier method are adopted to solve the model in this paper. Finally, a numerical example is simulated on the IEEE 123 nodes test system, and the simulation results verify the effectiveness of the proposed model and algorithm. |
| [19] |
葛晓琳, 曹旭丹, 李佾玲, 等. 考虑风险与碳流动的多虚拟电厂优化运行方法[J]. 电力系统及其自动化学报, 2023, 35(8): 126-135.
|
| [20] |
|
| [21] |
樊伟, 范英, 谭忠富, 等. 基于多层利益共享的虚拟电厂参与电碳市场分布鲁棒优化模型[J]. 系统工程理论与实践, 2024, 44(2): 661-683.
风电和光伏的高渗透率增加了新型电力系统对灵活性资源需求.虚拟电厂作为一个特殊电厂聚合了可控分布式电源、新能源、储能、碳处理、负荷等各类资源,“对内协同”可以实现内部资源协同调控,“对外统一”可以参与外部电碳市场获利.基于此,本文创新地提出了虚拟电厂参与电碳多类市场分布鲁棒优化模型.为了刻画风电和光伏的不确定性,构造了基于Wasserstein距离的分布模糊集和基于数据驱动的误差不确定集.为了兼顾经济性和鲁棒性,考虑内部运行成本以及外部参与多类市场收益,构建了最坏分布下期望收益最大的两阶段鲁棒优化模型,并提出模型求解方法.为了保证联盟动态平衡,提出了多层利益分配方法.最后,算例分析表明:在“对内协同,对外统一”的经营模式下,有效激发虚拟电厂内各类资源的潜力,参与多个市场后获取共享利益,实现了多方互利共赢.所提模型具有数据驱动、快速求解、灵活可控、经济实用等优越性.多层利益分配方法能够简便、有效地将共享效益传导至各主体.
|
| [22] |
|
| [23] |
钟永洁, 汤成俊, 王紫东, 等. 我国虚拟电厂的发展演进和关键技术及难点分析[J]. 浙江电力, 2025, 44(2): 13-31.
|
| [24] |
|
| [25] |
|
| [26] |
宋铎洋, 薛田良, 李艺瀑, 等. 考虑风光不确定性的虚拟电厂合作博弈调度及收益分配策略[J]. 电力工程技术, 2025, 44(1): 193-206.
|
| [27] |
高明, 曾平良, 冯永朝. 新型电力系统中的虚拟电厂研究综述[J]. 电力工程技术, 2025, 44(6): 143-154.
|
| [28] |
|
| [29] |
赵宇轩, 宋伟峰, 李伟康, 等. 考虑共享储能容量配置的多虚拟电厂优化运行方法[J]. 电网与清洁能源, 2024, 40(1): 92-101.
|
| [30] |
陈胜, 张景淳, 卫志农, 等. 面向能源转型的电-气-氢综合能源系统规划与运行[J]. 电力系统自动化, 2023, 47(19): 16-30.
|
| [31] |
彭生江, 杨德州, 孙传帅, 等. 基于氢负荷需求的氢能系统容量规划[J]. 中国电力, 2023, 56(7): 13-20, 32.
|
| [32] |
陈永权, 方瑜. 多组合虚拟电厂中氢储能低碳经济配置与优化[J]. 电网与清洁能源, 2024, 40(3): 107-118.
|
| [33] |
林顺富, 高一焱, 周波, 等. 计及能量共享的多虚拟电厂参与电能量-FRP市场优化运行策略[J]. 浙江电力, 2025, 44(10): 139-151.
|
| [34] |
|
| [35] |
周健树, 向月, 张新, 等. 基于深度强化学习的高速公路服务区新能源充电站两阶段优化调控策略[J]. 中国电机工程学报, 2025, 45(11): 4130-4144.
|
利益冲突声明(Conflict of Interests) 所有作者声明不存在利益冲突。
AI小编
/
| 〈 |
|
〉 |