PDF(6462 KB)
新型电力系统演化趋优下多元负荷绿色协同市场行为DQN仿真模型
张硕, 袁春辉, 李英姿, 肖阳明, 魏铭, 贺运政
电力建设 ›› 2025, Vol. 46 ›› Issue (7) : 191-204.
PDF(6462 KB)
PDF(6462 KB)
新型电力系统演化趋优下多元负荷绿色协同市场行为DQN仿真模型
DQN Simulation Model of Green Cooperative Market Behavior of Multi-Load Users for Optimal Evolution of New-Type Power System
【目的】 为探究和分析多元负荷绿色行为对新型电力系统演化的驱动作用，提升新型电力系统灵活性和资源优化配置能力，开展了多元负荷绿色协同市场行为仿真研究，构建了多元负荷在新型电力系统演化趋优下的绿色协同市场行为仿真模型。【方法】 首先，分析多元负荷的绿色协同市场行为与竞价行为，构建多元负荷市场竞价决策模型；其次，构建了多元负荷主体决策流程及电力市场全过程交易仿真流程，应用深度Q网络（deep Q-network，DQN）算法对多元负荷参与市场交易的行为进行分析，得到其最优竞价策略。最后，以某区域电力系统为案例，仿真模拟多元负荷电力市场全过程交易流程，得出多元负荷最优策略报价结果。【结果】 与传统Q-learning算法相比，构建的仿真模型可使多元负荷平均购电价格降低8.5%、需求响应收益提升13.7%、新能源的消纳率提升5%，此外，对新能源渗透率进行了敏感性分析，识别出需求响应对新能源渗透率敏感性更高。【结论】 研究结果表明所提模型可有效模拟多元负荷绿色协同市场行为，提升多元负荷的经济效益及新能源的消纳率，增强新型电力系统的资源优化配置能力，为新型电力系统多元负荷的市场化运营及政策制定提供了理论支持和应用参考。
[Objective] To investigate and analyze the driving role of the green behavior of multiple loads on the evolution of a new type of power system and to enhance the flexibility and resource optimization allocation capacity of a new type of power system， this study conducted a simulation study of the green cooperative market behavior of multiple loads and constructed a simulation model of the green cooperative market behavior of multiple loads under the optimization of the evolution of a new power system. [Methods] First， the green cooperative market behavior and bidding behavior of multiple loads were analyzed and a multiple-load market bidding decision model was constructed. Second， the decision-making process of multiple loads and the entire transaction simulation process of the power market were constructed， and the deep reinforcement learning DQN algorithm was applied to analyze the behavior of multiple loads participating in market transactions and obtain their optimal bidding strategies. Finally， using a regional power system as a case study， the simulation simulated the entire transaction process of the multiple-load power market and derived the optimal strategy that offers results of multiple loads. [Results] Compared to the traditional Q-learning algorithm， the simulation model constructed in this study can reduce the average power purchase price of multiple loads by 8.5%， increase the demand response revenue by 13.7%， and increase the consumption rate of new energy by 5%. In addition， a sensitivity analysis of the new energy penetration rate was conducted and a higher sensitivity of demand response to the new energy penetration rate was identified. [Conclusions] The results showed that the proposed model can effectively simulate the green cooperative market behavior of multiple loads， improve the economic benefits of multiple loads and the new energy consumption rate， enhance the resource optimization and allocation ability of the new power system， and provide theoretical support and application reference for the market-oriented operation of multiple loads in a new type of power system and policy formulation.
多元负荷 / 绿色协同行为 / 深度Q网络算法 / 电力市场交易仿真
multiple loads / green collaborative behavior / deep Q netwsrk（DQN） algorithm / electricity market trading simulation
| [1] |
国家能源局. 新型电力系统发展蓝皮书[EB/OL]. (2023-06-02) [2024-03-21]. http://www.nea.gov.cn/2023-06/02/c_1310724249.htm
|
| [2] |
解百臣, 章露, 郝鹏. 基于主从博弈的虚拟电厂实时定价需求响应机制研究[J]. 中国管理科学, 2024, 32(12):312-322.
|
| [3] |
李新国, 杨轩, 程少靖, 等. 面向多类型用户负荷的需求响应潜力量化评估[J]. 智慧电力, 2024, 52(9): 56-64.
|
| [4] |
陈湘元, 吴公平, 龙卓, 等. 考虑源荷不确定性及用户侧需求响应的综合能源系统多时间尺度优化调度[J]. 电力科学与技术学报, 2024, 39(3): 217-227.
|
| [5] |
张浩崇, 李培强, 杨鸽星, 等. 配电网新型负荷承载能力区间评估[J]. 智慧电力, 2024, 52(12): 12-18, 50.
|
| [6] |
王明富, 吴华华, 杨林华, 等. 电力市场环境下能源互联网发展现状与展望[J]. 电力需求侧管理, 2020, 22(2): 1-7.
|
| [7] |
冯天天, 李晏, 孙晓琪, 等. 大数据驱动下电-碳市场耦合及协同发展研究综述[J]. 智慧电力, 2024, 52(1): 55-64.
|
| [8] |
梁晨, 牛皓玮, 刘达, 等. 政府激励背景下企业绿色交易策略行为演化研究[J]. 智慧电力, 2024, 52(1): 23-29, 54.
|
| [9] |
阳婷婷, 李晓刚, 邹斌, 等. 促进灵活性资源的辅助服务市场分析[J]. 电力系统保护与控制, 2023, 51(7): 73-83.
|
| [10] |
|
| [11] |
叶小宁, 王彩霞, 李琼慧, 等. 国外新能源高占比电力系统电力供应保障措施及启示[J]. 中国电力, 2024, 57(4): 61-67.
|
| [12] |
袁泉, 张蔷, 禤培正, 等. 南方(以广东起步)电力现货市场双边交易仿真分析研究[J]. 广东电力, 2022, 35(6): 10-17.
|
| [13] |
宋莉, 刘敦楠, 庞博, 等. 需求侧资源参与电力市场机制及典型案例实践综述[J]. 全球能源互联网, 2021, 4(4): 401-410.
|
| [14] |
王彩霞, 时智勇, 梁志峰, 等. 新能源为主体电力系统的需求侧资源利用关键技术及展望[J]. 电力系统自动化, 2021, 45(16): 37-48.
|
| [15] |
张硕, 曾鸣, 李英姿, 等. 新能源电力系统用户需求响应复杂适应行为研究[J]. 电力建设, 2017, 38(11): 136-143.
摘 要：实施需求响应（demand response，DR）可有效消纳可再生能源，平衡新能源电力系统电量供应和需求。首先，分析了需求响应过程的复杂适应系统特征，描述了需求响应复杂交互过程；其次，构建了包含主体对象和环境对象的需求响应过程集成模型，定义了各构成对象；最后，结合多主体系统建模方法构建了用户主体响应行为模型，从主体行为的角度描述了需求响应过程。通过文章的研究，可为实施需求侧响应提供辅助决策，为用户需求响应的研究提供了新思路。
ABSTRACT： Demand response (DR) can effectively improve the renewable energy accommodation and balance the supply and demand of renewable energy power system. Firstly, this paper analyzes the characteristics of the complex adaptive system druring DR process and then describes its complex interactive process. Secondly, this paper constructes the integrated model of DR including agent object and environment object, and defines each constituent object. Finally, combined with the multi-agent system modeling method, this paper constructs the response behavior model of user agent and describes the DR process from the aspect of subject behavior. The research can provide assistant decision-making support for implementing DR and present innovative idea for the DR research of users.<div> </div>
|
| [16] |
吴明兴, 王宁, 王浩浩, 等. 双轨制下电力现货市场政府授权合约分解优化模型与方法[J]. 中国电力, 2023, 56(9): 15-26.
|
| [17] |
曾垂辉, 李文正, 李宝伟, 等. 含多分布式能源聚合商的本地电力市场最优能源交易策略[J]. 中国电力, 2023, 56(11): 236-245.
|
| [18] |
彭超逸, 徐苏越, 顾慧杰, 等. 基于主从博弈的虚拟电厂参与多元竞争市场投标策略研究[J]. 电力系统保护与控制, 2024, 52(7): 125-137.
|
| [19] |
张钧钊, 姜欣, 段世杰, 等. 虚拟电厂参与电-碳联合市场运行的竞价策略研究[J]. 电力系统保护与控制, 2023, 51(11): 108-118.
|
| [20] |
牛寅生, 姜曼, 叶泽, 等. 考虑省间壁垒的区域电力中长期市场统一出清模型及结算机制设计[J]. 电力科学与技术学报, 2023, 38(6): 33-43.
|
| [21] |
陈灏, 田琳, 盛剑胜, 等. 考虑风险规避和需求响应的电力市场可再生能源综合交易决策研究[J]. 电力科学与技术学报, 2023, 38(1): 27-34.
|
| [22] |
唐成鹏, 张粒子, 刘方, 等. 基于多智能体强化学习的电力现货市场定价机制研究(一): 不同定价机制下发电商报价双层优化模型[J]. 中国电机工程学报, 2021, 41(2): 536-553.
|
| [23] |
|
| [24] |
刘飞宇, 王吉文, 王正风, 等. 基于两阶段深度强化学习算法的多智能体自由合谋竞价机理研究[J]. 中国电机工程学报, 2024, 44(12): 4626-4639.
|
| [25] |
高宇, 李昀, 曹蓉蓉, 等. 基于多代理Double DQN算法模拟发电侧竞价行为[J]. 电网技术, 2020, 44(11): 4175-4183.
|
| [26] |
王蓓蓓, 刘飞宇, 杨朋朋, 等. 考虑轻微利他效用的发售电一体化集团成员合谋行为的多智能体深度双Q网络推演研究[J]. 中国电机工程学报, 2023, 43(7): 2640-2652.
|
| [27] |
许丹, 胡晓静, 胡斐, 等. 基于深度强化学习的电力市场量价组合竞价策略[J]. 电网技术, 2024, 48(8):3278-3286.
|
| [28] |
吴琪, 赵宣茗, 张佳诚, 等. 促进新能源消纳的电-碳市场耦合激励型出清机制[J]. 电力建设, 2023, 44(12): 14-27.
碳市场的推进有助于“双碳”目标的实现，电力行业将面临电-碳市场耦合的多主体多市场协同的交易格局，其中，环境成本通过碳市场传导进入电市场。然而，在当前的市场出清机制中，发电商的环境效益未能有效纳入，导致以新能源为主的低排放机组出清结果不理想，影响“双碳”目标实现。为此，通过引入激励因子调整机组发电投标策略，在电-碳市场耦合背景下，设计了以促进新能源消纳为目标的激励型出清机制。以此建立双层模型：上层对异质发电商投标策略建模，下层对激励型市场出清机制进行建模，通过深度强化学习算法求解模型。算例分析表明，设计模型能够正确描述耦合市场之间的相互影响，设计的出清机制提升了新能源机组的发电比重，促进了新能源消纳，提升了电力行业社会效益和减排效益。
Promoting the carbon market will help achieve carbon peaking and carbon neutrality goals. The power industry will face a multi-entity and multi-market coordinated trading pattern of electricity-carbon market coupling, in which environmental costs are transmitted into the electricity market through the carbon market. However, in the current market-clearing mechanism, the environmental benefits of power generators have not been effectively included, resulting in unsatisfactory results for low-emission units, mainly based on new energy sources, which affects the realization of goals. Therefore, by introducing incentive factors to adjust the bidding strategy of unit power generation, an incentive-clearing mechanism to promote the absorption of new energy was designed under the background of electricity-carbon market coupling. Thus, a two-layer model is established: the upper layer models the bidding strategy of heterogeneous power generators, and the lower layer models the incentive market-clearing mechanism. The model is solved with a deep reinforcement learning algorithm. The analysis indicates that the design model can correctly describe the interaction between the coupling markets and that the designed clearing mechanism increases the proportion of power generation in new energy units, promotes the absorption of new energy, and improves the social and emission reduction benefits of the power industry.
|
| [29] |
彭春华, 易泰洵, 孙惠娟, 等. 基于多智能体深度确定性策略梯度的电碳耦合市场发电商均衡竞价策略[J]. 电网技术, 2023, 47(10): 4229-4239.
|
| [30] |
|
| [31] |
张兴平, 王腾, 张馨月, 等. 基于多智能体深度确定策略梯度算法的火力发电商竞价策略[J]. 中国电力, 2024, 57(11): 161-172.
|
| [32] |
|
| [33] |
赵会茹, 赵一航, 武昭原, 等. 电力市场、碳排放权交易市场以及核证自愿减排市场耦合下发电商竞价策略[J]. 电力建设, 2024, 45(10): 123-135.
|
| [34] |
张硕, 张家源, 李英姿, 等. 多元主体驱动的新型电力系统市场化运行仿真关键技术[J]. 电力建设, 2024, 45(9): 178-188.
在“双碳”目标下,市场化是实现新型电力系统可持续发展的有效途径,建模仿真因其可分析动态过程且易于反复实验的优点,成为该领域的重要研究方法。目前电力市场仿真缺乏全场景全过程的研究,且难以满足多元市场主体的建模仿真需求。基于此,文章进行了多元主体参与电力市场的行为模型构建,提出了仿真系统开发的基本框架和关键技术并进行了应用。首先,对新型电力系统主要构成要素进行了定义,构建了智能主体层次行为模型,并结合强化学习原理对其决策过程进行了细化;其次,提出了模型构建、市场化运行仿真、数据库和仿真分析等关键技术,并进行仿真系统的开发和应用,完整模拟了多元主体驱动电力系统演化的过程;最后,对仿真技术的后续发展方向进行了展望。
Aiming at the dual-carbon goal, marketization is an effective approach to achieve the safe and stable operation as well as the sustainable development of a new power system. Simulation has become an important research method in this field because it enables the analysis of dynamic processes and facilitates the repetition of experiments. At present, power market simulations do not investigate the whole scene and process. Thus, it is difficult to meet the modeling and simulation specifications of multiple market players. Therefore, this study constructed a behavior model of multi-subject participation in the electricity market, setting and applying the basic framework and key technologies of simulation system development. First, the main components of a new power system were defined, the hierarchical behavior model of intelligent agents was constructed, and the decision-making process was refined based on the principle of reinforcement learning. Second, key technologies such as model construction, market-oriented operation simulation, database, and simulation analysis were explored, and the development and application of simulation systems were performed. The evolution process of a multi-subject driving power system was completely simulated. Finally, the prospects of future development directions of simulation technology were studied. |
| [35] |
张继行, 张一, 王旭, 等. 基于多代理强化学习的多新型市场主体虚拟电厂博弈竞价及效益分配策略[J]. 电网技术, 2024, 48(5): 1980-1991.
|
| [36] |
汪鹏. 风光新能源发电商参与多市场耦合交易模型研究[D]. 北京: 华北电力大学, 2023.
|
| [37] |
|
| [38] |
龚开, 王旭, 邓晖, 等. 现货市场环境下基于深度强化学习的光储联合电站储能系统最优运行方法[J]. 电网技术, 2022, 46(9): 3365-3377.
|
| [39] |
李超英. 基于多智能体的电力市场发电主体运营策略优化研究[D]. 北京: 华北电力大学, 2023.
|
AI小编
/
| 〈 |
|
〉 |
AI 导读
