PDF(9432 KB)
基于条件风险强化学习的梯级水光蓄联合优化调度
陈实, 唐国登, 刘艺洪, 许刘超, 朱钰杰, 周毅, 李华强, 臧天磊
电力建设 ›› 2025, Vol. 46 ›› Issue (9) : 84-97.
PDF(9432 KB)
PDF(9432 KB)
基于条件风险强化学习的梯级水光蓄联合优化调度
Condition-Based Risk Reinforcement Learning for Joint Optimal Scheduling of Cascade Hydropower and Solar-Powered Reservoirs
【目的】随着“双碳”目标的推进，水电开发商积极推进所在流域水力发电抽蓄化改造的同时，加大区域光伏等新能源的建设力度，逐渐形成高比例新能源与大规模抽水蓄能并存的局面。但在新能源出力与区间来水的双重不确定性影响下，系统运行配合更加复杂，尤其是考虑区间来水不确定后含蓄梯级水电调度方法鲜有研究。【方法】提出一种基于条件风险强化学习的梯级水光蓄联合优化调度方法，通过Informer深度神经网络预测流域区间来水，进一步将区间来水不确定性转化为灵活性供给指标；再结合风险理论，用条件风险价值（conditional value at risk， CVaR）量化灵活性缺额；最后提出考虑条件风险价值的近端策略优化（risk-managing proximal policy optimization， RM-PPO）强化学习算法进行求解以获得优化调度策略。【结果】以中国某实际梯级水光蓄基地为例进行验证，所提预测方法相较仅考虑时序的传统方法，均方误差（mean-square error，MSE）降低18.9%，平均绝对误差（mean absolute error， MAE）降低58.8%，尖峰事件捕获率为78.5%；所提RM-PPO调度算法实现了各站对系统灵活性的调控，抽蓄电站与光伏联动消纳，传统水电站与光伏协同源荷匹配。【结论】所提RM-PPO调度策略能够降低成本、控制风险，并保持系统运行灵活性，促进新能源消纳的同时提高水能利用率。
[Objective] Owing to the advancement of the dual-carbon strategy， hydropower developers are actively promoting the pumped-storage retrofitting of hydroelectric facilities while accelerating the regional construction of photovoltaic （PV） and other renewable-energy sources， thus gradually forming a scenario with high-penetration renewables and large-scale pumped storage. However， under the dual uncertainties of renewable-energy output and interval water inflow， the system operational coordination becomes more complex， particularly because scheduling methods for cascade hydropower systems that consider interval inflow uncertainty are rarely investigated. [Methods] This study proposes a conditional risk-aware reinforcement-learning-based optimal scheduling method for cascade hydro-PV-pumped storage systems. First， the “Informer” deep neural network is employed to predict basin-interval water inflow， where inflow uncertainty is transformed into flexibility supply indicators. Subsequently， risk theory is integrated to quantify flexibility deficits using the conditional-value-at-risk measure. Finally， an improved risk-managing proximal policy optimization （RM-PPO） reinforcement-learning algorithm is developed to derive optimized scheduling strategies. [Results] Validation using an actual cascade hydro-PV-pumped storage base in China shows that the proposed forecasting method reduces the MSE and MAE by 18.9% and 58.8%， respectively， compared with conventional time-series approaches， with a 78.5% capture rate for peak events. The RM-PPO scheduling algorithm achieves flexible system regulation through pumped-storage plants coordinated with PVs for surplus-energy absorption， whereas conventional hydropower stations synergize with PVs for source-load dynamic matching. [Conclusions] The RM-PPO-based scheduling strategy effectively reduces costs and controls operational risks while maintaining system flexibility， thus promoting renewable-energy accommodation and enhancing hydropower-utilization efficiency.
cascade hydropower-photovoltaic storage / conditional risk / reinforcement learning / scheduling optimization
| [1] |
尚博文, 徐铭铭, 张金帅, 等. 高比例分布式光伏接入背景下配电网电压调控方法研究综述[J]. 智慧电力, 2024, 52(12): 1-11.
|
| [2] |
孙志媛, 彭博雅, 孙艳. 考虑多能互补的电力电量平衡优化调度策略[J]. 中国电力, 2024, 57(1): 115-122.
|
| [3] |
任大伟, 侯金鸣, 肖晋宇, 等. 支撑双碳目标的新型储能发展潜力及路径研究[J]. 中国电力, 2023, 56(8): 17-25.
|
| [4] |
吴智泉, 贾纯超, 陈磊, 等. 新型电力系统中储能创新方向研究[J]. 太阳能学报, 2021, 42(10): 444-451.
|
| [5] |
姜爱军, 方品政, 胡南, 等. 抽水蓄能电站静止变频起动谐波特性分析及抑制策略[J]. 智慧电力, 2024, 52(8): 113-121.
|
| [6] |
申建建, 王月, 程春田, 等. 水风光多能互补发电调度问题研究现状及展望[J]. 中国电机工程学报, 2022, 42(11): 3871-3885.
|
| [7] |
赵明哲. 雅砻江流域清洁能源基地多尺度风光水互补调度与灵活性研究[D]. 西安: 西安理工大学, 2022.
|
| [8] |
|
| [9] |
程孟增, 唐一金, 商文颖, 等. 风-光-火-抽蓄联合系统中抽水蓄能电站最佳容量配置[J]. 电力建设, 2021, 42(11): 72-81.
针对含风电、光伏、火电和抽蓄联合系统中抽水蓄能电站的容量配置问题,构建了联合系统双层规划模型,并提出了求解流程。上层模型以弃风、弃光电量之和最小为目标,确定抽水蓄能电站的容量配置问题,下层模型旨在最大化联合系统的经济效益、环境价值,同时改善系统运行条件,解决抽水蓄能电站的运行调度问题,上下层目标采用基于Tent映射混沌优化的改进灰狼算法进行求解。通过对某地区冬季和夏季2个典型日场景进行仿真分析,验证了模型与算法的有效性,结果表明,所构建的双层规划模型对科学确定联合系统中抽水蓄能电站的容量是有效的,并在满足运行调度优化的条件下,能够改善系统运行条件,实现经济效益与环境价值最大化的预期目标。
Aiming at the capacity allocation of pumped-storage power stations in the association system of wind power, photovoltaic, thermal power and pumped storage, a bi-level programming of association system model is constructed and a solve loop is proposed. The upper-level model takes the minimum total curtailment of photovoltaic and wind power as the goal to determine the capacity allocation of the pumped-storage power station. The lower-level model aims to maximize the economic and environmental benefits of the association system, while improving the operating conditions of the system and solving the operation scheduling problem of pumped-storage power stations. The targets of the upper and lower-level are solved by the improved grey wolf optimizer (GWO) based on chaos optimization of Tent map. The effectiveness of the model and algorithm is verified by simulation analysis of two typical daily scenes in a certain area in winter and summer. The results show that the constructed bi-level programming model is effective for scientifically determining the capacity of pumped-storage power stations in the association system. And under the condition of satisfying the optimization of operation scheduling, it can improve the operating conditions of the system and realize the expected goal of maximizing economic and environmental benefits. |
| [10] |
陈雨潇, 龚锦霞, 赵文彬, 等. 面向新型电力系统的变速抽蓄电站调频策略研究[J]. 智慧电力, 2023, 51(3): 104-110.
|
| [11] |
周云海, 张智颖, 徐飞, 等. 含多座抽水蓄能电站的省级电网日前计划分层递进优化算法[J]. 中国电力, 2023, 56(5): 41-50.
|
| [12] |
刘德旭, 杨迎, 黄宏旭, 等. 新型电力系统大规模抽水蓄能调度运行与控制综述及展望[J]. 中国电机工程学报, 2025, 45(1): 80-98.
|
| [13] |
罗彬, 陈永灿, 刘昭伟, 等. 梯级水光互补系统最大化可消纳电量期望短期优化调度模型[J]. 电力系统自动化, 2023, 47(10): 66-75.
|
| [14] |
李咸善, 杨拯, 李飞, 等. 基于梯级水电调节的风光水联盟与区域电网联合运行优化调度策略[J]. 中国电机工程学报, 2023, 43(6): 2234-2248.
|
| [15] |
曹敏健, 胡泽春, 孟颖, 等. 含抽蓄电站与新能源发电的柔性直流系统日前优化调度方法[J]. 电力系统自动化, 2021, 45(15): 36-44.
|
| [16] |
夏依莎, 刘俊勇, 刘继春, 等. 基于电量共享的梯级水光蓄联合发电系统优化调度策略[J]. 电力自动化设备, 2021, 41(9): 118-125.
|
| [17] |
葛晓琳, 钟俊玲, 符杨. 考虑来水多区间不确定集的跨年梯级水电鲁棒优化调度[J]. 中国电机工程学报, 2018, 38(17): 5097-5106, 5306.
|
| [18] |
赵志鹏, 于志辉, 程春田, 等. 水风光综合基地多风险量化及长期多目标协调优化调度方法[J]. 电力系统自动化, 2024, 48(22): 118-130.
|
| [19] |
孙惠娟, 巩磊, 彭春华, 等. 考虑风光水多重不确定性置信风险的多目标动态分解优化调度[J]. 电网技术, 2022, 46(9): 3416-3428.
|
| [20] |
马宇航, 黄媛, 刘俊勇, 等. 考虑日内来水不确定和电网断面约束的梯级水电日前调度[J]. 电力建设, 2020, 41(9): 39-49.
日内来水的不确定性和传输通道断面约束是影响大容量梯级水电日前调度计划制定的重要因素。为提高日前调度对不同来水情景的适应性,首先在日均径流预测基础上,结合历史数据分析与概率分布拟合获取日内来水量的概率密度函数,通过准确刻画各时段来水规律,建立不同情景下的梯级发电量偏差来反映各时刻来水的不确定性。其次,在模型断面约束中引入模糊变量,并通过离线仿真将断面约束转化为发电机有功出力约束。针对含梯级电站发电量偏差的日前优化调度模型,采用多核并行动态规划算法进行求解,保证求解的收敛性及求解效率。最后,以西南地区某梯级水电站接入IEEE 39节点系统为例,验证了所提算法的合理性和可行性,所得调度计划在4种典型来水场景下均未出现水位及断面功率越限。
The uncertainties of daily interval flow and the restriction of the transmission section are important factors that affect the day-ahead dispatching plan of large-capacity cascade hydropower stations. In order to improve the adaptability of the daily dispatching to different inflow scenarios, the probability density function of daily interval flow is obtained on the basis of the forecast daily average interval flow firstly, combining historical data analysis and probability distribution fitting. Through accurately describing the rule of interval flow in each period, the deviation of cascade power generation under different scenarios is established to reflect the uncertainty of the interval flow at each period. Secondly, fuzzy variables are established to express the complex section constraint relationship in the day-ahead dispatching, and the section constraint is converted to the generator’s active power constraint through off-line simulation. The multi-core parallel dynamic programming algorithm is adopted to solve the day-ahead dispatching model to ensure the convergence and efficiency of the solution. Finally, the rationality and feasibility of the proposed algorithm are verified through the example of cascade hydropower stations in southwest China connected to IEEE 39-node system, and the water level and transmission section power are not out of limits in the four typical inflow scenarios. |
| [21] |
|
| [22] |
王辉, 王军杰, 王腾飞, 等. 出力不确定下基于CVaR的风电供应链优化决策[J]. 电力科学与技术学报, 2023, 38(4): 134-142.
|
| [23] |
周新颖. 数据驱动的电力批发市场竞价策略研究[D]. 北京: 华北电力大学, 2023.
|
| [24] |
高万胜, 蔺红. 考虑配电网灵活性不足风险的分布鲁棒低碳优化调度[J]. 电力系统保护与控制, 2024, 52(16): 49-61.
|
| [25] |
曾小青, 唐超雯. 分时电价环境下计及新能源消纳的虚拟电厂优化调度研究[J]. 电力科学与技术学报, 2023, 38(3): 24-34.
|
| [26] |
周耀辉, 司渭滨, 代立君, 等. 基于随机规划的新型电力系统优化调度方法[J/OL]. 自动化技术与应用, 2024: 1-6. (2024-11-26) [2025-01-12]. https://kns.cnki.net/KCMS/detail/detail.aspx?filename=ZDHJ20241122006&dbname=CJFD&dbcode=CJFQ.
|
| [27] |
杜倩昀, 周升, 李祖鑫, 等. 考虑天然来水量预报的小水电站富集型系统汛期优化调度策略[J]. 电力科学与技术学报, 2024, 39(6): 33-42.
|
| [28] |
何鑫, 刘俊勇, 高红均, 等. 计及光伏不确定性和低碳需求响应的工业园区光-储分布鲁棒规划模型[J]. 智慧电力, 2023, 51(2): 69-76.
|
| [29] |
闻昕, 刘凡骞, 谭乔凤, 等. 复杂水利-电力约束下梯级水电站短期多层级多目标优化调度研究[J]. 水利学报, 2023, 54(4): 392-404.
|
| [30] |
陈实, 朱亚斌, 刘艺洪, 等. 基于世界模型深度强化学习的含风电电力系统低碳经济调度[J]. 电网技术, 2024, 48(8): 3143-3154.
|
| [31] |
|
| [32] |
刘云飞, 张俊然. 深度神经网络学习率策略研究进展[J]. 控制与决策, 2023, 38(9): 2444-2460.
|
| [33] |
|
| [34] |
马帅, 夏俐. 风险敏感马氏决策过程与状态扩充变换[J]. 中山大学学报(自然科学版)(中英文), 2023, 62(1): 181-191.
|
| [35] |
李静轩, 周明, 朱凌志, 等. 可再生能源电力系统运行灵活性需求量化及优化调度方法[J]. 电网技术, 2021, 45(7): 2647-2656.
|
| [36] |
|
| [37] |
雷嘉明, 姜爱华, 吴新飞, 等. 计及源荷不确定性的综合能源系统近端策略优化调度[J]. 电力科学与技术学报, 2023, 38(5): 1-11.
|
| [38] |
潘超, 王锦鹏, 包钰婷, 等. 考虑源荷侧灵活性资源的风光消纳互动调控[J]. 智慧电力, 2023, 51(1): 1-8.
|
| [39] |
陈治铭, 刘建华, 柯添赐, 等. 一种对抗性的权重注意力机制Seq2Seq模型估算SOC[J/OL]. 电工技术学报, 2023: 1-12. (2023-11-20) [2025-01-12]. https://kns.cnki.net/KCMS/detail/detail.aspx?filename=DGJS20231113001&dbname=CJFD&dbcode=CJFQ.
|
| [40] |
国家能源局. 关于印发《水能利用率计算导则》的通知:国能发新能[2017]4号[EB/OL]. (2017-05-27)[2024-11-12]. https://zfxxgk.nea.gov.cn/auto87/201706/t20170608_2802.htm
|
AI小编
/
| 〈 |
|
〉 |
AI 导读
