基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化

闫庆友; 史超凡; 秦光宇; 许传博

doi:10.12204/j.issn.1000-7229.2022.08.003

PDF(3910 KB)

电力建设 ›› 2022, Vol. 43 ›› Issue (8) : 22-32. DOI: 10.12204/j.issn.1000-7229.2022.08.003

新型电力系统背景下储能系统规划配置与运行控制·栏目主持李相俊教授级高工、帅智康教授、颜宁副教授·

基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化

闫庆友 ¹^,² ,
史超凡 ¹^,² ,
秦光宇 ¹^,²^,³ ,
许传博 ¹^,²

作者信息 +

Research on Two-Layer Configuration and Operation Optimization Based on Proximal Policy Optimization for Electrochemical/Hydrogen Hybrid Energy Storage System

Qingyou YAN ¹^,² ,
Chaofan SHI ¹^,² ,
Guangyu QIN ¹^,²^,³ ,
Chuanbo XU ¹^,²

Author information +

文章历史 +

摘要

针对电化学储能和氢储能的互补特性,提出了一种包含电化学和氢储能的混合储能系统配置和运行的综合优化模型,并提出了智能算法进行求解。该模型基于双层决策优化问题,将混合储能系统配置及运行2个不同时间维度的问题分上下层进行综合求解,并考虑了两者间的相互影响,采用强化学习近端策略优化(proximal policy optimization,PPO)算法求解该双层优化模型。以甘肃省某地区的风光数据,通过对比应用多种传统算法求解结果,验证了所用算法在复杂环境下适应度最高且收敛速度最快。研究结果表明,应用该模型最大可降低24%的弃风、弃光率,有效提升系统综合效益。氢储能作为容量型储能配置不受地形因素限制,适用于多样的应用场景,从而为氢储能这一新型储能形态在全国的广泛配置提供了应用示范。

Abstract

According to the complementary characteristics of electrochemical energy storage and hydrogen storage, an integrated optimization model for the configuration and operation of a hybrid energy storage system is given, including electrochemical energy storage, hydrogen storage proposed and an intelligent algorithm. The model is based on a two-layer decision optimization problem, in which two different time dimensions of the hybrid energy storage system configuration and operation are solved in upper and lower layers, and the interaction between them is considered. A reinforcement learning proximal policy optimization (PPO) algorithm is used to solve the two-layer optimization model. By comparing the results of applying various traditional algorithms to solve the scenery data of a region in Gansu Province, it is verified that the used algorithm has the highest adaptability and the fastest convergence speed in a complex environment. The results show that the application of this model can reduce the abandoning rate of wind and solar power by 24% and effectively improve the comprehensive benefit of the system, and that hydrogen storage as a capacity-based energy storage configuration is not limited by topographical factors and is suitable for diverse application scenarios, thus providing an application demonstration for the widespread deployment of hydrogen storage, a new form of energy storage, in the whole country.

导出引用

闫庆友, 史超凡, 秦光宇, 等. 基于近端策略优化算法的电化学/氢混合储能系统双层配置及运行优化[J]. 电力建设. 2022, 43(8): 22-32 https://doi.org/10.12204/j.issn.1000-7229.2022.08.003

Qingyou YAN, Chaofan SHI, Guangyu QIN, et al. Research on Two-Layer Configuration and Operation Optimization Based on Proximal Policy Optimization for Electrochemical/Hydrogen Hybrid Energy Storage System[J]. Electric Power Construction. 2022, 43(8): 22-32 https://doi.org/10.12204/j.issn.1000-7229.2022.08.003

中图分类号： TM73

参考文献

列表( 原文顺序 | 文献年度倒序 | 文中引用次数倒序 ) 可视化分析

[1]	孙玉树, 杨敏, 师长立, 等. 储能的应用现状和发展趋势分析[J]. 高电压技术, 2020, 46(1): 80-89. SUN Yushu, YANG Min, SHI Changli, et al. Analysis of application status and development trend of energy storage[J]. High Voltage Engineering, 2020, 46(1): 80-89. 本文引用 [1]

[2]	王辉, 梁登香, 韩晓娟. 储能参与泛在电力物联网辅助服务应用综述[J]. 发电技术, 2021, 42(2): 171-179. WANG Hui, LIANG Dengxiang, HAN Xiaojuan. Reviews of energy storage participating in auxiliary services under ubiquitous internet of things[J]. Power Generation Technology, 2021, 42(2): 171-179. 本文引用 [1]

[3]	叶季蕾, 李斌, 张宇, 等. 基于全球能源互联网典型特征的储能需求及配置分析[J]. 发电技术, 2021, 42(1): 20-30. YE Jilei, LI Bin, ZHANG Yu, et al. Energy storage requirements and configuration analysis based on typical characteristics of global energy internet[J]. Power Generation Technology, 2021, 42(1): 20-30. 本文引用 [1]

[4]

国家电网公司“电网新技术前景研究”项目咨询组. 大规模储能技术在电力系统中的应用前景分析[J]. 电力系统自动化, 2013, 37(1): 3-8.

Consulting Group of State Grid Corporation of China to Prospects of New Technologies in Power Systems. An analysis of prospects for application of large-scale energy storage technology in power systems[J]. Automation of Electric Power Systems, 2013, 37(1): 3-8.

本文引用 [1]

[5]

胡娟, 杨水丽, 侯朝勇, 等. 规模化储能技术典型示范应用的现状分析与启示[J]. 电网技术, 2015, 39(4): 879-885.

Juan

, YANG

Shuili

, HOU

Chaoyong

, et al. Present condition analysis on typical demonstration application of large-scale energy storage technology and its enlightenment[J]. Power System Technology, 2015, 39(4): 879-885.

本文引用 [1]

[6]	严晓辉, 徐玉杰, 纪律, 等. 我国大规模储能技术发展预测及分析[J]. 中国电力, 2013, 46(8): 22-29. YAN Xiaohui, XU Yujie, JI Lü, et al. Forecasting and analysis on large-scale energy storage technologies in China[J]. Electric Power, 2013, 46(8): 22-29. 本文引用 [1]

[7]	郑乐, 胡伟, 陆秋瑜, 等. 储能系统用于提高风电接入的规划和运行综合优化模型[J]. 中国电机工程学报, 2014, 34(16): 2533-2543. ZHENG Le, HU Wei, LU Qiuyu, et al. Research on planning and operation model for energy storage system to optimize wind power integration[J]. Proceedings of the CSEE, 2014, 34(16): 2533-2543. 本文引用 [1]

[8]	许健, 刘念, 于雷, 等. 计及重要负荷的工业光伏微电网储能优化配置[J]. 电力系统保护与控制, 2016, 44(9): 29-37. XU Jian, LIU Nian, YU Lei, et al. Optimal allocation of energy storage system of PV microgrid for industries considering important load[J]. Power System Protection and Control, 2016, 44(9): 29-37. 本文引用 [1]

[9]

陶琼, 王德顺, 叶季蕾, 等. 考虑储能配置模式的多数据源融合分布式光伏发电并网接纳分析方法[J]. 高电压技术, 2018, 44(4): 1093-1098.

TAO

Qiong

, WANG

Deshun

, YE

Jilei

, et al. Capacity analysis of distributed photovoltaic generation integrated into power grid considering energy storage configuration mode based on fusion of multiple data sources[J]. High Voltage Engineering, 2018, 44(4): 1093-1098.

本文引用 [1]

[10]	徐飞, 陈磊, 金和平, 等. 抽水蓄能电站与风电的联合优化运行建模及应用分析[J]. 电力系统自动化, 2013, 37(1): 149-154. XU Fei, CHEN Lei, JIN Heping, et al. Modeling and application analysis of optimal joint operation of pumped storage power station and wind power[J]. Automation of Electric Power Systems, 2013, 37(1): 149-154. 本文引用 [1]

[11]

戴远航, 陈磊, 闵勇, 等. 风电场与含储热的热电联产联合运行的优化调度[J]. 中国电机工程学报, 2017, 37(12): 3470-3479, 3675.

DAI

Yuanhang

, CHEN

Lei

, MIN

Yong

, et al. Optimal dispatch for joint operation of wind farm and combined heat and power plant with thermal energy storage[J]. Proceedings of the CSEE, 2017, 37(12): 3470-3479, 3675.

本文引用 [1]

[12]	夏沛, 邓长虹, 龙志君, 等. 含抽水蓄能机组的风电消纳鲁棒机组组合[J]. 电力系统自动化, 2018, 42(19): 41-49. XIA Pei, DENG Changhong, LONG Zhijun, et al. Robust unit commitment with pumped storage units for wind power accommodation[J]. Automation of Electric Power Systems, 2018, 42(19): 41-49. 本文引用 [1]

[13]	张文建, 崔青汝, 李志强, 等. 电化学储能在发电侧的应用[J]. 储能科学与技术, 2020, 9(1): 287-295. ZHANG Wenjian, CUI Qingru, LI Zhiqiang, et al. Application of electrochemical energy storage in power generation[J]. Energy Storage Science and Technology, 2020, 9(1): 287-295. 本文引用 [1]

[14]

蔡霁霖, 徐青山, 袁晓冬, 等. 基于风电消纳时序场景的电池储能系统配置策略[J]. 高电压技术, 2019, 45(3): 993-1001.

CAI

Jilin

, XU

Qingshan

, YUAN

Xiaodong

, et al. Configuration strategy of large-scale battery storage system orienting wind power consumption based on temporal scenarios[J]. High Voltage Engineering, 2019, 45(3): 993-1001.

本文引用 [1]

[15]

刘闯, 孙同, 蔡国伟, 等. 基于同步机三阶模型的电池储能电站主动支撑控制及其一次调频贡献力分析[J]. 中国电机工程学报, 2020, 40(15): 4854-4866.

LIU

Chuang

, SUN

Tong

, CAI

Guowei

, et al. Third-order synchronous machine model based active support control of battery storage power plant and its contribution analysis for primary frequency response[J]. Proceedings of the CSEE, 2020, 40(15): 4854-4866.

本文引用 [1]

[16]	周荣昱. 基于小生境多目标粒子群优化算法的配电网混合储能配置研究[J]. 电气开关, 2021, 59(6): 35-41. ZHOU Rongyu. Optimal allocation of hybrid energy storage in distribution networks based on niche multi-objective particle swarm optimization algorithm[J]. Electric Switchgear, 2021, 59(6): 35-41. 本文引用 [1]

[17]	王玥娇, 张兴友, 郭俊山. 储能技术在高比例可再生能源电力系统中的应用[J]. 山东电力技术, 2021, 48(7): 19-25. WANG Yuejiao, ZHANG Xingyou, GUO Junshan. Review on the application of energy storage technology in power system with high proportion of renewable energy[J]. Shandong Electric Power, 2021, 48(7): 19-25. 本文引用 [1]

[18]	何颖源, 陈永翀, 刘勇, 等. 储能的度电成本和里程成本分析[J]. 电工电能新技术, 2019, 38(9): 1-10. HE Yingyuan, CHEN Yongchong, LIU Yong, et al. Analysis of cost per kilowatt-hour and cost per mileage for energy storage technologies[J]. Advanced Technology of Electrical Engineering and Energy, 2019, 38(9): 1-10. 本文引用 [1]

[19]	彭春华, 余愿, 孙惠娟. 基于源网荷协同优化的配电网光储联合系统规划[J]. 电网技术, 2019, 43(11): 3944-3951. PENG Chunhua, YU Yuan, SUN Huijuan. Planning of combined PV-ESS system for distribution network based on source-network-load collaborative optimization[J]. Power System Technology, 2019, 43(11): 3944-3951. 本文引用 [1]

[20]	刘冠群, 袁越, 王敏, 等. 考虑经济成本的光伏电站储能容量配置[J]. 可再生能源, 2014, 32(1): 1-5. LIU Guanqun, YUAN Yue, WANG Min, et al. Energy storage capacity determining of PV plant considering economic cost[J]. Renewable Energy Resources, 2014, 32(1): 1-5. 本文引用 [1]

[21]

蒋伟, 陈照光, 颜浩. 基于改进人工蜂群算法的家庭储能容量优化配置[J/OL]. 电测与仪表: 1-7 (2021-12-20)[2022-04-02]. http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html.

http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html

JANG

Wei

, CHEN

Zhaoguang

, YAN

Hao

. Optimal allocation of household energy storage capacity based on improved artificial bee colony algorithm[J/OL]. Electrical Measurement & Instrumentation: 1-7(2021-12-20)[2022-04-02]. http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html.

http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html

本文引用 [1]

[22]	王佳蕊, 李德鑫, 张家郡, 等. 微电网储能系统控制对电能质量的保障[J]. 哈尔滨理工大学学报, 2021, 26(6): 87-93. WANG Jiarui, LI Dexin, ZHANG Jiajun, et al. The control of microgrid energy storage system guarantees the power quality[J]. Journal of Harbin University of Science and Technology, 2021, 26(6): 87-93. 本文引用 [1]

[23]	杨晓雷, 丁磊明, 李逸鸿, 等. 计及风电不确定性的混合储能系统优化配置研究[J]. 电力需求侧管理, 2021, 23(6): 69-74. YANG Xiaolei, DING Leiming, LI Yihong, et al. Study on optimal allocation of hybrid energy storage system considering wind power uncertainty[J]. Power Demand Side Management, 2021, 23(6): 69-74. 本文引用 [1]

[24]

许传博, 赵云灏, 王晓晨, 等. 碳中和愿景下考虑电氢耦合的风光场站氢储能优化配置[J]. 电力建设, 2022, 43(1): 10-18.

Chuanbo

, ZHAO

Yunhao

, WANG

Xiaochen

, et al. Optimal configuration of hydrogen energy storage for wind and solar power stations considering electricity-hydrogen coupling under carbon neutrality vision[J]. Electric Power Construction, 2022, 43(1): 10-18.

本文引用 [3]

[25]	李红, 朱立位, 郭朝辉. 基于深度强化学习的混合储能系统最优控制[J]. 电器与能效管理技术, 2021(11): 42-49. LI Hong, ZHU Liwei, GUO Zhaohui. Optimal control of hybrid energy storage system based on deep reinforcement learning[J]. Electrical & Energy Management Technology, 2021(11): 42-49. 本文引用 [1]

[26]

聂欢欢, 张家琦, 陈颖, 等. 基于双层强化学习方法的多能园区实时经济调度[J]. 电网技术, 2021, 45(4): 1330-1336.

NIE

Huanhuan

, ZHANG

Jiaqi

, CHEN

Ying

, et al. Real-time economic dispatch of community integrated energy system based on a double-layer reinforcement learning method[J]. Power System Technology, 2021, 45(4): 1330-1336.

本文引用 [1]

[27]

C B

, KE

Y M

, LI

Y B

, et al. Data-driven configuration optimization of an off-grid wind/PV/hydrogen system based on modified NSGA-II and CRITIC-TOPSIS[J]. Energy Conversion and Management, 2020, 215: 112892.

https://doi.org/10.1016/j.enconman.2020.112892

https://linkinghub.elsevier.com/retrieve/pii/S0196890420304301

本文引用 [2]