Research on Two-Layer Configuration and Operation Optimization Based on Proximal Policy Optimization for Electrochemical/Hydrogen Hybrid Energy Storage System

Qingyou YAN; Chaofan SHI; Guangyu QIN; Chuanbo XU

doi:10.12204/j.issn.1000-7229.2022.08.003

PDF(3910 KB)

Electric Power Construction ›› 2022, Vol. 43 ›› Issue (8) : 22-32. DOI: 10.12204/j.issn.1000-7229.2022.08.003

Planning, Configuration and Operation Control of Energy Storage System under the Background of New Power Systems•Hosted by Professor-level Senior Engineer LI Xiangjun, Professor SHUAI Zhikang and Associate Professor YAN Ning•

Research on Two-Layer Configuration and Operation Optimization Based on Proximal Policy Optimization for Electrochemical/Hydrogen Hybrid Energy Storage System

Qingyou YAN ¹^,² ,
Chaofan SHI ¹^,² ,
Guangyu QIN ¹^,²^,³ ,
Chuanbo XU ¹^,²

Author information +

History +

Abstract

According to the complementary characteristics of electrochemical energy storage and hydrogen storage, an integrated optimization model for the configuration and operation of a hybrid energy storage system is given, including electrochemical energy storage, hydrogen storage proposed and an intelligent algorithm. The model is based on a two-layer decision optimization problem, in which two different time dimensions of the hybrid energy storage system configuration and operation are solved in upper and lower layers, and the interaction between them is considered. A reinforcement learning proximal policy optimization (PPO) algorithm is used to solve the two-layer optimization model. By comparing the results of applying various traditional algorithms to solve the scenery data of a region in Gansu Province, it is verified that the used algorithm has the highest adaptability and the fastest convergence speed in a complex environment. The results show that the application of this model can reduce the abandoning rate of wind and solar power by 24% and effectively improve the comprehensive benefit of the system, and that hydrogen storage as a capacity-based energy storage configuration is not limited by topographical factors and is suitable for diverse application scenarios, thus providing an application demonstration for the widespread deployment of hydrogen storage, a new form of energy storage, in the whole country.

Key words

wind-solar consumption / energy storage configuration / two-level optimization / hydrogen energy storage / proximal policy optimization (PPO) algorithm

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Qingyou YAN , Chaofan SHI , Guangyu QIN , et al. Research on Two-Layer Configuration and Operation Optimization Based on Proximal Policy Optimization for Electrochemical/Hydrogen Hybrid Energy Storage System[J]. Electric Power Construction. 2022, 43(8): 22-32 https://doi.org/10.12204/j.issn.1000-7229.2022.08.003

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]	孙玉树, 杨敏, 师长立, 等. 储能的应用现状和发展趋势分析[J]. 高电压技术, 2020, 46(1): 80-89. SUN Yushu, YANG Min, SHI Changli, et al. Analysis of application status and development trend of energy storage[J]. High Voltage Engineering, 2020, 46(1): 80-89. Cited in this article [1]

[2]	王辉, 梁登香, 韩晓娟. 储能参与泛在电力物联网辅助服务应用综述[J]. 发电技术, 2021, 42(2): 171-179. WANG Hui, LIANG Dengxiang, HAN Xiaojuan. Reviews of energy storage participating in auxiliary services under ubiquitous internet of things[J]. Power Generation Technology, 2021, 42(2): 171-179. Cited in this article [1]

[3]	叶季蕾, 李斌, 张宇, 等. 基于全球能源互联网典型特征的储能需求及配置分析[J]. 发电技术, 2021, 42(1): 20-30. YE Jilei, LI Bin, ZHANG Yu, et al. Energy storage requirements and configuration analysis based on typical characteristics of global energy internet[J]. Power Generation Technology, 2021, 42(1): 20-30. Cited in this article [1]

[4]

国家电网公司“电网新技术前景研究”项目咨询组. 大规模储能技术在电力系统中的应用前景分析[J]. 电力系统自动化, 2013, 37(1): 3-8.

Consulting Group of State Grid Corporation of China to Prospects of New Technologies in Power Systems. An analysis of prospects for application of large-scale energy storage technology in power systems[J]. Automation of Electric Power Systems, 2013, 37(1): 3-8.

Cited in this article [1]

[5]

胡娟, 杨水丽, 侯朝勇, 等. 规模化储能技术典型示范应用的现状分析与启示[J]. 电网技术, 2015, 39(4): 879-885.

Juan

, YANG

Shuili

, HOU

Chaoyong

, et al. Present condition analysis on typical demonstration application of large-scale energy storage technology and its enlightenment[J]. Power System Technology, 2015, 39(4): 879-885.

Cited in this article [1]

[6]	严晓辉, 徐玉杰, 纪律, 等. 我国大规模储能技术发展预测及分析[J]. 中国电力, 2013, 46(8): 22-29. YAN Xiaohui, XU Yujie, JI Lü, et al. Forecasting and analysis on large-scale energy storage technologies in China[J]. Electric Power, 2013, 46(8): 22-29. Cited in this article [1]

[7]	郑乐, 胡伟, 陆秋瑜, 等. 储能系统用于提高风电接入的规划和运行综合优化模型[J]. 中国电机工程学报, 2014, 34(16): 2533-2543. ZHENG Le, HU Wei, LU Qiuyu, et al. Research on planning and operation model for energy storage system to optimize wind power integration[J]. Proceedings of the CSEE, 2014, 34(16): 2533-2543. Cited in this article [1]

[8]	许健, 刘念, 于雷, 等. 计及重要负荷的工业光伏微电网储能优化配置[J]. 电力系统保护与控制, 2016, 44(9): 29-37. XU Jian, LIU Nian, YU Lei, et al. Optimal allocation of energy storage system of PV microgrid for industries considering important load[J]. Power System Protection and Control, 2016, 44(9): 29-37. Cited in this article [1]

[9]

陶琼, 王德顺, 叶季蕾, 等. 考虑储能配置模式的多数据源融合分布式光伏发电并网接纳分析方法[J]. 高电压技术, 2018, 44(4): 1093-1098.

TAO

Qiong

, WANG

Deshun

, YE

Jilei

, et al. Capacity analysis of distributed photovoltaic generation integrated into power grid considering energy storage configuration mode based on fusion of multiple data sources[J]. High Voltage Engineering, 2018, 44(4): 1093-1098.

Cited in this article [1]

[10]

徐飞, 陈磊, 金和平, 等. 抽水蓄能电站与风电的联合优化运行建模及应用分析[J]. 电力系统自动化, 2013, 37(1): 149-154.

Fei

, CHEN

Lei

, JIN

Heping

, et al. Modeling and application analysis of optimal joint operation of pumped storage power station and wind power[J]. Automation of Electric Power Systems, 2013, 37(1): 149-154.

Cited in this article [1]

[11]

戴远航, 陈磊, 闵勇, 等. 风电场与含储热的热电联产联合运行的优化调度[J]. 中国电机工程学报, 2017, 37(12): 3470-3479, 3675.

DAI

Yuanhang

, CHEN

Lei

, MIN

Yong

, et al. Optimal dispatch for joint operation of wind farm and combined heat and power plant with thermal energy storage[J]. Proceedings of the CSEE, 2017, 37(12): 3470-3479, 3675.

Cited in this article [1]

[12]	夏沛, 邓长虹, 龙志君, 等. 含抽水蓄能机组的风电消纳鲁棒机组组合[J]. 电力系统自动化, 2018, 42(19): 41-49. XIA Pei, DENG Changhong, LONG Zhijun, et al. Robust unit commitment with pumped storage units for wind power accommodation[J]. Automation of Electric Power Systems, 2018, 42(19): 41-49. Cited in this article [1]

[13]	张文建, 崔青汝, 李志强, 等. 电化学储能在发电侧的应用[J]. 储能科学与技术, 2020, 9(1): 287-295. ZHANG Wenjian, CUI Qingru, LI Zhiqiang, et al. Application of electrochemical energy storage in power generation[J]. Energy Storage Science and Technology, 2020, 9(1): 287-295. Cited in this article [1]

[14]

蔡霁霖, 徐青山, 袁晓冬, 等. 基于风电消纳时序场景的电池储能系统配置策略[J]. 高电压技术, 2019, 45(3): 993-1001.

CAI

Jilin

, XU

Qingshan

, YUAN

Xiaodong

, et al. Configuration strategy of large-scale battery storage system orienting wind power consumption based on temporal scenarios[J]. High Voltage Engineering, 2019, 45(3): 993-1001.

Cited in this article [1]

[15]

刘闯, 孙同, 蔡国伟, 等. 基于同步机三阶模型的电池储能电站主动支撑控制及其一次调频贡献力分析[J]. 中国电机工程学报, 2020, 40(15): 4854-4866.

LIU

Chuang

, SUN

Tong

, CAI

Guowei

, et al. Third-order synchronous machine model based active support control of battery storage power plant and its contribution analysis for primary frequency response[J]. Proceedings of the CSEE, 2020, 40(15): 4854-4866.

Cited in this article [1]

[16]	周荣昱. 基于小生境多目标粒子群优化算法的配电网混合储能配置研究[J]. 电气开关, 2021, 59(6): 35-41. ZHOU Rongyu. Optimal allocation of hybrid energy storage in distribution networks based on niche multi-objective particle swarm optimization algorithm[J]. Electric Switchgear, 2021, 59(6): 35-41. Cited in this article [1]

[17]	王玥娇, 张兴友, 郭俊山. 储能技术在高比例可再生能源电力系统中的应用[J]. 山东电力技术, 2021, 48(7): 19-25. WANG Yuejiao, ZHANG Xingyou, GUO Junshan. Review on the application of energy storage technology in power system with high proportion of renewable energy[J]. Shandong Electric Power, 2021, 48(7): 19-25. Cited in this article [1]

[18]

何颖源, 陈永翀, 刘勇, 等. 储能的度电成本和里程成本分析[J]. 电工电能新技术, 2019, 38(9): 1-10.

Yingyuan

, CHEN

Yongchong

, LIU

Yong

, et al. Analysis of cost per kilowatt-hour and cost per mileage for energy storage technologies[J]. Advanced Technology of Electrical Engineering and Energy, 2019, 38(9): 1-10.

Cited in this article [1]

[19]

彭春华, 余愿, 孙惠娟. 基于源网荷协同优化的配电网光储联合系统规划[J]. 电网技术, 2019, 43(11): 3944-3951.

PENG

Chunhua

, YU

Yuan

, SUN

Huijuan

. Planning of combined PV-ESS system for distribution network based on source-network-load collaborative optimization[J]. Power System Technology, 2019, 43(11): 3944-3951.

Cited in this article [1]

[20]	刘冠群, 袁越, 王敏, 等. 考虑经济成本的光伏电站储能容量配置[J]. 可再生能源, 2014, 32(1): 1-5. LIU Guanqun, YUAN Yue, WANG Min, et al. Energy storage capacity determining of PV plant considering economic cost[J]. Renewable Energy Resources, 2014, 32(1): 1-5. Cited in this article [1]

[21]

蒋伟, 陈照光, 颜浩. 基于改进人工蜂群算法的家庭储能容量优化配置[J/OL]. 电测与仪表: 1-7 (2021-12-20)[2022-04-02]. http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html.

http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html

JANG

Wei

, CHEN

Zhaoguang

, YAN

Hao

. Optimal allocation of household energy storage capacity based on improved artificial bee colony algorithm[J/OL]. Electrical Measurement & Instrumentation: 1-7(2021-12-20)[2022-04-02]. http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html.

http://kns.cnki.net/kcms/detail/23.1202.TH.20211218.0938.002.html

Cited in this article [1]

[22]	王佳蕊, 李德鑫, 张家郡, 等. 微电网储能系统控制对电能质量的保障[J]. 哈尔滨理工大学学报, 2021, 26(6): 87-93. WANG Jiarui, LI Dexin, ZHANG Jiajun, et al. The control of microgrid energy storage system guarantees the power quality[J]. Journal of Harbin University of Science and Technology, 2021, 26(6): 87-93. Cited in this article [1]

[23]	杨晓雷, 丁磊明, 李逸鸿, 等. 计及风电不确定性的混合储能系统优化配置研究[J]. 电力需求侧管理, 2021, 23(6): 69-74. YANG Xiaolei, DING Leiming, LI Yihong, et al. Study on optimal allocation of hybrid energy storage system considering wind power uncertainty[J]. Power Demand Side Management, 2021, 23(6): 69-74. Cited in this article [1]

[24]

许传博, 赵云灏, 王晓晨, 等. 碳中和愿景下考虑电氢耦合的风光场站氢储能优化配置[J]. 电力建设, 2022, 43(1): 10-18.

Chuanbo

, ZHAO

Yunhao

, WANG

Xiaochen

, et al. Optimal configuration of hydrogen energy storage for wind and solar power stations considering electricity-hydrogen coupling under carbon neutrality vision[J]. Electric Power Construction, 2022, 43(1): 10-18.

Cited in this article [3]

[25]	李红, 朱立位, 郭朝辉. 基于深度强化学习的混合储能系统最优控制[J]. 电器与能效管理技术, 2021(11): 42-49. LI Hong, ZHU Liwei, GUO Zhaohui. Optimal control of hybrid energy storage system based on deep reinforcement learning[J]. Electrical & Energy Management Technology, 2021(11): 42-49. Cited in this article [1]

[26]

聂欢欢, 张家琦, 陈颖, 等. 基于双层强化学习方法的多能园区实时经济调度[J]. 电网技术, 2021, 45(4): 1330-1336.

NIE

Huanhuan

, ZHANG

Jiaqi

, CHEN

Ying

, et al. Real-time economic dispatch of community integrated energy system based on a double-layer reinforcement learning method[J]. Power System Technology, 2021, 45(4): 1330-1336.

Cited in this article [1]

[27]

C B

, KE

Y M

, LI

Y B

, et al. Data-driven configuration optimization of an off-grid wind/PV/hydrogen system based on modified NSGA-II and CRITIC-TOPSIS[J]. Energy Conversion and Management, 2020, 215: 112892.

https://doi.org/10.1016/j.enconman.2020.112892

https://linkinghub.elsevier.com/retrieve/pii/S0196890420304301

Cited in this article [2]