Transmission Network Planning Method Based on Deep Reinforcement Learning

doi:10.12204/j.issn.1000-7229.2021.07.012

Abstract

Abstract:

In order to solve the problem that the existing transmission network planning methods are not flexible enough in multiple scenarios, and to improve the operation efficiency of the planning methods, this paper proposes a transmission network planning method based on deep reinforcement learning. Firstly, this paper uses scenario information entropy to generate a variety of typical scenarios, and establishes a flexible transmission network planning model suitable for multiple scenarios. Secondly, an improved pointer network model suitable for transmission network planning is proposed by using the deep reinforcement learning method and the actor critical method, and the revised pointer network with Actor-Critic (RPNAC) method is used to solve the planning model. Finally, an example verifies the effectiveness and feasibility of the proposed method.

Key words: transmission network planning, information entropy, multiple scenarios, pointer network, deep reinforcement learning

CLC Number:

TM715

LIU Shuai, KONG Liang, LIU Zifa, LI Yuwen, CHEN Yixuan. Transmission Network Planning Method Based on Deep Reinforcement Learning[J]. ELECTRIC POWER CONSTRUCTION, 2021, 42(7): 101-109.

Figures/Tables 10

Fig.1 Typical scenarios extraction method based on information entropy

Fig.2 Structure of revised pointer network

Table 1 Comparison of different methods with respect to approximating the original scenarios

Table 2 Comparison of different methods with respect to scenario diversity

Fig.3 Network structure of the Garver-6 system

Table 3 Comparison of planning results of typical scenarios

Table 4 Comparison of PSO and RPNAC based on test set

Fig.4 Some of the results of the planning problems

Table 5 Comparison of planning results of Garver-6 system between RPNAC and PSO

Table 6 Data of evaluation indices of experiment 2

References 19

[1]	刘盾盾, 程浩忠, 刘佳, 等. 输电网鲁棒规划研究综述与展望[J]. 电网技术, 2019, 43(1):135-143.
	LIU Dundun, CHENG Haozhong, LIU Jia, et al. Review and prospects of robust transmission expansion planning[J]. Power System Technology, 2019, 43(1):135-143.
[2]	梁子鹏, 陈皓勇, 郑晓东, 等. 考虑风电极限场景的输电网鲁棒扩展规划[J]. 电力系统自动化, 2019, 43(16):58-67.
	LIANG Zipeng, CHEN Haoyong, ZHENG Xiaodong, et al. Robust expansion planning of transmission network considering extreme scenario of wind power[J]. Automation of Electric Power Systems, 2019, 43(16):58-67.
[3]	黄启航, 王秀丽, 齐世雄, 等. 考虑负荷和风电相关性的多场景鲁棒输电网规划[J]. 电力建设, 2018, 39(6):63-70.
	HUANG Qihang, WANG Xiuli, QI Shixiong, et al. Multi-scenario robust transmission planning considering correlation between wind power and load demand[J]. Electric Power Construction, 2018, 39(6):63-70.
[4]	侍红兵, 胥峥, 陈涛, 等. 一种含风电-电池储能的多场景输电网规划方法[J]. 导航与控制, 2020, 19(3):108-114.
	SHI Hongbing, XU Zheng, CHEN Tao, et al. A multi-scenario transmission network planning method with wind power and battery energy storage[J]. Navigation and Control, 2020, 19(3):108-114.
[5]	翟海保, 程浩忠, 陈春霖, 等. 基于最小期望投资悔值的柔性约束电网灵活规划方法[J]. 上海交通大学学报, 2005, 39(S1):27-30.
	ZHAI Haibao, CHENG Haozhong, CHEN Chunlin, et al. The minimal expectant regret concerned electric power network planning under flexible constraints[J]. Journal of Shanghai Jiao Tong University, 2005, 39(S1):27-30.
[6]	张立波, 程浩忠, 曾平良, 等. 基于不确定理论的输电网规划[J]. 电力系统自动化, 2016, 40(16):159-167.
	ZHANG Libo, CHENG Haozhong, ZENG Pingliang, et al. Transmission network planning approaches based on uncertainty theories[J]. Automation of Electric Power Systems, 2016, 40(16):159-167.
[7]	WU P, CHENG H Z, XING J. The interval minimum load cutting problem in the process of transmission network expansion planning considering uncertainty in demand[J]. IEEE Transactions on Power Systems, 2008, 23(3):1497-1506. doi: 10.1109/TPWRS.2008.922573 URL
[8]	宋新甫, 徐龙秀, 张艳, 等. 结合灰色关联分析的熵权VIKOR法输电网规划方案综合评价方法[J]. 电工技术, 2019(11):11-14.
	SONG Xinfu, XU Longxiu, ZHANG Yan, et al. Comprehensive evaluation method of transmission network planning scheme based on entropy weight VIKOR method combined with grey relational analysis[J]. Electric Engineering, 2019(11):11-14.
[9]	MOCANU E, MOCANU D C, NGUYEN P H, et al. On-line building energy optimization using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(4):3698-3708. doi: 10.1109/TSG.5165411 URL
[10]	FRANÇOIS-LAVET V, TARALLA D, ERNST D, et al. Deep reinforcement learning solutions for energy microgrids management[M/OL]. Spain: Pompeu Fabra University, 2016. http://hdl.handle.net/2268/203831.
[11]	FORUZAN E, SOH L K, ASGARPOOR S. Reinforcement learning approach for optimal distributed energy management in a microgrid[J]. IEEE Transactions on Power Systems, 2018, 33(5):5749-5758. doi: 10.1109/TPWRS.59 URL
[12]	李澎, 彭敏放. 基于改进遗传算法的含风电场电力系统无功优化[J]. 现代电子技术, 2020, 43(13):167-171.
	LI Peng, PENG Minfang. Reactive optimization of improved genetic algorithm based power system with wind farm[J]. Modern Electronics Technique, 2020, 43(13):167-171.
[13]	杨宏, 闫玉杰, 王瑜. Beta分布在风电预测误差模型中的适用性[J]. 电测与仪表, 2020, 57(11):37-41, 48.
	YANG Hong, YAN Yujie, WANG Yu. Applicability of Beta distribution on wind power forecast error modeling[J]. Electrical Measurement & Instrumentation, 2020, 57(11):37-41, 48.
[14]	李浩, 钟声远, 王永真, 等. 基于能量与信息耦合的分布式能源系统配置优化方法[J]. 中国电机工程学报, 2020, 40(17):5467-5476.
	LI Hao, ZHONG Shengyuan, WANG Yongzhen, et al. Optimization method on the distributed energy system based on energy and information coupled[J]. Proceedings of the CSEE, 2020, 40(17):5467-5476.
[15]	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1):1-27.
	LIU Quan, ZHAI Jianwei, ZHANG Zongzhang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1):1-27.
[16]	王功鹏, 段萌, 牛常勇. 基于卷积神经网络的随机梯度下降算法[J]. 计算机工程与设计, 2018, 39(2):441-445.
	WANG Gongpeng, DUAN Meng, NIU Changyong. Stochastic gradient descent algorithm based on convolution neural network[J]. Computer Engineering and Design, 2018, 39(2):441-445.
[17]	车兵, 李轩, 郑建勇, 等. 基于LHS与BR的风电出力场景分析研究[J]. 电力工程技术, 2020, 39(6):213-219.
	CHE Bing, LI Xuan, ZHENG Jianyong, et al. Scenario analysis of wind power output based on LHS and BR[J]. Electric Power Engineering Technology, 2020, 39(6):213-219.
[18]	王蓓蓓, 刘小聪, 李扬. 面向大容量风电接入考虑用户侧互动的系统日前调度和运行模拟研究[J]. 中国电机工程学报, 2013, 33(22):35-44.
	WANG Beibei, LIU Xiaocong, LI Yang. Day-ahead generation scheduling and operation simulation considering demand response in large-capacity wind power integrated systems[J]. Proceedings of the CSEE, 2013, 33(22):35-44.
[19]	王群, 董文略, 杨莉. 基于Wasserstein距离和改进K-medoids聚类的风电/光伏经典场景集生成算法[J]. 中国电机工程学报, 2015, 35(11):2654-2661.
	WANG Qun, DONG Wenlue, YANG Li. A wind power/photovoltaic typical scenario set generation algorithm based on Wasserstein distance metric and revised K-medoids cluster[J]. Proceedings of the CSEE, 2015, 35(11):2654-2661.