基于深度强化学习的输电网网架规划方法

doi:10.12204/j.issn.1000-7229.2021.07.012

摘要/Abstract

摘要：

为解决现有输电网规划方法在多场景情况下存在的灵活性不足的问题,同时进一步提高规划方法的运算效率,文章提出一种基于深度强化学习的输电网规划方法。首先,通过聚类方法,以系统信息熵最小为目标,生成用于规划的电网典型场景,并建立适用于多场景的输电网灵活规划模型。其次,综合运用深度强化学习方法及Actor-Critic方法,提出适用于输电网规划的改进指针网络模型,并采用改进指针网络与Actor-Critic结合的方法(revised pointer network with Actor-Critic, RPNAC)对规划模型进行求解。最后,基于IEEE标准算例进行计算及数据分析,验证了文章所提方法的科学性和高效性。

关键词: 输电网网架规划, 信息熵, 多场景, 指针网络, 深度强化学习

Abstract:

In order to solve the problem that the existing transmission network planning methods are not flexible enough in multiple scenarios, and to improve the operation efficiency of the planning methods, this paper proposes a transmission network planning method based on deep reinforcement learning. Firstly, this paper uses scenario information entropy to generate a variety of typical scenarios, and establishes a flexible transmission network planning model suitable for multiple scenarios. Secondly, an improved pointer network model suitable for transmission network planning is proposed by using the deep reinforcement learning method and the actor critical method, and the revised pointer network with Actor-Critic (RPNAC) method is used to solve the planning model. Finally, an example verifies the effectiveness and feasibility of the proposed method.

Key words: transmission network planning, information entropy, multiple scenarios, pointer network, deep reinforcement learning

中图分类号:

TM715

刘帅, 孔亮, 刘自发, 李玉文, 陈逸轩. 基于深度强化学习的输电网网架规划方法[J]. 电力建设, 2021, 42(7): 101-109.

LIU Shuai, KONG Liang, LIU Zifa, LI Yuwen, CHEN Yixuan. Transmission Network Planning Method Based on Deep Reinforcement Learning[J]. ELECTRIC POWER CONSTRUCTION, 2021, 42(7): 101-109.

图/表 10

图1 基于信息熵的典型场景提取方法

Fig.1 Typical scenarios extraction method based on information entropy

图2 改进指针网络模型结构

Fig.2 Structure of revised pointer network

表1 不同方法提取结果对原场景逼近能力比较

Table 1 Comparison of different methods with respect to approximating the original scenarios

表2 不同方法提取结果场景多样性比较

Table 2 Comparison of different methods with respect to scenario diversity

图3 IEEE Garver-6系统网络结构

Fig.3 Network structure of the Garver-6 system

表3 典型场景规划结果对比

Table 3 Comparison of planning results of typical scenarios

表4 PSO与RPNAC测试集规划结果

Table 4 Comparison of PSO and RPNAC based on test set

图4 测试集问题规划结果

Fig.4 Some of the results of the planning problems

表5 RPNAC和PSO对Garver-6系统规划结果对比

Table 5 Comparison of planning results of Garver-6 system between RPNAC and PSO

表6 试验2结果评价指标数据

Table 6 Data of evaluation indices of experiment 2

参考文献 19

[1]	刘盾盾, 程浩忠, 刘佳, 等. 输电网鲁棒规划研究综述与展望[J]. 电网技术, 2019, 43(1):135-143.
	LIU Dundun, CHENG Haozhong, LIU Jia, et al. Review and prospects of robust transmission expansion planning[J]. Power System Technology, 2019, 43(1):135-143.
[2]	梁子鹏, 陈皓勇, 郑晓东, 等. 考虑风电极限场景的输电网鲁棒扩展规划[J]. 电力系统自动化, 2019, 43(16):58-67.
	LIANG Zipeng, CHEN Haoyong, ZHENG Xiaodong, et al. Robust expansion planning of transmission network considering extreme scenario of wind power[J]. Automation of Electric Power Systems, 2019, 43(16):58-67.
[3]	黄启航, 王秀丽, 齐世雄, 等. 考虑负荷和风电相关性的多场景鲁棒输电网规划[J]. 电力建设, 2018, 39(6):63-70.
	HUANG Qihang, WANG Xiuli, QI Shixiong, et al. Multi-scenario robust transmission planning considering correlation between wind power and load demand[J]. Electric Power Construction, 2018, 39(6):63-70.
[4]	侍红兵, 胥峥, 陈涛, 等. 一种含风电-电池储能的多场景输电网规划方法[J]. 导航与控制, 2020, 19(3):108-114.
	SHI Hongbing, XU Zheng, CHEN Tao, et al. A multi-scenario transmission network planning method with wind power and battery energy storage[J]. Navigation and Control, 2020, 19(3):108-114.
[5]	翟海保, 程浩忠, 陈春霖, 等. 基于最小期望投资悔值的柔性约束电网灵活规划方法[J]. 上海交通大学学报, 2005, 39(S1):27-30.
	ZHAI Haibao, CHENG Haozhong, CHEN Chunlin, et al. The minimal expectant regret concerned electric power network planning under flexible constraints[J]. Journal of Shanghai Jiao Tong University, 2005, 39(S1):27-30.
[6]	张立波, 程浩忠, 曾平良, 等. 基于不确定理论的输电网规划[J]. 电力系统自动化, 2016, 40(16):159-167.
	ZHANG Libo, CHENG Haozhong, ZENG Pingliang, et al. Transmission network planning approaches based on uncertainty theories[J]. Automation of Electric Power Systems, 2016, 40(16):159-167.
[7]	WU P, CHENG H Z, XING J. The interval minimum load cutting problem in the process of transmission network expansion planning considering uncertainty in demand[J]. IEEE Transactions on Power Systems, 2008, 23(3):1497-1506. doi: 10.1109/TPWRS.2008.922573 URL
[8]	宋新甫, 徐龙秀, 张艳, 等. 结合灰色关联分析的熵权VIKOR法输电网规划方案综合评价方法[J]. 电工技术, 2019(11):11-14.
	SONG Xinfu, XU Longxiu, ZHANG Yan, et al. Comprehensive evaluation method of transmission network planning scheme based on entropy weight VIKOR method combined with grey relational analysis[J]. Electric Engineering, 2019(11):11-14.
[9]	MOCANU E, MOCANU D C, NGUYEN P H, et al. On-line building energy optimization using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(4):3698-3708. doi: 10.1109/TSG.5165411 URL
[10]	FRANÇOIS-LAVET V, TARALLA D, ERNST D, et al. Deep reinforcement learning solutions for energy microgrids management[M/OL]. Spain: Pompeu Fabra University, 2016. http://hdl.handle.net/2268/203831.
[11]	FORUZAN E, SOH L K, ASGARPOOR S. Reinforcement learning approach for optimal distributed energy management in a microgrid[J]. IEEE Transactions on Power Systems, 2018, 33(5):5749-5758. doi: 10.1109/TPWRS.59 URL
[12]	李澎, 彭敏放. 基于改进遗传算法的含风电场电力系统无功优化[J]. 现代电子技术, 2020, 43(13):167-171.
	LI Peng, PENG Minfang. Reactive optimization of improved genetic algorithm based power system with wind farm[J]. Modern Electronics Technique, 2020, 43(13):167-171.
[13]	杨宏, 闫玉杰, 王瑜. Beta分布在风电预测误差模型中的适用性[J]. 电测与仪表, 2020, 57(11):37-41, 48.
	YANG Hong, YAN Yujie, WANG Yu. Applicability of Beta distribution on wind power forecast error modeling[J]. Electrical Measurement & Instrumentation, 2020, 57(11):37-41, 48.
[14]	李浩, 钟声远, 王永真, 等. 基于能量与信息耦合的分布式能源系统配置优化方法[J]. 中国电机工程学报, 2020, 40(17):5467-5476.
	LI Hao, ZHONG Shengyuan, WANG Yongzhen, et al. Optimization method on the distributed energy system based on energy and information coupled[J]. Proceedings of the CSEE, 2020, 40(17):5467-5476.
[15]	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1):1-27.
	LIU Quan, ZHAI Jianwei, ZHANG Zongzhang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1):1-27.
[16]	王功鹏, 段萌, 牛常勇. 基于卷积神经网络的随机梯度下降算法[J]. 计算机工程与设计, 2018, 39(2):441-445.
	WANG Gongpeng, DUAN Meng, NIU Changyong. Stochastic gradient descent algorithm based on convolution neural network[J]. Computer Engineering and Design, 2018, 39(2):441-445.
[17]	车兵, 李轩, 郑建勇, 等. 基于LHS与BR的风电出力场景分析研究[J]. 电力工程技术, 2020, 39(6):213-219.
	CHE Bing, LI Xuan, ZHENG Jianyong, et al. Scenario analysis of wind power output based on LHS and BR[J]. Electric Power Engineering Technology, 2020, 39(6):213-219.
[18]	王蓓蓓, 刘小聪, 李扬. 面向大容量风电接入考虑用户侧互动的系统日前调度和运行模拟研究[J]. 中国电机工程学报, 2013, 33(22):35-44.
	WANG Beibei, LIU Xiaocong, LI Yang. Day-ahead generation scheduling and operation simulation considering demand response in large-capacity wind power integrated systems[J]. Proceedings of the CSEE, 2013, 33(22):35-44.
[19]	王群, 董文略, 杨莉. 基于Wasserstein距离和改进K-medoids聚类的风电/光伏经典场景集生成算法[J]. 中国电机工程学报, 2015, 35(11):2654-2661.
	WANG Qun, DONG Wenlue, YANG Li. A wind power/photovoltaic typical scenario set generation algorithm based on Wasserstein distance metric and revised K-medoids cluster[J]. Proceedings of the CSEE, 2015, 35(11):2654-2661.