A Novel Declaration Strategy for Integrated Energy Servicer Based on Q-Learning Algorithm in Power Spot Market

doi:10.12204/j.issn.1000-7229.2020.09.015

Abstract

Abstract:

With the development of integrated energy system and power market reform, integrated energy servicer is expected to become a new market member in power market transaction. In order to solve the problem that the limited reference information in the declaration stage restricts the formulation of the declaration strategy, a declaration strategy based on Q-learning for integrated energy servicer is proposed to improve the ideal degree of the declaration strategy. The core idea of the proposed strategy is to make full use of the huge historical operation information and train the declaration strategy agent by artificial intelligence algorithms to establish the inherent relationship between the limited reference information grasped by integrated energy servicer during the market bidding process and its optimal declaration strategy. The declaration agent can realize automatic generation and intelligent improvement of declaration policies, which takes energy market public information, social public information and enterprise private information as environment variables. Finally, a case study based on the actual data of a provincial power grid shows that the proposed method can better match the declaration strategy under the cooperative game and has the characteristics of fast convergence, high ideal degree and high computational efficiency, which is more suitable for the actual needs of integrated energy servicer.

Key words: integrated energy servicer, power spot market, Q-learning algorithm, declaration strategy

CLC Number:

TM743

HAO Xudong, SUN Wei, CHENG Dingyi, ZHANG Guoqiang, KUANG Honghui. A Novel Declaration Strategy for Integrated Energy Servicer Based on Q-Learning Algorithm in Power Spot Market[J]. ELECTRIC POWER CONSTRUCTION, 2020, 41(9): 132-138.

Figures/Tables 10

Table 1 Current market declaration mechanism

Fig.1 Principle of reinforcement learning

Fig.2 Training process of Q-reinforcement learning

Table 2 Environment variable set

Fig.3 Typical energy demand curve

Table 3 Main equipment list

Fig.4 Value changes of return function

Fig.5 Value changes in day of return function

Table 4 Declaration strategy

Fig.6 Revenue comparison

References 21

[1]	韩宇, 彭克, 王敬华 , 等. 多能协同综合能源系统协调控制关键技术研究现状与展望[J]. 电力建设, 2018,39(12):81-87.
	HAN Yu, PENG Ke, WANG Jinghua , et al. Research status and prospect of key technologies for coordinated control of multi-energy synergic integrated energy systems[J]. Electric Power Construction, 2018,39(12):81-87.
[2]	葛睿, 陈龙翔, 汤俊 , 等. 跨区域省间可再生能源增量现货市场设计与实践[J]. 电力建设, 2019,40(1):11-18.
3	GE Rui, CHEN Longxiang, TANG Jun , et al. Design and practice of cross-regional incremental spot market for renewable energy[J]. Electric Power Construction, 2019,40(1):11-18.
[3]	谈金晶, 李扬 . 多能源协同的交易模式研究综述[J]. 中国电机工程学报, 2019,39(22):6483-6497.
	TAN Jinjing, LI Yang . Review on transaction mode in multi-energy collaborative market[J]. Proceedings of the CSEE, 2019,39(22):6483-6497.
[4]	戚艳, 刘敦楠, 徐尔丰 , 等. 面向园区能源互联网的综合能源服务关键问题及展望[J]. 电力建设, 2019,40(1):123-132.
	QI Yan, LIU Dunnan, XU Erfeng , et al. Key issues and prospects of integrated energy service for energy Internet in park[J]. Electric Power Construction, 2019,40(1):123-132.
[5]	WANG D, UNIVERSITY T, LIU L , et al. Review of key problems related to integrated energy distribution systems[J]. CSEE Journal of Power and Energy Systems, 2018,4(2):130-145.
[6]	黄伟, 葛良军, 华亮亮 , 等. 参与双重市场的区域综合能源系统日前优化调度[J]. 电力系统自动化, 2019,43(12):68-82.
	HUANG Wei, GE Liangjun, HUA Liangliang , et al. Day-ahead optimal scheduling of regional integrated energy system participating in dual market[J]. Automation of Electric Power Systems, 2019,43(12):68-82.
[7]	吴嘉豪, 曾成碧, 苗虹 . 计及子区域间能量交换的多区域综合能源系统协调经济调度[J]. 电力建设, 2019,40(11):39-47.
	WU Jiahao, ZENG Chengbi, MIAO Hong . Coordinated economic scheduling for multi-regional integrated energy system considering energy exchange between sub-regions[J]. Electric Power Construction, 2019,40(11):39-47.
[8]	刘珮云, 丁涛, 贺元康 , 等. 基于综合需求响应的负荷聚合商最优市场交易策略[J]. 电力自动化设备, 2019,39(8):224-231.
	LIU Peiyun, DING Tao, HE Yuankang , et al. Optimal trading strategy for load aggregator based on integrated demand response[J]. Electric Power Automation Equipment, 2019,39(8):224-231.
[9]	TAHIR M F, CHEN H Y, MEHMOOD K , et al. Integrated energy system modeling of China for 2020 by incorporating demand response, heat pump and thermal storage[J]. IEEE Access, 2019(7):40095-40108.
[10]	武赓, 武庆国, 王昊婧 , 等. 考虑风险规避的综合能源服务商能源购置策略[J]. 电力自动化设备, 2019,39(6):12-20.
	WU Geng, WU Qingguo, WANG Haojing , et al. Energy purchasing strategy of multi-energy service provider considering risk aversion[J]. Electric Power Automation Equipment, 2019,39(6):12-20.
[11]	尹龙, 刘继春, 高红均 , 等. 考虑多种用户价格机制下的综合型能源售电公司购电竞价策略[J]. 电网技术, 2018,42(1):88-97.
	YIN Long, LIU Jichun, GAO Hongjun , et al. Study on bidding strategy of comprehensive power retailer under multiple user-price mechanisms[J]. Power System Technology, 2018,42(1):88-97.
[12]	耿琪, 胡炎, 何建宗 , 等. 基于纳什谈判的区域综合能源系统运行优化[J]. 电力建设, 2020,41(1):114-125.
	GENG Qi, HU Yan, HE Jianzong , et al. Optimal operation of integrated community energy system based on Nash bargaining[J]. Electric Power Construction, 2020,41(1):114-125.
[13]	练依情, 郭祚刚, 喻磊 , 等. 市场出清机制下综合能源服务商竞争策略[J]. 南方电网技术, 2019,13(7):10-16, 42.
	LIAN Yiqing, GUO Zuogang, YU Lei , et al. Competition strategy of integrated energy service provider under pool-based market mechanism[J]. Southern Power System Technology, 2019,13(7):10-16, 42.
[14]	郭祚刚, 喻磊, 胡洋 , 等. 基于合作博弈的综合能源服务商现货市场风险规避策略[J]. 中国电力, 2019,52(11):28-34.
	GUO Zuogang, YU Lei, HU Yang , et al. Research on competition strategy of integrated energy service provider under the pool-based market mechanism[J]. Electric Power, 2019,52(11):28-34.
[15]	LIU N, WANG J, WANG L F . Hybrid energy sharing for multiple microgrids in an integrated heat-electricity energy system[J]. IEEE Transactions on Sustainable Energy, 2019,10(3):1139-1151.
[16]	赵晓东, 杜小泽, 曾鸣 . 商业综合能源系统经济性分析[J]. 电力需求侧管理, 2019,21(4):87-92.
	ZHAO Xiaodong, DU Xiaoze, ZENG Ming . The influence of market electricity price signal on the economy of integrated energy system[J]. Power Demand Side Management, 2019,21(4):87-92.
[17]	马辉, 陈雨果, 陈晔 , 等. 南方(以广东起步)电力现货市场机制设计[J]. 南方电网技术, 2018,12(12):42-48.
	MA Hui, CHEN Yuguo, CHEN Ye , et al. Mechanism design of Southern China(starting from Guangdong Province)electric spot market[J]. Southern Power System Technology, 2018,12(12):42-48.
[18]	HWANG K S, JIANG W C, CHEN Y J , et al. Model learning for multistep backward prediction in dyna- mic Q-learning[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018,48(9):1470-1481.
[19]	SPANO S, CARDARILLI G C, DI NUNZIO L , et al. An efficient hardware implementation of reinforcement learning: The Q-learning algorithm[J]. IEEE Access, 2019(7):186340-186351.
[20]	DENG Z, GUAN H J, HUANG R , et al. Combining model-based Q-learning with structural knowledge transfer for robot skill learning[J]. IEEE Transactions on Cognitive and Developmental Systems, 2019,11(1):26-35.
[21]	张健, 曹志东 . 负荷预测在电力市场中的应用[J]. 山东电力技术, 2000,27(2):24-68.