基于Spark平台和并行随机森林回归算法的短期电力负荷预测

doi:10.3969/j.issn.1000－7229.2017.10.012

电力建设 ›› 2017, Vol. 38 ›› Issue (10): 84-.doi: 10.3969/j.issn.1000－7229.2017.10.012

基于Spark平台和并行随机森林回归算法的短期电力负荷预测

刘琪琛1, 雷景生2, 郝珈玮2,黄燕刚1,李强1,罗海波1

(1．国网四川省电力公司眉山供电公司, 四川省眉山市620010分号.
2.上海电力学院计算机科学与技术学院, 上海市200090)

出版日期:2017-10-01
作者简介:刘琪琛(1987)，男，本科，工程师，主要研究方向为智能电网、电力大数据和电力系统调度自动化技术分号雷景生(1966)，男，博士，教授，主要研究方向为智能电网、电力大数据和无线传感网络分号郝珈玮(1990)，男，硕士，通信作者，主要从事电力大数据和电力监测无线传感网络等方面的研究工作分号黄燕刚(1974)，男，本科，助理工程师，主要研究方向为电力大数据和电力系统调度自动化技术分号李强(1980)，男，本科，技师，主要研究方向为智能电网、电力大数据和电力系统调度自动化技术分号罗海波(1986)，男，本科，工程师，主要研究方向为智能电网、电力大数据和电力系统调度自动化技术。
基金资助:
国家自然科学基金项目（61472236，61672337）；国网眉山供电公司雄鹰创新攻关团队项目（基于调度技术支持系统的大数据分析与应用）

Short-Term Power Load Forecasting Based on Spark Platform and Parallel Random Forest Regression Algorithm Model

LIU Qichen1, LEI Jingsheng2, HAO Jiawei2, HUANG Yangang1, LI Qiang1, LUO Haibo1

(1. Meishan Power Supply Company, State Grid Sichuan Electric Power Company, Meishan 620010, Sichuan Province, China分号.
2. School of Computer Science and Technology, Shanghai University of Electric Power, Shanghai 200090, China)

Online:2017-10-01
Supported by:
Project supported by National Natural Science Foundation of China(61472236，61672337 )

摘要/Abstract

摘要： 摘要：随着智能电网、全球能源互联网的建设与相关技术的发展，现代电力系统中电力大数据的格局已经形成，如何对高维海量数据进行深度挖掘以实现数据的充分利用，成为当前电力工作者们关心的问题。该文针对电力大数据环境下高精度和实时性的负荷预测展开了研究，提出了基于Spark平台和并行随机森林回归算法(Spark platform and parallel random forest regression, SP-RFR)的短期电力负荷预测方法，通过3次弹性分布式数据集(resilient distributed datasets, RDD)转换实现单机随机森林算法的并行化改进，并在Spark分布式集群环境下实现部署。结合某区域实际电力负荷数据设计试验，进行模型训练和回归预测，通过试验证明，对同等的数据集，基于Spark平台的并行随机森林回归算法预测精度高于单机负荷预测算法；并行随机森林算法受离群数据干扰较小，且随着数据集的增大，并行随机森林算法表现出良好的鲁棒性；与单机算法在运行时间上相比，随着数据集的增大，基于分布式集群的方法优势明显。该文提出的方法能够有效地在分布式环境中进行电力负荷预测，为负荷预测提供了一种新思路。

关键词: , 电力大数据, 分布式计算, 并行随机森林回归算法, Spark平台, 短期电力负荷预测

Abstract: ABSTRACT： With the development of smart grid, global energy Internet and related technologies, the structure of power big data is already formed. How to make full use of the high-dimensional massive data through data mining to make full use of data has aroused widespread concern of power workers. Aiming at the high precision and real-time load forecasting with the background of power big data, this paper proposes the short-term power load forecasting based on Spark platform and parallel random forest regression (SP-RFR) algorithm. The parallelization improvement of single machine random forest algorithm is realized by three transforms of resilient distributed datasets(RDD), and can be deployed on a Spark distributed cluster. Experiments are designed by using actual power load data of a transformer substation, and model training and regression prediction are implemented. The conclusions are as follows, for the same testing data set, the short-term power load forecasting method based on SP-RFR model is superior to single machine regression forecasting model； SP-RFR model is less disturbed by outlier data, and SP-RFR model has good robustness with the increase of data set； compared with the single machine model, with the increase of the data set, the SP-RFR, which is based on the distributed cluster, has obvious advantages. The proposed method can effectively forecast power load in distributed background, which can provide a new idea for power load forecasting.

Key words: , power big data, distributed computing, parallel random forest regression algorithm, Spark platform, short-term power load forecasting.

中图分类号:

刘琪琛, 雷景生, 郝珈玮,黄燕刚,李强,罗海波. 基于Spark平台和并行随机森林回归算法的短期电力负荷预测[J]. 电力建设, 2017, 38(10): 84-.

LIU Qichen, LEI Jingsheng, HAO Jiawei, HUANG Yangang, LI Qiang, LUO Haibo. Short-Term Power Load Forecasting Based on Spark Platform and Parallel Random Forest Regression Algorithm Model[J]. Electric Power Construction, 2017, 38(10): 84-.

参考文献

［1］张小平,李佳宁,付灏. 全球能源互联网对话工业4.0［J］. 电网技术,2016,40(6):1607-1611..

ZHANG Xiaoping, LI Jianing, FU Hao. Global energy interconnection dialogue industry 4.0［J］. Power System Technology, 2016, 40(6): 1607-1611.

［2］王钦,蒋怀光,文福拴,等. 智能电网中大数据的概念、技术与挑战［J］. 电力建设, 2016, 37(12): 1-10..

WANG Qin, JIANG Huaiguang, WEN Fushuan, et al. Concept, technology and challenge of big data in smart grids［J］. Electric Power Construction, 2016, 37(12):1-10．

［3］张东霞,苗新,刘丽萍,等. 智能电网大数据技术发展研究［J］. 中国电机工程学报,2015,35(1):2-12..

ZHANG Dongxia, MIAO Xin, LIU Liping, et al. Research on development strategy for smart grid big data［J］.Proceedings of the CSEE, 2015,35(1):2-12.

［4］刘振亚. 智能电网技术［M］. 北京: 中国电力出版社, 2010:1-16.

［5］王大鹏. 灰色预测模型及中长期电力负荷预测应用研究［D］. 武汉：华中科技大学, 2013..

WANG Dapeng. Research on grey prediction models and their applications in medium-and long-term power load forecasting［D］. Wuhan: Huazhong University of Science and Technology, 2013.

［6］陈科. 基于模糊神经网络的电力系统短期负荷预测［D］. 南京: 东南大学,2015..

CHEN Ke. Short-term load forecasting in power system based on fuzzy neural network［D］. Nanjing: Southeast University, 2015.

［7］ NAZARKO J, ZALEWSKI W. The fuzzy regression approach to peak load estimation in power distribution systems［J］. IEEE Transactions on Power Systems, 1999, 14(3): 809-814.

［8］万昆,柳瑞禹. 区间时间序列向量自回归模型在短期电力负荷预测中的应用［J］. 电网技术,2012,36(11):77-81..

WAN Kun, LIU Ruiyu. Application of interval time-series vector autoregression model in short-term load forecasting［J］. Power System Technology, 2012, 36(11): 77-81.

［9］张刚, 刘福潮, 王维洲, 等．电网短期负荷预测的BP-ANN方法及应用［J］. 电力建设, 2014, 35(3): 54-58..

ZHANG Gang, LIU Fuchao, WANG Weizhou, et al. BP-ANN method for grid short-term load forecasting and its application［J］. Electric Power Construction, 2014, 35(3): 54-58.

［10］任震. 小波分析及其在电力系统中的应用［M］. 北京: 中国电力出版社, 2003:218-223.

［11］ BI Y, ZHAO J, ZHANG D. Power load forecasting algorithm based on wavelet packet analysis［C］//2004 International Conference on Power System Technology. Singapore, IEEE: 2004, 987-990.

［12］皮志贤. 基于决策树的电力负荷预测模型研究［D］. 北京：华北电力大学,2011..

PI Zhixian. Research on load forecasting model based on decision tree［D］.Beijing: North China Electric Power University, 2011.

［13］王新,孟玲玲. 基于EEMD-LSSVM的超短期负荷预测［J］. 电力系统保护与控制, 2015, 43(1): 61-66..

WANG Xin, MENG Lingling. Ultra-short-term load forecasting based on EEMD-LSSVM［J］. Power System Protection and Control, 2015, 43(1): 61-66.

［14］ ZAHARIA M, CHOWDHURY M, FRANKLIN M J, et al. Spark: Cluster computing with working sets［C］//Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing. Boston: USENIX Association Berkeley, 2010:10.

［15］Apache Software Foundation.Machine learning example ［EB/OL］ (2016-12-25)［2017-05-20］.http：//Spark.apache.org/exam- ples.html.

［16］冯琳. 集群计算引擎Spark中的内存优化研究与实现［D］.北京:清华大学,2013..

FENG Lin. Research and implementation of memory optimization based on parallel computing engine Spark［D］. Beijing: Tsinghua University,2013.

［17］袁宝超,刘道伟,刘丽平,等. 基于Spark的大电网广域时空序列分析平台构建［J］.电力建设,2016,37(11):48-54..

YUAN Baochao, LIU Daowei, LIU Liping, et al. Platform building for wide-area spatiotemporal sequences analysis of large-scale power grid based on Spark［J］. Electric Power Construction, 2016, 37(11): 48-54.

［18］李文栋. 基于Spark的大数据挖掘技术的研究与实现［D］.济南:山东大学,2015..

LI Wendong. The research and implementation of mining large data based on Spark［D］. Jinan: Shandong University,2015.

［19］刘泽燊, 潘志松. 基于Spark的并行SVM算法研究［J］. 计算机科学, 2016, 43(5): 238-242..

LIU Zeshen, PAN Zhisong. Research on parallel SVM algorithm based on Spark［J］. Computer Science, 2016, 43(5):238-242.

［20］李坤, 刘鹏, 吕雅洁. 基于SPARK的LIBSVM参数优选并行化算法［J］.南京大学学报（自然科学版), 2016,52(2):343-352..

LI Kun, LIU Peng, LYU Yajie. The parallel algorithms for LIBSVM parameter optimization based on Spark［J］. Journal of Nanjing University (Nature Sciences), 2016, 52(2): 343-352.

［21］王保义,王冬阳,张少敏. 基于Spark和IPPSO_LSSVM的短期分布式电力负荷预测算法［J］. 电力自动化设备,2016,36(1):117-122. .

WANG Baoyi, WANG Dongyang, ZHANG Shaomin. Distributed short-term load forecasting algorithm based on Spark and IPPSO-LSSVM［J］.Electric Power Automation Equipment, 2016,36(1):117-122.

［22］马天男, 牛东晓, 黄雅莉, 等．基于Spark平台和多变量L2-Bossting回归模型的分布式能源系统短期负荷预测［J］. 电网技术, 2016, 40(6): 1642-1649..

MA Tiannan, NIU Dongxiao, HUANG Yali, et al. Short-term load forecasting for distributed energy system based on Spark platform and multi-variable L2-Boosting regression model ［J］. Power System Technology, 2016, 40(6): 1642-1649.

［23］ PANG-NING T, MICHAEL S, VIPIN K. 数据挖掘导论［M］. 范明, 范宏建，译. 北京:人民邮电出版社, 2015: 150-168.

［24］张素香,刘建明,赵丙镇,等.基于云计算的居民用电行为分析模型研究［J］.电网技术,2013,37(6): 1542-1546．.

ZHANG Suxiang, LIU Jianming, ZHAO Bingzhen, et al. Cloud computing-based analysis on residential electricity consumption behavior［J］. Power System Technology, 2013, 37(6): 1542-1546.

［25］ ZAHARIA M, CHOWDHURY M, TATHAGATA D, et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing［C］// Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation. San Jose , USENIX Association Berkeley：2012: 2..

[1]	汤吉鸿, 章德, 丛凡超, 田国梁, 吴贵烘, 李宇骏, 禹海峰. 基于线性二次型最优控制的多端直流最优功率调制方案[J]. 电力建设, 2020, 41(8): 32-39.
[2]	赵洱岽, 王浩, 林弘杨. 现货市场中基于演化博弈的火电企业阶梯报价策略[J]. 电力建设, 2020, 41(8): 68-71.
[3]	宋建，束洪春，董俊，梁雨婷，李雨龙，杨博. 基于GM(1，1)与BP神经网络的综合负荷预测[J]. 电力建设, 2020, 41(5): 75-80.
[4]	祁兵，叶欣，李彬，陈宋宋，李源非，石坤. 考虑IEC标准的用户侧电力物联网体系架构研究[J]. 电力建设, 2020, 41(5): 92-99.
[5]	钟永洁，谢东亮，孙永辉，翟苏巍，常福刚，张明理. 考虑能源子系统与储能的综合能源系统优化分析[J]. 电力建设, 2019, 40(8): 59-68.
[6]	吴金城，董树锋，张舒鹏，韩荣杰，寿挺，李建斌. 基于分布式计算技术的火电厂辅助调频储能系统容量及功率规划方法[J]. 电力建设, 2019, 40(6): 57-64.
[7]	杨军，林洋佳，陈杰军，李逸欣，詹祥澎. 未来城市共享电动汽车发展模式[J]. 电力建设, 2019, 40(4): 49-59.
[8]	江帆，杨洪耕. 基于选择性贝叶斯分类的非侵入式负荷识别方法[J]. 电力建设, 2019, 40(2): 94-.
[9]	刘中原，王维庆，王海云，袁成玉，王亮，李永钦，丁文彬. 基于VSG技术的VSC-HVDC输电系统受端换流器控制策略[J]. 电力建设, 2019, 40(2): 100-.
[10]	林伟芳，易俊，贾俊川，王安斯，余芳芳，方诗卉. 故障后半波长输电线路的稳态过电压控制措施[J]. 电力建设, 2019, 40(2): 109-.
[11]	崔树银，陆奕，常啸. 考虑信用评分机制的电力碳排放交易区块链模型[J]. 电力建设, 2019, 40(1): 104-111.
[12]	黄裕春，王宏，王珂，文福拴. 考虑电动汽车充电设施接入的居民配电系统供电容量最优规划[J]. 电力建设, 2018, 39(6): 1-.
[13]	常巩, 邓友均, 杨洪明, 文福拴, 赖明勇. 充换电站模式下电动汽车参与物流配送的电能补给方式规划[J]. 电力建设, 2018, 39(6): 7-.
[14]	刘洵源, 齐峰, 文福拴, 冯昌森, 王蕾，邹波，张西竹. 光伏不平衡接入的配电系统中电动汽车有序充电策略[J]. 电力建设, 2018, 39(6): 21-.
[15]	杨文涛, 王蕾, 邹波, 张西竹, 文福拴, 程敏 . 基于大数据服务平台的电动汽车有序充放电管理[J]. 电力建设, 2018, 39(6): 28-.

基于Spark平台和并行随机森林回归算法的短期电力负荷预测

Short-Term Power Load Forecasting Based on Spark Platform and Parallel Random Forest Regression Algorithm Model

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价