Analysis on Charging Behaviors of Electric Vehicles Based on Markov Decision Processes

doi:10.3969/j.issn.1000-7229.2018.07.016

Abstract

Abstract: In view of the uncertainty of electric vehicle charging behavior, models of electric vehicle travel and battery electricity change based on trip chain theory are established, and an analysis method for charging behavior of electric vehicles by introducing the Markov decision processes（MDP） is proposed. The method takes the user's charging behavior as a Markov decision set, constructs a state transfer matrix according to the transfer probability between various regions. The user satisfaction index is set up as the reward function of the decision process. The optimal charging decision of the electric vehicle users at every decision point is obtained by solving the finite stage total reward criterion. The example is simulated by extracting characteristic data of electric vehicles, results show the time and space distribution of electric vehicle charging load. Compared with the traditional Monte Carlo method, the proposed MDP method can simulate user charging behavior more accurately and reflect the temporal and spatial distribution characteristics of charging demand. At the same time, different charging behavior of electric vehicles in different areas and different parking hours is analyzed, which can provide support for the planning and construction of electric vehicle charging piles.

Key words: electric vehicles, trip chain, Markov decision processes（MDP）, charging behavior

CLC Number:

PAN Yinji, QIU Xiaoyan, WU Jiawu, XIAO Jiankang. Analysis on Charging Behaviors of Electric Vehicles Based on Markov Decision Processes[J]. Electric Power Construction, 2018, 39(7): 129-137.

References

［1］BOULANGER A G，CHU A C，MAXX S，et al．Vehicle electrification：Status and issues［J］．Proceedings of the IEEE，2011，99（6）：1116-1138．

［2］高赐威，张亮．电动汽车充电对电网影响的综述［J］．电网技术，2011，32（2）：127-131.

GAO Ciwei，ZHANG Liang．A survey of influence of electrics vehicle charging on power grid［J］．Power System Technology，2011，32（2）：127-131．

［3］王锡凡，邵成成，王秀丽，等．电动汽车充电负荷与调度控制策略综述［J］．中国电机工程学报，2013，33（1）：1-10.

WANG Xifan，SHAO Chengcheng，WANG Xiuli，et al．Survey of electric vehicle charging load and dispatch control strategies［J］．Proceedings of the CSEE，2013，33（1）：1-10．

［4］罗卓伟，胡泽春，宋永华，等．电动汽车充电负荷计算方法［J］．电力系统自动化，2011，35（14）：36-42.

LUO Zhuowei，HU Zechun，SONG Yonghua，et al．Study on plug-in electric vehicles charging load calculating［J］．Automation of Electric Power System，2011，35（14）：36-42．

［5］艾学勇，顾洁，解大，等．电动汽车日充电曲线预测方法［J］．电力系统及其自动化学报，2013，25（6）：25-30.

AI Xueyong，GU Jie，XIE Da，et al．Forecasting method for electric vehicle daily charging curve［J］．Proceedings of the CSU-EPSA，2013，25（6）：25-30．

［6］杨冰，王丽芳，廖承林．大规模电动汽车充电需求及影响因素［J］．电工技术学报，2013，28（2）：22-27.

YANG Bing，WANG Lifang，LIAO Chenglin．Research on power-charging demand of large-scale electric vehicles and its impacting factors［J］．Transactions of China Electrotechnical Society，2013，28（2）：22-27．

［7］LI Gan，ZHANG Xiaoping．Modelling of plug-in hybrid electric vehicle charging demand in probabilistic power flow calculations［J］．IEEE Transactions on Smart Grid，2012，3（1）：492-499．

［8］陈丽丹，聂涌泉，钟庆．基于出行链的电动汽车充电负荷预测模型［J］．电工技术学报，2015，30（4）：216-225.

CHEN Lidan，NIE Yongquan，ZHONG Qing．A model for electric vehicle charging load forecasting based on trip chains［J］．Transactions of China Electrotechnical Society，2015，30（4）：216-225．

［9］赵书强，周靖仁，李志伟，等．基于出行链理论的电动汽车充电需求分析方法［J］．电力自动化设备，2017，37（8）：105-112.

ZHAO Shuqiang，ZHOU Jingren，LI Zhiwei，et al．EV charging demand analysis based on trip chain theory［J］．Electric Power Automation Equipment，2017，37（8）：105-112．

［10］徐青山，蔡婷婷，刘瑜俊，等．考虑驾驶人行为习惯及出行链的电动汽车充电站站址规划［J］．电力系统自动化，2016，40（4）：59-65.

XU Qingshan，CAI Tingting，LIU Yujun，et al．Location planning of charging stations for electric vehicles based on drivers’ behaviours and travel chain［J］．Automation of Electric Power Systems，2016，40（4）：59-65．

［11］唐飞，王波，查晓明，等．基于双阶段并行隐马尔科夫模型的电力系统暂态稳定评估［J］．中国电机工程学报，2013，33（10）：90-97.

TANG Fei，WANG Bo，ZHA Xiaoming，et al．Power system transient stability assessment based on two-stage parallel hidden Markov model［J］．Proceedings of the CSEE，2013，33（10）：90-97．

［12］林少伯，韩民晓，赵国鹏，等．基于随机预测误差的分布式光伏配网储能系统容量配置方法［J］．中国电机工程学报，2013，33（4）：25-33.

LIN Shaobo，HAN Minxiao，ZHAO Guopeng，et al．Capacity allocation of energy storage in distributed photovoltaic power system based on stochastic prediction error［J］．Proceedings of the CSEE，2013，33（4）：25-33．

［13］周松林，茆美琴，苏建徽．考虑风力发电随机性的微电网潮流预测［J］．中国电机工程学报，2013，37（22）：26-34.

ZHOU Songlin，MAO Meiqin，SU Jianhui．Power flow forecasting of micro-grid considering randomness of wind power［J］．Proceedings of the CSEE，2013，37（22）：26-34．

［14］许威，吕林，许立雄，等．基于马尔科夫链的电动汽车充电需求计算［J］．电力系统及其自动化学报，2017，29（3）：12-19.

XU Wei，LYU Lin，XU Lixiong，et al．Calculation of charging demand from electric vehicles based on Markov chain［J］．Proceedings of the CSU-EPSA，2017，29（3）：12-19．

［15］BOWMAN J L，BEN-AKIVA M E．Activity-based disaggregate travel demand model system with activity schedules［J］．Transportation Research Part A Policy & Practice，2001，35（1）：1-28．

［16］U．S．Department of Transportation，Federal Highway Administration．2009 national household travel survey［EB/OL］．http://nhts.ornl.gov

［17］田立亭，史双龙，贾卓．电动汽车充电功率需求的统计学建模方法［J］．电网技术，2010，34（11）：126-130.

TIAN Liting，SHI Shuanglong，JIA Zhuo．A statistical model for charging power demand of electrical vehicles［J］．Power System Technology，2010，34（11）：126-130．

［18］温剑锋，陶顺，肖湘宁，等．基于出行链随机模拟的电动汽车充电需求分析［J］．电网技术，2015，39（6）：1477-1484.

WEN Jianfeng，TAO Shun，XIAO Xiangning，et al．Analysis on charging demand of EV based on stochastic simulation of trip chain［J］．Power System Technology，2015，39（6）：1477-1484．

［19］胡奇英，刘建庸．马尔可夫决策过程引论［M］．西安：西安电子科技大学，2000：3-10．

［20］HINDERER K．Estimates for finite-stage dynamic programs［J］．Journal of Mathematical Analysis & Applications，1976，55（1）：207-238．

［21］BELLMAN R E，LASALLE J P．On non-zero games and stochastic processes［J］．1949．

［22］SAUER D U，WENZL H．Comparison of different approaches for lifetime prediction of electrochemical systems-using lead acid batteries as example［J］．Journal of Power Sources，2008，176（2）：477-483．

［23］SCHALTZ E，KHALIGH A，RASMUSSEN P O．Influence of battery/ultra-capacitor energy-storage sizing on battery lifetime in a fuel cell hybrid electric vehicle［J］．IEEE Transactions on Vehicular Technology，2009，58（8）：3882-3891．

［24］韩晓娟，程成，籍天明，等．计及电池使用寿命的混合储能系统容量优化模型［J］．中国电机工程学报，2013，33（34）：91-97.

HAN Xiaojuan，CHENG Cheng，JI Tianming，et al．Capacity optimal modelling of hybrid energy storage systems considering battery life［J］．Proceedings of the CSEE，2013，33（34）：91-97．

［25］麻秀范，李颖，王皓，等．基于电动汽车出行随机模拟的充电桩需求研究［J］．电工技术学报，2017，32（2）：190-202.

MA Xiufan，LI Ying，WANG Hao，et al．Research on demand of charging piles based on stochastic simulation of EV trip chain［J］．Transactions of China Electrotechnical Society，2017，32（2）：190-202．

[1]	LIU Hui， XIE Liangfeng， XIE Haimin， YANG Yude， QIN Zhijun. Network Loss Evaluation of Power Systems Considering Participation of Electric Vehicles in AGC [J]. Electric Power Construction, 2020, 41(6): 52-59.
[2]	ZHANG Yuwei, YANG Jun, WU Fuzhang, ZHAN Xiangpeng, LONG Xuemei, ZHANG Jun, XU Jian. Analysis on Disturbance Propagation of Charging Station Fault Considering Grid-Traffic Network Coupling [J]. Electric Power Construction, 2020, 41(1): 13-22.
[3]	TAN Kai， LI Zhongwei， GUAN Yadong， YE Lin， TONG Weiming， ZHANG Baojun. Lightweight Key Management Scheme Based on Communication System of Electric Vehicle Charging Piles [J]. Electric Power Construction, 2019, 40(9): 73-81.
[4]	DU Lijia， JIN Xiaolong， HE Wei， MU Yunfei， JIA Hongjie， WEI Wei. A Tie-line Power Smoothing Control Method for an Office Building Microgrid by Scheduling Thermal Mass of the Building and Plug-in Electric Vehicles [J]. Electric Power Construction, 2019, 40(8): 26-33.
[5]	XIA Shiwei， LU Tao， DU Ting， LI Gengyin， BU Siqi. Distributed Coordinated Operation Method of Plug-in Electric Vehicles in Micro Grid Considering Fluctuation of Wind Power [J]. Electric Power Construction, 2019, 40(6): 86-94.
[6]	CHEN Jiamin,XU Yonghai， ZHANG Xueyin. Energy Storage Sizing Method Considering V2G Response Capability [J]. Electric Power Construction, 2019, 40(3): 34-41.
[7]	WANG Chenglong， WANG Yufei， XUE Hua. Control Strategy of Flywheel Energy Storage System Based on Load Current Compensation and Speed Feedback [J]. Electric Power Construction, 2019, 40(3): 51-58.
[8]	MEI Zhe, ZHAN Hongxia, YUAN Jihe2 HUANG Hu, ZHANG Xi, DENG Qiang. Multi-objective Optimal Operating Strategy of Distribution Network Considering V2G on the Basis of Grouping Method of Electric Vehicles [J]. Electric Power Construction, 2018, 39(8): 59-68.
[9]	WANG Dan，SUN Dongdong，HU Qinge，YUAN Kai， SUN Chongbo，SONG Yi，XU Jing，DING Chengdi . Demand Response of Electric Vehicles Based on Optimal Energy Status Regulation and Its Impact on Distribution Network [J]. Electric Power Construction, 2018, 39(5): 86-.
[10]	RONG Xiaoxue， LIU Xiyuan， SUN Qiming， BIE Zhaohong. Coordinated Charging Strategy for Battery in EV Charging and Switching Station [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 120-125.
[11]	LIU Haixuan，ZENG Pingliang，MA Jun，XU Xiaohui. Operation Strategy of Electric Vehicles for Interacting with the Grid in Battery to Grid Mode [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 126-132.
[12]	TANG Jia， GAO Shuang， WANG Dan， SONG Yi， YANG Zhanyong. A Hierarchical Control Algorithm for Aggregated Electric Vehicles in Distribution Networks [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 146-152.
[13]	ZHANG Qian， LI Yuzhe， ZHOU Lin， CAI JiaJia， ZHOU Louwei. Evaluation of Changing Controllable Number of Different Type of EVs when Access into Frequency Regulation [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 160-166.
[14]	WANG Xiaolei， YAO Weifeng， WEN Fushuan， ZENG Pingliang， ZHANG Xizhu， WANG Lei. Comprehensive Benefit Analysis of Coordinated Charging Management of Electric Vehicles [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 194-201.
[15]	LIU Nian， CHENG Minyang， ZOU Fuqiang， ZHANG Jianhua. Operation Mode and Comprehensive Evaluation of PV Charging Station for Commercial Building [J]. ELECTRIC POWER CONSTRUCTION, 2015, 36(7): 202-208.