Expert Experience-Guided Hierarchical Reinforcement Learning for Coordinated Post-Disaster Restoration of Distribution Networks

XUE Zhang, LI Xiaolu, LIU Jinsong, LIN Shunfu

Electric Power Construction ›› 2026, Vol. 47 ›› Issue (10) : 1-14.

PDF(4274 KB)
PDF(4274 KB)
Electric Power Construction ›› 2026, Vol. 47 ›› Issue (10) : 1-14. DOI: 10.12204/j.issn.1000-7229.2026.10.001
Key Technologies of Global Security Defense and Disaster Resistance for High-Proportion New Energy Power Grid·Hosted by WANG Yuanyuan,WANG Jinyu,HUANG Can,XIAO Hao,BIAN Jing·

Expert Experience-Guided Hierarchical Reinforcement Learning for Coordinated Post-Disaster Restoration of Distribution Networks

Author information +
History +

Abstract

[Objective] Extreme disasters are prone to trigger multi-point faults in distribution networks, and the lack of coordination between network reconfiguration and fault emergency repair will degrade the efficiency of fault restoration. This paper proposes a coordinated post-disaster restoration strategy for distribution networks based on expert experience-guided hierarchical reinforcement learning (HRL). [Methods] Firstly, a decision-making model for post-disaster coordinated restoration of distribution networks is constructed. Taking the minimization of load loss, switch operation cost and maintenance travel cost as the optimization objectives, the model comprehensively considers the operation constraints, network topology constraints and fault repair constraints of distribution networks, as well as the time-varying traffic accessibility constraints of transportation networks. Secondly, targeting the sequential decision-making characteristics of post-disaster restoration, a hierarchical reinforcement learning decision framework consisting of high-level task selection and low-level action execution is established. The high-level layer selects network reconfiguration or repair scheduling tasks according to real-time system states, while the low-level layer generates topology reconfiguration operations, repair task allocation schemes and route adjustment strategies under corresponding constraint conditions. Finally, expert trajectories are adopted to pre-train the policy network, and the parameters of the hierarchical policy are continuously updated through online interactive training. [Results] Random disaster scenario tests on a coupled system of the modified IEEE 33-bus distribution network and the Sioux-Falls traffic network demonstrate that the average cumulative load loss of the proposed method is 5.45 MWh, which is reduced by 22.43% and 29.57% compared with the mixed-integer second-order cone programming (MISOCP) and binary particle swarm optimization (BPSO) algorithms, respectively. The average online solution time of the proposed method is 0.032 7 s, which is significantly lower than that of the comparative methods. [Conclusions] The proposed method realizes the coordinated optimization of post-disaster network reconfiguration and repair scheduling for distribution networks. It exhibits superior performance in reducing cumulative load loss and improving online decision-making efficiency, and can provide auxiliary decision support for the rapid restoration of distribution networks under extreme disaster conditions.

Key words

distribution network restoration / hierarchical reinforcement learning / behavior cloning / fault emergency repair / network reconfiguration

Cite this article

Download Citations
XUE Zhang , LI Xiaolu , LIU Jinsong , et al. Expert Experience-Guided Hierarchical Reinforcement Learning for Coordinated Post-Disaster Restoration of Distribution Networks[J]. Electric Power Construction. 2026, 47(10): 1-14 https://doi.org/10.12204/j.issn.1000-7229.2026.10.001

附录A

一个区域的交通网可以由有限个节点集合与边集合构成的图表示,即GTN=(NTN, BTN),在DSR问题中,根据RC所处的初始节点及故障点,可形成由Nrd集合所构成的交通调度子网模型。RC的交通调度网示例如图A1所示,其中RC的交通调度网由节点集Nrd={N0,…,N8,N10}及其所构成的路段组成,RC驻点N0与故障点N10空间位置是固定的,但灾后路面极可能受到破坏影响RC移动速度,甚至部分路段无法通行,在灾后抢修的急迫情况下,等待交通恢复势必延长供电恢复过程,因此,在规划RC调度方案时需要考虑道路通行风险因素,本文将道路广义距离与路阻函数相结合,对任意路段两节点的转移时间 ${\mathit{t}}_{\mathit{m}\mathit{n}}^{\mathrm{t}\mathrm{r}}$进行估计。

${\mathit{D}}_{\mathit{m}\mathit{n}}=\left\{\begin{array}{l}{\mathit{d}}_{\mathit{m}\mathit{n}},{\mathit{\theta }}_{\mathit{m}\mathit{n}}=0\\ {\mathit{d}}_{\mathit{m}\mathit{n}}(1+{\mathit{\theta }}_{\mathit{m}\mathit{n}}),0<{\mathit{\theta }}_{\mathit{m}\mathit{n}}<\mathit{\eta }\\ \mathrm{\infty },{\mathit{\theta }}_{\mathit{m}\mathit{n}}\ge \mathit{\eta }\end{array}\right.$
$t_{m n}^{\mathrm{tr}}=\frac{D_{m n}}{\bar{v}}\left[1+\alpha\left(\frac{X_{m n, t}}{C_{m n}}\right)^{\beta}\right]$

式中:dmn为路段实际距离;Dmn为灾后道路广义距离;η为可安全通行的最高阈值;θmn为路段的损毁率,取值范围均为[0,1],当损毁率大于安全通行阈值时,道路中断; $\stackrel{-}{\mathit{v}}$为平均车速;Xmn,t为路段实时车流量;Cmn为路段的通行能力;α、β为性能参数,一般情况下α=0.15,β=4。

在多个路由可达情况下,选择总行程时间T(l,t)最短者:

${\mathcal{l}}_{\mathit{o}\to \mathit{k}}\left(\mathit{t}\right)=\underset{\mathit{l}\in {\mathit{R}}_{\mathit{o}\to \mathit{k}}}{\mathrm{m}\mathrm{i}\mathrm{n}}\mathit{T}(\mathit{l},\mathit{t})=\underset{\mathit{l}\in {\mathit{R}}_{\mathit{o}\to \mathit{k}}}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{(\mathit{m},\mathit{n})\in \mathit{l}}{\mathit{t}}_{\mathit{m}\mathit{n}}^{\mathit{t}\mathit{r}}\left(\mathit{t}\right)$

式中:Ro→k为起点o至终点k可行路由集合,在图A1的调度网络中,RC可以通过三条路由集合到达抢修点,即l1,l2,l3∈R0→10,但距离最短的路由l1部分道路损毁(图A1中N6—N7路段)无法安全通行,同时,路由l2上所需要的转移时间相较l3要长。在这种情况下,RC将选择路由l3作抢修路线。

图A1 RC的交通调度网

Fig.A1 RC traffic scheduling network

附录B

图B1 灾后路网通行能力

Fig.B1 Post-disaster road network accessibility

图B2 RC调度策略

Fig.B2 RC dispatch strategy

图B3 网络重构策略

Fig.B3 Network reconfiguration strategy

表B1 基于PPO两阶段分层训练伪代码

Table B1 Two-stage hierarchical training pseudocode with PPO

Input: 预训练参数 ${\mathit{\theta }}_{\mathrm{H}}^{\mathit{\text{'}}}$、 ${\mathit{\theta }}_{\mathrm{L}}^{\mathit{\text{'}}}$,裁剪参数 $\mathit{ϵ}$,折扣因子 $\mathit{\gamma }$
Output:训练完成的策略网参数 ${\mathit{\theta }}_{\mathrm{H}}^{\mathrm{*}}$, ${\mathit{\theta }}_{\mathrm{L}}^{\mathrm{*}}$模型
1: 加载预训练保存的策略网络参数 ${\mathit{\theta }}_{\mathrm{H}}^{\mathit{\text{'}}}$, ${\mathit{\theta }}_{\mathrm{L}}^{\mathit{\text{'}}}$模型
2: 初始化critic networks及PPO相关参数
3: For each training episode do
4: 初始化环境状态s0
/* ----------阶段一:低层执行策略训练---------- */
5: For each $\mathit{o}\in \mathcal{O}$do
6: 初始化低层经验池 ${\mathcal{B}}_{\mathrm{L}}=\left\{\right\}$, 轨迹 ${\mathit{\tau }}_{\mathrm{L}}=\left[\right]$
7: Repeat
8: 选择动作 ${\mathit{a}}_{\mathit{t},\mathit{k}}\sim {\mathit{\pi }}_{\mathit{o}}\left({\mathit{a}}_{\mathit{t},\mathit{k}}\right|{\mathit{s}}_{\mathit{t},\mathit{k}}^{\mathrm{L}},{\mathit{\theta }}_{\mathrm{L}}^{\mathit{\text{'}}})$
9: $({\mathit{s}}_{\mathit{t},\mathit{k}+1}^{\mathrm{L}},{\mathit{r}}_{\mathit{t},\mathit{k}}^{\mathrm{L}})\leftarrow $交互环境 $({\mathit{s}}_{\mathit{t},\mathit{k}}^{\mathrm{L}},{\mathit{a}}_{\mathit{t},\mathit{k}})$
10: ${\mathit{\tau }}_{\mathrm{L}}\leftarrow [{\mathit{s}}_{\mathit{t},\mathit{k}}^{\mathrm{L}},{\mathit{a}}_{\mathit{t},\mathit{k}},{\mathit{s}}_{\mathit{t},\mathit{k}+1}^{\mathrm{L}},{\mathit{r}}_{\mathit{t},\mathit{k}}^{\mathrm{L}}]$
11: Until 经验池达到预设长度
12: PPO算法按照式(27)计算更新低层参数 ${\mathit{\theta }}_{\mathrm{L}}^{\mathit{\text{'}}}$
13: End for
14: 保存训练后最优低层策略参数 ${\mathit{\theta }}_{\mathrm{L}}^{\mathrm{*}}$模型
/* ----------阶段二:高层选项策略训练 ---------- */
15: 初始化高层经验池 ${\mathcal{B}}_{\mathrm{H}}=\left\{\right\}$, 轨迹 ${\mathit{\tau }}_{\mathrm{H}}=\left[\right]$
16: 加载训练保存的低层策略网络参数 ${\mathit{\theta }}_{\mathrm{L}}^{\mathrm{*}}$模型
17: Repeat
18: 选择选项 ${\mathit{o}}_{\mathit{t}}\sim {\mathit{\pi }}_{\mathcal{O}}\left({\mathit{o}}_{\mathit{t}}\right|{\mathit{s}}_{\mathit{t}}^{\mathrm{H}},{\mathit{\theta }}_{\mathrm{H}}^{\mathit{\text{'}}})$
19: 在低层策略网络参数 ${\mathit{\theta }}_{\mathrm{L}}^{\mathrm{*}}$模型下执行选项ot
20: 获得高层状态 ${\mathit{s}}_{\mathit{t}+\mathrm{\Delta }{\mathit{t}}_{\mathit{m}}}^{\mathrm{H}}$及选项持续时间 $\mathrm{\Delta }{\mathit{t}}_{\mathit{m}}$
21: ${\mathit{\tau }}_{\mathrm{H}}\leftarrow [{\mathit{s}}_{\mathit{t}}^{\mathrm{H}},{\mathit{o}}_{\mathit{t}},{\mathit{r}}_{\mathit{t}}^{\mathrm{H}},\mathrm{\Delta }{\mathit{t}}_{\mathit{m}},{\mathit{s}}_{\mathit{t}+1}^{\mathrm{H}}]$
22: 更新环境状态 ${\mathit{s}}_{\mathit{t}+\mathrm{\Delta }{\mathit{t}}_{\mathit{m}}}^{\mathrm{H}}\leftarrow ({\mathit{s}}_{\mathit{t}}^{\mathrm{H}},{\mathit{o}}_{\mathit{t}},\mathrm{\Delta }{\mathit{t}}_{\mathit{m}})$
23: Until 经验池达到预设长度
24: 按照式(30)计算更新高层策略参数 ${\mathit{\theta }}_{\mathrm{H}}^{\mathit{\text{'}}}$
25: End for
26: 保存训练完成的策略网参数 ${\mathit{\theta }}_{\mathrm{H}}^{\mathrm{*}}$, ${\mathit{\theta }}_{\mathrm{L}}^{\mathrm{*}}$模型
表B2 IEEE 33系统DG参数

Table B2 IEEE 33 system DG parameters

编号 接入点 有功限制/kW 无功限制/kvar
DG1 4 [0,170] [-83,83]
DG2 8 [0,410] [-198,198]
DG3 13 [0,600] [-290,290]
DG4 20 [0,260] [-126,126]
DG5 23 [0,220] [-106,106]
DG6 32 [0,350] [-169,169]
表B3 超参数设置

Table B3 Hyperparameter settings

超参数 值
学习率(learning rate) 0.000 3
折扣因子(discount factor) 0.99
批量大小(batch size) 128
最大回合步长(max episode steps) 50
训练周期(training episodes) 300 000
每轮更新步数(rollout) 4096
表B4 故障候选点参数

Table B4 Fault candidate point parameters

编号 DN位置 TN位置 修复时间/min
M1 线路23-24 9 75
M2 线路27-28 4 55
M3 线路16-17 18 50
M4 线路6-26 3 60
M5 线路6-7 12 80
M6 线路2-3 10 50
M7 线路19-20 20 30

References

[1]
孙为民, 孙华东, 何剑, 等. 面向严重自然灾害的电力系统韧性评估技术综述[J]. 电网技术, 2024, 48(1): 129-139.
Sun Weimin, Sun Huadong, He Jian, et al. Review of power system resilience assessment techniques for severe natural disasters[J]. Power System Technology, 2024, 48(1): 129-139.
[2]
Xu L, Lin N, Poor H V, et al. Quantifying cascading power outages during climate extremes considering renewable energy integration[J]. Nature Communications, 2025, 16: 2582.
[3]
王乐, 唐捷, 黄远飞, 等. 考虑微地形的配电系统雨涝淹没停电风险区划分方法[J]. 电力科学与技术学报, 2024, 39(4): 84-92.
Wang Le, Tang Jie, Huang Yuanfei, et al. Risk zone assessment of rainstorm induced waterlogging associated outage of distribution system with consideration of micro-topography[J]. Journal of Electric Power Science and Technology, 2024, 39(4): 84-92.
[4]
宋梦, 周佳妮, 高赐威, 等. CPSS视角下城市建筑与配电网高韧性协调运行: 研究述评与展望[J]. 电力系统自动化, 2023, 47(23): 105-121.
Song Meng, Zhou Jiani, Gao Ciwei, et al. High-resilience coordinated operation of urban buildings and distribution networks from cyber-physical-social system perspective: research review and prospect[J]. Automation of Electric Power Systems, 2023, 47(23): 105-121.
[5]
侯慧, 徐海峰, 王少华, 等. 含降雨量修正的台风灾害下输电杆塔数据机理联合故障概率预测[J]. 高电压技术, 2025, 51(4): 1654-1662.
Hou Hui, Xu Haifeng, Wang Shaohua, et al. Data mechanism joint failure probability prediction of transmission tower with rainfall correction under typhoon disaster[J]. High Voltage Engineering, 2025, 51(4): 1654-1662.
[6]
舒印彪, 汤涌, 张正陵, 等. 欧洲伊比利亚半岛电网“4·28”大停电事故分析及启示[J]. 中国电机工程学报, 2025, 45(17): 6603-6612.
Shu Yinbiao, Tang Yong, Zhang Zhengling, et al. Analysis and lessons of the April 28, 2025 blackout in the Iberian Peninsula power system[J]. Proceedings of the CSEE, 2025, 45(17): 6603-6612.
[7]
田书欣, 苏鹏斌, 赵昊星, 等. 冰灾场景下考虑无人机应用的综合能源系统韧性提升方法[J]. 浙江电力, 2025, 44(11): 1-13.
Tian Shuxin, Su Pengbin, Zhao Haoxing, et al. A resilience enhancement method for integrated energy systems considering UAV response in ice storm scenarios[J]. Zhejiang Electric Power, 2025, 44(11): 1-13.
[8]
耿宁, 覃智君, 黄镇. 台风灾害下含多元储能的电-气-热综合能源系统韧性提升策略[J]. 电网与清洁能源, 2025, 41(9): 64-73.
Geng Ning, Qin Zhijun, Huang Zhen. A resilience enhancement strategy for electricity-gas-heat integrated energy systems with multi-energy storage under typhoon disaster[J]. Power System and Clean Energy, 2025, 41(9): 64-73.
[9]
袁家海, 张凯, 张健, 等. 极端高温天气下电力系统韧性提升策略[J]. 发电技术, 2025, 46(4): 694-704.
Abstract
目的 为应对极端高温天气对电力系统供需平衡造成的冲击,提出一种面向中长期的电力系统韧性提升方法。 方法 首先,模拟持续性极端高温场景下“源-荷”耦合出力特性,并利用时序运行模拟模型对系统韧性表现进行初评估;其次,基于运行模拟结果,在电力供需平衡的基础上以供电成本与负荷削减成本之和最小为目标,建立面向韧性提升的电力资源规划模型,得到韧性提升组合措施;最后,对韧性提升策略场景进行运行模拟,以验证其有效性。 结果 以2025年广东省为案例场景,与未实施电力系统韧性提升策略的场景相比,实施策略后极端高温天气下系统累积缺供电量、最大失负荷规模和因电力短缺而造成的经济损失分别降低了86.71%、60.72%和91.55%,系统最小供电水平提升了11.07%。 结论 通过协同供电资源扩容与负荷资源部署,电力系统韧性提升策略有效地提升了极端高温天气下系统的供电能力,同时降低了因电力短缺而造成的经济损失和社会代价。
Yuan Jiahai, Zhang Kai, Zhang Jian, et al. Resilience enhancement strategies for power systems under extreme high-temperature weather[J]. Power Generation Technology, 2025, 46(4): 694-704.

Objectives To address the impact of extreme high-temperature weather on the supply and demand balance of power systems, a medium- to long-term method for enhancing power system resilience is proposed. Methods First, the coupled “source-load” output characteristics under sustained extreme high-temperature scenarios are simulated, and an initial evaluation of system resilience is performed using a chronological operation simulation model. Second, based on the operation simulation results, a resilience enhancement-oriented power resource planning model is established, with the objective of minimizing the combined cost of power supply and load shedding under the constraint of supply and demand balance, thereby deriving combined measures for resilience enhancement. Finally, operation simulations of the resilience enhancement strategy scenarios are conducted to verify their effectiveness. Results Taking Guangdong Province in 2025 as a case study, compared with the scenario without the implementation of the resilience enhancement strategies for power systems, the application of such strategies under extreme high-temperature weather decreases the cumulative power shortage, maximum lost load scale, and economic losses caused by power shortage by 86.71%, 60.72%, and 91.55%, respectively. The system’s minimum power supply level increases by 11.07%. Conclusions By coordinating the expansion of power supply resources and deployment of load resources, the resilience enhancement strategies effectively improve the power system’s supply capacity under extreme high-temperature weather, while reducing the economic loss and social cost caused by power shortages.

[10]
谭静, 王东, 张英华, 等. 台风灾害下电网多维韧性评估研究[J]. 山东电力技术, 2024, 51(3): 27-35.
Tan Jing, Wang Dong, Zhang Yinghua, et al. Multidimensional resilience assessment of power grids under typhoon disasters[J]. Shandong Electric Power, 2024, 51(3): 27-35.
[11]
陈磊, 邓欣怡, 陈红坤, 等. 电力系统韧性评估与提升研究综述[J]. 电力系统保护与控制, 2022, 50(13): 11-22.
Chen Lei, Deng Xinyi, Chen Hongkun, et al. Review of the assessment and improvement of power system resilience[J]. Power System Protection and Control, 2022, 50(13): 11-22.
[12]
吴桂联, 朱乃璇, 陈浩, 等. 基于多尺度注意力机制与多资源协同优化的台风灾害下配电网韧性提升方法[J]. 电力科学与技术学报, 2025, 40(6): 90-100.
Wu Guilian, Zhu Naixuan, Chen Hao, et al. Method for enhancing distribution network resilience under typhoon disasters based on multi-scale attention mechanisms and multi-resource coordinated optimization[J]. Journal of Electric Power Science and Technology, 2025, 40(6): 90-100.
[13]
刘自发, 薛文博. 考虑智能软开关及多种应急资源协同的配电网韧性提升策略[J]. 电力科学与技术学报, 2025, 40(3): 1-16.
Liu Zifa, Xue Wenbo. Resilience enhancement strategy for distribution network considering coordination of intelligent soft open points and multiple emergency resources[J]. Journal of Electric Power Science and Technology, 2025, 40(3): 1-16.
[14]
麻灿皓, 陈丽娟, 吴志. 考虑多元灵活性资源协同的配电网韧性提升策略[J]. 电力工程技术, 2025, 44(1): 115-125.
Ma Canhao, Chen Lijuan, Wu Zhi. Resilience enhancement strategy for distribution networks considering multiple flexibility resources collaboration[J]. Electric Power Engineering Technology, 2025, 44(1): 115-125.
[15]
许寅, 王颖, 和敬涵, 等. 多源协同的配电网多时段负荷恢复优化决策方法[J]. 电力系统自动化, 2020, 44(2): 123-131.
Xu Yin, Wang Ying, He Jinghan, et al. Optimal decision-making method for multi-period load restoration in distribution network with coordination of multiple sources[J]. Automation of Electric Power Systems, 2020, 44(2): 123-131.
[16]
Shen Y C, Gu C H, Ma Z B, et al. A two-stage resilience enhancement for distribution systems under hurricane attacks[J]. IEEE Systems Journal, 2021, 15(1): 653-661.
[17]
高兆丽, 胥明凯, 丁素英, 等. 基于改进人工蜂群算法的配电网多点故障应急抢修优化调度[J]. 电力系统保护与控制, 2019, 47(13): 107-114.
Gao Zhaoli, Xu Mingkai, Ding Suying, et al. Optimization scheduling of multi-fault rush repair for distribution networks based on modified artificial bee colony algorithm[J]. Power System Protection and Control, 2019, 47(13): 107-114.
[18]
郁琛, 李尚轩, 谢云云, 等. 考虑交通网与配电网信息融合的台风后配电网抢修策略优化[J]. 电力系统自动化, 2022, 46(4): 15-24.
Yu Chen, Li Shangxuan, Xie Yunyun, et al. Optimization of post-typhoon rush repair strategy for distribution network considering information integration of traffic network and distribution network[J]. Automation of Electric Power Systems, 2022, 46(4): 15-24.
[19]
Yao S H, Wang P, Liu X C, et al. Rolling optimization of mobile energy storage fleets for resilient service restoration[J]. IEEE Transactions on Smart Grid, 2020, 11(2): 1030-1043.
[20]
黄婧杰, 聂焕, 张荐, 等. 主动配电网灾后韧性提升的孤岛划分与故障抢修协同方法[J]. 电力自动化设备, 2025, 45(10): 128-134.
Huang Jingjie, Nie Huan, Zhang Jian, et al. Collaborative method of island partition and fault repair for post-disaster resilience enhancement in active distribution network[J]. Electric Power Automation Equipment, 2025, 45(10): 128-134.
[21]
王涵, 孙磊, 文福拴. 考虑孤岛动态划分与融合的配电系统故障恢复策略[J]. 电力自动化设备, 2025, 45(10): 92-99.
Wang Han, Sun Lei, Wen Fushuan. Service restoration strategy of distribution system considering dynamic islanding partition and merging[J]. Electric Power Automation Equipment, 2025, 45(10): 92-99.
[22]
徐岩, 郭佳睿, 马天祥. 考虑韧性提升的配电网故障恢复与抢修协调优化[J]. 高电压技术, 2024, 50(12): 5516-5528.
Xu Yan, Guo Jiarui, Ma Tianxiang. Coordinated optimization of fault recovery and emergency repair for distribution network considering resilience improvement[J]. High Voltage Engineering, 2024, 50(12): 5516-5528.
[23]
徐岩, 张荟, 孙易洲. 基于变异粒子群算法的主动配电网故障恢复策略[J]. 电力自动化设备, 2021, 41(12): 45-53.
Xu Yan, Zhang Hui, Sun Yizhou. Fault recovery strategy of active distribution network based on mutation particle swarm optimization algorithm[J]. Electric Power Automation Equipment, 2021, 41(12): 45-53.
[24]
萧文聪, 陈俊斌, 余涛, 等. 基于专家知识嵌入强化学习的配电系统灾后恢复决策方法[J]. 电力系统自动化, 2025, 49(12): 91-100.
Xiao Wencong, Chen Junbin, Yu Tao, et al. Decision-making method for post-disaster distribution system restoration based on expert knowledge-embedded reinforcement learning[J]. Automation of Electric Power Systems, 2025, 49(12): 91-100.
[25]
江昌旭, 林骏驰, 郭辰, 等. 图强化学习在配电网优化中的应用综述[J]. 电网技术, 2026, 50(3): 922-936.
Jiang Changxu, Lin Junchi, Guo Chen, et al. A review on application of graph reinforcement learning in power distribution network optimization[J]. Power System Technology, 2026, 50(3): 922-936.
[26]
胡维昊, 曹迪, 黄琦, 等. 深度强化学习在配电网优化运行中的应用[J]. 电力系统自动化, 2023, 47(14): 174-191.
Hu Weihao, Cao Di, Huang Qi, et al. Application of deep reinforcement learning in optimal operation of distribution network[J]. Automation of Electric Power Systems, 2023, 47(14): 174-191.
[27]
张沛, 陈玉鑫, 王光华, 等. 基于图强化学习的配电网故障恢复决策[J]. 电力系统自动化, 2024, 48(2): 151-158.
Zhang Pei, Chen Yuxin, Wang Guanghua, et al. Fault recovery decision of distribution network based on graph reinforcement learning[J]. Automation of Electric Power Systems, 2024, 48(2): 151-158.
[28]
何小龙, 高红均, 王仁浚, 等. 基于图深度强化学习的有源配电网故障恢复方法[J]. 电网技术, 2025, 49(10): 4342-4352.
He Xiaolong, Gao Hongjun, Wang Renjun, et al. Fault recovery method of active distribution network based on graph deep reinforcement learning[J]. Power System Technology, 2025, 49(10): 4342-4352.
[29]
江昌旭, 周龙灿, 庄鹏威, 等. 基于数据-模型混合驱动方法的多类型移动应急资源优化调度策略[J]. 电网技术, 2026, 50(2): 858-868.
Jiang Changxu, Zhou Longcan, Zhuang Pengwei, et al. Scheduling optimization strategy for multi-type mobile emergency resources based on a data-model hybrid driven approach[J]. Power System Technology, 2026, 50(2): 858-868.
[30]
陈逸飞, 郑子萱, 肖先勇, 等. 数据-物理混合驱动的配电网运行韧性评估方法与提升策略[J]. 电力系统保护与控制, 2025, 53(10): 13-22.
Chen Yifei, Zheng Zixuan, Xiao Xianyong, et al. A data-physical hybrid-driven method for evaluating and enhancing the operational resilience of distribution networks[J]. Power System Protection and Control, 2025, 53(10): 13-22.
[31]
田启东, 张家琦, 陈颖, 等. 基于强化学习的灾后配电网应急抢修决策方法[J]. 电工电能新技术, 2023, 42(3): 66-75.
Abstract
灾后故障配电网的应急抢修及重构需求,面临着灾情的不确定性、多抢修队调配特性、抢修与重构耦合等多个挑战。本文基于强化学习构建了灾后多抢修队的抢修调配模型,设计了状态粗筛机制以固定强化学习状态及动作空间大小,搭建了配电网抢修恢复测试环境。在不同恢复模式、多个故障场景下的测试结果表明:强化学习在提出的多个评价指标下综合表现稳定,可以作为灾后动态抢修的可选方案。
Tian Qidong, Zhang Jiaqi, Chen Ying, et al. Post-disaster emergency repair and reconfiguration plan of faulty distribution system based on reinforcement learning[J]. Advanced Technology of Electrical Engineering and Energy, 2023, 42(3): 66-75.
The post-disaster emergency repair and reconfiguration plan of the faulty distribution system are confronted with the challenges from the uncertainty of the disaster, the deployment of multiple emergency repair teams and the coupling of emergency repair and reconfiguration. This paper builds a model for emergency repair deployment of multiple repair teams based on reinforcement learning. A state coarse screening mechanism is designed to fix the size of the state and action space of reinforcement learning. An environment based on OpenAI Gym is constructed for the repair and reconfiguration of the distribution system. The experimental results show that reinforcement learning has a stable comprehensive performance under the proposed multiple evaluation indicators, and it can be used as an optional solution for dynamic repair after a disaster.
[32]
Zhang Y C, Qiu F, Hong T Q, et al. Hybrid imitation learning for real-time service restoration in resilient distribution systems[J]. IEEE Transactions on Industrial Informatics, 2022, 18(3): 2089-2099.
[33]
赖俊, 魏竞毅, 陈希亮. 分层强化学习综述[J]. 计算机工程与应用, 2021, 57(3): 72-79.
Abstract
近年来强化学习愈发体现其强大的学习能力,2017年AlphaGo在围棋上击败世界冠军,同时在复杂竞技游戏星际争霸2和DOTA2中人类的顶尖战队也败于AI之手,但其自身又存在着自身的弱点,在不断的发展中瓶颈逐渐出现。分层强化学习因为能够解决其维数灾难问题,使得其在环境更为复杂,动作空间更大的环境中表现出更加优异的处理能力,对其的研究在近几年不断升温。对强化学习的基本理论进行简要介绍,对Option、HAMs、MAXQ这3种经典分层强化学习算法进行介绍,之后对近几年在分层的思想下提出的分层强化学习算法从3个方面进行综述,并对其进行分析,讨论了分层强化学习的发展前景和挑战。
Lai Jun, Wei Jingyi, Chen Xiliang. Overview of hierarchical reinforcement learning[J]. Computer Engineering and Applications, 2021, 57(3): 72-79.

In recent years, reinforcement learning has increasingly reflected its strong learning ability. In 2017, AlphaGo beat the world champion in go. Meanwhile, in the complex competitive games StarCraft 2 and dota2, the top human teams are also defeated by AI. However, it has its own weaknesses, and the bottleneck gradually appears in the continuous development. Hierarchical reinforcement learning can solve the problem of dimension disaster, which makes it show more excellent processing ability in the environment with more complex environment and larger action space. This paper briefly introduces the basic theory of reinforcement learning. It introduces three classical hierarchical reinforcement learning algorithms, option, hams and MAXQ. It summarizes and analyzes the hierarchical reinforcement learning algorithm proposed in recent years under the idea of stratification from three aspects. It discusses the development prospects and challenges of hierarchical reinforcement learning.

[34]
彭佳盛, 文云峰, 梁晓锐, 等. 基于弹性系数的配电网重构与故障抢修协同优化方法[J]. 电力系统自动化, 2025, 49(6): 88-95.
Peng Jiasheng, Wen Yunfeng, Liang Xiaorui, et al. Resilience coefficient based collaborative optimization method of distribution network reconfiguration and fault repair[J]. Automation of Electric Power Systems, 2025, 49(6): 88-95.
[35]
Huang Y X, Li G F, Chen C, et al. Resilient distribution networks by microgrid formation using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2022, 13(6): 4918-4930.
[36]
陈潇瑞, 李晓露, 林顺富. 考虑交通网延误的配电网灾后滚动抢修策略优化[J]. 现代电力, 2023, 40(5): 807-816.
Chen Xiaorui, Li Xiaolu, Lin Shunfu. Optimization of post-disaster rolling emergency repair strategy for distribution network considering transportation network delay[J]. Modern Electric Power, 2023, 40(5): 807-816.
[37]
Duan J L, Li S B, Guan Y, et al. Hierarchical reinforcement learning for self-driving decision-making without reliance on labelled driving data[J]. IET Intelligent Transport Systems, 2020, 14(5): 297-305.
[38]
王宏威, 阮晓波, 李雨龙, 等. 基于日变交通流量降雨损伤LSTM模型的城市路网韧性分析[J]. 中国安全科学学报, 2025, 35(12): 213-220.
Abstract
为探究常规降雨作用下城市路网性能时变性,建立合理的路网韧性评价体系,首先,构建城市道路路段交通流量降雨损伤的长短期记忆网络(LSTM)模型;其次,定义具有时变特性的路网性能函数;然后,基于韧性理念推导出路网时变韧性计算模型;最后,基于某城市降雨信息与路网交通数据及Sioux Falls网络,分析降雨对路段与路网时变特性的作用效果。结果表明:当降雨持续时间或降雨量增加时,路段工作日交通流量损伤均有增加趋势;路网韧性是各相关路段协同作用的综合体现,因而其对降雨作用呈现出较交通流量损伤更为迟钝的响应,且其值在常规降雨条件下通常不小于0.9;少雨水季节明显强于多雨水季节的路网韧性,而路网年韧性在未受到降雨灾害影响时一般会维持在一个较高水平上,且其在相近年份之间的差异并不显然。
Wang Hongwei, Ruan Xiaobo, Li Yulong, et al. Resilience analysis of urban road networks based on LSTM model for rainfall-induced day-to-day traffic flow degradation[J]. China Safety Science Journal, 2025, 35(12): 213-220.

To explore the time-varying performance of urban road networks under regular rainfall and establish a reasonable evaluation system for road network resilience, an LSTM model for the rainfall-induced traffic flow degradation in urban road segments was first constructed, and a network performance function with time-varying characteristics was second defined. Then, based on the resilience concept, a time-varying resilience calculation model for road networks was derived. Finally, the effect of rainfall on the time-varying characteristics of road segments and networks was investigated based on rainfall information and traffic data of a certain city and the Sioux Falls network. The results show that when the duration or amount of rainfall increases, the traffic flow degradation of road segments increases on weekdays. The road network resilience is a comprehensive reflection of the synergistic effect of various related road segments, and therefore its response to rainfall is slower than that of traffic flow degradation, and its value is generally not less than 0.9 under regular rainfall conditions. The road network resilience during low rainfall seasons is significantly stronger than that during high rainfall seasons, and the annual resilience of road networks is usually maintained at a higher level when not affected by rainfall disasters, and the difference between adjacent years is not significant.

Footnotes

利益冲突声明(Conflict of Interests): 所有作者声明不存在利益冲突。

作者贡献声明(Authors' Contributions): 薛章负责研究设计、资料收集及分析,撰写论文;李晓露负责研究指导,修订论文,审核论文;柳劲松负责研究指导,审核论文;林顺富负责审核论文。所有作者均阅读并同意了论文终稿内容。

Funding

National Natural Science Foundation of China(52577119)
PDF(4274 KB)

Accesses

Citation

Detail

Recommended

/

〈 〉