Super-Resolution Reconstruction of Thermal Imaging of Power Equipment Based on Edge-Enhancement Generative Adversarial Network

doi:10.12204/j.issn.1000-7229.2021.07.010

Abstract

Abstract:

The temperature data of non-contact infrared thermal imaging is of great significance for condition monitoring and health assessment of power equipment. However, the high cost and technical barriers of high-resolution infrared imager limit the application of high-resolution thermal imaging in on-line monitoring of equipment in power internet of things. Super-resolution reconstruction meets the resolution requirements and reduces the cost at the same time. In this paper, the enhancement of thermal imaging of power equipment is realized by constructing an improved generative adversarial network of edge enhancement. The network adds a depth residual shrinkage network module on the basis of SRGAN (super-resolution generative adversarial networks), which improves the stability of training on the basis of reducing image noise, enhances the reconstruction of image peak information through edge-extraction technology, and improves the effect of edge restoration. The example analysis shows that, PSNR (peak signal-to-noise ratio ) and SSIM (structural similarity) indicators are used to analyze the overall data and edge data after reconstruction, both are significantly improved, and the subjective visual effect after reconstruction is more clear, which has high engineering practical value.

Key words: power equipment thermal imaging, super-resolution reconstruction, deep residual shrinkage network, edge extraction

CLC Number:

TM769

LIU Yunfeng, YANG Jinbiao, HAN Jinfeng, PENG Yihao, ZHAO Hongshan. Super-Resolution Reconstruction of Thermal Imaging of Power Equipment Based on Edge-Enhancement Generative Adversarial Network[J]. ELECTRIC POWER CONSTRUCTION, 2021, 42(7): 83-89.

Figures/Tables 8

Fig.1 Model structure of EE-SRGAN

Fig.2 Residual shrinkage network (RSN)

Fig.3 Change process of PSNR with the number of iterations

Fig.4 Change process of SSIM with the number of iterations

Table 1 Evaluation of reconstructed infrared image magnified 4 times with different algorithms

Table 2 Evaluation of edge data of reconstructed infrared image magnified 4 times with different algorithms

Fig.5 Comparison of reconstruction results

Fig.6 Edge comparison of reconstruction results

References 23

[1]	裴少通. 基于红外紫外成像检测技术的绝缘子运行状态分析与评估[D]. 北京: 华北电力大学, 2019.
	PEI Shaotong. Analysis and evaluation of insulator operation status based on infrared and ultraviolet imaging detection technology[D]. Beijing: North China Electric Power University, 2019.
[2]	唐艳秋, 潘泓, 朱亚平, 等. 图像超分辨率重建研究综述[J]. 电子学报, 2020, 48(7):1407-1420.
	TANG Yanqiu, PAN Hong, ZHU Yaping, et al. A survey of image super-resolution reconstruction[J]. Acta Electronica Sinica, 2020, 48(7):1407-1420.
[3]	李伟, 张旭东. 基于卷积神经网络的深度图像超分辨率重建方法[J]. 电子测量与仪器学报, 2017, 31(12):1918-1928.
	LI Wei, ZHANG Xudong. Depth image super-resolution reconstruction based on convolution neural network[J]. Journal of Electronic Measurement and Instrumentation, 2017, 31(12):1918-1928.
[4]	端木春江, 沈碧婷. 基于邻域回归的医学图像超分辨率重建[J]. 计算机应用研究, 2020, 37(12):3792-3794,3802.
	DUANMU Chunjiang, SHEN Biting. Medical image super-resolution reconstruction based on neighborhood regression[J]. Application Research of Computers, 2020, 37(12):3792-3794,3802.
[5]	卢涛, 陈冲, 许若波, 等. 基于边缘增强生成对抗网络的人脸超分辨率重建[J]. 华中科技大学学报(自然科学版), 2020, 48(1):87-92.
	LU Tao, CHEN Chong, XU Ruobo, et al. Face hallucination based on edge enhanced generative adversarial network[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2020, 48(1):87-92.
[6]	JIANG X H, XU Y F, WEI P P, et al. CT image super resolution based on improved SRGAN[C]//2020 5th International Conference on Computer and Communication Systems (ICCCS). Shanghai, China: IEEE, 2020: 363-367.
[7]	赵秀影, 苏耘, 董艳芹, 等. 一种基于小波与双三次插值的CCD图像超分辨方法[J]. 计算机应用研究, 2009, 26(6):2365-2367.
	ZHAO Xiuying, SU Yun, DONG Yanqin, et al. Kind of super-resolution method of CCD image based on wavelet and bicubic interpolation[J]. Application Research of Computers, 2009, 26(6):2365-2367.
[8]	YANG J, WRIGHT J, HUANG T, et al. Image superresolution as sparse representation of raw image patches[C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA:IEEE, 2008.
[9]	DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307. doi: 10.1109/TPAMI.2015.2439281 URL
[10]	DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network[M]//Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 391-407.
[11]	SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[J]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 1874-1883.
[12]	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016: 1646-1654.
[13]	LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 105-114.
[14]	WANG X T, YU K, WU S X, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[DB/OL]. [2020-04-12]. https://arxiv.org/abs/1809.00219.
[15]	SHANG T Z, DAI Q J, ZHU S C, et al. Perceptual extreme super resolution network with receptive field block[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2020: 1778-1787.
[16]	CAI J R, ZENG H, YONG H W, et al. Toward real-world single image super-resolution: A new benchmark and a new model[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE, 2019: 3086-3095.
[17]	YUAN Y, LIU S Y, ZHANG J W, et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Salt Lake City, UT, USA: IEEE, 2018: 814-81409.
[18]	GUAN J W, PAN C, LI S N, et al. SRDGAN: Learning the noise prior for Super Resolution with Dual Generative Adversarial Networks[DB/OL]. [2020-04-12]..
[19]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[C]Proceedings of the 34^th International Conference on Machine Learnings . 2017:214-223.
[20]	CHEN Y, TAI Y, LIU X M, et al. FSRNet: end-to-end learning face super-resolution with facial priors[J]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 2492-2501.
[21]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11):139-144. doi: 10.1145/3422622 URL
[22]	ZHAO M H, ZHONG S S, FU X Y, et al. Deep residual shrinkage networks for fault diagnosis[J]. IEEE Transactions on Industrial Informatics, 2020, 16(7):4681-4690. doi: 10.1109/TII.9424 URL
[23]	Free FLIR thermal dataset for algorithm training[EB/OL]. [2020-06-12]. https://www.flir.com/oem/adas/adas-dataset-form/