基于改进U型神经网络的路面裂缝检测方法

惠冰; 李远见

doi:10.3963/j.jssn.1674-4861.2023.01.011

基于改进U型神经网络的路面裂缝检测方法

doi: 10.3963/j.jssn.1674-4861.2023.01.011

惠冰^{1, 2,},
李远见³

1.
长安大学公路学院西安 710064
2.
长安大学民航机场智慧建造与维养重点实验室西安 710064
3.
内蒙古高速公路养护有限责任公司呼和浩特 010010

基金项目:

国家重点研发计划项目 2021YFB2601000

国家自然科学基金项目 52178409

内蒙古自治区交通运输科技项目 NJ-2021-17

详细信息

通讯作者:
惠冰（1982—），博士，副教授.研究方向：路面检测与养护管理

中图分类号: U491.2
计量
- 文章访问数: 577
- HTML全文浏览量: 205
- PDF下载量: 37
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-06
- 网络出版日期: 2023-05-13

A Detection Method for Pavement Cracks Based on an Improved U-Shaped Network

HUI Bing^{1, 2
,},
LI Yuanjian³

1.
School of Highway, Chang'an University, Xi'an 710064, China
2.
Key Laboratory of Intelligent Construction and Maintenance of CAAC, Chang'an University, Xi'an 710064, China
3.
Inner Mongolia Expressway Maintenance Co., Ltd, Hohhot 010010, China

摘要

摘要: 针对传统的裂缝分割算法难以识别狭窄裂缝且分割边缘不精准，从而造成识别精度较低的问题，研究了基于改进U型神经网络（Unet）的路面裂缝检测方法。由于传统Unet特征提取网络是层次较浅的浅层神经网络，难以提取更复杂的裂缝特征信息，故本文以牛津大学视觉几何组网络（VGG16）作为传统Unet的特征提取网络，提高网络的裂缝特征提取能力；为抑制高低阶特征融合时产生的无用特征，本文在模型解码部分添加压缩与激励单元（SE block），构建裂缝注意力单元，使得网络可以关注不同通道下的裂缝特征，建立了基于SE block和VGG16的改进Unet网络（SE-VUnet）。研究采用迁移学习的方法，将在ImageNet上预训练好的VGG16网络权重迁移到裂缝检测中。通过挑选Crack500数据集，并使用摄像头采集图片构建1 600张路面裂缝数据集，再次训练SE-VUnet模型，获得裂缝区域分割结果。以查准率（precision）与查全率（recall）的加权调和平均值F1和雅卡尔（Jaccard）相似系数作为量化评价指标。将SE-VUnet分别与Unet、SOLO v2、Mask R-CNN以及Deeplabv3+进行分割效果和实时性对比。研究结果表明：SE-VUnet模型的综合F1和雅卡尔系数分别为0.840 3和0.722 1，相比于Unet分别高出了1.04%和1.51%，且均高于其他3种对比模型；SE-VUnet的单帧图片预测时间为89 ms，在分割效果提升明显的情况下仅比Unet慢5 ms，优于其他模型。
- 信息工程 /
- 裂缝检测 /
- U型神经网络 /
- 深度学习 /
- 语义分割
Abstract: Due to the traditional crack segmentation algorithm is difficult to identify narrow cracks and the segmentation edge is not accurate. This paper proposes a pavement crack detection method based on improved U-Shaped Network (Unet) to increase detection accuracy. Since traditional Unet is a type of"shallow"neural network, it is not good for extracting complex crack features. The Oxford University Visual Geometry Group Network (VGG16) is therefore used for feature extraction, in order to improve the accuracy of crack feature extraction. In addition, the fusion of high- and low-order features generate several useless features. The compression and excitation unit (SE block) is added to the decoding part of the model to develop a crack attention unit which allows the network to focus on the crack features under different channels. Moreover, an improved Unet is proposed by combining SE block with VGG16 (SE-VUnet). In addition, a transfer learning method is used to transfer the pre-trained VGG16 network weight on ImageNet for crack detection. By selecting the Crack500 data set and using the camera to collect images to develop1600 pavement crack data sets, the SE-VUnet model is trained again to obtain the crack segmentation results. The weighted harmonic mean F1 of Precision and Recall and Jaccard similarity coefficient are used as quantitative evaluation indicators. The segmentation effect and real-time performance of SE-VUnet are compared with Unet and three other representative models. Study results show that the comprehensive F1 and the Jaccard coefficient of SE-VUnet model is 0.840 3 and 0.722 1, which is 1.04% and 1.51% higher than Unet respectively, as well as other three comparison models. The time for the SE-VUnet to screen a single-frame image is 89 ms, which is only 5ms slower than the Unet but with a significant improvement over the crack segmentation and detection process.
- information engineering /
- crack detection /
- Unet /
- deep learning /
- semantic segmentation

HTML全文

图 1 整体流程图

Figure 1. Overall flow chart

下载: 全尺寸图片幻灯片

图 2 网络模型结构

Figure 2. Structures of network models

下载: 全尺寸图片幻灯片

图 3 SE-VUnet模型

Figure 3. The SE-VUnet model

下载: 全尺寸图片幻灯片

图 4 Unet编码结构与本文特征提取网络

Figure 4. Unet encoding structure and feature extraction network of this paper

下载: 全尺寸图片幻灯片

图 5 SE block结构

Figure 5. SE block

下载: 全尺寸图片幻灯片

图 6 Unet解码结构改进

Figure 6. Improvement of Unet decoding structure

下载: 全尺寸图片幻灯片

图 7 传统机器学习和迁移学习对比

Figure 7. Comparison of traditional machine learning and transfer learning

下载: 全尺寸图片幻灯片

图 8 图像采集平台

Figure 8. Image acquisition platform

下载: 全尺寸图片幻灯片

图 9 路面裂缝图像示例

Figure 9. Pavement crack image

下载: 全尺寸图片幻灯片

图 10 SE-VUnet与Unet的F1和雅卡尔系数

Figure 10. F1 and Jaccard coefficients of SE-VUnet and Unet

下载: 全尺寸图片幻灯片

图 11 狭长裂缝分割结果

Figure 11. Long and narrow crack segmentation result

下载: 全尺寸图片幻灯片

图 12 宽裂缝预测结果

Figure 12. Non-narrow crack segmentation result

下载: 全尺寸图片幻灯片

图 13 路面裂缝在多个模型下的测试结果

Figure 13. Test results of pavement cracks under multiple models

下载: 全尺寸图片幻灯片

图 14 多个模型在裂缝宽度较窄和对比度较低情况下的分割结果

Figure 14. Segmentation results of multiple models in the case of narrow crack width and low contrast

下载: 全尺寸图片幻灯片

表 1 试验数据

Table 1. Experimental data

训练验证集	数量/张	测试集	数量/张
狭长裂缝	498	狭长裂缝	91
宽裂缝	852	宽裂缝	159

下载: 导出CSV

表 2 不同模型的推理时间

Table 2. Inference time of different models

模型	推理时间/s
SE-VUnet	0.089
Unet	0.084
SOLO v2	0.130
Mask R-CNN	0.162
Deeplabv3+	0.093
文献[10]	0.360

下载: 导出CSV

表 3 不同模型的F1和Jaccard指标

Table 3. F1 and Jaccard coefficients of different models

模型	Precision	Recall	F1	Jaccard
SE-VUnet	0.803 1	0.881 2	0.840 3	0.722 1
Unet	0.805 3	0.856 4	0.829 9	0.707 0
SOLO v2	0.835 3	0.813 9	0.824 5	0.702 1
Mask R-CNN	0.633 2	0.733 1	0.679 5	0.511 7
Deeplabv3+	0.863 4	0.533 3	0.659 4	0.501 0

下载: 导出CSV

参考文献(24)

[1]	马建, 赵祥模, 贺拴海, 等. 路面检测技术综述[J]. 交通运输工程学报, 2017, 17(5): 121-137. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201705012.htm MA J, ZHAO X M, HE S H, et al. Review of pavement detection technology[J]. Journal of Traffic and Transportation Engineering, 2017, 17(5): 121-137. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201705012.htm
[2]	AMHAZ R, CHAMBON S, IDIER J, et al. Automatic crack detection on 2D pavement images: An algorithm based on minimal path selection[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(10): 2718-2729. doi: 10.1109/TITS.2015.2477675
[3]	PENG L, CHAO W, LI S, et al. Research on crack detection method of airport runway based on Twice-Threshold segmentation[C]. 5^th International Conference on Instrumentation & Measurement, Computer, Communication and Control (IMCCC), Qinhuangdao, China: IEEE, 2015.
[4]	王兴建, 秦国锋, 赵慧丽. 基于多级去噪模型的路面裂缝检测方法[J]. 计算机应用, 2010, 30(6): 1606-1609, 1612. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY201006051.htm WANG X J, QIB G F, ZHAO H L. Pavement crack detection method based on multilevel denoising model[J]. Journal of Computer Applications, 2010, 30(6): 1606-1609, 1612. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY201006051.htm
[5]	张志华, 邓砚学, 张新秀. 基于改进SegNet的沥青路面病害提取与分类方法[J]. 交通信息与安全, 2022, 40(3): 127-135. doi: 10.3963/j.jssn.1674-4861.2022.03.013 ZHANG Z H, DENG Y X, ZHANG X X. A method for detecting and differentiating asphalt pavement distress based on an improved SegNet algorithm[J]. Journal of Transport Information and Safety, 2022, 40(3): 127-135. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.03.013
[6]	ZHAO H, QIN G, WANG X. Improvement of canny algorithm based on pavement edge detection[C]. 3^rd International Congress on Image and Signal Processing, Yantai, China: IEEE, 2010.
[7]	SALMAN M, MATHAVAN S, KAMAL K, et al. Pavement crack detection using the Gabor filter[C]. 16^th international IEEE conference on intelligent transportation systems(ITSC 2013), New York, America: IEEE, 2013
[8]	章天杰, 韩海航. 基于残差神经网络的沥青路面裂缝识别分类研究[J]. 公路, 2021, 66(10): 24-29. https://www.cnki.com.cn/Article/CJFDTOTAL-GLGL202110004.htm ZHANG T J, HAN H H. Research on identification and classification of asphalt pavement cracks using residual neural network[J]. Highway, 2021, 66(10): 24-29. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-GLGL202110004.htm
[9]	沙爱民, 童峥, 高杰. 基于卷积神经网络的路表病害识别与测量[J]. 中国公路学报, 2018, 31(1): 1-10. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm SHA A M, TONG Z, GAO J. Recognition and measurement of pavement disasters based on convolutional neural networks[J]. China Journal of Highway and Transport, 2018, 31 (1): 1-10. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm
[10]	孙朝云, 裴莉莉, 李伟, 等. 基于改进Faster R-CNN的路面灌封裂缝检测方法[J]. 华南理工大学学报(自然科学版), 2020, 48(2): 84-93. SUN Z Y, PEI L L, LI W, et al. Pavement potting crack detection method based on improved Faster R-CNN[J]. Journal of South China University of Technology(Natural Science Edition), 2020, 48(2): 84-93. (in Chinese)
[11]	晏班夫, 徐观亚, 栾健, 等. 基于Faster R-CNN与形态法的路面病害识别[J]. 中国公路学报, 2021, 34(9): 181-193. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm YAN B F, XU G Y, LUAN J, et al. Pavmenet distress detection based on Faster R-CNN and morphological operations[J]. China Journal of Highway and Transport, 2021, 34 (9): 181-193. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm
[12]	RONNEBERGER O, FISCHER P, BROX T, et al. U-Net: Convolutional networks for biomedical image segmentation[C]. Medical Image Computing and Computer Assisted Intervention, Berlin, Germany: Springer, 2015.
[13]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL](2015-4-10)[2023-2-27]. https://doi.org/10.48550/arXiv.1409.1556.
[14]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[J]. IEEE Transcations on Pattern Analysis and Machine Intelligence. 2017, 42(8): 7132-7141.
[15]	杨炜, 黄立红, 赵祥模, 等. 基于FRRN注意力监督的沥青路面积水区域分割[J]. 交通运输工程学报, 2021, 21(5): 309-322. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202105029.htm YANG W, HUANG L H, ZHAO X M, et al. Puddle area segmentation of asphalt pavements based on FRRN attention and supervision[J]. Journal of Traffic and Transportation Engineering, 2021, 21(5): 309-322. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202105029.htm
[16]	林禹, 赵泉华, 李玉. 1种基于深度传递迁移学习的遥感影像分类方法[J]. 地球信息科学学报, 2022, 24(3): 495-507. LIN Y, ZHAO Q H, LI Y. A remote sensing image classification method based on deep transitive transfer learning[J]. Journal of Geo-information Science, 2022, 24(3): 495-507. (in Chinese)
[17]	YANG F, ZHANG L, YU S, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(4): 1525-1535.
[18]	WANG X, ZHANG R, KONG T, et al. SOLOv2: Dynamic and fast instance segmentation[J]. Advances in Neural Information Processing Systems, 2020(33): 17721-17732.
[19]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]. The IEEE International Conference on Computer Vision, Venice, Italy: ICCV, 2017.
[20]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. The European Conference on Computer Vision, Munich, Germany: ECCV, 2018.
[21]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[22]	张继凯, 赵君, 张然, 等. 深度学习的图像实例分割方法综述[J]. 小型微型计算机系统, 2021, 42(1): 161-171. ZHANG J K, ZHAO J, ZHANG R, et al. Survey of image instance segmentation methods using deep learning[J]. Journal of Chinese Computer Systems, 2021, 42(1): 161-171. (in Chinese)
[23]	吴忧, 袁雪. 基于改进SOLOv2的复杂场景下智能机器人巡检识别算法[J]. 北京交通大学学报, 2022, 46(5): 95-106. WU Y, YUAN X. Inspection and identification algorithm based on improved SOLOv2 of intelligent robot in complex environment[J]. Journal of Beijing Jiaotong University, 2022, 46(5): 95-106. (in Chinese)
[24]	邱实, 陈斌, 胡文博, 等. 基于深度学习和虚拟模型的路面全域伤损状态自动化感知[J/OL]. 中国公路学报: (2022-11)[2023-02-25]. http://kns.cnki.net/kcms/detail/61.1313.U.20221124.0917.004.html. QIU S, CHEN B, HU W B, et al. Automated pavement-wide injury state sensing based on deep learning and virtual models[J/OL]. China Journal of Highway and Transport, (2022-11)[2023-02-25]. http://kns.cnki.net/kcms/detail/61.1313.U.20221124.0917.004.html. (in Chinese)