融合改进YOLO和背景差分的道路抛洒物检测算法

周勇; 张炳振; 张枭勇; 刘宇鸣

doi:10.3963/j.jssn.1674-4861.2022.05.012

融合改进YOLO和背景差分的道路抛洒物检测算法

doi: 10.3963/j.jssn.1674-4861.2022.05.012

深圳市城市交通规划设计研究中心股份有限公司广东深圳 518000

基金项目:

深圳市工业和信息化产业发展专项资金项目 20190830020003

详细信息

通讯作者:
周勇（1990—），硕士，中级工程师. 研究方向：交通数字孪生、车路协同等. E-mail: zhouyong@sutpc.com

中图分类号: U491
计量
- 文章访问数: 1102
- HTML全文浏览量: 290
- PDF下载量: 125
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-02
- 网络出版日期: 2022-12-05

A Detection Method for Abandoned Materials on Road Surface Based on an Improved YOLO and Background Differencing Algorithm

Shenzhen Urban Transport Planning Center Co., Ltd, Shenzhen 518000, Guangdong, China

摘要

摘要: 针对现有道路抛洒物检测算法识别准确率低、识别种类有限、实时检测效率低的问题，探索了将深度学习目标检测和传统图像处理相结合的抛洒物检测算法。提出在YOLOv5s目标检测算法基础上，对模型结构进行修改以满足实时性需求。具体地，使用卷积优化YOLO中的降采样模块，采用Ghost网络替代原始的特征提取网络以减少计算量，根据抛洒物检测对象的特点设计符合数据集的锚框以提高目标识别准确度。使用优化后的YOLO检测道路场景中车辆、行人作为交通参与者得到检测框，在检测框周围设定感兴趣区域，并在感兴趣区域内用背景差分算法实现前景目标识别。计算前景目标与YOLO检测结果的交并比，排除交通参与者目标后实现道路抛洒物的识别。针对交通参与者检测的实验结果表明，改进后的YOLO检测算法在整体识别精度没有损失的情况下单帧检测速度为20.67 ms，比原始YOLO检测算法速度提升16.42%。真实道路抛洒物实验结果表明，在没有抛洒物训练数据情况下，传统混合高斯模型算法平均精度值为0.51，采用融合改进YOLO和背景差分的抛洒物检测算法平均精度值为0.78，算法检测精度提高52.9%。改进后算法可适用于没有抛洒物数据或正样本数据稀少的情况。该算法在嵌入式设备Jetson Xavier NX上单帧检测速度达到24.4 ms，可实现抛洒物的实时检测。
- 交通工程 /
- 抛洒物识别 /
- 目标检测 /
- YOLO算法 /
- 混合高斯模型
Abstract: Due to the problems such as low detection accuracy, limited types of materials that can be detected, and slow speed of detection algorithms for abandoned materials, a detection algorithm combining target detection based on deep learning and traditional image processing is proposed. The structure of the YOLOv5s detection algorithm is modified, in order to have a capacity of real-time detection. The downsampling module in YOLO is optimized using convolution; the original feature extraction network is replaced with a Ghost network to reduce the computational burden, and the anchor frame is designed to match the dataset according to the characteristics of the detected objects to improve the detection accuracy. The optimized YOLO algorithm is used to detect vehicles and pedestrians as traffic participants in the road scenes and the region of interest is set based on the detection results. By detecting foreground targets in the region of interest with a background differencing algorithm, and calculating the intersection and merger ratio between the foreground target and the detection results from the YOLO algorithm, the detection of road abandoned object can be completed after excluding the detected traffic participants. In the experiments of target detection, the improved YOLO algorithm has a detection speed of 20.67 ms for each frame without any drop in the detection accuracy, which is 16.42% faster than that of the original YOLO detection algorithm. Experimental results indicate that the mean average precision (mAP) of the traditional mixed Gaussian model algorithm is 0.51, while the mAP of the detection algorithm using the improved YOLO and background differencing is 0.78. The detection accuracy of the algorithm improves by 52.9%. The improved algorithm can be applied to scenarios where there is no data or sample data is limited. The detection time required for each frame is only 24.4 ms when the proposed algorithm is installed on a Jetson Xavier NX computer, and therefore it can be used to carry out real-time detection of abandoned materials.
- traffic engineering /
- abandoned object detection /
- object detection /
- YOLO algorithm /
- Gaussian mixture model

HTML全文

图 1 抛洒物检测算法流程

Figure 1. Abandoned object detection algorithm flowchart

下载: 全尺寸图片幻灯片

图 2 YOLO backbone网络结构图

Figure 2. YOLO backbone network structure diagram

下载: 全尺寸图片幻灯片

图 3 Focus模块示意图

Figure 3. Focusmodule schematic

下载: 全尺寸图片幻灯片

图 4 CSP模块网络结构图

Figure 4. CSP module network structure diagram

下载: 全尺寸图片幻灯片

图 5 Ghost Bottleneck模块网络结构

Figure 5. Ghost Bottleneck network structure diagram

下载: 全尺寸图片幻灯片

图 6 ROI示意图

Figure 6. ROI schematic

下载: 全尺寸图片幻灯片

图 7 IoU示意图

Figure 7. IoU schematic

下载: 全尺寸图片幻灯片

图 8 Precision-Recall曲线计算AP值示意图

Figure 8. Precision-Recall curvecalculation AP diagram

下载: 全尺寸图片幻灯片

图 9 数据集图像示例

Figure 9. Example dataset image

下载: 全尺寸图片幻灯片

图 10 检测系统示意图

Figure 10. Schematic diagram of detection system

下载: 全尺寸图片幻灯片

图 11 不同场景洒物检测算法结果

Figure 11. Algorithm results of abandoned object detection in different scenes

下载: 全尺寸图片幻灯片

图 12 不同抛洒物检测算法结果对比图

Figure 12. Comparison ofthe results of different abandoned object detection algorithms

下载: 全尺寸图片幻灯片

表 1 Bottleneck和Ghost Bottleneck参数量对比

Table 1. Parameters of Bottleneck and Ghost Bottleneck

模块	参数量/MB
BottleneckCSP	7.5
Ghost Bottleneck 1	4.9
Ghost Bottleneck 2	2.5

下载: 导出CSV

表 2 交通参与者数据集各类别分布

Table 2. The distribution of each category in the traffic participant dataset

交通参与者	数量	占比/%
小汽车Car	16 720	45
行人Pedestrian	8 916	24
公交车Bus	6 315	17
货车Truck	5 201	14

下载: 导出CSV

表 3 YOLO交通参与者检测算法结果对比

Table 3. Comparison of YOLO traffic participant detection algorithm results

算法	mAP	小汽车	公交车	行人	货车	检测时间/ms
YOLO	0.825 5	0.839	0.820	0.824	0.819	24.73
YOLO+卷积降采样	0.823 5	0.836	0.821	0.825	0.812	22.94
YOLO+Ghost Bottleneck	0.833 3	0.852	0.836	0.827	0.818	22.18
本文算法	0.831 0	0.857	0.830	0.826	0.811	20.67
注：mAP指4类交通参与者AP值的平均值，反映检测模型整体性能。

下载: 导出CSV

表 4 抛洒物检测算法结果对比

Table 4. Comparison of abandoned object detection algorithmresults

算法	mAP	检测时间/ms
传统混合高斯算法	0.51	18.1
改进混合高斯算法	0.62	20.5
实例分割模型	0.76	290
本文算法	0.78	24.4

下载: 导出CSV

参考文献(23)

[1]	李杰, 曾叙砜, 李平, 等. 道路交通安全文献的知识可视化综述[J]. 交通信息与安全, 2020, 38(1): 13-19. doi: 10.3963/j.jssn.1674-4861.2020.01.002 LI J, ZENG X F, LI P, et al. Visualization review of road traffic safety literature[J]. Journal of Transport Information and Safety, 2020, 38(1): 13-19. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2020.01.002
[2]	MCHUGH J M, KONRAD J, SALIGRAMA V, et al. Foreground-adaptive background subtraction[J]. IEEE Signal Processing Letters, 2009, 16(5): 390-393. doi: 10.1109/LSP.2009.2016447
[3]	SZWOCH G. Extraction of stable foreground image regions for unattended luggage detection[J]. Multimedia Tools and Applications, 2016, 75(2): 761-786. doi: 10.1007/s11042-014-2324-4
[4]	ZENG Y, LAN J, RAN B, et al. A novel abandoned object detection system based on three-dimensional image information[J]. Sensors, 2015, 15(3): 6885-6904. doi: 10.3390/s150306885
[5]	汪贵平, 马力旺, 郭璐, 等. 高速公路抛洒物事件图像检测算法[J]. 长安大学学报(自然科学版), 2017, 37(5): 81-88. doi: 10.3969/j.issn.1671-8879.2017.05.011 WANG G P, MA L W, GUO L, et al. Image detection algorithm for incident of discarded things in highway[J]. Journal of Chang'an University(Natural Science Edition), 2017, 37(5): 81-88. (in Chinese) doi: 10.3969/j.issn.1671-8879.2017.05.011
[6]	杨杰超, 许江淳, 陆万荣, 等. 嵌入式高速公路异物侵限的检测与跟踪研究[J]. 自动化仪表, 2018, 39(12): 70-73. https://www.cnki.com.cn/Article/CJFDTOTAL-ZDYB201812017.htm YANG J C, XU J C, LU W R, et al. Study on embedded expressway clearance intrusion detection and tracking[J]. Process Automation Instrumentation, 2018, 39(12): 70-73. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZDYB201812017.htm
[7]	宁正, 牛宏侠, 张肇鑫. 基于改进混合高斯模型的铁轨异物入侵检测方法[J]. 传感器与微系统, 2021, 40(5): 146-149. https://www.cnki.com.cn/Article/CJFDTOTAL-CGQJ202105041.htm NING Z, NIU H X, ZHANG Z X. Railway foreign body intrusion detection method based on improved mixed Gaussian model[J]. Transducer and Microsystem Technologies, 2021, 40(5): 146-149. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CGQJ202105041.htm
[8]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]. Conference on Computer Vision and Pattern Recognition, Las Vegas, USA: IEEE, 2016.
[9]	金瑶, 张锐, 尹东. 城市道路视频中小像素目标检测[J]. 光电工程, 2019, 46(9): 76-83. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201909009.htm JIN Y, ZHANG R, YIN D. Object detection for small pixel in urban roads videos[J]. Opto-Electronic Engineering, 2019, 46(9): 76-83. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201909009.htm
[10]	章悦, 张亮, 谢非, 等. 基于实例分割模型优化的道路抛洒物检测算法[J]. 计算机应用, 2021, 41(11): 3228-3233. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY202111021.htm ZHANG Y, ZHANG L, XIE F, et al. Road abandoned object detection algorithm based on optimized instance segmentation model[J]. Journal of Computer Applications, 2021, 41(11): 3228-3233. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY202111021.htm
[11]	周雯, 史天运, 李平, 等. 基于深度学习的动车组运行安全图像异物检测[J]. 交通信息与安全, 2019, 37(6): 48-55. doi: 10.3963/j.issn.1674-4861.2019.06.006 ZHOU W, SHI T Y, LI P, et al. Foreign objects detection of safety image of EMU operation based on deep learning[J]. Journal of Transport Information and Safety, 2019, 37(6): 48-55. (in Chinese) doi: 10.3963/j.issn.1674-4861.2019.06.006
[12]	何文玉, 杨杰, 张天露. 基于深度学习的轨道异物入侵检测算法[J]. 计算机工程与设计, 2020, 41(12): 3376-3383. https://www.cnki.com.cn/Article/CJFDTOTAL-SJSJ202012046.htm HE W Y, YANG J, ZHANG T L. Orbital foreign object intrusion detection algorithm based on deep learning[J]. Computer Engineering and Design, 2020, 41(12): 3376-3383. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-SJSJ202012046.htm
[13]	ZHUANG F, QI Z, DUAN K, et al. A comprehensive survey on transfer learning[J]. Proceedings of the IEEE, 2020, 109(1): 43-76.
[14]	SUNG F, YANG Y, ZHANG L, et al. Learning to compare: Relation network for few-shot learning[C]. Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018.
[15]	ZHOU F, ZHAO H, NIE Z. Safety helmet detection based on YOLOv5[C]. 2021 IEEE International Conference on Power Electronics, Computer Applications(ICPECA), Shenyang, China: IEEE, 2021.
[16]	程健, 王东伟, 杨凌凯, 等. 一种改进的高斯混合模型煤矸石视频检测方法[J]. 中南大学学报(自然科学版), 2018, 49(1): 118-123. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD201801016.htm CHENG J, WANG D W, YANG L K, et al. An improved Gaussian mixture model for coal gangue video detection[J]. Journal of Central South University(Science and Technology), 2018, 49(1): 118-123. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD201801016.htm
[17]	郭保青, 杨柳旭, 史红梅, 等. 基于快速背景差分的高速铁路异物侵入检测算法术[J]. 仪器仪表学报, 2016, 37(6): 1371-1378. GUO B Q, YANG L X, SHI H M, et al. High-speed railway clearance intrusion detection algorithm with fast background subtraction[J]. Chinese Journal of Scientific Instrument, 2016, 37(6): 1371-1378. (in Chinese)
[18]	蔡念, 陈世文, 郭文婷, 等. 融合高斯混合模型和小波变换的运动目标检测[J]. 中国图象图形学报, 2011, 16(9): 1716-1721. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201109025.htm CAI N, CHEN S W, GUO W T, et al. Moving object detection using Gaussian mixture model and wavelet transform[J]. Journal of Image and Graphics, 2011, 16(9): 1716-1721. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201109025.htm
[19]	WANG C Y, LIAO H Y M, WU Y H, et al. CSPNet: A new backbone that can enhance learning capability of CNN[C]. Conference on Computer Vision and Pattern Recognition Workshops, Piscataway: IEEE, 2020.
[20]	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(9): 1904-1916.
[21]	HAN K, WANG Y, TIAN Q, et al. Ghostnet: More features from cheap operations[C]. Conference on Computer Vision and Pattern Recognition, Piscataway: IEEE, 2020.
[22]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: Common objects in context[J]. Lecture Notes in Computer Science, 2014(8693): 740-755.
[23]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]. Conference on Computer Vision and Pattern Recognition, Honolulu: IEEE, 2017.