面向数据脱敏的交通场景车牌检测方法

应申; 曾卓源; 张纪元

doi:10.3963/j.jssn.1674-4861.2024.06.009

面向数据脱敏的交通场景车牌检测方法

doi: 10.3963/j.jssn.1674-4861.2024.06.009

武汉大学资源与环境科学学院武汉 430079

基金项目:

国家重点研发计划项目 2021YFB2501101

湖北重大科技攻关项目 2023BAA017

详细信息

通讯作者:
应申（1979—），博士，教授. 研究方向：地图学与地理信息系统、自动驾驶高精地图. E-mail：shy@whu.edu.cn

中图分类号: U412
计量
- 文章访问数: 51
- HTML全文浏览量: 12
- PDF下载量: 4
- 被引次数: 0
出版历程
- 收稿日期: 2024-06-11
- 网络出版日期: 2025-03-08

A Detection Method of Traffic Scene License Plate for Data Desensitization

School of Resource and Environmental Sciences, Wuhan University, Wuhan 430079, China

摘要

摘要: 从车载相机图像中快速准确地检测车牌对于保护交通敏感信息具有重要意义。针对传统YOLOv8算法对交通场景下车牌检测存在的小目标特征提取能力弱、背景信息误检等问题，研究了基于改进YOLOv8的交通场景车牌检测方法TLP-YOLO。为增强主干网络提取图像特征信息能力，引入多尺度注意力模块（effi-cient multi-scale attention，EMA），加强对不同尺度目标区域的关注，提升模型对背景信息的甄别能力；提出跳阶加权特征金字塔网络，丰富小目标特征，通过跳阶连接和加权融合方式，避免特征金字塔结构不同层级间的信息损失，提升模型的多尺度特征融合能力；为降低模型计算量并保持检测精度，对检测头进行轻量化处理，引入部分卷积（partial convolution，PConv）和逐点卷积（pointwise，PWConv）模块代替常规卷积结构，以高效利用空间特征，避免冗余计算。基于中国城市停车数据集（Chinese city parking dataset，CCPD）和中国道路车牌数据集（Chinese road plate dataset，CRPD）构建用于验证模型性能的多交通场景数据集并进行了算法验证。实验结果表明：①TLP-YOLO的AP50-95和AP70分别达到了83.6%，和97.7%，相较于基准算法YOLOv8，其AP50-95和AP70分别提高了2%和0.8%。② TLP-YOLO模型的计算复杂度（floating point opera-tions，FLOPs）为7.5 G，模型参数量为1.67 M，检测速度达到了101 fps，相较于基准算法YOLOv8，计算复杂度降低了8.5%，模型参数量减少了45%，平均检测速度则与之相当。改进后的算法能够在保证模型轻量化的同时，提高检测精度，满足车载设备对交通场景中车牌检测准确性与部署要求。
- 数据脱敏 /
- 车牌检测 /
- YOLOv8 /
- 交通场景 /
- 特征融合 /
- 高效多尺度注意力
Abstract: Rapid and accurate detection of license plates in vehicle-based images is significant for protecting privacy information in smart transportation. However, the original YOLOv8 algorithm has limitations on the license plate detection in traffic scenes, such as weak feature extraction ability of small targets and misdetection of background information, etc. To fill these gaps, an improved traffic scene license plate detection method based on YOLOv8 (TLP-YOLO) is proposed. The efficient multi-scale attention (EMA) module is adopted to enhance the ability of the backbone network to extract image characteristics. It makes the backbone network pay more attention to target regions of different scales and improves the recognition ability of the model to background information. A new feature pyramid network with skip connection and weighted fusion (SW-FPN) is designed. It enriches the features of small targets and avoid the information loss between different levels of the feature pyramid network, which improving the multi-scale feature fusion ability. In order to reduce the floating-point operations (FLOPs) and maintain the detection accuracy, the partial convolution (PConv) and pointwise convolution (PWConv) modules are introduced to replace the conventional convolution structure in detection head, which reduces redundant calculations and improves the utilization efficiency of spatial features. Based on Chinese city parking dataset (CCPD) and Chinese road plate dataset (CRPD), a dataset with multiple traffic scenes is constructed to verify the property of the model. Experimental results show that: ①The average precision (IOU changes from 0.5 to 0.95) of the proposed network is 83.6%, which is 2% higher than that of YOLOv8. The average precision (IOU is 0.7) of the proposed network is 97.7%, which is 0.8% higher than that of YOLOv8. ②The FLOPs of TLP-YOLO model is 7.5 G, the number of parameters is 1.67 M, and the detection speed reaches 101 fps. In comparison to the original YOLOv8, the FLOPs and the number of parameters is reduced by 8% and 45%, the detection speed is about the same. The improved algorithm can not only ensure the lightweight of the model, but also meet the requirements of vehicle equipment for the accuracy and deployment of license plate detection in traffic scenes.
- data desensitization /
- license plate detection /
- YOLOv8 /
- traffic scene /
- feature fusion /
- efficient multi-scale attention

HTML全文

图 1 TLP-YOLO网络框架

Figure 1. Network structure of TLP-YOLO

下载: 全尺寸图片幻灯片

图 2 C2f_EMA结构

Figure 2. Structure of C2f_EMA

下载: 全尺寸图片幻灯片

图 3 EMA结构

Figure 3. Structure of EMA

下载: 全尺寸图片幻灯片

图 4 加入EMA注意力前后效果对比

Figure 4. Comparison of effects before and after adding EMA attention

下载: 全尺寸图片幻灯片

图 5 改进前后的Neck网络

Figure 5. Improved Neck network before and after

下载: 全尺寸图片幻灯片

图 6 加权融合方式

Figure 6. weighted fusion method

下载: 全尺寸图片幻灯片

图 7 Detect-Faster结构

Figure 7. Structure of Detect-Faster

下载: 全尺寸图片幻灯片

图 8 卷积模块对比

Figure 8. Convolution module comparison

下载: 全尺寸图片幻灯片

图 9 数据集车牌尺寸分布情况

Figure 9. License plate size distribution of the dataset

下载: 全尺寸图片幻灯片

图 10 CCPD数据集检测效果对比

Figure 10. Comparison of detection effect of CCPD

下载: 全尺寸图片幻灯片

图 11 CRPD数据集检测效果对比

Figure 11. Comparison of detection effect of CRPD

下载: 全尺寸图片幻灯片

表 1 数据分配情况

Table 1. Data distribution situation

子数据集	训练集样本量/个	测试集样本量/个	描述
CCPD_base	1 000	500	常见情况下的车牌
CCPD_blur	1 000	500	较模糊的车牌
CCPD_weather	1 000	500	雨雪雾等复杂天气
CCPD_rotate & tilt	1 000	500	倾斜、形变较大的车牌
CCPD_db	1 000	500	车牌区域光照不均匀
CCPD_fn	1 000	500	车牌拍摄距离相对较近或较远
CCPD_challenge	0	2 000	车牌检测中较有挑战性的图片
CRPD_single	3 000	1 500	仅有1个车牌的图片
CRPD_double	3 000	1 500	有2个车牌的图片
CRPD_multi	0	1 585	包含多个车牌的图片

下载: 导出CSV

表 2 注意力模块对比实验

Table 2. Comparative experiments on attention modules

模块名称	AP_50-95/%	AP₇₀/%	FLOPs/G
C2f	81.6	96.9	8.2
C2f+CBAM	81.5	96.9	8.2
C2f+GAM	81.9	97.1	13.5
C2f+GC	81.7	96.8	8.2
C2f+EMA	82.0	97.1	8.4

下载: 导出CSV

表 3 特征融合方式对比实验

Table 3. Comparative experiments on feature fusion method

融合方式	AP_50-95/%	AP₇₀/%	FLOPs/G
拼接融合	81.6	96.9	8.2
加权融合	83.0	97.2	16.5
本文方式	83.2	97.3	12.3
注：拼接融合为YOLOv8采用的基础融合方式，加权融合为为完全采用加权的方式进行特征融合，本文方式为拼接结合加权，2特征融合采用拼接，3特征融合采用加权。

下载: 导出CSV

表 4 YOLOv8与TLP-YOLO的性能对比

Table 4. Performance comparison between YOLOv8 and TLP-YOLO

算法	r /%	p /%	AP_50-95/%	AP₇₀/%	FLOPs/G	FPS/（f/s）	Parameters/M	模型所占空间/MB
YOLOv8	96.1	97.4	81.6	96.9	8.2	104.2	3.01	14.4
TLP-YOLO	98.2	98.3	83.6	97.7	7.5	101.0	1.67	9.8

下载: 导出CSV

表 5 CCPD数据集上YOLOv8与TLP-YOLO的性能对比

Table 5. Performance comparison between YOLOv8 and TLP-YOLO on CCPD

子数据集	AP_50-95/%		AP₇₀/%
子数据集	YOLOv8	TLP-YOLO	YOLOv8	TLP-YOLO
CCPD_base	88.2	88.5	99.2	99.2
CCPD_blur	88.6	89	98.7	98.8
CCPD_db	84.9	85.6	98.6	98.9
CCPD_fn	86.7	86	98.6	98.3
CCPD_rotate	92.3	93	99.5	99.5
CCPD_weather	95.3	94.9	99.4	99.5
CCPD_challenge	71.9	72.4	96.5	96.9
CCPD整体	87.7	88.1	99	99.2

下载: 导出CSV

表 6 CRPD数据集上YOLOv8与TLP-YOLO的性能对比

Table 6. Performance comparison between YOLOv8 and TLP-YOLO on CRPD

子数据集	AP_50-95/%		AP₇₀/%
子数据集	YOLOv8	TLP-YOLO	YOLOv8	TLP-YOLO
CRPD_single	78.6	81.6	95.6	97.4
CRPD_double	78.2	81.2	95.7	97
CRPD_multi	77.6	80.7	95.3	96.6
CRPD整体	77.9	80.9	95.4	96.8

下载: 导出CSV

表 7 不同优化方式对模型性能的影响

Table 7. Impact of different optimization methods on model performance

实验序号	C2f_EMA	SW-FPN	Detect-Faster	AP_50-95/%	AP₇₀/%	FLOPs/G
1				81.6	96.9	8.2
2	√			82.0	97.1	8.4
3		√		83.2	97.3	12.3
4			√	81.6	96.8	5.8
5	√	√		83.5	97.7	12.7
6	√		√	82.1	97.1	6.1
7		√	√	83.4	97.5	7.4
8	√	√	√	83.6	97.7	7.5

下载: 导出CSV

表 8 实验数据集上不同检测算法性能对比

Table 8. Performance comparison of different detection algorithms on the experimental dataset

检测算法	AP₇₀/%	FLOPs/G	FPS/（帧/s）
Faster-RCNN	92.9		26.7
SSD	94.4		58.0
TE2E	94.2		4.5
RPNet	94.5		61.0
YOLOv5s	94.8	15.9	77.4
YOLOv7	95.3	13.9	83.1
YOLOv8	96.9	8.2	104.2
TLP-YOLO	97.7	7.5	101.0

下载: 导出CSV

参考文献(27)

[1]	邱彬, 李广友. 智能网联汽车数据安全管理研究[J]. 汽车工程学报, 2022, 12 (3): 307-313. doi: 10.3969/j.issn.2095-1469.2022.03.11 QIU B, LI G Y. Research on data security management of intelligent connected vehicle[J]. Chinese Journal of Automotive Engineering, 2022, 12 (3): 307-313. (in Chinese) doi: 10.3969/j.issn.2095-1469.2022.03.11
[2]	唐迪, 顾健, 张凯悦, 等. 数据脱敏技术发展趋势[J]. 保密科学技术, 2021, 127 (4): 4-11. TANG D, GU J, ZHANG K Y, et al. Development trend of data desensitization technology[J]. Science and Technology of Secrecy, 2021, 127 (4): 4-11. (in Chinese)
[3]	REN S Q, HE K M, GIRSHICK R, et al. Faster r-cnn: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[4]	LIU W, ANGUELOV D, ERHAN D, et al. Ssd: single shot multiBox detector[C]. The European Conference on Computer Vision, Amsterdam, Netherlands: Springer, 2016.
[5]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA: IEEE, 2016.
[6]	艾曼. 基于Faster-RCNN的车牌检测[J]. 计算机与数字工程, 2020, 48 (1): 174-177. doi: 10.3969/j.issn.1672-9722.2020.01.033 AI M. License plate detection based on faster-rcnn[J]. Computer and Digital Engineering, 2020, 48 (1): 174-177. (in Chinese) doi: 10.3969/j.issn.1672-9722.2020.01.033
[7]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA: IEEE, 2016.
[8]	马巧梅, 王明俊, 梁昊然. 复杂场景下基于改进YOLOv3的车牌定位检测算法[J]. 计算机工程与应用, 2021, 57(7): 198-208. MA Q M, WANG M J, LIANG H R. License plate location detection algorithm based on improved Yolov3 in complex scenes[J]. Computer Engineering and Applications, 2021, 57 (7): 198-208. (in Chinese)
[9]	XU Z B, YANG W, MENG A J, et al. Towards end-to-end license plate detection and recognition: a large dataset and baseline[C]. The 2018 European Conference on Computer Vision, Heidelberg, Germany: Springer, 2018.
[10]	REDMON J, FARHADI A. Yolov3: an incremental improvement[EB/OL]. (2018-04-08)[2023-06-25]. https://arxiv.org/pdf/1804.02767.pdf.
[11]	HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018.
[12]	孙世昕, 马蕾, 李千目, 等. 基于深度学习的实时车牌检测与识别[J]. 现代交通与冶金材料, 2022, 2 (4): 61-67. SUN S X, MA L, LI Q M, et al. Real-time license plate detection and recognition based on deep learning[J]. Modern Transportation and Metallurgical Materials. 2022, 2(4): 61-67. (in Chinese)
[13]	陈婷, 姚大春, 高涛, 等. 基于PReNet和YOLOv4融合的雨天交通目标检测网络[J]. 交通运输工程学报, 2022, 22 (3): 225-237. CHEN T, YAO D C, GAO T, et al. A fused network based on PReNet and YOLOv4 for traffic object detection in rainy environment[J]. Journal of Traffic and Transportation Engineering, 2022, 22 (3): 225-237. (in Chinese)
[14]	FAN X D, ZHAO W. Improving robustness of license plates automatic recognition in natural scenes[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(10): 18845-18854. doi: 10.1109/TITS.2022.3151475
[15]	张逸凡, 聂琳真, 黄灏然, 等. 基于改进YOLOv5算法的道路交通参与者实时检测方法[J]. 交通信息与安全, 2024, 42 (1): 115-123. doi: 10.3963/j.jssn.1674-4861.2024.01.013 ZHANG Y F, NIE L Z, HUANG H R, et al. A method of real-time detection for road traffic participants based on an improved yolov5 algorithm[J]. Journal of Transport Information and Safety, 2024, 42 (1): 115-123. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2024.01.013
[16]	OUYANG D, HE S, ZHAN J, et al. Efficient multi-scale attention module with cross-spatial learning[C]. 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece: IEEE, 2023.
[17]	CHEN J R, KAO S H, HE H, et al. Run, don't walk: chasing higher flops for faster neural networks[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada: IEEE, 2023.
[18]	WANG C Y, LIAO H Y M, WU Y H, et al. Cspnet: a new backbone that can enhance learning capability of cnn[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, USA: IEEE, 2020.
[19]	LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA: IEEE, 2017.
[20]	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018.
[21]	TAN M, PANG R, LE Q V. Efficientdet: scalable and efficient object detection[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA: IEEE, 2020.
[22]	邱天衡, 王玲, 王鹏, 等. 基于改进YOLOv5的目标检测算法研究[J]. 计算机工程与应用, 2022, 58 (13): 63-73. QIU T H, WANG L, WANG P, et al. Research on object detection algorithm based on improved yolov5[J]. Computer Engineering and Applications, 2022, 58(13): 63-73. (in Chinese)
[23]	GONG Y X, DENG L J, TAO S, et al. Unified Chinese license plate detection and recognition with high efficiency[J]. Journal of Visual Communication and Image Representation, 2022, 86: 103541. http://arxiv.org/abs/2205.03582?context=cs.CV
[24]	WOO S H, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]. The 15th European Conference on Computer Vision, Munich, Germany: Springer, 2018.
[25]	CAO Y, XU J R, LIN S, et al. Gcnet: non-local networks meet squeeze-excitation networks and beyond[C]. 2019 IEEE/CVF International Conference on Computer Vision Workshop, Seoul, South Korea: IEEE, 2019.
[26]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada: IEEE, 2023.
[27]	LI H, WANG P, SHEN C H. Toward end-to-end car license plate detection and recognition with deep neural networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20 (3): 1126-1136.