留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度学习的船舶驾驶员疲劳检测算法

王鹏 神和龙 尹勇 吕红光

王鹏, 神和龙, 尹勇, 吕红光. 基于深度学习的船舶驾驶员疲劳检测算法[J]. 交通信息与安全, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008
引用本文: 王鹏, 神和龙, 尹勇, 吕红光. 基于深度学习的船舶驾驶员疲劳检测算法[J]. 交通信息与安全, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008
WANG Peng, SHEN Helong, YIN Yong, LYU Hongguang. A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008
Citation: WANG Peng, SHEN Helong, YIN Yong, LYU Hongguang. A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008

基于深度学习的船舶驾驶员疲劳检测算法

doi: 10.3963/j.jssn.1674-4861.2022.01.008
基金项目: 

国家重点研发计划项目 2019YFB1600602

详细信息
    作者简介:

    王鹏(1996—),硕士研究生. 研究方向:计算机视觉航海应用. E-mail:Arieswp1996@163.com

    通讯作者:

    神和龙(1984—),博士,副教授. 研究方向:航海动态仿真、计算机视觉航海应用. E-mail:shenhelong@126.com

  • 中图分类号: U676.1

A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique

  • 摘要: 针对日益凸显的船舶值班人员疲劳驾驶问题,为有效预警值班驾驶员的疲劳状态,保障船舶航行安全,研究了基于深度学习的疲劳检测算法。考虑到船舶驾驶台空间大、背景复杂等特点,使用深度可分离卷积改进RetinaFace人脸检测模型,优化模型的检测速度;基于Channel Split和Channel Shuffle思想,结合批量归一化、全局平均池化等技术搭建改进的ShuffleNetV2网络,自动提取图像特征,识别眼睛、嘴巴的开闭状态;根据PERCLOS准则融合眼睛、嘴巴2个特征参数综合判定驾驶员是否疲劳。实验结果表明:改进后RetinaFace模型的检测速度由9.33帧/s提升至22.60帧/s,人脸检测精度和速度均优于多任务卷积神经网络(MTCNN);改进的ShuffleNetV2网络识别眼睛、嘴巴状态的准确率高达99.50%以上;算法在模拟驾驶台环境中识别疲劳状态的精确率达到95.70%,召回率达到96.73%,均高于目前常见的Haar-like+Adaboost以及MTCNN+CNN疲劳检测算法。算法检测每帧图片仅需0.083 s,基本满足实时检测的要求。

     

  • 图  1  算法流程图

    Figure  1.  Overall flow chart of algorithm

    图  2  RetinaFace人脸检测模型示意图

    Figure  2.  Schematic diagram of RetinaFace model

    图  3  深度可分离卷积示意图

    Figure  3.  Schematic diagram of depthwise separable convolution

    图  4  RetinaFace改进效果示意图

    Figure  4.  Improvement effect of RetinaFace model

    图  5  眼睛、嘴巴裁剪区域示意图

    Figure  5.  Cropped area of eyes and mouth

    图  6  ShuffleNet V2常用卷积块

    Figure  6.  Convolution blocks of ShuffleNet V2

    图  7  部分眼睛、嘴巴样本

    Figure  7.  Partial samples of eyes and mouth

    图  8  眼睛分类网络的训练曲线

    Figure  8.  Training curve of eyes classification network

    图  9  嘴巴分类网络的训练曲线

    Figure  9.  Training curve of mouth classification network

    图  10  眼睛、嘴巴状态识别结果

    Figure  10.  Recognition results of eye and mouth status

    图  11  对比实验结果

    Figure  11.  Compare experimental results

    图  12  部分疲劳检测结果

    Figure  12.  Partial results of fatigue detection

    表  1  不同模型在自建眼睛数据集上的分类对比

    Table  1.   Classification comparison of different models on self-built eye dataset

    算法 准确率 速度/(ms/帧)
    LeNet 95.53 12
    VGG16 97.88 16
    MobileNetV2 97.43 9
    本文模型 99.71 7
    下载: 导出CSV

    表  2  不同疲劳检测算法结果对比

    Table  2.   Comparison of different fatigue detection algorithms

    算法 Nt Nd Np 精确率p/% 召回率R/%
    Haar-like+Adaboost 92 96 75 78.12 81.52
    MTCNN+CNN 92 94 84 89.36 91.30
    本文算法 92 93 89 95.70 96.73
    下载: 导出CSV
  • [1] 郭其顺. 世界海难事故的现状与原因分析[J]. 中国船检, 2000(3): 44-46. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGCJ200003030.htm

    GUO Q S. Current situation and cause analysis of maritime accidents in the world[J]. China Ship Survey, 2000(3): 44-46. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGCJ200003030.htm
    [2] 张堃. 船舶机舱值班人员眼部视觉识别及疲劳状态监测[D]. 大连: 大连海事大学, 2019.

    ZHANG K. Eye visual recognition and fatigue monitoring of engineering watch-keepers on board ship[D]. Dalian: Dalian Maritime University, 2019. (in Chinese)
    [3] 刘建蓓, 马小龙, 张志伟, 等. 基于心电分析的青藏高原驾驶人疲劳特性[J]. 交通运输工程学报, 2016, 16(4): 151-158. doi: 10.3969/j.issn.1671-1637.2016.04.016

    LIU J B, MA X L, ZHANG Z W, et al. Fatigue characteristics of driver in Qinghai-Tibet Plateau based on electrocardiogram analysis[J]. Journal of Traffic and Transportation Engineering, 2016, 16(4): 151-158. (in Chinese) doi: 10.3969/j.issn.1671-1637.2016.04.016
    [4] LAWOYIN S, FEI D Y, BAI O. Accelerometer-based steering-wheel movement monitoring for drowsy-driving detection[J]. Journal of automobile engineering, 2015, 229(2): 163-173. doi: 10.1177/0954407014536148
    [5] 刘梦佳, 赵建国. 基于SVM的疲劳驾驶人脸图像特征检测算法[J]. 半导体光电, 2020, 41(2): 278-282. https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202002026.htm

    LIU M J, ZHAO J G. Research on feature detection algorithm of fatigue driving face image based on SVM[J]. Semiconductor Optoelectronics, 2020, 41(2): 278-282. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202002026.htm
    [6] 王迪. 基于人眼状态的疲劳检测算法研究与应用[D]. 成都: 电子科技大学, 2020.

    WANG D. Research and application of fatigue detection algorithm based on human eye state[D]. Chengdu: University of Electronic Science and Technology of China, 2020. (in Chinese)
    [7] 黄玲, 洪佩鑫, 吴泽荣, 等. 基于防疫常态化的驾驶员疲劳状态检测方法[J]. 交通信息与安全, 2021, 39(4): 26-34. doi: 10.3963/j.jssn.1674-4861.2021.04.004

    HUANG L, HONG P X, WU Z R, et al. A detection method for drivers' fatigue states based on normalization of epidemic prevention[J]. Journal of Transport Information and Safety, 2021, 39(4): 26-34. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2021.04.004
    [8] 张博雅. 基于深度学习的驾驶安全评估方法研究[D]. 西安: 西安科技大学, 2019.

    ZHANG B Y. Research on driving safety assessment method based on deep learing[D]. Xi'an: Xi'an University Of Science And Technology, 2019. (in Chinese)
    [9] LIU Z M, PENG Y X, HU W J. Driver fatigue detection based on deeply-learned facial expression representation[J]. Journal of Visual Communication and Image Representation, 2020 (71): 102-107.
    [10] 郭佳伟. 基于计算机视觉的驾驶员异常行为识别与预警[D]. 大连: 大连海事大学, 2019.

    GUO J W. Driver's abnormal behavior recognition and early-warning based on computation vision[D]. Dalian: Dalian Maritime University, 2019. (in Chinese)
    [11] LI K N, GONG Y B, REN Z L. A fatigue driving detection algorithm based on facial multi-feature fusion[J]. IEEE Access, 2020, 8(11): 244-259.
    [12] 陈军, 张黎, 周博, 等. 基于级联卷积神经网络的驾驶员分心驾驶行为检测[J]. 科学技术与工程, 2020, 20(14): 5702-5708. doi: 10.3969/j.issn.1671-1815.2020.14.037

    CHEN J, ZHANG L, ZHOU B, et al. Driver distracted driving behavior detection based on cascaded convolutional neural network[J]. Science Technology and Engineering, 2020, 20(14): 5702-5708. (in Chinese) doi: 10.3969/j.issn.1671-1815.2020.14.037
    [13] DENG J K, GUO J, VERVERAS E, et al. Retinaface: Single-shot multi-level face localisation in the wild[C]. The IEEE conference on computer vision and pattern recognition, Seattle: IEEE, 2020.
    [14] GAVAI N R, JAKHADE Y A, TRIBHUVAN S A, et al. MobileNets for flower classification using TensorFlow[C]. 2017 International Conference on Big Data, IoT and Data Science (BID), Pune: IEEE, 2017.
    [15] YANG S, LUO P, LOY C C, et al. Wider face: A face detection benchmark[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: IEEE, 2016.
    [16] ZHANG K P, ZHANG Z P, LI Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503. doi: 10.1109/LSP.2016.2603342
    [17] MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]. European Conference on Computer Vision, Munich: ECCV, 2018.
    [18] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate Shift[C]. International Conference on Machine Learning, Lille: PMLR, 2015.
    [19] PENG Y, DONG Y, CHENG D. Design and implementation of a driver's eye state recognition algorithm based on PERCLOS[J]. Chinese Journal of Electronics, 2014, 23(4): 669-672.
    [20] ABTAHI S, OMIDYEGANEH M, SHIRMOHAMMADI S, et al. YawDD: A yawning detection dataset[C]. The 5th ACM multimedia systems conference, Singapore: ACM, 2014.
  • 加载中
图(12) / 表(2)
计量
  • 文章访问数:  987
  • HTML全文浏览量:  531
  • PDF下载量:  105
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-08-12
  • 网络出版日期:  2022-03-31

目录

    /

    返回文章
    返回