A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique

Aiming at preventing Officers on Watch (OOW) from fatigue driving, a fatigue detection and alert algorithm based on deep learning technique is developed. Considering the large space and complex background of the ship bridge, the RetinaFace model is improved by using Depthwise Separable Convolution to optimize the detection speed. An upgraded ShuffleNetV2 network is then developed by adopting the concepts of Channel Split, Channel Shuffle, and other techniques such as batch normalization and global average pooling. The proposed algorithm can extract image features and automatically identify the opening and closing of the eyes and mouth of the OOW. According to the PERCLOS criteria, the two features of the eyes and mouth are integrated to determine whether the OOW is fatigued. Experimental results show that the detection speed of the improved RetinaFace model improves from 9.33 to 22.60 frames/s. The detection accuracy and speed for the face detection are superior to the multi-task convolutional neural network. The upgraded ShuffleNetV2 network achieves over 99.50% accuracy in recognizing the states of eyes and mouth. The algorithm has an accuracy of 95.70% and a recall rate of 96.73% in identifying the fatigue state in a simulated ship bridge scenario, which are higher than Haar-like+Adaboost and MTCNN+CNN fatigue detection algorithmsused in practice. It only takes 0.083 s for the algorithm to complete the process, which indicates that the algorithm is capable of carrying out real-time detection.