留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于异步交互聚合网络的港船作业区域人员异常行为识别

陈信强 郑金彪 凌峻 王梓创 吴建军 阎莹

陈信强, 郑金彪, 凌峻, 王梓创, 吴建军, 阎莹. 基于异步交互聚合网络的港船作业区域人员异常行为识别[J]. 交通信息与安全, 2022, 40(2): 22-29. doi: 10.3963/j.jssn.1674-4861.2022.02.003
引用本文: 陈信强, 郑金彪, 凌峻, 王梓创, 吴建军, 阎莹. 基于异步交互聚合网络的港船作业区域人员异常行为识别[J]. 交通信息与安全, 2022, 40(2): 22-29. doi: 10.3963/j.jssn.1674-4861.2022.02.003
CHEN Xinqiang, ZHENG Jinbiao, LING Jun, WANG Zichuang, WU Jianjun, YAN Ying. Detecting Abnormal Behaviors of Workers at Ship Working Fields via Asynchronous Interaction Aggregation Network[J]. Journal of Transport Information and Safety, 2022, 40(2): 22-29. doi: 10.3963/j.jssn.1674-4861.2022.02.003
Citation: CHEN Xinqiang, ZHENG Jinbiao, LING Jun, WANG Zichuang, WU Jianjun, YAN Ying. Detecting Abnormal Behaviors of Workers at Ship Working Fields via Asynchronous Interaction Aggregation Network[J]. Journal of Transport Information and Safety, 2022, 40(2): 22-29. doi: 10.3963/j.jssn.1674-4861.2022.02.003

基于异步交互聚合网络的港船作业区域人员异常行为识别

doi: 10.3963/j.jssn.1674-4861.2022.02.003
基金项目: 

国家重点研发计划项目 2019YFB1600602

国家自然科学基金项目 52102397

国家自然科学基金项目 51978069

陕西省重点研发计划项目 2021KWZ-09

中国博士后科学基金项目 2021M700790

详细信息
    作者简介:

    陈信强(1987—),博士,讲师. 研究方向:自动化码头、智能船舶、交通大数据挖掘.E-mail: chenxinqiang@stu.shmtu.edu.cn

    通讯作者:

    阎莹(1981—),博士,教授. 研究方向:道路安全设计与评估、驾驶行为辨识与检测技术、人-车-路系统安全、事故预测与建模. E-mail: yanying2199@chd.edu.cn

  • 中图分类号: U697.33

Detecting Abnormal Behaviors of Workers at Ship Working Fields via Asynchronous Interaction Aggregation Network

  • 摘要: 港船作业区域人员的异常行为识别可为智能航运的管控与决策提供重要数据支撑,有利于推动智慧港口和智能船舶的发展。基于异步交互聚合网络开展了面向港船工作环境下的人员异常行为识别研究。基于YOLO模型对港船图像进行卷积操作,利用特征金字塔优化卷积结果得到图像序列中每一帧的人员位置,结合联合学习检测和嵌入范式输出港船图像序列中的人、物体特征信息以及时序信息;利用异步交互聚合网络中的交互聚合结构更新特征池的多维度特征信息,以识别港区与船舶工作环境下的人员异常行为。实验结果表明:提出的港船作业区域人员异常行为识别方法的平均识别精度为91%,在港区工作环境下的人员异常行为识别精度为85%,在船舶驾驶台环境下,提出的异常行为识别框架对船员的不安全行为识别精度达到97%。所提出的识别框架在不同港船作业区域环境中都能获得较好的精度,验证了其有效性和可靠性。

     

  • 图  1  港船环境下人员异常行为识别流程图

    Figure  1.  Flow chart for detecting abnormal behaviors of workers at ship working fields

    图  2  串行密集交互聚合结构

    Figure  2.  The serial dense interaction aggregation structure

    图  3  不同场景下的异常行为动作识别效果图

    Figure  3.  The proposed framework performance on recognizing abnormal behavior from different video clips

    图  4  SlowFast算法对视频3序列识别结果

    Figure  4.  The recognition results of SlowFast for video 3 clips

    表  1  港船人员异常行为的视频片段信息

    Table  1.   Details for the collected video clips involved with anomaly behavior

    视频序列 帧率/ (帧/s) 分辨率 时长/s 实验场景
    1 25 926x522 10 港口环境, 名工作人员
    2 25 814x458 10 港口环境, 多名工作人员
    3 25 720x480 7 复杂港口环境, 多名工作人员
    4 25 704x576 10 船舶驾驶台,多名工作人员
    下载: 导出CSV

    表  2  视频1序列的异常行为识别结果

    Table  2.   Abnormal behavior recognition results for video 1 clips  单位: %

    人员 J1 J2 J3 J4
    #1 100 97 100 76
    下载: 导出CSV

    表  3  视频2序列的异常行为识别结果

    Table  3.   Abnormal behavior recognition results for video 2 clips 单位: %

    人员 J1 J2 J3 J4
    #1 100 97
    #2 88 100 100
    #3 79 100 100 90
    下载: 导出CSV

    表  4  视频3序列的异常行为识别结果

    Table  4.   Abnormal behavior recognition results for video 3 clips 单位: %

    人员 J1 J2 J5 J3
    #1 97 97
    #2 75 75
    #3 89 100 72
    #4 87 97 99 100
    #5 62 62
    #6 70 70 69
    下载: 导出CSV

    表  5  视频4序列的异常行为识别结果

    Table  5.   Abnormal behavior recognition results for video 4 clips  单位: %

    人员 J1 J2 J5 J6
    #1 100 97
    #2 100 86 100
    #3 100 89 100
    #4 100 100
    下载: 导出CSV
  • [1] 张永锋, 龚建伟, 殷明. 新冠肺炎疫情对中国港航业的影响及其对策[J]. 交通运输工程学报, 2020, 20(3): 159-167. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202003019.htm

    ZHANG Y F, GONG J W, YIN M. Influences and response measures of COVID-19 epidemic on shipping and port industry in china[J]. Journal of Traffic and Transportation Engineering, 2020, 20(3): 159-167. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC202003019.htm
    [2] 周翔宇, 吴兆麟, 王凤武, 等. 自主船舶的定义及其自主水平的界定[J]. 交通运输工程学报, 2019, 19(6): 149-162. https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201906016.htm

    ZHOU X Y, WU Z L, WANG F W, et al. Definition of autonomous ship and its autonomy level[J]. Journal of Traffic and Transportation Engineering, 2019, 19(6): 149-162. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JYGC201906016.htm
    [3] 郑松, 吴晓林, 王飞跃, 等. 平行系统方法在自动化集装箱码头中的应用研究[J]. 自动化学报, 2019, 45(3): 490-504. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201903005.htm

    ZHENG S, WU X L, WANG F Y, et al. Applying the parallel systems approach to automatic container terminal[J]. Acta Automatica Sinica, 2019, 45(3): 490-504(. in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201903005.htm
    [4] ANDREA Z, JACOPO C, RICCARDO V, et al. Predicting intentions from motion: the subject-adversarial adaptation approach[J]. International Journal of Computer Vision, 2020, 128(1): 220-239. doi: 10.1007/s11263-019-01234-9
    [5] WANG H, KLÄSER A, SCHMID C, et al. Dense trajectories and motion boundary descriptors for action recognition[J]. International Journal of Computer Vision, 2013, 103 (1): 60-79. doi: 10.1007/s11263-012-0594-8
    [6] 秦宇龙, 王永雄, 胡川飞, 等. 结合注意力与多尺度时空信息的行为识别算法[J]. 小型微型计算机系统, 2021, 42(9): 1802-1809. doi: 10.3969/j.issn.1000-1220.2021.09.002

    QIN Y L, WANG Y X, HU C F, et al. Action recognition algorithm based on attention and multiscale channels separation spatiotemporal information[J]. Journal of Chinese Computer Systems, 2021, 42(9): 1802-1809. (in Chinese) doi: 10.3969/j.issn.1000-1220.2021.09.002
    [7] YANG L H, LIU J, WANG Y M, et al. Online updating extended belief rule-based system for sensor-based activity recognition[J]. Expert Systems with Applications, 2021(186): 1-14.
    [8] 谭等泰, 李世超, 常文文, 等. 多特征融合的行为识别模型[J]. 中国图像图形学报, 2020, 25(12): 2541-2552. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202012007.htm

    TAN D T, LI S C, CHANG W W, et al. Multi-feature fusion behavior recognition model[J]. Journal of Image and Graphics, 2020, 25(12): 2541-2552. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202012007.htm
    [9] ALWANDO E H P, CHEN Y T, FANG W H. CNN-Based multiple path search for action tube detection in videos[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(1): 104-116. doi: 10.1109/TCSVT.2018.2887283
    [10] SHU X B, ZHANG L Y, SUN Y L, et al. Host-Parasite: graph LSTM-in-LSTM for group activity recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(2): 663-674. doi: 10.1109/TNNLS.2020.2978942
    [11] WANG P C, LI W Q, GAO Z M, et al. Action recognition from depth maps using deep convolutional neural networks[J]. IEEE Transactions on Human-Machine Systems, 2016, 46(4): 498-509. doi: 10.1109/THMS.2015.2504550
    [12] 陈影玉, 杨神化, 索永峰. 船舶行为异常检测研究进展[J]. 交通信息与安全, 2020, 38(5): 1-11. doi: 10.3963/j.jssn.1674-4861.2020.05.001

    CHEN Y Y, YANG S H, SUO Y F. Research progress of ship behavior anomaly detection[J]. Journal of Transport Information and Safety, 2020, 38(5): 1-11(. in Chinese) doi: 10.3963/j.jssn.1674-4861.2020.05.001
    [13] CHEN X Q, QI L, YANG Y S, et al. Video-based detection infrastructure enhancement for automated ship recognition and behavior analysis[J]. Journal of Advanced Transportation, 2020(2020): 1-12.
    [14] WEI Z K, XIE X L, ZHANG X J. AIS trajectory simplification algorithm considering ship behaviours[J]. Ocean Engineering, 2020(216): 1-10.
    [15] XUE J, CHEN Z J, PAPADIMITRIOU E, et al. Influence of environmental factors on human-like decision-making for intelligent ship[J]. Ocean Engineering, 2019(186): 1-14.
    [16] TANG J J, XIA J, MU X Z, et al. Asynchronous interaction aggregation for action detection[C]. European Conference on Computer Vision-ECCV 2020, Lecture Notes in Computer Science, Glasgow, UK: Springer, 2020.
    [17] CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]. 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA: IEEE, 2018.
    [18] GU C H, SUN C, ROSS D A, et al. AVA: A video dataset of spatio-temporally localized atomic visual actions[C]. 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA: IEEE, 2018.
    [19] FEICHTENHOFER C, FAN H Q, MALIK J, et al. SlowFast networks for video recognition[C]. 2019 International Conference on Computer Vision (ICCV), Seoul, Korea: IEEE, 2019.
  • 加载中
图(4) / 表(5)
计量
  • 文章访问数:  169
  • HTML全文浏览量:  90
  • PDF下载量:  55
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-10-25
  • 网络出版日期:  2022-05-18

目录

    /

    返回文章
    返回