A Recognition Model for Passenger Boarding and Alighting Action Based on Improved Temporal Pyramid Network

Traditional algorithms for identifying illegal passenger-carrying behavior, which rely on image processing techniques, utilize manually crafted human-vehicle interaction rules to discern boarding and alighting actions. However, these rule sets often fall short due to the intricate nature of traffic scenarios, resulting in suboptimal recognition performance. Therefore, a deep learning model based on a temporal pyramid network(TPN) is introduced for boarding and alighting action recognition. By training on a large dataset, more complete features of taxi passenger boarding and alighting behaviors are extracted to improve recognition accuracy. To address the issue of the TPN model not distinguishing between driver and passenger roles, the output layer is redesigned based on door area perception. This modification enhances the efficiency of multi-dimensional feature extraction. To tackle the issue of the large spatiotemporal span in boarding and alighting actions, which leads to interference from irrelevant movements, a sliding window mechanism is introduced. This mechanism, based on dynamic window weights, captures key video frames of the actions, enhancing recognition efficiency. Based on the above improvement measures, a boarding and alighting neural network(BANN) model, based on door area perception and dynamic weights, is proposed to efficiently and accurately recognize illegal passenger-carrying behaviors. A training dataset with 4, 047 annotated video clips and a test dataset with 810 unannotated video clips are constructed for model performance validation based on surveillance videos from Beijing Capital Airport. Experimental results demonstrate that the BANN model achieves precision and recall rates of 90.21% and 88.53%, respectively, representing improvements of 9.78% and 11.04% over the baseline TPN model. These results indicate that the BANN model can effectively meet the needs of traffic order supervision in transportation hubs.