A Spatio-temporal Transformer-STGNN Hybrid Reinforcement Learning Method for Bus Cooperative Optimization

To address multi-objective conflicts and global ripple delays caused by local decisions, a dynamic cooperative optimization model is investigated. This model integrates a Spatio-Temporal Transformer and a spatiotemporal graph neural network (STGNN). A bidirectional interactive architecture for the perception and anticipation of spatiotemporal features is constructed. Dynamic dependencies of bus stops are extracted using the spatial attention mechanism of the Transformer. An adaptive weight matrix is generated and subsequently inputted into the STGNN as a dynamic adjacency matrix. This network combines graph convolutional networks and causal dilated convolutions to predict risks of passenger accumulation and delay propagation. The anticipated risk scores are reversely transmitted to the temporal attention layer. Driven by risks, this closed-loop feedback mechanism adjusts the weights. Thereby, the nonlinear ripple effects in space and time generated by single scheduling decisions are quantified. A hybrid reinforcement learning framework is designed. An improved genetic algorithm (GA) and a Deep Double Q-Network (DDQN) are nested in this framework. A Pareto front solution set is generated using the global broad search of the GA. This set serves as the initial strategy space for the DDQN. Simultaneously, the risk scores from the STGNN are introduced into the multi-objective reward function as safety constraint penalties. Consequently, the operational efficiency of vehicles and the travel costs of passengers are balanced. Thus, optimal coordinated control strategies for stop-skipping and holding are generated. Simulation experiments are conducted based on the operational data of Foshan Bus Route 101 during peak hours. The results indicate that the fluctuations in headways are controlled by this strategy. Furthermore, the continuous bus bunching phenomenon is successfully avoided. Under the premise of a 0.83% increase in the average passenger waiting time, the average passenger in-vehicle time is reduced by approximately 24.7%. Meanwhile, the overall vehicle travel time is shortened, and the driving speed between stops is improved. Compared with traditional single deep reinforcement learning algorithms, this hybrid algorithm exhibits fewer iteration counts and smaller optimization errors. The proposed model is applicable to the real-time cooperative scheduling for urban trunk buses with high-frequency departures. A dynamic balance of multiple scheduling strategies is achieved under the premise of controlling risk propagation.