A Merging Model Based on Piecewise Deep Reinforcement Learning for Connected and Autonomous Vehicle in Work Zone under Mixed Autonomy

The classical early and late merge work worse under dynamic demand, and render conflict merging gap due to large speed differences at the upstream. To this end, a piece-wise deep reinforcement learning-based merging model is proposed for connected and autonomous vehicles (CAVs) in work zones under mixed autonomy. Above all, the merging conflicts and efficiency reduction caused by many vehicles in closed lanes trying to merge into one gap on the open lane are addressed by the model with speed guidance, gap creation, and positional alignment. Such a model consists of the soft Actor-Critic algorithm-based longitudinal control and the rule-based lane-changing decision-making. For longitudinal control, 9 features are selected as the agent state to describe surrounding traffic conditions from both local and global views. The mentioned features include the speed and acceleration of the ego vehicle, the speed of and the distance to the lead vehicle, the speed of and the distance to the lead and lag vehicles on the adjacent left lane, and the distance to the merging point. Subsequently, a piecewise reward function for CAVs in the work zone is established by optimizing comfort, safety, and efficiency simultaneously. Such a reward function combines minimizing acceleration and jerk, preventing collisions, generating merging gaps, aligning with the gap center on the open lane, mitigating vehicular speed differences, adhering to advisory speed, and encouraging following vehicles with yield behavior. Particularly, an item of reward function with respect to driving efficiency is shaped on the basis of the speed difference between the lag vehicle on the adjacent lane and the ego vehicle, such that halting of both the CAV and the human-driving vehicle can be alleviated at the merging point. Simulation results illustrate that the proposed model increases by about 4.76% of average speed, and 19.71% of minimal time-to-collision under medium/heavy demand in work zone, in contrast to early merge, late merge and New England merge. In addition, the average speed, minimum time-to-collision, and successful merging rate in mixed autonomy with heterogeneous human-driving vehicles, increase with the increase of the CAV market penetration rate, while all the vehicles merge without halting.