基于Q学习模型的无信号交叉口离散车队控制

doi:10.19562/j.chinasae.qcgc.2022.09.006

Abstract

Abstract:

In order to enhance the driving efficiency of connected and automated vehicles （CAVs） at traffic bottlenecks， a platoon cooperative control strategy at an unsignalized intersection is proposed. Firstly， a control framework for the allocation of the right of way for platoons is put forward based on the traffic flow model and occupied time of platoons at the intersection. Then， a Q-learning model is designed to conditionally select platoon sizes， with instantaneous efficiency and travel delays as compound indicators. Finally， an online trajectory planning simulation is carried out for the grouped vehicles based on vehicle following model. The results show that the Q-learning model can flexibly allocate the platooning commands according to different working conditions and ensure the overall safety of platoons during driving process. Compared with the nonplatoon scheme， the traffic capacity of the intersection is increased by around 36.1%.

Key words: connected and automated vehicles, unsignalized intersection, multi-platoon cooperation, Q-learning model, trajectory planning

Lijun Qian,Chen Chen,Jian Chen,Xinyu Chen,Chi Xiong. Discrete Platoon Control at an Unsignalized Intersection Based on Q-learning Model[J].Automotive Engineering, 2022, 44(9): 1350-1358.

Figures/Tables 14

变量名称	变量说明	取值
$ω 1$	$X 1$ 权重	0.481
$ω 2$	$X 2$ 权重	0.267
$ω 3$	$X 3$ 权重	0.148
$ω 4$	$X 4$ 权重	0.106
$n m a x$	最大车队规模	3辆
$α$	模型学习率	0.001
$γ$	折扣因子	0.800

变量名称	变量说明	取值
$d m$	合流区尺寸	50 m
$d c$	协同区尺寸	30 m
$d b$	观察区尺寸	80 m
$d s$	子区间尺寸	15 m
$d r$	车道宽度	3.5 m
$d 0$	跟车安全距离	2.0 m
$x ˙$	车速范围	0~40 km/h
$x ¨$	加速度范围	-5~5 m/s²
$v 0$	初始车速	40 km/h
$v m$	到达合流区时车速	25 km/h
$l$	车辆长度	4.5 m

References 20

1	AHMANE M， ABBAS-TURKI A， PERRONNET F， et al. Modeling and controlling an isolated urban intersection based on cooperative vehicles ［J］. Transportation Research Part C： Emerging Technologies， 2013， 28： 44-62.
2	BIAN Y， LI S E， REN W， et al. Cooperation of multiple connected vehicles at unsignalized intersections： Distributed observation， optimization， and control ［J］. IEEE Transactions on Industrial Electronics， 2019， 67（12）： 10744-10754.
3	MENG Y， LI L， WANG F Y， et al. Analysis of cooperative driving strategies for nonsignalized intersections ［J］. IEEE Transactions on Vehicular Technology， 2017， 67（4）： 2900-2911.
4	LIANG K Y， MARTENSSOM J， JOHANSSON K H. Heavy-duty vehicle platoon formation for fuel efficiency ［J］. IEEE Transactions on Intelligent Transportation Systems， 2015， 17（4）： 1051-1061.
5	赵祥模，张心睿，王润民，等. 网联交叉口信号-车辆轨迹协同优化控制方法［J］. 汽车工程， 2021， 43（11）： 1577-1586.
	ZHAO X M， ZHANG X R， WANG R M， et al. Cooperative optimization control method of traffic signals and vehicle trajectories at connected intersection ［J］. Automotive Engineering， 2021， 43（11）：1577-1586.
6	BASHIRI M， FLEMING C H. A platoon-based intersection management system for autonomous vehicles ［C］. 2017 IEEE Intelligent Vehicles Symposium （IV）， 2017.
7	LIORIS J， PEDARSANI R， TASCIKARAOGLU F Y， et al. Platoons of connected vehicles can double throughput in urban roads ［J］. Transportation Research Part C： Emerging Technologies， 2017， 77： 292-305.
8	YU C， SUN W， LIU H X， et al. Managing connected and automated vehicles at isolated intersections： from reservation-to optimization-based methods ［J］. Transportation Research Part B： Methodological， 2019， 122： 416-435.
9	BASHIRI M， JAFARZADEH H， FLEMING C H. Paim： platoon-based autonomous intersection management ［C］. 21st International Conference on Intelligent Transportation Systems （ITSC）， 2018.
10	YOUNES M B， BOUKERCHE A. Intelligent traffic light controlling algorithms using vehicular networks ［J］. IEEE Transactions on Vehicular Technology， 2015， 65（8）： 5887-5899.
11	余伶俐，邵玄雅，龙子威，等.智能车辆深度强化学习的模型迁移轨迹规划方法［J］. 控制理论与应用， 2019， 36（9）： 1409-1422.
	YU L L， SHAO X Y， LONG Z W， et al. Intelligent land vehicle model transfer trajectory planning method of deep reinforcement learning ［J］. Control Theory & Applications， 2019， 36（9）： 1409-1422.
12	HOEL C J， DRIGGS-CAMPBELL K， WOLFF K， et al. Combining planning and deep reinforcement learning in tactical decision making for autonomous driving ［J］. IEEE Transactions on Intelligent Vehicles， 2019， 5（2）： 294-305.
13	LIN P， LIU J， JIN P J， et al. Autonomous vehicle-intersection coordination method in a connected vehicle environment ［J］. IEEE Intelligent Transportation Systems Magazine， 2017， 9（4）： 37-47.
14	XU B， LI S E， BIAN Y， et al. Distributed conflict-free cooperation for multiple connected vehicles at unsignalized intersections ［J］. Transportation Research Part C： Emerging Technologies， 2018， 93： 322-334.
15	LI B， ZHANG Y， ZHANG Y， et al. Near-optimal online motion planning of connected and automated vehicles at a signal-free and lane-free intersection ［C］. 2018 IEEE Intelligent Vehicles Symposium （IV）， 2018.
16	COLOMBO A， DEL VECCHIO D. Least restrictive supervisors for intersection collision avoidance： a scheduling approach ［J］. IEEE Transactions on Automatic Control， 2014， 60（6）： 1515-1527.
17	AMIRGHOLY M， NOURINEJAD M， GAO H O. Optimal traffic control at smart intersections： automated network fundamental diagram ［J］. Transportation Research Part B： Methodological， 2020， 137： 2-18.
18	TOUHBI S， BABRAM M A， NGUYEN-HUU T， et al. Adaptive traffic signal control： exploring reward definition for reinforcement learning ［J］. Procedia Computer Science， 2017， 109： 513-520.
19	李茹，马育林，田欢，等.基于熵值和G1法的自动驾驶车辆综合智能定量评价［J］. 汽车工程， 2020， 42（10）： 1327-1334.
	LI R， MA Y L， TIAN H， et al. Comprehensive intelligent quantitative evaluation of autonomous vehicle based on entropy and G1 methods ［J］. Automotive Engineering， 2020， 42（10）： 1327-1334.
20	CHEN C， WU B， XUAN L， et al. A discrete control method for the unsignalized intersection based on cooperative grouping ［J］. IEEE Transactions on Vehicular Technology， 2022， 71（1）： 123-136.

[1]	Shurui Guan,Keqiang Li,Junyu Zhou,Jia Shi,Weiwei Kong,Yugong Luo. A Cooperative Lane Change Strategy for Intelligent Connected Vehicles Oriented to Mandatory Lane Change Scenarios [J]. Automotive Engineering, 2024, 46(2): 201-210.
[2]	Jizheng Liu,Zhenpo Wang,Fengchun Sun,Lei Zhang. Research on Delay Compensation Control for Heterogeneous Connected and Automated Vehicle Platoons [J]. Automotive Engineering, 2023, 45(9): 1573-1582.
[3]	Jun Li, Wei Zhou, Shuang Tang. Lane Change and Obstacle Avoidance Trajectory Planning of Intelligent Vehicle Based on Adaptive Fitting [J]. Automotive Engineering, 2023, 45(7): 1174-1183.
[4]	Lijun Qian,Chen Chen,Jian Chen. Centralized Trajectory Planning in an Unsignalized Intersection Environment Considering Driver Error [J]. Automotive Engineering, 2023, 45(5): 768-776.
[5]	Hongchang Zhang,Peng Ning,Jie Yang,Jianwei Song,Lin Hao,Juan Zeng. Parallel Parking Trajectory Planning Based on Double-Layer Solution Strategy [J]. Automotive Engineering, 2023, 45(12): 2299-2309.
[6]	Yihe Chen,Weiwei Kong,Jie Yu,Keqiang Li,Yugong Luo. Reservation-based Vehicle Platoon Control at Unsignalized Intersections Under Mixed Traffic Condition [J]. Automotive Engineering, 2022, 44(7): 953-959.
[7]	Bin Tang,Zhanxiang Xu,Haobin Jiang,Yingfeng Cai,Zitian Hu,Zhengyi Yang. Trajectory Planning of Intelligent Vehicles in Lane Change for Collision Avoidance Based on Segmented Optimization [J]. Automotive Engineering, 2022, 44(6): 831-841.
[8]	Jiaxing Yu,Arab Aliasghar,Xiaofei Pei,Xuexun Guo. Research on Cornering Trajectory Planning for Intelligent Vehicle Considering Trajectory Smoothness and Stability for Collision Avoidance [J]. Automotive Engineering, 2022, 44(5): 656-663.
[9]	Jie Hu,Minchao Zhang,Wencai Xu,Ruinan Chen,Xinkai Zhong,Linglei Zhu. Parallel Parking Trajectory Planning for Autonomous Vehicles [J]. Automotive Engineering, 2022, 44(3): 330-339.
[10]	Guochen Niu,Wenshuai Li,Hongxu Wei. Intelligent Vehicle Lane Changing Trajectory Planning Based on Double Quintic Polynomials [J]. Automotive Engineering, 2021, 43(7): 978-986.
[11]	Zhijun Gao,Jiangfeng Wang,Lei Chen,Jiakuan Dong,Dongyu Luo,Xuedong Yan. Trajectory Planning Algorithm for CAV at Intersections Based on Dynamic Distance Windows [J]. Automotive Engineering, 2021, 43(4): 537-545.
[12]	Zhang Jiaxu, Shi Zhengtang, Zhao Jian, Zhu Bing. Optimal Control Strategy Design for Vehicle High-speed Emergency Lane Change Collision Avoidance Based on Radau Pseudospectral Method [J]. Automotive Engineering, 2020, 42(8): 1040-1049.
[13]	Gao Zhenhai, Zhu Naixuan, Gao Fei, Mei Xingtai, Zhang Jin, He Lei. Self-learning Lane-change Trajectory Planning System with Driver Characteristics [J]. Automotive Engineering, 2020, 42(12): 1710-1717.
[14]	Hu Yanping, Tian Bo, Chen Wuwei, Zhang Ruichen. Obstacle Avoidance Trajectory Planning for Intelligent VehicleBased on Derived State Lattice from Obstacle [J]. Automotive Engineering, 2019, 41(8): 967-974.

Discrete Platoon Control at an Unsignalized Intersection Based on Q-learning Model

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 20

Related Articles 14

Metrics

Comments

Recommended 10