面向狭窄环境的安全泊车路径规划算法研究

doi:10.19562/j.chinasae.qcgc.2025.05.001

Abstract

Abstract:

For safe and feasible path-planning in real time of autonomous parking system， a parking path planning algorithm based on constrained reinforcement learning with a hybrid action space is proposed in this paper. Specifically， the proposed algorithm employs a hybrid action space reinforcement learning framework that integrates discrete actions with continuous parameters to achieve parameterized trajectory planning， thereby enhancing the executability of planned paths. On this basis， a constrained reinforcement learning algorithm within the hybrid action space is designed to optimize safe policy execution， ensuring the safety of parking paths. Moreover， a curriculum learning mechanism is introduced during model training to guide exploration progressively， improving training stability and convergence speed. Finally， extensive comparative and ablation experiments are conducted on both perpendicular and parallel parking scenarios. The experimental results show that the proposed parking path planning algorithm outperforms existing state-of-the-art methods in terms of success rate， safety， and real-time performance， exhibiting superior overall effectiveness.

Key words: autonomous parking, hybrid-action reinforcement learning, motion-planning, constraint optimization

Jiayi Guan,Bin Li,Ao Zhou,Zhiguo Zhao,Qiao Lin,Guang Chen. Study on Safe Parking Path Planning Algorithm for Narrow Environment[J].Automotive Engineering, 2025, 47(5): 797-808.

Figures/Tables 17

算法1：混合动作安全泊车路径规划算法
1：输入：初始化策略网络参数 $θ 0$ ；初始化奖励值函数网络参数 $? 0$ ；初始化成本值函数网络参数 $φ 0$
2：for 每个回合do：
3：通过策略 $π θ t$ 在环境中采集多组轨迹 $D t = {s t, a t, r t, c t i}$
4：计算后续轨迹奖励和成本 $R ? t$ 、 $C ? t$
5：基于当前奖励值网络 $V ? t r$ 计算奖励的优势函数 $A r, d π θ t$ 和 $A r, p π θ t$
6：基于当前成本值网络 $V φ t c$ 计算成本的优势函数 $A c, d π θ t$ 和 $A c, p π θ t$
7：基于式（7）估算轨迹成本值 $L c π θ t$
8：如果 $L c π θ t ≤ c ˉ$ 条件满足则通过奖励优势函数更新策略：
$θ t + 1 = a r g m a x ???????? θ E π θ t K ? (θ) (A r, d π θ t + A r, p π θ t)$
9：反之则通过成本优势函数更新策略：
$θ t + 1 = a r g m i n ???????? θ E π θ t K ? (θ) (A c, d π θ t + A c, p π θ t)$
10：通过均方误差拟合奖励和成本值函数：
$? t + 1 = a r g m i n ? 1 D t T ∑ τ ~ ? D t ∑ t = 0 T V ? r s t - R ? t 2$
$φ t + 1 = a r g m i n ????????? φ 1 D t T ∑ τ ~ ? D t ∑ t = 0 T (V φ c (s t) - C ? t) 2$
11：终止for
12：输出：策略网络 $π θ$

方法	Reeds-Shepp	Hybrid A*	SAC	PPO	HCRL（Ours）
$S r ?$ /% $↑$	1.00±0.00	1.00±0.00	1.00±0.00	1.00±0.00	1.00±0.00
$C s$ $↓$	52.98±26.78	8.75±3.32	26.00±12.72	27.94±13.55	8.67±3.89
$T c ?$ /s $↓$	0.01±0.00	2.74±3.86	0.50±0.03	0.56±0.03	0.07±0.01
$L p ?$ /m $↓$	8.91±0.36	9.68±1.16	14.73±2.07	15.98±2.21	9.78±0.86
$N g$ $↓$	1.00±0.00	1.33±0.58	5.84±2.30	6.16±2.77	1.66±0.71
$N c$ $↓$	2.87±0.51	3.68±1.03	14.18±3.04	13.37±3.19	3.82±1.50
$S o$ $↑$	83.33	79.23	44.47	40.58	94.05

方法	Reeds-Shepp	Hybrid A*	SAC	PPO	HCRL（Ours）
$S r ?$ /% $↑$	1.00 ±0.00	0.83 ±0.12	1.00 ±0.00	1.00 ±0.00	1.00±0.00
$C s$ $↓$	65.49 ±29.72	10.70 ±4.66	16.22 ±7.27	18.02±7.82	7.92±3.76
$T c ?$ /s $↓$	0.01 ±0.00	23.93 ±7.20	0.31 ±0.06	0.32 ±0.08	0.06±0.02
$L p ?$ /m $↓$	8.80 ±0.34	9.34 ±0.84	11.02 ±1.65	11.64 ±1.81	9.96±0.92
$N g$ $↓$	2.00 ±0.00	2.26 ±1.14	2.60±0.34	2.72 ±1.16	2.31±1.01
$N c$ $↓$	3.82 ±1.38	4.51 ±1.94	13.29±2.53	12.56 ±2.25	4.69 ±1.22
$S o$ $↑$	83.33	55.46	53.80	48.14	84.45

方法	PADDPG-Lag	HPPO-Lag	PDQN-Lag-R	HCRL（Ours）
$S r ?$ /% $↑$	0.98 ± 0.01	0.99± 0.01	1.00 ± 0.00	1.00 ± 0.00
$C s$ $↓$	11.33 ± 5.10	10.04 ± 4.86	12.58 ± 5.77	8.67 ± 3.89
$T c ?$ /s $↓$	0.10± 0.04	0.09± 0.05	0.11± 0.05	0.07± 0.01
$L p ?$ /m $↓$	10.30 ± 1.05	12.67 ± 1.49	11.08 ± 0.88	9.78± 0.86
$N g$ $↓$	1.56 ± 0.53	1.74± 0.89	1.67 ± 0.70	1.66 ± 0.71
$N c$ $↓$	3.61 ± 1.36	4.14 ± 1.86	3.95± 1.64	3.82 ± 1.50
$S o$ $↑$	75.61	77.54	89.05	94.05

方法	PADDPG-Lag	HPPO-Lag	PDQN-Lag-R	HCRL（Ours）
$S r ?$ /% $↑$	0.99 ± 0.01	1.00 ± 0.00	0.97 ± 0.01	1.00 ± 0.00
$C s$ $↓$	12.09 ± 4.65	10.31 ±4.30	13.17 ± 5.83	7.92 ± 3.76
$T c ?$ /s $↓$	0.07 ± 0.02	0.06 ± 0.01	0.07 ± 0.03	0.06 ± 0.02
$L p ?$ /m $↓$	10.62 ± 1.69	10.03 ± 1.37	11.87 ± 1.82	9.96 ± 1.14
$N g$ $↓$	2.67 ± 1.68	2.43 ± 1.23	2.91 ± 1.75	2.31± 1.01
$N c$ $↓$	6.29 ± 4.26	5.03 ± 0.96	6.67 ± 6.95	4.69 ± 1.22
$S o$ $↑$	67.23	79.97	57.15	84.45

References 41

1	ZHAO C， LIAO F， LI X， et al. Macroscopic modeling and dynamic control of on-street cruising-for-parking of autonomous vehicles in a multi-region urban road network［J］. Transportation Research Part C： Emerging Technologies， 2021，128： 103176.
2	BOCK F， DI MARTINO S， ORIGLIA A. Smart parking： using a crowd of taxis to sense on-street parking space availability［J］. IEEE Transactions on Intelligent Transportation Systems， 2019， 21（2）： 496-508.
3	XIE M， ZHANG X， WU Z， et al. A shared parking optimization framework based on dynamic resource allocation and path planning［J］. Physica A： Statistical Mechanics and Its Applications， 2023， 616：128649.
4	KHALID M， WANG K， ASLAM N， et al. From smart parking towards autonomous valet parking： a survey， challenges and future works［J］. Journal of Network and Computer Applications， 2021， 175： 102935.
5	胡杰，朱令磊，陈瑞楠，等. 狭小车位平行泊车路径规划方法研究［J］. 汽车工程， 2022， 44（7）： 1040-1048.
	HU J， ZHU L， CHEN R， et al. Research on path planning method for parallel parking in narrow spaces ［J］. Automotive Engineering， 2022， 44（7）： 1040-1048.
6	QIN Z， CHEN X， HU M， et al. A novel path planning methodology for automated valet parking based on directional graph search and geometry curve ［J］. Robotics and Autonomous Systems， 2020， 132：103606.
7	REEDS J， LAWRENCE S. Optimal paths for a car that goes both forwards and backwards ［J］. Pacific Journal of Mathematics， 1990，145（2）： 367-393.
8	胡杰，张敏超，徐文才，等. 自动驾驶车辆的平行泊车轨迹规划［J］. 汽车工程， 2022， 44 （3）： 330-339.
	HU J， ZHANG M， XU W， et al. Trajectory planning for parallel parking of autonomous vehicles ［J］. Automotive Engineering， 2022， 44（3）： 330-339.
9	CHEN G， HOU J， DONG J， et al. Multi-objective scheduling strategy with genetic algorithm and time-enhanced A* planning for autonomous parking robotics in high-density unmanned parking lots ［J］. IEEE/ASME Transactions on Mechatronics， 2020， 26（3）： 1547-1557.
10	SHENG W， LI B， ZHONG X. Autonomous parking trajectory planning with tiny passages： a combination of multistage hybrid A-star algorithm and numerical optimal control ［J］. IEEE Access， 2021，9：102801-102810.
11	DU Z， MIAO Q， ZONG C. Trajectory planning for automated parking systems using deep reinforcement learning ［J］. International Journal of Automotive Technology， 2020， 21（4）： 881-887.
12	ZHANG P， XIONG L， YU Z， et al. Reinforcement learning-based end-to-end parking for automatic parking system［J］. Sensors. 2019， 19（18）： 3996.
13	SONG S， CHEN H， SUN H， et al. Data efficient reinforcement learning for integrated lateral planning and control in automated parking system［J］. Sensors， 2020， 20（24）：7297.
14	CHAN K H， MUSTAPHA A， JUBAIR M A. Comparative analysis of loss functions in TD3 for autonomous parking ［J］. Journal of Soft Computing and Data Mining， 2024，5（1）：1-4.
15	高强，陆洲，段晨东，等. 汽车垂直泊车路径规划与路径跟踪研究［J］. 汽车工程， 2021， 43 （7）： 987-994，1012.
	GAO Q， LU Z， DUAN C， et al. Research on path planning and path tracking for vertical parking of vehicles ［J］. Automotive Engineering， 2021， 43（7）： 987-994，1012.
16	LI B， WANG K， SHAO Z. Time-optimal maneuver planning in automatic parallel parking using a simultaneous dynamic optimization approach ［J］. IEEE Transactions on Intelligent Transportation Systems， 2016， 17（11）： 3263-3274.
17	胡文，谭运生，康龙云，等. 基于驾驶员经验的自动泊车规划算法研究［J］. 汽车工程， 2019， 41 （12）： 1394-1400，1415.
	HU W， TAN Y S， KANG L Y， et al. Study on automatic parking planning algorithm based on driver's experience［J］. Automotive Engineering， 2019， 41（12）： 1394-1400，1415.
18	张家旭，王晨，赵健. 面向狭小平行泊车位的路径规划与跟踪控制［J］. 吉林大学学报（工学版）， 2021， 51（5）： 1879-1886.
	ZHANG J， WANG C， ZHAO J. Path planning and tracking control for narrow parallel parking spaces ［J］. Journal of Jilin University （Engineering and Technology Edition）， 2021， 51（5）： 1879-1886.
19	DOLGOV D， THRUN S， MONTEMERLO M， et al. Path planning for autonomous vehicles in unknown semi-structured environm-ents ［J］. The International Journal of Robotics Research， 2010， 29（5）：485-501.
20	JHANG J H， LIAN F L. An autonomous parking system of optimally integrating bidirectional rapidly-exploring random trees and parking-oriented model predictive control ［J］. IEEE Access， 2020， 8： 163502-163523.
21	KARAMAN S， FRAZZOLI E. Sampling-based algorithms for optimal motion planning［J］. The International Journal of Robotics Research， 2011，30（7）： 846-894.
22	GUAN J， SHEN L， ZHOU A， et al. POCE： primal policy optimization with conservative estimation for multi-constraint offline reinforcement learning ［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2024： 26243-26253.
23	LIU H X， FENG S. Curse of rarity for autonomous vehicles［J］. Nature Communications， 2024，15（1）：4808.
24	TAKEHARA R， GONSALVES T. Autonomous car parking system using deep reinforcement learning ［C］. 2021 2nd International Conference on Innovative and Creative Information Technology （ICITech）. IEEE， 2021： 85-89.
25	SHI J， LI K， PIAO C， et al. Model-based predictive control and reinforcement learning for planning vehicle-parking trajectories for vertical parking spaces［J］. Sensors， 2023， 23 （16）： 7124.
26	WU Y， WANG L， LU X， et al. Reinforcement learning-based autonomous parking with expert demonstrations［C］. 2023 7th CAA International Conference on Vehicular Control and Intelligence （CVCI）. IEEE， 2023： 1-6.
27	DEN HENGST F， FRANÇOIS-LAVET V， HOOGENDOORN M， et al. Planning for potential： efficient safe reinforcement learning［J］. Machine Learning， 2022， 111（6）：2255-2274.
28	TIONG T， SAAD I， TEO K T， et al. Autonomous valet parking with asynchronous advantage actor-critic proximal policy optimization ［C］. 2022 IEEE 12th Annual Computing and Communication Workshop and Conference （CCWC）. IEEE， 2022： 334-340.
29	JUNZUO L， QIANG L. An automatic parking model based on deep reinforcement learning［J］. In Journal of Physics： Conference Series， 2021，1883（1）.
30	TANG X， YANG Y， LIU T， et al. Path planning and tracking control for parking via soft actor-critic under non-ideal scenarios ［J］. IEEE/CAA Journal of Automatica Sinica，2023，11（1）： 181-195.
31	WÖHLKE J， SCHMITT F， VAN HOOF H. Hierarchies of planning and reinforcement learning for robot navigation ［C］. 2021 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2021： 10682-10688.
32	YUAN Z， WANG Z， LI X， et al. Hierarchical trajectory planning for narrow-space automated parking with deep reinforcement learning： a federated learning scheme［J］. Sensors， 2023，23 （8）：4087.
33	CAI L， GUAN H， ZHOU Z Y， et al. Parking planning under limited parking corridor space ［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 24（2）：1962-1981.
34	LIU W， LI Z， LI L， et al. Parking like a human： a direct trajectory planning solution ［J］. IEEE Transactions on Intelligent Transpor-tation Systems， 2017，18（12）：3388-3397.
35	SOUSA B， RIBEIRO T， COELHO J， et al. Parallel， angular and perpendicular parking for self-driving cars using deep reinforcement learning ［C］. 2022 IEEE International Conference on Autonomous Robot Systems and Competitions （ICARSC）. IEEE， 2022： 40-46.
36	HAARNOJA T， ZHOU A， ABBEEL P， et al. Soft actor-critic： off-policy maximum entropy deep reinforcement learning with a stochastic actor ［C］. International Conference on Machine Learning， PMLR. 2018： 1861-1870.
37	HAUSKNECHT M， STONE P. Deep reinforcement learning in parameterized action space ［C］. International Conference on Learning Representations， 2015.
38	WANG Y， HE H， TAN X. Truly proximal policy optimization ［C］. Uncertainty in Artificial Intelligence， 2020：113-122.
39	HAARNOJA T， ZHOU A， HARTIKAINEN K， et al. Soft actor-critic algorithms and applications ［C］. Proceedings of the 35th International Conference on Machine Learning， 2018.
40	XIONG J， WANG Q， YANG Z， et al. Parametrized deep Q-networks learning： reinforcement learning with discrete-continuous hybrid action space ［J］. arXiv preprint arXiv：1810. 06394. 2018.
41	FAN Z， SU R， ZHANG W， et al. Hybrid actor-critic reinforcement learning in parameterized action space［C］. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence， 2019： 2279-2285.

[1]	Wang Yongsheng, Luo Yugong, Gu Yanchen, Chen Rui, Qi Yunlong, Jiang Fachao. Precise Adjustment Strategy of Autonomous Parking for Automatic Charging [J]. Automotive Engineering, 2020, 42(12): 1701-1709.
[2]	Qian Lijun, Wu Bing, Qiu Duoyang, Hu Weilong. Path Planning for Autonomous Parallel Parking Based on Piecewise Gauss Pseudospectral Method [J]. Automotive Engineering, 2019, 41(12): 1401-1409.

Study on Safe Parking Path Planning Algorithm for Narrow Environment

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 41

Related Articles 2

Metrics

Comments

Recommended 0