基于效用理论的运动规划奖励函数设计方法

doi:10.19562/j.chinasae.qcgc.2023.08.008

Abstract

Abstract:

Personalized and driver-preferred motion planning is of great importance in enhancing the acceptance of autonomous driving systems by drivers. This paper proposes a method for designing a motion planning reward function that considers driver preferences. Firstly， a two-layer structure model for quantifying driver trajectory preferences is proposed based on utility theory. The upper-layer utility evaluation model quantifies the driver's trade-off process between safety， comfort， and efficiency， while the lower-layer driver perception model quantifies the relationship between the driver's subjective feelings about safety， comfort， and efficiency and trajectory feature indicators. Then， two estimation methods for the trajectory preference model are proposed based on rating and pairwise comparison methods， respectively. Finally， the model estimation method is verified through a driver simulator evaluation test. Each participant in the experiment subjectively evaluates multiple trajectories using both rating and pairwise comparison approaches. Based on the evaluation results from the two evaluation methods and the computed trajectory features， the driver trajectory preference model is estimated using the two approaches. The results show that the proposed model can accurately describe the driver's preference evaluation process， with the estimation results based on comparison more accurate.

Key words: utility theory, motion planning, reward function, driver preference, personalization

Wei Ran,Hui Chen,Jiaxin Yang,Nishimura Yosuke,Chaopeng Guo,Youyu Yin. Design Method of Motion Planning Reward Function Based on Utility Theory[J].Automotive Engineering, 2023, 45(8): 1373-1382.

Figures/Tables 16

References 34

1	HASENJÄGER M， WERSING H. Personalization in advanced driver assistance systems and autonomous vehicles： a review［C］. 2017 IEEE 20th International Conference on Intelligent Transportation Systems （ITSC）. IEEE， 2017： 1-7.
2	YI D， SU J， HU L， et al. Implicit personalization in driving assistance： state-of-the-art and open issues［J］. IEEE Transactions on Intelligent Vehicles， 2019， 5（3）： 397-413.
3	XU D， DING Z， HE X， et al. Learning from naturalistic driving data for human-like autonomous highway driving［J］. IEEE Transactions on Intelligent Transportation Systems， 2020， 22（12）： 7341-7354.
4	ZIEGLER J， BENDER P， DANG T， et al. Trajectory planning for Bertha—A local， continuous method［C］. 2014 IEEE Intelligent Vehicles Symposium Proceedings. IEEE， 2014： 450-457.
5	MALIK S， KHAN M A， EL-SAYED H， et al. How do autonomous vehicles decide？［J］. Sensors， 2022， 23（1）： 317.
6	SUTTON R S， BARTO A G. Reinforcement learning： an introduction［M］. MIT Press， 2018.
7	LU C， GONG J， LV C， et al. A personalized behavior learning system for human-like longitudinal speed control of autonomous vehicles［J］. Sensors， 2019， 19（17）： 3672.
8	ZHU M， WANG X， WANG Y. Human-like autonomous car-following model with deep reinforcement learning［J］. Transportation Research Part C： Emerging Technologies， 2018， 97： 348-368.
9	XIE J， XU X， WANG F， et al. Modeling human-like longitudinal driver model for intelligent vehicles based on reinforcement learning［J］. Proceedings of the Institution of Mechanical Engineers， Part D： Journal of Automobile Engineering， 2021， 235（8）： 2226-2241.
10	REDDY S， DRAGAN A， LEVINE S， et al. Learning human objectives by evaluating hypothetical behavior［C］. International Conference on Machine Learning. PMLR， 2020： 8020-8029.
11	ABBEEL P， NG A Y. Apprenticeship learning via inverse reinforcement learning［C］. Proceedings of the Twenty-First International Conference on Machine Learning， 2004： 1.
12	WU Z， QU F， YANG L， et al. Human-like decision making for autonomous vehicles at the intersection using inverse reinforcement learning［J］. Sensors， 2022， 22（12）： 4500.
13	KUDERER M， GULATI S， BURGARD W. Learning driving styles for autonomous vehicles from demonstration［C］. 2015 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2015： 2641-2646.
14	NAGAHAMA A， SAITO T， WADA T， et al. Autonomous driving learning preference of collision avoidance maneuvers［J］. IEEE Transactions on Intelligent Transportation Systems， 2020， 22（9）： 5624-5634.
15	ZIEBART B D， MAAS A L， BAGNELL J A， et al. Maximum entropy inverse reinforcement learning［C］. Aaai. 2008， 8： 1433-1438.
16	宋东鉴，朱冰，赵健，等. 基于驾驶行为生成机制的智能汽车类人行为决策［J］. 汽车工程， 2022， 44（12）： 1797-1808.
	SONG D J， ZHU B， ZHAO J， et al. Human-like behavior decision-making of intelligent vehicles based on driving behavior generation mechanism［J］. Automotive Engineering， 2022， 44（12）： 1797-1808.
17	KÄTHNER D， GRIESCHE S. Should my vehicle drive as I do？ A methodology to determine drivers， preference for automated driving styles［C］. TEAP 2017. 2017.
18	BASU C， YANG Q， HUNGERMAN D， et al. Do you want your autonomous car to drive like you？［C］. Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction， 2017： 417-425.
19	YUSOF N M， KARJANTO J， TERKEN J， et al. The exploration of autonomous vehicle driving styles： preferred longitudinal， lateral， and vertical accelerations［C］. Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications， 2016： 245-252.
20	HORSWILL M S， MCKENNA F P. The effect of perceived control on risk taking 1［J］. Journal of Applied Social Psychology， 1999， 29（2）： 377-391.
21	赵斌，陈慧，冉巍，等.车道对中控制系统的驾驶员自适应需求验证［J］.汽车技术， 2021（3）：1-6.
	ZHAO B， CHEN H， RAN W， et al. Study on the necessity of designing personalized lane centering control system ［J］.Automobile Technolody， 2021（3）：1-6.
22	KEENEY R L， RAIFFA H. Decisions with multiple objectives： preferences and value trade-offs［M］. Cambridge University Press， 1993.
23	BEN-AKIVA M E， LERMAN S R， LERMAN S R. Discrete choice analysis： theory and application to travel demand［M］. MIT Press， 1985.
24	DEUBEL C， ERNST S， PROKOP G. Objective evaluation methods of vehicle ride comfort-a literature review［J］. Journal of Sound and Vibration， 2022： 117515.
25	ZONG C， GUO K， GUAN H. Research on closed-loop comprehensive evaluation method of vehicle handling and stability［C］. SAE Paper 2000-01-0694.
26	JOHANNES F， EYKE H. Preference learning［M］. Berlin & Heidelberg： Springer， 2010.
27	MARTINEZ C M， HEUCKE M， WANG F Y， et al. Driving style recognition for intelligent vehicle control and advanced driver assistance： a survey［J］. IEEE Transactions on Intelligent Transportation Systems， 2017， 19（3）： 666-676.
28	郭子彬，陈慧，夏韬锴，等. 弯道工况下驾驶员主观风险感知的量化研究［J］. 汽车工程， 2022， 44（9）： 1447-1455.
	GUO Z B， CHEN H， XIA T K，et al. Study on quantification of driver’s subjective risk perception in curve driving condition［J］. Automotive Engineering， 2022， 44（9）： 1447-1455.
29	余志生. 汽车理论［M］. 5版.北京：机械工业出版社， 2009.
	YU Z S. Automobile theory ［M］. 5th ed. Beijing： China Machine Press， 2009.
30	MANSFIELD N J， WHITING-LEWIS E. Correlation between objective and subjective measures of automobile ride comfort for 1203 drivers［C］. 18th International Congress on Acoustics， Kyoto， Japan. 2004.
31	KIM M S， KIM K W， YOO W S. Method to objectively evaluate subjective ratings of ride comfort［J］. International Journal of Automotive Technology， 2011， 12： 831-837.
32	AO D， WONG P K， HUANG W， et al. Analysis of co-relation between objective measurement and subjective assessment for dynamic comfort of vehicles［J］. International Journal of Automotive Technology， 2020， 21： 1553-1567.
33	CAO H， ZHAO S， SONG X， et al. An optimal hierarchical framework of the trajectory following by convex optimisation for highly automated driving vehicles［J］. Vehicle System Dynamics， 2019， 57（9）： 1287-1317.
34	HOFFMANN G M， TOMLIN C J， MONTEMERLO M， et al. Autonomous automobile trajectory tracking for off-road driving： controller design， experimental validation and racing［C］. 2007 American Control Conference. IEEE， 2007： 2296-2301.

类别	变量	特征个数：特征
安全	横向偏移量	6：左/右偏移最大/平均值，左右偏移范围大小，最大偏移量
	跨道时间（STLC & CTLC，2种）	8：左侧/右侧车道线跨道时间最小值/均值及其倒数
	TAD	2：最小值，平均值
舒适	横向加速度，纵向加、减速度	2：最大值，平均值
	加加速度	2：最大值，平均值
	横摆角速度	2：最大值，平均值
	横摆角加速度	2：最大值，平均值
	WARMS	1
效率	速度	3：最大值，最小值，平均值

项目	年平均驾驶里程/km	年龄
均值	19 620	32.75
标准差	22 714	10.34

评价项	打分表	分数
1.感到很安全	1（不符合）- 5（非常符合）
2.乘坐很舒适	1（不符合）- 5（非常符合）
3.过弯很快有效率	1（不符合）- 5（非常符合）
4.喜欢这种驾驶方式	1（不符合）- 5（非常符合）

评价项	备选项
1.哪一个更安全	第一个	第二个	差不多
2.哪一个乘坐更舒适	第一个	第二个	差不多
3.哪一个过弯更有效率	第一个	第二个	差不多
4.更喜欢哪种驾驶方式	第一个	第二个	差不多

[1]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[2]	Weiguo Liu,Zhiyu Xiang,Rui Liu,Guodong Li,Zixu Wang. Research on End-to-End Vehicle Motion Planning Method Based on Deep Learning [J]. Automotive Engineering, 2023, 45(8): 1343-1352.
[3]	Ming Wang,Xiaolin Tang,Kai Yang,Guofa Li,Xiaosong Hu. A Motion Planning Method for Autonomous Vehicles Considering Prediction Risk [J]. Automotive Engineering, 2023, 45(8): 1362-1372.
[4]	Feng Gao,Defu Feng,Qiuxia Hu. Accelerating Technologies of Numerical Optimization for Motion Planning Designed by Nonlinear Model Predictive Control [J]. Automotive Engineering, 2023, 45(8): 1438-1447.
[5]	Cheng Lin, Bowen Wang, Lü Peiyuan, Xinle Gong, Xiao Yu. Research on Motion Planning and Cooperative Control for Autonomous Vehicles with Lane Change Gaming Maneuvers Under the Curved Road [J]. Automotive Engineering, 2023, 45(7): 1099-1111.
[6]	Fengchong Lan,Yingjie Liu,Jiqing Chen,Zhaolin Liu. Study on Motion Planning of Autonomous Vehicles in Cut-in Scenes Based on Dynamic Game Algorithm [J]. Automotive Engineering, 2023, 45(1): 9-19.
[7]	Fengchong Lan,Shicheng Li,Jiqing Chen,Zongmao Shen. Identification Method for Occupant Personalized Ride Comfort of Autonomous Vehicles [J]. Automotive Engineering, 2021, 43(8): 1168-1176.
[8]	Anjie Wang,Ling Zheng,Yinong Li,Kan Wang. Motion Planning for Active Collision Avoidance of Intelligent Vehicles Based on Predictive Risk Field [J]. Automotive Engineering, 2021, 43(7): 1096-1104.
[9]	Hongqing Tian,Feng Ding,Xunjia Zheng,Heye Huang,Jianqiang Wang. Motion Planning Based on Virtual Force of Potential Field for Intelligent Connected Vehicles [J]. Automotive Engineering, 2021, 43(4): 518-526.
[10]	Zhang Yiming, Zhou Bing, Wu Xiaojian, Cui Qingjia, Chai Tian. Motion Planning of High Speed Intelligent Vehicle Based on Front Vehicle Trajectory Prediction [J]. Automotive Engineering, 2020, 42(5): 574-580.

Design Method of Motion Planning Reward Function Based on Utility Theory

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 34

Related Articles 10

Metrics

Comments

Recommended 10