不确定性环境下的自动驾驶汽车行为决策方法

doi:10.19562/j.chinasae.qcgc.2024.02.003

Abstract

Abstract:

In the context of real-world driving environments， due to the perturbation of perception data and the unpredictable behavior of other traffic participants， rational decision-making in highly interactive and intricate driving scenarios considering the impact of uncertainty factors is one of the main concerns that decision-making and planning systems for autonomous vehicles must address. A behavioral decision-making method for autonomous vehicles navigating in uncertain environments is proposed in this paper. To mitigate the impact of uncertainty， the behavioral decision-making process is transformed into a partially observable Markov decision process （POMDP）. Furthermore， to tackle the computational complexity of the POMDP model， the complex network theory is applied for the first time for dynamically modeling the microscopic driving environment surrounding the autonomous vehicle， which allows for the effective characterization of interaction relationship between vehicle nodes and the scientific selection of significant vehicle nodes， guiding the autonomous vehicle's decision-making process， enabling precise identification of critical vehicle nodes， and pruning the decision space. The effectiveness of the proposed method is verified in a simulation environment， and the experimental results show that the proposed method has higher computational efficiency， superior performance， and enhanced flexibility in comparison to existing state-of-the-art behavioral decision-making methods.

Key words: autonomous vehicles, decision-making, POMDP, complex network

Xinke Fu,Yingfeng Cai,Long Chen,Hai Wang,Qingchao Liu. Decision-Making for Autonomous Driving in Uncertain Environment[J].Automotive Engineering, 2024, 46(2): 211-221.

Figures/Tables 9

References 35

1	SCHWARTING W， ALONSO-MORA J， RUS D. Planning and decision-making for autonomous vehicles ［J］. Annual Review of Control， Robotics， and Autonomous Systems， 2018， 1（1）： 187-210.
2	LIKMETA A， METELLI A M， TIRINZONI A， et al. Combining reinforcement learning with rule-based controllers for transparent and general decision-making in autonomous driving ［J］. Robotics and Autonomous Systems， 2020， 131： 103568.
3	ZHAO L， ICHISE R， SASAKI Y， et al. Fast decision making using ontology-based knowledge base［C］. Proceedings of the 2016 IEEE Intelligent Vehicles Symposium （IV）， F 19-22 June 2016.
4	VACEK S， GINDELE T， ZOLLNER J M， et al. Using case-based reasoning for autonomous vehicle guidance［C］. Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems， F 29 Oct.-2 Nov. 2007.
5	CHEN C， SEFF A， KORNHAUSER A， et al. DeepDriving： learning affordance for direct perception in autonomous driving［C］. Proceedings of the 2015 IEEE International Conference on Computer Vision （ICCV）， F 7-13 Dec. 2015.
6	DESJARDINS C， CHAIB-DRAA B. Cooperative adaptive cruise control： a reinforcement learning approach ［J］. IEEE Transactions on Intelligent Transportation Systems， 2011， 12（4）： 1248-1260.
7	SHU H， LIU T， MU X， et al. Driving tasks transfer using deep reinforcement learning for decision-making of autonomous vehicles in unsignalized intersection ［J］. IEEE Transactions on Vehicular Technology， 2022， 71（1）： 41-52.
8	MNIH V， KAVUKCUOGLU K， SILVER D， et al. Human-level control through deep reinforcement learning ［J］. Nature， 2015， 518（7540）： 529-533.
9	VITELLI M， NAYEBI A. Carma： a deep reinforcement learning approach to autonomous driving ［D］. Tech Rep Stanford University， Tech Rep， 2016.
10	FU Y， LI C， YU F R， et al. Hybrid autonomous driving guidance strategy combining deep reinforcement learning and expert system ［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（8）： 11273-11286.
11	KAELBLING L P， LITTMAN M L， CASSANDRA A R. Planning and acting in partially observable stochastic domains ［J］. Artificial Intelligence， 1998， 101（1）： 99-134.
12	CAI P， LUO Y， HSU D， et al. HyP-DESPOT： a hybrid parallel algorithm for online planning under uncertainty ［J］. The International Journal of Robotics Research， 2021， 40（2-3）： 558-573.
13	SILVER D， VENESS J. Monte-Carlo planning in large POMDPs ［J］. Advances in Neural Information Processing Systems， 2010， 23.
14	YE N， SOMANI A， HSU D， et al. DESPOT： online POMDP planning with regularization ［J］. Journal of Artificial Intelligence Research， 2017， 58： 231-266.
15	KURNIAWATI H， YADAV V. An online POMDP solver for uncertainty planning in dynamic environment ［M］. Robotics Research. Springer， 2016： 611-629.
16	HUBMANN C， SCHULZ J， XU G， et al. A belief state planner for interactive merge maneuvers in congested traffic［C］. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems （ITSC）， F 4-7 Nov. 2018.
17	BAI H， CAI S， YE N， et al. Intention-aware online POMDP planning for autonomous driving in a crowd［C］. Proceedings of the 2015 IEEE International Conference on Robotics and Automation （ICRA）， F 26-30 May 2015.
18	HUBMANN C， BECKER M， ALTHOFF D， et al. Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles［C］. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium （IV）， F 11-14 June 2017.
19	HUBMANN C， SCHULZ J， BECKER M， et al. Automated driving in uncertain environments： planning with interaction and uncertain maneuver prediction ［J］. IEEE Transactions on Intelligent Vehicles， 2018， 3（1）： 5-17.
20	GALCERAN E， CUNNINGHAM A G， EUSTICE R M， et al. Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction： theory and experiment ［J］. Autonomous Robots， 2017， 41（6）： 1367-1382.
21	CUNNINGHAM A G， GALCERAN E， EUSTICE R M， et al. MPDM： multipolicy decision-making in dynamic， uncertain environments for autonomous driving［C］. Proceedings of the 2015 IEEE International Conference on Robotics and Automation （ICRA）， F 26-30 May 2015.
22	NISHI T， DOSHI P， PROKHOROV D. Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning ［J］. IEEE Transactions on Intelligent Vehicles， 2019， 4（2）： 287-297.
23	DING W， ZHANG L， CHEN J， et al. EPSILON： an efficient planning system for automated vehicles in highly interactive environments ［J］. IEEE Transactions on Robotics， 2022， 38（2）： 1118-1138.
24	ZHANG L， DING W， CHEN J， et al. Efficient uncertainty-aware decision-making for automated driving using guided branching［C］. Proceedings of the 2020 IEEE International Conference on Robotics and Automation （ICRA）， F 31 May-31 Aug. 2020.
25	WATTS D J， STROGATZ S H. Collective dynamics of ‘small-world’networks ［J］. Nature， 1998， 393（6684）： 440-442.
26	BOCCALETTI S， LATORA V， MORENO Y， et al. Complex networks： structure and dynamics ［J］. Physics Reports， 2006， 424（4-5）： 175-308.
27	LIU Y Y， SLOTINE J J， BARABáSI A L. Controllability of complex networks ［J］. Nature， 2011， 473（7346）： 167-173.
28	YUAN Z， ZHAO C， DI Z， et al. Exact controllability of complex networks ［J］. Nature Communications， 2013， 4（1）： 2447.
29	BAGGIO G， BASSETT D S， PASQUALETTI F. Data-driven control of complex networks ［J］. Nature Communications， 2021， 12（1）： 1429.
30	LI A， CORNELIUS S P， LIU Y Y， et al. The fundamental advantages of temporal networks ［J］. Science， 2017， 358（6366）： 1042-1046.
31	UNICOMB S， IñIGUEZ G， GLEESON J P， et al. Dynamics of cascades on burstiness-controlled temporal networks ［J］. Nature Communications， 2021， 12（1）： 133.
32	STROGATZ S H. Exploring complex networks ［J］. Nature， 2001， 410（6825）： 268-276.
33	ZOU Y， DONNER R V， MARWAN N， et al. Complex network approaches to nonlinear time series analysis ［J］. Physics Reports， 2019， 787：1-97.
34	TREIBER M， KESTING A. Traffic flow dynamics ［J］. Traffic Flow Dynamics： Data， Models Simulation， Springer-Verlag Berlin Heidelberg， 2013， 983-1000.
35	COULTER R C. Implementation of the pure pursuit path tracking algorithm［M］. Carnegie Mellon University， The Robotics Institute， 1992.

[1]	ZHOU Wei, ZHANG Cheng-Ning, LI Jun-Qiu. [J]. , 2016, 38(12): 1407 -1414 .
[2]	LI Huan, HUANG Ying, ZHANG Fu-Jun, ZHAO Yu, GE Yan-Wu. [J]. , 2016, 38(12): 1420 -1426 .
[3]	HAO Han, WANG Si-南, LI Xiao, LIU Zong-Wei, ZHAO Fu-Quan. [J]. , 2017, 39(1): 1 -8 .
[4]	XU Cheng-Shan, JIANG Fa-Chao, SONG Sen-Nan, TIAN Guang-Yu. [J]. , 2017, 39(1): 9 -14 .
[5]	JIANG Ting, DAI Bing. [J]. , 2016, 38(12): 1459 -1466 .
[6]	CUI An, LI Bin, WANG Xue-Liang, ZHANG Shi-Zhan. [J]. , 2016, 38(12): 1521 -1525 .
[7]	CHU Liang, ZHAO Di, LI Wen-Hui. [J]. , 2017, 39(1): 61 -65 .
[8]	YUAN Zhi-Qun, GU Zheng-Qi, YANG Ming-Zhi, PENG Qian, LIU Xian-Gui. [J]. , 2017, 39(1): 28 -34 .
[9]	ZHANG Guan-Jun, ZHAO Xin-Feng, CAO Li-Bo. [J]. , 2017, 39(2): 150 -158 .
[10]	CAO Li-Bo, HU Yuan, YAN Ling-Bo, PENG Yu, SHI Xiang-南. [J]. , 2017, 39(2): 174 -180 .

方法	速度/（km·h^-1）	舒适度/（m·s^-2）
EPSILON	42.21	4.22
本文算法	47.56	1.90

[1]	Shengbo Eben Li,Guojian Zhan,Yuxuan Jiang,Zhiqian Lan,Yuhang Zhang,Wenjun Zou,Chen Chen,Bo Cheng,Keqiang Li. Key Technologies of Brain-Inspired Decision and Control Intelligence for Autonomous Driving Systems [J]. Automotive Engineering, 2023, 45(9): 1499-1515.
[2]	Gaoshi Zhao,Long Chen,Yingfeng Cai,Yubo Lian,Hai Wang,Qingchao Liu,Chenglong Teng. Trajectory Prediction Technology Integrating Complex Network and Memory-Augmented Network [J]. Automotive Engineering, 2023, 45(9): 1608-1616.
[3]	Cheng Lin, Bowen Wang, Lü Peiyuan, Xinle Gong, Xiao Yu. Research on Motion Planning and Cooperative Control for Autonomous Vehicles with Lane Change Gaming Maneuvers Under the Curved Road [J]. Automotive Engineering, 2023, 45(7): 1099-1111.
[4]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[5]	Yanyan Chen,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Efficient Automatic Driving Instance Segmentation Method Based on Detection [J]. Automotive Engineering, 2023, 45(4): 541-550.
[6]	Lü Ying,Xu Qi,Qiuzheng Liu,Xinyu Wang,Guoying Chen. Path Tracking Control Method with Steering Lag for Autonomous Vehicles [J]. Automotive Engineering, 2023, 45(12): 2234-2241.
[7]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[8]	Long Chen,Chen Yang,Yingfeng Cai,Hai Wang,Yicheng Li. Pedestrian Crossing Intention Prediction Method Based on Multimodal Feature Fusion [J]. Automotive Engineering, 2023, 45(10): 1779-1790.
[9]	Fengchong Lan,Yingjie Liu,Jiqing Chen,Zhaolin Liu. Study on Motion Planning of Autonomous Vehicles in Cut-in Scenes Based on Dynamic Game Algorithm [J]. Automotive Engineering, 2023, 45(1): 9-19.
[10]	Zhenhai Gao,Xiangtong Yan,Fei Gao. A Decision-making Method for Longitudinal Autonomous Driving Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2022, 44(7): 969-975.
[11]	Yingfeng Cai,Ziheng Lu,Yicheng Li,Long Chen,Hai Wang. Tightly Coupled SLAM System Based on Multi-Sensor Fusion [J]. Automotive Engineering, 2022, 44(3): 350-361.
[12]	Jingwei Zhang,Tiejun Liu,Rengang Li,Dan Liu,Jinglin Zhan,Hongwei Kan. A Temporal Calibration Method for Multi-Sensor Fusion of Autonomous Vehicles [J]. Automotive Engineering, 2022, 44(2): 215-224.
[13]	Dongjian Song,Bing Zhu,Jian Zhao,Jiayi Han,Yanchen Liu. Human-Like Behavior Decision-Making of Intelligent Vehicles Based on Driving Behavior Generation Mechanism [J]. Automotive Engineering, 2022, 44(12): 1797-1808.
[14]	Chaoyang Jiang,Tianran Lan,Xiaoni Zheng,Jiulong Gao,Xuetong Ye. Distributed Multi-vehicle Collaborative Visual SLAM System [J]. Automotive Engineering, 2022, 44(12): 1809-1817.
[15]	Yuande Jiang,Bing Zhu,Xiangmo Zhao,Jian Zhao,Bingbing Zheng. Modeling of Traffic Vehicle Interaction for Autonomous Vehicle Testing [J]. Automotive Engineering, 2022, 44(12): 1825-1833.

Decision-Making for Autonomous Driving in Uncertain Environment

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 35

Related Articles 15

Metrics

Comments

Recommended 10