基于单目视觉运动估计的周边多目标轨迹预测方法

doi:10.19562/j.chinasae.qcgc.2022.09.003

Abstract

Abstract:

Based on a low-cost monocular visual perception system， a multi-target trajectory prediction method considering the influence of vehicle motion is proposed in this paper. Firstly， the ego vehicle motion estimation model composed of the depth estimation network and the position and orientation estimation network is established to achieve effective calculation of ego vehicle visual odometer in image sequence. Then， a prediction model is built by using the historical position and orientation sequences of ego vehicle， and a normalization processing on the historical positions of surrounding targets is fulfilled under the current perspective of ego vehicle camera. Finally， the prediction network is constructed based on the historical trajectory information and regional image information to realize the effective prediction of surrounding multi-target trajectories around intelligent vehicles. The innovation points of this paper are combining visual SLAM method with trajectory predictive model and putting forward the new motion estimation model and ConvLSTM-based multi-target trajectory predictive network. The model proposed overcomes the adverse influence of existing research on the trajectory prediction accuracy of the surrounding target caused by ignoring the ego vehicle motion state， and achieves a better prediction results under the condition of using monocular vision perception only. The results of test on public data sets show that with a prediction time step of 1.5s， the model proposed has a MSEcenter of 321， i.e. a 52% lower than that of the existing baseline model， with an excellent performance also in the long-time-step trajectory prediction in the future.

Key words: intelligent vehicles, trajectory prediction, monocular vision, depth estimation

Zihao Wang,Yingfeng Cai,Hai Wang,Long Chen,Xiaoxia Xiong. Surrounding Multi-Target Trajectory Prediction Method Based on Monocular Visual Motion Estimation[J].Automotive Engineering, 2022, 44(9): 1318-1326.

Figures/Tables 8

模型

M S E b b o x

M S E c e n t e r

M S E f u l l

IoU

Linear

847/2 861/4 982

767/2 282/5 680

29 244

0.48/0.24/0.23

Seq2Seq

313/653/1 187

160/333/669

4 363

0.60/0.54/0.43

本文

方法

196/423/838

43/103/321

1 553

0.75/0.65/0.53

模型	$M S E b b o x$	$M S E c e n t e r$ （1.5 s）	$M S E f u l l$
Linear^［15］	233/857/2 303	1 565	6 111
LSTM^［15］	289/569/1 558	1 474	5 766
PIETraj^［15］	110/399/1 248	1 183	4 780
BiTraP-D^［17］	93/378/1 206	1 105	4 565
本方法（KITTI）	196/423/838	321	1 553

训练阶段	$M S E b b o x$	$M S E c e n t e r$	$M S E f u l l$	IoU
一阶段	288/726/1 556	135/406/1 039	5 489	0.63/0.50/0.39
二阶段	226/532/1 040	72/203/523	2 784	0.67/0.53/0.43
完整训练	196/423/838	43/103/321	1 553	0.75/0.65/0.53

References 27

1	DEO N， TRIVEDI M M. Convolutional social pooling for vehicle trajectory prediction［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018： 1468-1476.
2	MESSAOUD K， YAHIAOUI I， VERROUST-BLONDET A， et al. Non-local social pooling for vehicle trajectory prediction［C］.2019 IEEE Intelligent Vehicles Symposium （IV）. IEEE， 2019： 975-980.
3	ZHAO T， XU Y， MONFORT M， et al. Multi-agent tensor fusion for contextual trajectory prediction［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019： 12126-12134.
4	JEON H， CHOI J， KUM D. Scale-net： scalable vehicle trajectory prediction network under random number of interacting vehicles via edge-enhanced graph convolutional neural network［C］.2020 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2020： 2095-2102.
5	CHEN Y， SCHMID C， SMINCHISESCU C. Self-supervised learning with geometric constraints in monocular video： Connecting flow， depth， and camera［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019： 7063-7072.
6	YIN Z， SHI J. Geonet： unsupervised learning of dense depth， optical flow and camera pose［C］.Proceedings of the IEEE Conference on Computer Vision and PATTERN RECOgnition. 2018： 1983-1992.
7	GARG R， BG V K， CARNEIRO G， et al. Unsupervised CNN for single view depth estimation： geometry to the rescue［C］. European Conference on Computer Vision. Springer， Cham， 2016.
8	ZHOU T， BROWN M， SNAVELY N， et al. Unsupervised learning of depth and ego-motion from video［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017： 1851-1858.
9	ZOU Y， JI P， TRAN Q H， et al. Learning monocular visual odometry via self-supervised long-term modeling［C］.Computer Vision–ECCV 2020： 16th European Conference， Glasgow， UK， August 23–28， 2020， Proceedings， Part XIV 16. Springer International Publishing， 2020： 710-727.
10	CASSER V， PIRK S， MAHJOURIAN R， et al. Depth prediction without the sensors： leveraging structure for unsupervised learning from monocular videos［C］.Proceedings of the AAAI conference on artificial intelligence. 2019， 33（1）： 8001-8008.
11	CASSER V， PIRK S， MAHJOURIAN R， et al. Unsupervised monocular depth and ego-motion learning with structure and semantics［J］. CVPR Workshops， 2019： 381-388.
12	GODARD C， AODHA O M， BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency［C］. Computer Vision & Pattern Recognition. IEEE， 2017.
13	GODARD C， MAC AODHA O， FIRMAN M， et al. Digging into self-supervised monocular depth estimation［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019： 3828-3838.
14	BHATTACHARYYA A， FRITZ M， SCHIELE B. Long-term on-board prediction of people in traffic scenes under uncertainty［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018： 4194-4202.
15	RASOULI A， KOTSERUBA I， KUNIC T， et al. PIE： a large-scale dataset and models for pedestrian intention estimation and trajectory prediction［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. IEEE， 2020.
16	YAO Y， XU M， CHOI C， et al. Egocentric vision-based future vehicle localization for intelligent driving assistance systems［C］. 2019 International Conference on Robotics and Automation （ICRA）. 2019.
17	YAO Y， ATKINS E， JOHNSON-ROBERSON M， et al. Bitrap： bi-directional pedestrian trajectory prediction with multi-modal goal estimation［J］. IEEE Robotics and Automation Letters， 2021， 6（2）： 1463-1470.
18	SHI X， CHEN Z， WANG H， et al. Convolutional LSTM network： a machine learning approach for precipitation nowcasting［J］. MIT Press， 2015.
19	GU J， BRADBURY J， XIONG C， et al. Non-autoregressive neural machine translation［C］. 2017 ICLR， 2017.
20	GAO J， SUN C， ZHAO H， et al. VectorNet： encoding HD maps and agent dynamics from vectorized representation［J］. arXiv： Computer Vision and Pattern Recognition， 2020.
21	GEIGER A， LENZ P， STILLER C， et al. Vision meets robotics： the KITTI dataset［J］. International Journal of Robotics Research， 2013， 32（11）：1231-1237.
22	ZHAO H， GALLO O， FROSIO I， et al. Loss functions for image restoration with neural networks［J］. IEEE Transactions on Computational Imaging， 2017.
23	WANG Z. Image quality assessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing， 2004.
24	KINGMA D， BA J. Adam： a method for stochastic optimization［J］. Computer Science， 2014.
25	PARK S H， KIM B D， KANG C M， et al. Sequence-to-sequence prediction of vehicle trajectory via lstm encoder-decoder architecture［J］. IEEE， 2018.
26	YU J， JIANG Y， WANG Z， et al. UnitBox： an advanced object detection network［M］. ACM， 2016.
27	RASOULI A， KOTSERUBA I， TSOTSOS J K. Are they going to cross？ a benchmark dataset and baseline for pedestrian crosswalk behavior［C］. IEEE International Conference on Computer Vision Workshop. IEEE， 2017.nternational Conference on Computer Vision Workshops. 2017.

[1]	Jianping Hao,Yanzhao Su,Zhihua Zhong,Jin Huang. Service-Oriented Architecture and Service Scheduling Mechanism for Intelligent Vehicles [J]. Automotive Engineering, 2023, 45(9): 1563-1572.
[2]	Gaoshi Zhao,Long Chen,Yingfeng Cai,Yubo Lian,Hai Wang,Qingchao Liu,Chenglong Teng. Trajectory Prediction Technology Integrating Complex Network and Memory-Augmented Network [J]. Automotive Engineering, 2023, 45(9): 1608-1616.
[3]	Jing Lian,Shuoxian Li,Yidi Liu,Dongfang Yang,Linhui Li. Goal Supervised Attention Network for Vehicle Trajectory Prediction [J]. Automotive Engineering, 2023, 45(8): 1353-1361.
[4]	Qihui Hu,Yingfeng Cai,Hai Wang,Long Chen,Zhaozhi Dong,Qingchao Liu. Heterogeneous Multi-object Trajectory Prediction Method Based on Hierarchical Graph Attention [J]. Automotive Engineering, 2023, 45(8): 1448-1456.
[5]	Shiju Pan, Jianshi Li, Hua Li, Jingtao Lou, Youchun Xu. Path Following Method of Intelligent Vehicles Based on Feedback Pure Tracking Method [J]. Automotive Engineering, 2023, 45(7): 1134-1144.
[6]	Zhenhai Gao, Mingxi Bao, Fei Gao, Minghong Tang. The Method of Probabilistic Multi-modal Expected Trajectory Prediction Based on LSTM [J]. Automotive Engineering, 2023, 45(7): 1145-1152.
[7]	Jun Li, Wei Zhou, Shuang Tang. Lane Change and Obstacle Avoidance Trajectory Planning of Intelligent Vehicle Based on Adaptive Fitting [J]. Automotive Engineering, 2023, 45(7): 1174-1183.
[8]	Yong Han,Xujie Lin,Hongwu Huang,Hongyu Cai,Jinrong Luo,Yangting Li. An Approach for Predicting Pedestrian Trajectories in Typical Car Crash Scenarios [J]. Automotive Engineering, 2023, 45(6): 1022-1030.
[9]	Zixian Li,Shiju Pan,Yuan Zhu,Binbing He,Youchun Xu. Semi-active Suspension Control for Intelligent Vehicles Based on State Feedback and Preview Feedforward [J]. Automotive Engineering, 2023, 45(5): 735-745.
[10]	Xiang Gao,Long Chen,Xinye Wang,Xiaoxia Xiong,Yicheng Li,Yuexia Chen. Intelligent Vehicle Driving Risk Assessment Method Based on Trajectory Prediction [J]. Automotive Engineering, 2023, 45(4): 588-597.
[11]	Qin Shi,Yafang Chen,Teng Cheng,Qiang Zhang,Wenchong Wang,Benyi Shi. Unsupervised Monocular Infrared Image Depth Estimation Based on Local Plane Guidance Layer [J]. Automotive Engineering, 2023, 45(12): 2291-2298.
[12]	Shiju Pan,Yongle Li,Zixian Li,Binbing He,Yuan Zhu,Youchun Xu. Path Following Method of Intelligent Vehicles Based on Improved Pure Tracking [J]. Automotive Engineering, 2023, 45(1): 1-8.
[13]	Wenbo Shao,Jun Li,Yuxin Zhang,Hong Wang. Key Technologies to Ensure the Safety of the Intended Functionality for Intelligent Vehicles [J]. Automotive Engineering, 2022, 44(9): 1289-1304.
[14]	Zewu Deng,Zhaozheng Hu,Zhe Zhou, LiuYulin,Chao Peng. Intelligent Vehicle Positioning by Fusing LiDAR and Double-layer Map Model [J]. Automotive Engineering, 2022, 44(7): 1018-1026.
[15]	Xinyu Chen,Lijun Qian,Qidong Wang. Eco-driving Control at Signalized Intersections with Consideration of Time Delay [J]. Automotive Engineering, 2022, 44(7): 960-968.

Surrounding Multi-Target Trajectory Prediction Method Based on Monocular Visual Motion Estimation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 27

Related Articles 15

Metrics

Comments

Recommended 10