基于单目视觉运动估计的周边多目标轨迹预测方法

doi:10.19562/j.chinasae.qcgc.2022.09.003

摘要/Abstract

摘要：

本文中基于低成本单目视觉感知系统，提出一种考虑自车运动影响的周边多目标轨迹预测方法。首先，建立了由深度估计网络和位姿估计网络构成的自车运动估计模型，实现图像序列中自车视觉里程计的有效计算；其次，利用自车历史位姿序列构建了预测模型，在自车相机当前视角下完成周边目标历史位置的归一化处理；最后，基于目标历史轨迹信息和区域图像信息构建了预测网络，实现智能汽车周边多目标运动轨迹的有效预测。本文的创新点是将视觉SLAM方法与轨迹预测模型相结合，提出了新的自车运动估计模型和基于ConvLSTM的多目标轨迹预测网络。所提模型克服了现有研究因忽视自车运动状态对周边目标轨迹预测精度带来的不利影响，并在仅使用单目视觉的感知条件下达到了较好的预测效果。公开数据集的测试表明，所提模型在1.5 s预测轨迹中心像素点的平均均方误差为321，比现有基线模型降低了52%，在对未来较长时步轨迹预测方面也具有优异表现。

关键词: 智能汽车, 轨迹预测, 单目视觉, 深度估计

Abstract:

Based on a low-cost monocular visual perception system， a multi-target trajectory prediction method considering the influence of vehicle motion is proposed in this paper. Firstly， the ego vehicle motion estimation model composed of the depth estimation network and the position and orientation estimation network is established to achieve effective calculation of ego vehicle visual odometer in image sequence. Then， a prediction model is built by using the historical position and orientation sequences of ego vehicle， and a normalization processing on the historical positions of surrounding targets is fulfilled under the current perspective of ego vehicle camera. Finally， the prediction network is constructed based on the historical trajectory information and regional image information to realize the effective prediction of surrounding multi-target trajectories around intelligent vehicles. The innovation points of this paper are combining visual SLAM method with trajectory predictive model and putting forward the new motion estimation model and ConvLSTM-based multi-target trajectory predictive network. The model proposed overcomes the adverse influence of existing research on the trajectory prediction accuracy of the surrounding target caused by ignoring the ego vehicle motion state， and achieves a better prediction results under the condition of using monocular vision perception only. The results of test on public data sets show that with a prediction time step of 1.5s， the model proposed has a MSEcenter of 321， i.e. a 52% lower than that of the existing baseline model， with an excellent performance also in the long-time-step trajectory prediction in the future.

Key words: intelligent vehicles, trajectory prediction, monocular vision, depth estimation

汪梓豪,蔡英凤,王海,陈龙,熊晓夏. 基于单目视觉运动估计的周边多目标轨迹预测方法[J]. 汽车工程, 2022, 44(9): 1318-1326.

Zihao Wang,Yingfeng Cai,Hai Wang,Long Chen,Xiaoxia Xiong. Surrounding Multi-Target Trajectory Prediction Method Based on Monocular Visual Motion Estimation[J]. Automotive Engineering, 2022, 44(9): 1318-1326.

图/表 8

图1

图2

图3

图4

表1

所提模型在KITTI Tracking数据集的对比测试"

模型

M S E b b o x

M S E c e n t e r

M S E f u l l

IoU

Linear

847/2 861/4 982

767/2 282/5 680

29 244

0.48/0.24/0.23

Seq2Seq

313/653/1 187

160/333/669

4 363

0.60/0.54/0.43

本文

方法

196/423/838

43/103/321

1 553

0.75/0.65/0.53

表1

表2

几种模型在JAAD数据集上测试结果的对比"

模型	$M S E b b o x$	$M S E c e n t e r$ （1.5 s）	$M S E f u l l$
Linear^［15］	233/857/2 303	1 565	6 111
LSTM^［15］	289/569/1 558	1 474	5 766
PIETraj^［15］	110/399/1 248	1 183	4 780
BiTraP-D^［17］	93/378/1 206	1 105	4 565
本方法（KITTI）	196/423/838	321	1 553

表2

表3

所提模型在不同训练阶段的指标对比"

训练阶段	$M S E b b o x$	$M S E c e n t e r$	$M S E f u l l$	IoU
一阶段	288/726/1 556	135/406/1 039	5 489	0.63/0.50/0.39
二阶段	226/532/1 040	72/203/523	2 784	0.67/0.53/0.43
完整训练	196/423/838	43/103/321	1 553	0.75/0.65/0.53

表3

图5

参考文献 27

1	DEO N， TRIVEDI M M. Convolutional social pooling for vehicle trajectory prediction［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018： 1468-1476.
2	MESSAOUD K， YAHIAOUI I， VERROUST-BLONDET A， et al. Non-local social pooling for vehicle trajectory prediction［C］.2019 IEEE Intelligent Vehicles Symposium （IV）. IEEE， 2019： 975-980.
3	ZHAO T， XU Y， MONFORT M， et al. Multi-agent tensor fusion for contextual trajectory prediction［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019： 12126-12134.
4	JEON H， CHOI J， KUM D. Scale-net： scalable vehicle trajectory prediction network under random number of interacting vehicles via edge-enhanced graph convolutional neural network［C］.2020 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2020： 2095-2102.
5	CHEN Y， SCHMID C， SMINCHISESCU C. Self-supervised learning with geometric constraints in monocular video： Connecting flow， depth， and camera［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019： 7063-7072.
6	YIN Z， SHI J. Geonet： unsupervised learning of dense depth， optical flow and camera pose［C］.Proceedings of the IEEE Conference on Computer Vision and PATTERN RECOgnition. 2018： 1983-1992.
7	GARG R， BG V K， CARNEIRO G， et al. Unsupervised CNN for single view depth estimation： geometry to the rescue［C］. European Conference on Computer Vision. Springer， Cham， 2016.
8	ZHOU T， BROWN M， SNAVELY N， et al. Unsupervised learning of depth and ego-motion from video［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017： 1851-1858.
9	ZOU Y， JI P， TRAN Q H， et al. Learning monocular visual odometry via self-supervised long-term modeling［C］.Computer Vision–ECCV 2020： 16th European Conference， Glasgow， UK， August 23–28， 2020， Proceedings， Part XIV 16. Springer International Publishing， 2020： 710-727.
10	CASSER V， PIRK S， MAHJOURIAN R， et al. Depth prediction without the sensors： leveraging structure for unsupervised learning from monocular videos［C］.Proceedings of the AAAI conference on artificial intelligence. 2019， 33（1）： 8001-8008.
11	CASSER V， PIRK S， MAHJOURIAN R， et al. Unsupervised monocular depth and ego-motion learning with structure and semantics［J］. CVPR Workshops， 2019： 381-388.
12	GODARD C， AODHA O M， BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency［C］. Computer Vision & Pattern Recognition. IEEE， 2017.
13	GODARD C， MAC AODHA O， FIRMAN M， et al. Digging into self-supervised monocular depth estimation［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019： 3828-3838.
14	BHATTACHARYYA A， FRITZ M， SCHIELE B. Long-term on-board prediction of people in traffic scenes under uncertainty［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018： 4194-4202.
15	RASOULI A， KOTSERUBA I， KUNIC T， et al. PIE： a large-scale dataset and models for pedestrian intention estimation and trajectory prediction［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. IEEE， 2020.
16	YAO Y， XU M， CHOI C， et al. Egocentric vision-based future vehicle localization for intelligent driving assistance systems［C］. 2019 International Conference on Robotics and Automation （ICRA）. 2019.
17	YAO Y， ATKINS E， JOHNSON-ROBERSON M， et al. Bitrap： bi-directional pedestrian trajectory prediction with multi-modal goal estimation［J］. IEEE Robotics and Automation Letters， 2021， 6（2）： 1463-1470.
18	SHI X， CHEN Z， WANG H， et al. Convolutional LSTM network： a machine learning approach for precipitation nowcasting［J］. MIT Press， 2015.
19	GU J， BRADBURY J， XIONG C， et al. Non-autoregressive neural machine translation［C］. 2017 ICLR， 2017.
20	GAO J， SUN C， ZHAO H， et al. VectorNet： encoding HD maps and agent dynamics from vectorized representation［J］. arXiv： Computer Vision and Pattern Recognition， 2020.
21	GEIGER A， LENZ P， STILLER C， et al. Vision meets robotics： the KITTI dataset［J］. International Journal of Robotics Research， 2013， 32（11）：1231-1237.
22	ZHAO H， GALLO O， FROSIO I， et al. Loss functions for image restoration with neural networks［J］. IEEE Transactions on Computational Imaging， 2017.
23	WANG Z. Image quality assessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing， 2004.
24	KINGMA D， BA J. Adam： a method for stochastic optimization［J］. Computer Science， 2014.
25	PARK S H， KIM B D， KANG C M， et al. Sequence-to-sequence prediction of vehicle trajectory via lstm encoder-decoder architecture［J］. IEEE， 2018.
26	YU J， JIANG Y， WANG Z， et al. UnitBox： an advanced object detection network［M］. ACM， 2016.
27	RASOULI A， KOTSERUBA I， TSOTSOS J K. Are they going to cross？ a benchmark dataset and baseline for pedestrian crosswalk behavior［C］. IEEE International Conference on Computer Vision Workshop. IEEE， 2017.nternational Conference on Computer Vision Workshops. 2017.

[1]	郝建平,苏炎召,钟志华,黄晋. 面向智能汽车的SOA架构及服务调度机制研究[J]. 汽车工程, 2023, 45(9): 1563-1572.
[2]	赵高士,陈龙,蔡英凤,廉玉波,王海,刘擎超,滕成龙. 融合复杂网络和记忆增强网络的轨迹预测技术[J]. 汽车工程, 2023, 45(9): 1608-1616.
[3]	连静,李硕贤,刘一荻,杨东方,李琳辉. 基于车道目标引导的车辆轨迹预测[J]. 汽车工程, 2023, 45(8): 1353-1361.
[4]	胡启慧,蔡英凤,王海,陈龙,董钊志,刘擎超. 基于层次图注意的异构多目标轨迹预测方法[J]. 汽车工程, 2023, 45(8): 1448-1456.
[5]	高镇海, 鲍明喜, 高菲, 唐明弘. 基于LSTM概率多模态预期轨迹预测方法[J]. 汽车工程, 2023, 45(7): 1145-1152.
[6]	韩勇,林旭洁,黄红武,蔡鸿瑜,罗金镕,李燕婷. 典型汽车碰撞事故场景中行人运动轨迹预测方法[J]. 汽车工程, 2023, 45(6): 1022-1030.
[7]	高翔,陈龙,王歆叶,熊晓夏,李祎承,陈月霞. 基于轨迹预测的智能汽车行驶风险评估方法[J]. 汽车工程, 2023, 45(4): 588-597.
[8]	胡杰,朱琪,陈锐鹏,张敏超,张志豪,刘昊岩. 引入必经点约束的智能汽车全局路径规划[J]. 汽车工程, 2023, 45(3): 350-360.
[9]	张紫微,郑玲,李以农,乔旭强,郑浩,王戡. 考虑前车运动不确定性的多目标自适应巡航控制[J]. 汽车工程, 2023, 45(3): 361-371.
[10]	石琴,陈雅芳,程腾,张强,王文冲,石本义. 基于局部平面引导层的无监督单目红外图像深度估计[J]. 汽车工程, 2023, 45(12): 2291-2298.
[11]	刘正发,吴亚,刘佩根,顾荣琦,陈广. 基于特征和标签联合分布匹配的智能驾驶跨域自适应目标检测[J]. 汽车工程, 2023, 45(11): 2082-2091.
[12]	赵树廉,来飞,李克强,陈涛,孟璋劼,唐逸超,吴思宇,田浩东. 基于数字孪生技术的智能汽车测试方法研究[J]. 汽车工程, 2023, 45(1): 42-51.
[13]	邵文博,李骏,张玉新,王红. 智能汽车预期功能安全保障关键技术[J]. 汽车工程, 2022, 44(9): 1289-1304.
[14]	梁旺,秦兆博,陈亮,边有钢,胡满江. 基于改进BP神经网络的智能车纵向控制方法[J]. 汽车工程, 2022, 44(8): 1162-1172.
[15]	赵健,宋东鉴,朱冰,吴杭哲,韩嘉懿,刘宇翔. 数据机理混合驱动的交通车意图识别方法[J]. 汽车工程, 2022, 44(7): 997-1008.