基于动作预测与环境条件的行人过街意图识别

doi:10.19562/j.chinasae.qcgc.2021.07.014

汽车工程 ›› 2021, Vol. 43 ›› Issue (7): 1066-1076.doi: 10.19562/j.chinasae.qcgc.2021.07.014

基于动作预测与环境条件的行人过街意图识别

杨彪¹,范福成²,杨吉成²,蔡英凤³(),王海⁴

^1.常州大学微电子与控制工程学院，常州 213016
^2.常州大学计算机与人工智能学院，常州 213016
^3.江苏大学汽车工程研究院，镇江 212013
^4.江苏大学汽车与交通工程学院，镇江 212013

收稿日期:2021-01-14 修回日期:2021-03-03 出版日期:2021-07-25 发布日期:2021-07-20
通讯作者: 蔡英凤 E-mail:caicaixiao0304@126.com
基金资助:
常州市应用基础研究项目(CJ20200083)

Recognition of Pedestrians’ Street⁃crossing Intentions Based on Action Prediction and Environment Context

Biao Yang¹,Fucheng Fan²,Jicheng Yang²,Yingfeng Cai³(),Hai Wang⁴

^1.School of Microelectronics and Control Engineering，Changzhou University，Changzhou 213016
^2.School of Computer Science and Artificial Intelligence，Changzhou University，Changzhou 213016
^3.Institude of Automotive Engineering，Jiangsu University，Zhenjiang 212013
^4.Institude of Automotive and Transportation Engineering，Jiangsu University，Zhenjiang 212013

Received:2021-01-14 Revised:2021-03-03 Online:2021-07-25 Published:2021-07-20
Contact: Yingfeng Cai E-mail:caicaixiao0304@126.com

摘要/Abstract

摘要：

考虑到人车冲突多发于行人过街过程，本文中提出了一种基于行人动作预测与环境条件的过街意图识别网络MIFRN，它通过结构各异的子网络分别对行人的未来动作信息、行人周围的局部交通场景、车速和人车距离信息进行编码，并在信息融合的基础上预测行人是否有过街意图。最后在公共数据集PIE和JAAD上验证了算法的性能。结果表明：本文提出的方法可准确并鲁棒地识别行人的过街意图。

关键词: 智能网联车, 行人意图识别, 动作预测, 环境条件

Abstract:

In view of that pedestrian?vehicle collisions often happen in the process of pedestrians’ street crossing， a street?crossing intention recognition network MIFRN is proposed based on pedestrians’ action prediction and environment contexts in this paper. MIFRN encodes pedestrians’ future actions information， local traffic scenes surrounding pedestrians， vehicle speeds， and pedestrian?vehicle distance information respectively through structure?varying sub?networks， and predicts pedestrians’ intention of street crossing on the basis of information fusion. Finally， the performance of algorithm is verified based on two public databases PIE and JAAD. The results indicate that the method proposed can recognize pedestrians’ street?crossing intentions accurately and robustly.

Key words: intelligent and connected vehicles, pedestrian crossing intention, action prediction, environment context

杨彪,范福成,杨吉成,蔡英凤,王海. 基于动作预测与环境条件的行人过街意图识别[J]. 汽车工程, 2021, 43(7): 1066-1076.

Biao Yang,Fucheng Fan,Jicheng Yang,Yingfeng Cai,Hai Wang. Recognition of Pedestrians’ Street⁃crossing Intentions Based on Action Prediction and Environment Context[J]. Automotive Engineering, 2021, 43(7): 1066-1076.

图/表 5

参考文献 36

1	世界卫生组织. 道路安全全球现状报告2018 ［EB/OL］. .
	World Healthy Organization. Global status report on road safety 2018 ［EB/OL］. .
2	NHTSA. Pedestrian safety ［EB/OL］. .
3	GSUL M，胡予红，周旋，等. 道路交通运输安全发展报告（2017）［J］. 中国应急管理， 2018（2）： 48-58.
	GSUL M， HU Y， ZHOU X， et al. Road traffic and transport safety development report （2017）［J］. Chinese Emergency Management， 2018（2）： 48-58.
4	RAJAMANI R. Vehicle dynamics and control［M］. Springer⁃Verlag， New York， USA. 2006.
5	LIN C F， ULSOY A G. Vehicle dynamics and externaldisturbance estimation for future vehicle path prediction［J］. IEEE Transactions on Control Systems Technology， 2000， 8（3）： 508-518.
6	BRAND M， OLIVER N， PENTLAND A. Coupled hidden Markov models for complex action recognition［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 1997：994-999.
7	娄新雨，王海，蔡英凤，等. 采用64线激光雷达的实时道路障碍物检测与分类算法的研究［J］.汽车工程，2019，41（8）：779-784.
	LOU X， WANG H， CAI Y， et al. A research on an algorithm for real⁃time detection and classification of road obstacle by using 64-line lidar ［J］. Automotive Engineering， 2019， 41（8）： 779-784.
8	SONG X， ZHAN W， CHE X， et al. Scale⁃aware attention⁃based PillarsNet （SAPN） based 3D object detection for point cloud［J］. Mathematical Problems in Engineering， 2020， 2020.
9	种衍文，匡湖林，李清泉，等. 一种基于多特征和机器学习的分级行人检测方法［J］. 自动化学报， 2012， 38（3）：375-381.
	CHONG Y， KUANG H， LI Q，et al. Two⁃stage pedestrian detection based on multiple features and machine learning ［J］. Acta Automatica Sinica， 2012， 38（3）： 375-381.
10	TIAN Y， LUO P， WANG X， et al. Deep learning strong parts for pedestrian detection［C］.Proceedings of the IEEE International Conference on Computer Vision.2015： 1904-1912.
11	MAO J， XIAO T， JIANG Y， et al. What can help pedestrian detection？［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017：3127-3136.
12	刘国辉，张伟伟，吴训成，等. 基于VGG⁃M网络模型的前方车辆跟踪［J］. 汽车工程， 2019， 41（1）：61-67.
	LIU G， ZHANG W， WU X，et al. Front vehicle tracking based on VGG⁃M network model ［J］. Automotive Engineering， 2019， 41（1）： 61-67.
13	KALANTAROV S， RIEMER R， ORON⁃GILAD T. Pedestrians’ road crossing decisions and body parts’ movements［J］. Transportation research part F： Traffic Psychology and Behaviour， 2018， 53： 155-171.
14	KELLER C G， GAVRILA D M. Will the pedestrian cross？a study on pedestrian path prediction［J］. IEEE Transactions on Intelligent Transportation Systems， 2013，15（2）： 494-506.
15	GU Y， HASHIMOTO Y， HSU L T， et al. Human⁃like motion planning model for driving in signalized intersections［J］. IATSS Research， 2017， 41（3）： 129-139.
16	LEE N， CHOI W， VERNAZA P， et al. Desire： distant future prediction in dynamic scenes with interacting agents［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017： 336-345.
17	SHEN M， HABIBI G， HOW J P. Transferable pedestrian motion prediction models at intersections［C］.2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2018： 4547-4553.
18	CAO Z， SIMON T， WEI S E， et al. Realtime multi⁃person 2D pose estimation using part affinity fields［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017： 7291-7299.
19	FANG Z， LÓPEZ A M. Is the pedestrian going to cross？ answering by 2D pose estimation［C］.2018 IEEE Intelligent Vehicles Symposium （IV）. IEEE， 2018： 1271-1276.
20	QUINTERO R， PARRA I， LORENZO J， et al. Pedestrian intention recognition by means of a hidden markov model and body language［C］.2017 IEEE 20th International Conference on Intelligent Transportation Systems（ITSC）. IEEE， 2017： 1-7.
21	KINGMA D P， WELLING M. Stochastic gradient VB and the variational auto⁃encoder［C］.Second International Conference on Learning Representations， ICLR. 2014， 19.
22	NOH H， HONG S， HAN B. Learning deconvolution network for semantic segmentation［C］.Proceedings of the IEEE International Conference on Computer Vision.2015： 1520-1528.
23	KOHL S， ROMERA⁃PAREDES B， MEYER C， et al. A probabilistic U⁃net for segmentation of ambiguous images［C］.Advances in Neural Information Processing Systems， 2018： 6965-6975.
24	BADRINARAYANAN V， KENDALL A， CIPOLLA R. Segnet： a deep convolutional encoder⁃decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495.
25	PASZKE A， CHAURASIA A， KIM S， et al. Enet： a deepneural network architecture for real⁃time semantic segmentation［J］. arXiv Preprint arXiv：， 2016.
26	CHUNG J， GULCEHRE C， CHO K， et al. Empirical evaluation of gated recurrent neural networks on sequen⁃cemodeling［C］.NIPS 2014 Workshop on Deep Learning，December2014. 2014.
27	MEDSKER L R， JAIN L C. Recurrent neural networks［J］. Design and Applications， 2001， 5.
28	HOCHREITER S， SCHMIDHUBER J. Long short⁃term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780.
29	BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： optimal speed and accuracy of object detection［J］. arXiv Preprint arXiv：， 2020.
30	YAMAGUCHI K， KATO T， NINOMIYA Y.Vehicle egomotion estimation and moving object detection using a monocular camera［C］. 18th International Conference on Pattern Recognition （ICPR’06）， vol. 4. IEEE， 2006：610-613.
31	RASOULI A， KOTSERUBA I， TSOTSOS J K. Agreeing to cross： how drivers and pedestrians communicate［C］.2017 IEEE Intelligent Vehicles Symposium （IV）. IEEE，2017： 264-269.
32	RASOULI A， KOTSERUBA I， KUNIC T， et al. PIE： a large⁃scale dataset and models for pedestrian intention estimation and trajectory prediction［C］.Proceedings of the IEEE International Conference on Computer Vision. 2019： 6262-6271.
33	RASOULI A， KOTSERUBA I， TSOTSOS J K.Are they going to cross？ a benchmark dataset and baseline for pedestrian crosswalk behavior［C］. Proceedings of the IEEE International Conference on Computer Vision， 2017： 206-213.
34	KATAOKA H，SATOH Y， AOKI Y， et al.Temporal and fifine⁃grained pedestrian action recognition on driving recorder database［J］. Sensors， 2018，18（2）： 627.
35	CADENA P R G， YANG M， QIAN Y， et al.Pedestrian graph： pedestrian crossing prediction based on 2D pose estimation and graph convolutional networks［C］. International Conference on Intelligent Transportation Systems （ITSC）. IEEE， 2019： 2000-2005.
36	LIU B， ADELI E， CAO Z， et al.Spatiotemporal relationship reasoning for pedestrian intent prediction［C］.2020 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2020： 3485-3492.

[1]	田洪清,丁峰,郑讯佳,黄荷叶,王建强. 基于势能场虚拟力的智能网联车辆运动规划[J]. 汽车工程, 2021, 43(4): 518-526.
[2]	高志军,王江锋,陈磊,董佳宽,罗冬宇,闫学东. 基于动态距离窗的交叉口CAV轨迹规划算法[J]. 汽车工程, 2021, 43(4): 537-545.

基于动作预测与环境条件的行人过街意图识别

Recognition of Pedestrians’ Street⁃crossing Intentions Based on Action Prediction and Environment Context

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 36

相关文章 2

Metrics

本文评价

推荐阅读 10