基于动作条件交互的高效行人过街意图预测

doi:10.19562/j.chinasae.qcgc.2024.01.004

Abstract

Abstract:

With acceleration of the urbanization process， pedestrian-vehicle conflicts have become a significant issue that modern society urgently needs to solve. In complex traffic scenarios， pedestrian crossing behavior leads to frequent traffic accidents. Accurately and timely anticipating pedestrian crossing intentions is crucial for avoiding pedestrian-vehicle conflicts， improving driving safety， and ensuring pedestrian safety. An Efficient Action-Conditioned Interaction Pedestrian Crossing Intention Anticipation Framework （EAIPF） is proposed in this paper to anticipate pedestrian crossing intention. EAIPF introduces in a pedestrian action encoding module to enhance the representation ability of multimodal action patterns and discover deep skeletal context information. At the same time， the scene object interaction module is introduced to explore interaction information with objects and understand advanced semantic clues in traffic scenes. Finally， the intention anticipation module fuses pedestrian action and object interaction features to achieve robust anticipation of pedestrian crossing intentions. The proposed method is verified on two public datasets， JAAD and PIE， achieving the accuracy of 89% and 90%， respectively， indicating that the proposed method can accurately anticipate pedestrian crossing intentions in complex traffic scenarios.

Key words: pedestrian-vehicle conflict, crossing intention anticipation, graph convolution network, pedestrian action encoding, scene understanding

Biao Yang, Zhiwen Wei, Rongrong Ni, Hai Wang, Yingfeng Cai, Changchun Yang. Efficient Pedestrian Crossing Intention Anticipation Based on Action-Conditioned Interaction[J].Automotive Engineering, 2024, 46(1): 29-38.

Figures/Tables 11

References 34

1	CHEN B， SUN D， ZHOU J， et al. A future intelligent traffic system with mixed autonomous vehicles and human-driven vehicles［J］. Information Sciences， 2020， 529： 59-72.
2	连静，王欣然，李琳辉，等. 基于人-车交互的行人轨迹预测［J］. 中国公路学报， 2021， 34（5）： 215.
	LIAN J， WANG X R， LI L H， et al. Pedestrian trajectory prediction based on human-vehicle interaction［J］. China Journal of Highway and Transport， 2021， 34（5）： 215.
3	LI J， SU W， WANG Z. Simple pose： rethinking and improving a bottom-up approach for multi-person pose estimation［C］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020： 11354-11361.
4	LUO Z， WANG Z， HUANG Y， et al. Rethinking the heatmap regression for bottom-up human pose estimation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 13264-13273.
5	LIU J， ROJAS J， LI Y， et al. A graph attention spatio-temporal convolutional network for 3D human pose estimation in video［C］. 2021 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2021： 3374-3380.
6	LI Y， YANG S， LIU P， et al. SimCC： a simple coordinate classification perspective for human pose estimation［C］. Computer Vision-ECCV 2022： 17th European Conference， Tel Aviv， Israel， October 23-27， 2022， Proceedings， Part VI. Cham： Springer Nature Switzerland， 2022： 89-106.
7	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 5693-5703.
8	YU B， YIN H， ZHU Z. Spatio-temporal graph convolutional networks： a deep learning framework for traffic forecasting［J］. arXiv preprint arXiv：， 2017.
9	SHI L， ZHANG Y， CHENG J， et al. Two-stream adaptive graph convolutional networks for skeleton-based action recognition［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 12026-12035.
10	YE F， PU S， ZHONG Q， et al. Dynamic GCN： context-enriched topology learning for skeleton-based action recognition［C］. Proceedings of the 28th ACM International Conference on Multimedia， 2020： 55-63.
11	杨彪，范福成，杨吉成，等. 基于动作预测与环境条件的行人过街意图识别［J］. 汽车工程， 2021， 43（7）： 1066-1076.
	YANG Biao， FAN Fucheng， YANG Jicheng， et al. Recognizing pedestrians’ crossing intentions based on action prediction and environment context ［J］. Automotive Engineering， 2021， 43（7）： 1066-1076.
12	CADENA P R G， QIAN Y， WANG C， et al. Pedestrian Graph+： a fast pedestrian crossing prediction model based on graph convolutional networks［J］. IEEE Transactions on Intelligent Transportation Systems， 2022.
13	YANG D， ZHANG H， YURTSEVER E， et al. Predicting pedestrian crossing intention with feature fusion and spatio-temporal attention［J］. IEEE Transactions on Intelligent Vehicles， 2022， 7（2）： 221-230.
14	NI R， YANG B， WEI Z， et al. Pedestrians crossing intention anticipation based on dual‐channel action recognition and hierarchical environmental context［J］. IET Intelligent Transport Systems， 2023， 17（2）： 255-269.
15	RASOULI A， KOTSERUBA I， TSOTSOS J K. Are they going to cross？ a benchmark dataset and baseline for pedestrian crosswalk behavior［C］. Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017： 206-213.
16	RASOULI A， KOTSERUBA I， TSOTSOS J K. Pedestrian action anticipation using contextual feature fusion in stacked RNNs［J］. arXiv preprint arXiv：， 2020.
17	SHI J， LIU C， ISHI C T， et al. Skeleton-based emotion recognition based on two-stream self-attention enhanced spatial-temporal graph convolutional network［J］. Sensors， 2020， 21（1）： 205.
18	STERGIOU A， POPPE R. Adapool： exponential adaptive pooling for information-retaining downsampling［J］. IEEE Transactions on Image Processing， 2022， 32： 251-266.
19	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 770-778.
20	CHEN Y， ZHANG Z， YUAN C， et al. Channel-wise topology refinement graph convolution for skeleton-based action recognition［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 13359-13368.
21	GIRSHICK R. Fast R-CNN［C］. Proceedings of the IEEE International Conference on Computer Vision， 2015： 1440-1448.
22	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2017： 2117-2125.
23	RASOULI A， KOTSERUBA I， KUNIC T， et al. PIE： a large-scale dataset and models for pedestrian intention estimation and trajectory prediction［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 6262-6271.
24	ZHOU Z， FAN X， SHI P， et al. R-MSFM： recurrent multi-scale feature modulation for monocular depth estimating［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 12777-12786.
25	BOUAZIZI A， HOLZBOCK A， KRESSEL U， et al. Motionmixer： MLP-based 3D human body pose forecasting［J］. arXiv preprint arXiv：， 2022.
26	KOTSERUBA I， RASOULI A， TSOTSOS J K. Benchmark for evaluating pedestrian action prediction［C］. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， 2021： 1258-1268.
27	SHI X， CHEN Z， WANG H， et al. Convolutional LSTM network： a machine learning approach for precipitation nowcasting［J］. Advances in Neural Information Processing Systems， 2015， 28.
28	KOTSERUBA I， RASOULI A， TSOTSOS J K. Do they want to cross？ understanding pedestrian intention for behavior prediction［C］. 2020 IEEE Intelligent Vehicles Symposium （IV）. IEEE， 2020： 1688-1693.
29	BHATTACHARYYA A， FRITZ M， SCHIELE B. Long-term on-board prediction of people in traffic scenes under uncertainty［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 4194-4202.
30	NG J Y， HAUSKNECHT M， VIJAYANARASIMHAN S， et al. Beyond short snippets： deep networks for video classification［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2015： 4694-4702.
31	LIN R， LIU S， YANG M， et al. Hierarchical recurrent neural network for document modeling［C］. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing， 2015： 899-907.
32	TRAN D， BOURDEV L， FERGUS R， et al. Learning spatiotemporal features with 3D convolutional networks［C］. Proceedings of the IEEE International Conference on Computer Vision， 2015： 4489-4497.
33	LI J， WANG C， ZHU H， et al. CrowdPose： efficient crowded scenes pose estimation and a new benchmark［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 10863-10872.
34	CARREIRA J， ZISSERMAN A. Quo vadis， action recognition？ a new model and the kinetics dataset［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2017： 6299-6308.

[1]	ZHOU Wei, ZHANG Cheng-Ning, LI Jun-Qiu. [J]. , 2016, 38(12): 1407 -1414 .
[2]	LI Huan, HUANG Ying, ZHANG Fu-Jun, ZHAO Yu, GE Yan-Wu. [J]. , 2016, 38(12): 1420 -1426 .
[3]	HAO Han, WANG Si-南, LI Xiao, LIU Zong-Wei, ZHAO Fu-Quan. [J]. , 2017, 39(1): 1 -8 .
[4]	ZHANG Ying-Chao, ZHAN Da-Peng, ZHAO Jing, ZHANG Zhe, LI Jie. [J]. , 2016, 38(12): 1434 -1439 .
[5]	SHEN Zhe, WANG Yi-Gang, YANG Zhi-Gang, LI Fang-Xu. [J]. , 2016, 38(12): 1440 -1445 .
[6]	ZENG Bi-Qiang, GAO Ji-Dong, PENG Wei, SUN Zhen-Dong. [J]. , 2016, 38(12): 1446 -1451 .
[7]	ZHANG Yi-Bing, 吕Jian-Jun , YUAN Guang-Hui. [J]. , 2016, 38(12): 1467 -1470 .
[8]	LIU Hui, WANG Xiao-Jie, XIANG Chang-Le. [J]. , 2016, 38(12): 1483 -1487 .
[9]	GAO Ji-Dong, GAO Bo-Lin, XIE Shu-Gang. [J]. , 2016, 38(12): 1515 -1520 .
[10]	DONG Zhu-Rong, ZHANG Xin, HU Song-Hua, QIU Hao. [J]. , 2017, 39(1): 79 -85 .

2D 动作编码	3D 动作编码	预测模块	场景对象交互模块	Acc	AUC	F1	Pre	Rec
√				84.15	72.66	55.04	55.22	54.87
	√			84.04	74.59	57.07	54.43	59.98
√		√		86.27	82.88	66.43	58.03	77.66
	√	√		86.01	79.95	63.85	54.27	77.57
√			√	85.47	76.63	60.27	57.75	63.02
	√		√	84.61	77.23	59.96	55.01	65.88
√		√	√	88.96	84.10	78.81	85.66	72.98
	√	√	√	88.01	79.23	65.70	65.67	65.73

2D 动作编码	3D 动作编码	预测模块	场景对象交互模块	Acc	AUC	F1	Pre	Rec
√				63.76	55.48	75.12	86.76	66.18
	√			63.85	55.72	75.62	86.79	63.85
√		√		67.86	67.89	72.51	77.98	67.76
	√	√		68.05	67.74	72.93	77.49	68.97
√			√	64.15	61.77	71.31	71.41	71.21
	√		√	65.69	63.53	72.44	72.81	72.08
√		√	√	68.64	67.72	74.01	76.86	71.37
	√	√	√	69.10	65.03	76.69	72.64	81.21

2D 动作编码	3D 动作编码	预测模块	场景对象交互模块	Acc	AUC	F1	Pre	Rec
√				78.57	66.69	50.92	71.81	39.45
	√			79.80	69.60	56.34	72.07	46.24
√		√		87.75	81.26	75.31	86.98	66.41
	√	√		88.86	82.92	77.79	88.64	69.31
√			√	82.62	74.06	63.82	77.05	54.47
	√		√	84.10	78.98	70.43	73.91	67.26
√		√	√	88.93	82.59	77.58	90.16	68.08
	√	√	√	90.33	86.57	81.95	86.35	77.97

模型名称	模型变体	PIE					JAAD_beh					JAAD_all
模型名称	模型变体	Acc	AUC	F1	Pre	Rec	Acc	AUC	F1	Pre	Rec	Acc	AUC	F1	Pre	Rec
ATGC^［14］	VGG16	0.71	0.60	0.41	0.49	0.36	0.59	0.52	0.71	0.63	0.82	0.82	0.75	0.55	0.49	0.63
ATGC^［14］	ResNet50	0.70	0.59	0.38	0.47	0.32	0.46	0.45	0.54	0.58	0.51	0.81	0.72	0.52	0.47	0.56
ConvLSTM^［27］	VGG16	0.58	0.55	0.39	0.32	0.49	0.53	0.49	0.64	0.64	0.64	0.63	0.57	0.32	0.24	0.48
ConvLSTM^［27］	ResNet50	0.54	0.46	0.26	0.23	0.29	0.59	0.55	0.69	0.68	0.70	0.63	0.58	0.33	0.25	0.49
SingleRNN^［28］	GRU	0.83	0.77	0.67	0.70	0.64	0.58	0.54	0.67	0.67	0.68	0.65	0.59	0.34	0.26	0.49
SingleRNN^［28］	LSTM	0.81	0.75	0.64	0.67	0.61	0.51	0.48	0.61	0.63	0.59	0.78	0.75	0.54	0.44	0.70
MultiRNN^［29］	GRU	0.83	0.80	0.71	0.69	0.73	0.61	0.50	0.74	0.64	0.86	0.79	0.79	0.58	0.45	0.79
StackedRNN^［30］	GRU	0.82	0.78	0.67	0.67	0.68	0.60	0.60	0.66	0.73	0.61	0.79	0.79	0.58	0.46	0.79
HierarchicalRNN^［31］	GRU	0.82	0.77	0.67	0.68	0.66	0.53	0.50	0.63	0.64	0.61	0.80	0.79	0.59	0.47	0.79
SFRNN^［15］	GRU	0.82	0.79	0.69	0.67	0.70	0.51	0.45	0.63	0.61	0.64	0.84	0.84	0.65	0.54	0.84
C3D^［32］	3DConv	0.77	0.67	0.52	0.63	0.44	0.61	0.51	0.75	0.63	0.91	0.84	0.81	0.65	0.57	0.75
I3D^［33］	3DConv	0.80	0.73	0.62	0.67	0.58	0.62	0.56	0.73	0.68	0.79	0.81	0.74	0.63	0.66	0.61
I3D^［33］	Optical	0.81	0.83	0.72	0.60	0.90	0.62	0.51	0.75	0.65	0.88	0.84	0.80	0.63	0.55	0.73
PCPA^［34］	3DConv	0.86	0.91	0.78	0.69	0.89	0.50	0.47	0.59	0.61	0.58	0.70	0.85	0.51	0.36	0.87
Ped Graph+^［11］	GCN	0.89	0.90	0.81	0.83	0.79	0.70	0.70	0.76	0.77	0.75	0.86	0.88	0.65	0.58	0.75
Ours	GCN	0.90	0.87	0.82	0.86	0.78	0.69	0.65	0.77	0.73	0.81	0.89	0.84	0.79	0.86	0.73

Efficient Pedestrian Crossing Intention Anticipation Based on Action-Conditioned Interaction

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 34

Related Articles 2

Metrics

Comments

Recommended 10

[1]	Gege Cui,Lü Chao,Jinghang Li,Zheyu Zhang,Guangming Xiong,Jianwei Gong. Data-Driven Personalized Scenario Risk Map Construction for Intelligent Vehicles [J]. Automotive Engineering, 2023, 45(2): 231-242.
[2]	Bai Zhonghao, Wang Yunyu, Zhang Linwei. Driver Distraction Behavior Detection with Multi-information Fusion Based on Graph Convolution Networks [J]. Automotive Engineering, 2020, 42(8): 1027-1033.