基于改进长短时记忆网络的驾驶行为检测方法研究

doi:10.19562/j.chinasae.qcgc.2021.08.011

Abstract

Abstract:

Fatigue driving and unsafe driving behavior are the main causes of traffic accidents. With the progress of intelligent transportation technology， using deep learning algorithm to detect driving behavior has become one of the hotspots of research. On the basis of convolution neural network （CNN） and long?term memory neural network， a hybrid dual stream convolution neural network algorithm is proposed by combining attention mechanism to improve network structure. In spatial flow channel， CNN is used to extract the spatial feature values of video image and the traditional mean pooling is replaced by spatial pyramid pooling is used to replace mean pooling， with the transformation of feature map unified. In time stream channel， single shot detection algorithm is adopted to calculate two adjacent frames of optical flow images of video sequence for detecting small facial targets such as human eyes. Then the fusion and classification of image features are carried out. Finally， experiments are performed on LFW dataset and self?built dataset. The results show that with the method adopted the accuracy of face recognition and fatigue driving detection is 1.36 and 2.5 percentage points higher than other methods respectively.

Key words: safe driving, CNN, LSTM, SSD, face recognition, fatigue driving detection

Dongmei Shi,Feng Xiao. Study on Driving Behavior Detection Method Based on Improved Long and Short⁃term Memory Network[J].Automotive Engineering, 2021, 43(8): 1203-1209.

Figures/Tables 13

References 18

1	张丽霞，刘涛，潘福全，等.驾驶员因素对道路交通事故指标的影响分析［J］.中国安全科学学报，2014，24（5）：79-84.
	ZHANG L X， LIU T， PAN F Q， et al. Analysis of effects of driver factors on road traffic accident indexes［J］.China Safety Science Journal，2014，24（5）：79-84.
2	LIU L F， WU S Q， XU W M. Real⁃time fatigue driving detection based on analysis of facial landmarks ［J］.Video Engineering， 2018，42（ 12）： 27-30.
3	陈骥驰，王宏，王翘秀，等. 基于脑电信号的疲劳驾驶状态研究［J］. 汽车工程， 2018， 40（5）：515-520.
	CHEN J C， WANG H， WANG Q X， et al. A study on drowsy driving state based on EEG signals［J］. Automotive Engineering， 2018， 40（5）：515-520.
4	KURIAN N K， RISHIKESH D. Real time based driver's safeguard system by analyzing human physiological signals ［J］. International Journal of Engineering Trends and Technology， 2013， 4（1）： 41-45.
5	LI F， WANG X W， LU B L. Detection of driving fatigue based on grip force on steering wheel with wavelet transformation and support vector machine［C］. International Conference on Neural Information Processing. Springer Berlin Heidelberg， 2013.
6	GUO M， LI S. Online detection of driver fatigue based on reaction time： designing a monitoring and early⁃warning system［C］.20th COTA International Conference of Transportation Professionals， 2020.
7	TRAVIESO⁃GONZALEZ C M ， ALONSO⁃HEMANDEZ J B ， CANINO⁃RODRIGUEZ J M ， et al. Robust detection of fatigue parameters based on infrared information［J］. IEEE Access， 2021， 9：18209-18221.
8	LI Z J， LI S B， LI R J， et al. Online detection of driver fatigue using steering wheel angles for real driving conditions ［J］. Sensors， 2017， 17（3）： 495-507.
9	FENG Y， ZHAO D L. Research on driver fatigue monitoring algorithm based on deep learning ［J］. Journal of Wuhan University of Technology （Transportation Science & Engineering）， 2018，42（3）： 417-422.
10	HAO F F， LIU J， CHEN X D. A review of human behavior recognition based on deep learning［C］. 2020 International Conference on Artificial Intelligence and Education （ICAIE）， 2020.
11	巩晓倩，蒲亦非，杨智勇，等.基于有限状态自动机的人眼开度PERCLOS实现算法［J］.计算机应用研究，2014，31（1）：307-310.
	GONG X Q， PU Y F， YANG Z Y，et al. Perclos implementation computing method of eye open degree based on finite state machine［J］， Application Research of Computers，2014，31（1）：307-310.
12	MANDAI B， LI L， GANG S W， et al. Towards detection of bus driver fatigue based on robust visual analysis of eye state ［J］. IEEE Transactions on Intelligent Transportation Systems， 2017， 3（3）：1-13.
13	CHEN P. Research on driver fatigue detection strategy based on human eye state［C］. 2017 Chinese Automation Congress （CAC）. IEEE， 2017.
14	LI Y L， HU B， WANG S J， et al. Real⁃time eye locating and tracking for driver fatigue detection［J］. Applied Mechanics & Materials， 2010， 20-23：1359-1364.
17	宗长富，陈双，冯刚，等.基于频率加权滤波的汽车平顺性评价［J］.吉林大学学报（工学版）， 2011，41（6）：517-521.
	ZONG C F， CHEN S， FENG G， et al. Evaluation of vehicle ride comfort based on frequency weighted filtering ［J］. Journal of Jilin University （Engineering and Technology Edition）， 2011，41（6）：517-521.
18	兰凤崇，李诗成，陈吉清，等.考虑操纵稳定性的自动驾驶汽车轨迹综合优化方法［J］.湖南大学学报（自然科学版）， 2019（10）：36-45.
	LAN F C， LI S C， CHEN J Q， et al. Comprehensive approach for trajectory optimization of autopilot vehicles considering handling stability ［J］. Journal of Hunan University （Natural Sciences）， 2019（10）：36-45.

疲劳状态	PERCLOS值	E_open	FE_open
正常	≤0.15	≥0.8	≤0.2
轻度	0.15~0.4	0.65~0.8	0.2~0.3
中度	0.4~0.6	0.45~0.65	0.3~0.5
严重	≥0.6	≤0.45	≥0.5

方法	训练时间/s	参数/MB	FLOPS	准确率/%
CNN	24.3	1.185	2.112	78.52
LRCN	27.7	1.377	2.101	80.16
SSD	26.1	1.265	2.057	81.23
LSTM	28.8	1.436	1.995	81.92
本文	30.2	1.581	1.863	83.28

视频编号	FE_open	识别状态	实际状态	测试样本数	准确率/%
S16	0.35	轻度疲劳	轻度疲劳	763	78.38
S17	0.75	重度疲劳	重度疲劳	755	77.38
S18	0.31	轻度疲劳	轻度疲劳	762	78.25
S19	0.42	中度疲劳	中度疲劳	720	74.00
S20	0.12	正常	正常	726	74.75
S21	0.87	重度疲劳	重度疲劳	730	75.25

方法	网络架构	准确率%
方法	网络架构	疲劳	用手机	吃东西	扭头	说话
CNN	VGG16	60.56	90.32	96.65	98.88	91.55
LRCN	VGG16	66.78	94.53	97.55	98.92	92.11
SSD	Inception Net	68.75	93.82	97.35	99.05	94.26
LSTM	ResNet50	70.78	95.76	97.93	98.82	93.51
本文	VGG16+Inception Net	73.36	96.45	98.22	99.31	95.15

[1]	Weiguo Liu,Zhiyu Xiang,Rui Liu,Guodong Li,Zixu Wang. Research on End-to-End Vehicle Motion Planning Method Based on Deep Learning [J]. Automotive Engineering, 2023, 45(8): 1343-1352.
[2]	Zhenhai Gao, Mingxi Bao, Fei Gao, Minghong Tang. The Method of Probabilistic Multi-modal Expected Trajectory Prediction Based on LSTM [J]. Automotive Engineering, 2023, 45(7): 1145-1152.
[3]	Yong Han,Xujie Lin,Hongwu Huang,Hongyu Cai,Jinrong Luo,Yangting Li. An Approach for Predicting Pedestrian Trajectories in Typical Car Crash Scenarios [J]. Automotive Engineering, 2023, 45(6): 1022-1030.
[4]	Guihong Bi,Xu Xie,Zilong Cai,Zhao Luo,Chenpeng Chen,Xin Zhao. Capacity Estimation of Lithium-ion Battery Based on Deep Learning Under Dynamic Conditions [J]. Automotive Engineering, 2022, 44(6): 868-878.
[5]	Jinghua Guo,Zhifei He,Yugong Luo,Keqiang Li. Vehicle Cut-in Trajectory Prediction Based on Deep Learning in a Human-machine Mixed Driving Environment [J]. Automotive Engineering, 2022, 44(2): 153-160.
[6]	Guanqun Liang,Tong Zhao,Yan Wang,Yintao Wei. Road Unevenness Identification Based on LSTM Network [J]. Automotive Engineering, 2021, 43(4): 509-517.
[7]	Yanqiu Xiao,Kun Zhou,Guangzhen Cui,Zhanpeng Fang,Qidi Liu Yanqi Sun. Joint Detection of Pedestrian and Rider Based on Diversity Region Prediction [J]. Automotive Engineering, 2021, 43(1): 77-85.
[8]	Li Da, Zhang Zhaosheng, Liu Peng, Wang Zhenpo, Dong Haotian. Vehicle Classification Method in Multi-climates Based on Modified LSTM-AdaBoost Algorithm [J]. Automotive Engineering, 2020, 42(9): 1248-1255.
[9]	Wang Hai, Wang Kuan, Cai Yingfeng, Liu Ze, Chen Long. Traffic Sign Recognition Based on Improved Cascade Convolution Neural Network [J]. Automotive Engineering, 2020, 42(9): 1256-1262.
[10]	Feng Tianpei, Sun Yuedong, Wang Yansong, Zhang Boqiang, Liu Ningning, Guo Hui. Annoyance Evaluation Model of Vehicle Interior Noise Based onTime-series Smoothed Excitation Level Spectrum CNN Model [J]. Automotive Engineering, 2020, 42(6): 784-792.

Study on Driving Behavior Detection Method Based on Improved Long and Short⁃term Memory Network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 18

Related Articles 10

Metrics

Comments

Recommended 10