基于图像识别与动力学融合的路面附着系数估计方法

doi:10.19562/j.chinasae.qcgc.2023.07.013

Abstract

Abstract:

Accurate estimation of tire-road friction is a prerequisite for vehicle active safety control. Firstly， a single-wheel dynamics model is established， and precise estimation of the longitudinal tire force is realized using the Kalman filter. Then a particle filter （PF）-based tire-road friction estimator is developed based on the Magic Formula tire model. Secondly， a forward road adhesion coefficient prediction method based on image recognition is proposed. The DeeplabV3+， semantic segmentation network and the MobilNetV2 lightweight convolutional neural network are used for road segmentation and classification， based on which the tire-road friction is obtained through table look-up. Finally， the spatiotemporal synchronization method and fusion mechanism of dynamics and image recognition are established to realize effective correlation and reliable fusion of the two estimation methods. The Carsim-Simulink co-simulation shows that the proposed estimation method based on image recognition and dynamics fusion can efficiently improve the tire-road friction estimation accuracy.

Key words: tire-road friction, particle filter, image recognition, semantic segmentation, convolutional neural network

Lei Zhang, Keren Guan, Xiaolin Ding, Pengyu Guo, Zhenpo Wang, Fengchun Sun. Tire-Road Friction Estimation Method Based on Image Recognition and Dynamics Fusion[J].Automotive Engineering, 2023, 45(7): 1222-1234.

Figures/Tables 19

References 46

1	KHALEGHIAN S， EMAMI A， TAHERI S. A technical survey on tire-road friction estimation［J］. Friction， 2017， 5（2）： 123-146.
2	王震坡，丁晓林，张雷. 四轮轮毂电机驱动电动汽车驱动防滑控制关键技术综述［J］. 机械工程学报， 2019， 55： 99-120.
	WANG Z P， DING X L， ZHANG L. Overview on key technologies of Acceleration slip regulation for four-wheel-independently-actuated electric vehicles［J］. Journal of Mechanical Engineering， 2019， 55： 99-120.
3	SWART P L， LACQUET B M， BLOM C. An acoustic sensor system for determination of macroscopic surface roughness［J］. IEEE Transactions on Instrumentation and Measurement， 1996， 45（5）： 879-884.
4	CASSELGREN J， ROSENDAHL S， SJÖDAHL M， et al. Road condition analysis using NIR illumination and compensating for surrounding light［J］. Optics and Lasers in Engineering， 2016， 77： 175-182.
5	YANG H J， JANG H， JEONG D S. Detection algorithm for road surface condition using wavelet packet transform and SVM［C］. The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision. IEEE， 2013： 323-326.
6	ZHAO J， WU H， CHEN L. Road surface state recognition based on svm optimization and image segmentation processing［J］. Journal of Advanced Transportation， 2017， 2017： 1-21.
7	王海，蔡柏湘，蔡英凤，等. 基于语义分割网络的路面积水与湿滑区域检测［J］. 汽车工程， 2021， 43（4）： 485-491.
	WANG H， CAI B X， CAI Y F， et al. Detection of water⁃covered and wet areas on road pavement based on semantic segmentation network［J］. Automotive Engineering， 2021， 43（4）： 485-491.
8	ŠABANOVIČ E， ŽURAULIS V， PRENTKOVSKIS O， et al. Identification of road‐surface type using deep neural networks for friction coefficient estimation［J］. Sensors， 2020， 20（3）： 612-628.
9	NOLTE M， KISTER N， MAURER M. Assessment of deep convolutional neural networks for road surface classification［C］. 2018 21st International Conference on Intelligent Transportation Systems. IEEE， 2018： 381-386.
10	RENUKA D S M， SUDEEPINI D. Road surface detection using FMCW 77GHz automotive RADAR using MFCC［C］. 2021 6th International Conference on Inventive Computation Technologies. IEEE， 2021： 794-799.
11	余卓平，曾德全，熊璐，等. 基于激光雷达的无人车路面附着系数估计［J］. 华中科技大学学报， 2019， 47（7）： 124-127.
	YU Z P， ZENG D Q， XIONG L， et al. Road adhesion coefficient estimation for unmanned vehicle based on lidar［J］. J．Huazhong Univ．of Sci．& Tech， 2019， 47（7）： 124-127.
12	HOLZMANN F， BELLINO M， SIEGWART R， et al. Predictive estimation of the road-tire friction coefficient［C］. 2006 IEEE Conference on Computer Aided Control System Design， 2006 IEEE International Conference on Control Applications， 2006 IEEE International Symposium on Intelligent Control. IEEE， 2006： 885-890.
13	ERDOGAN G， ALEXANDER L， RAJAMANI R. Estimation of tire-road friction coefficient using a novel wireless piezoelectric tire sensor［J］. IEEE Sensors Journal， 2011， 11（2）： 267-279.
14	卢俊辉，巫世晶. 基于车轮振动的路面实时识别研究［J］. 振动与冲击， 2008（4）： 19-22.
	LU J H， WU S J. Real-time pavement recognition based on wheel vibration［J］. Journal of Vibration and Shock， 2008（4）： 19-22.
15	GUSTAFSSON F. Slip-based tire-road friction estimation［J］. Automatica， 1997， 33（6）： 1087-1099.
16	ONO E， ASANO K， SUGAI M， et al. Estimation of automotive tire force characteristics using wheel velocity［J］. Control Engineering Practice， 2003， 11（12）： 1361-1370.
17	袁朝春，张龙飞，陈龙，等. 路面峰值附着系数辨识算法研究［J］. 汽车工程， 2017， 39（11）： 1268-1273.
	YUAN C C， ZHANG L F， CHEN L， et al. A research on the algorithm for identifying the peak adhesion coefficient of road surface［J］. Automotive Engineering， 2017， 39（11）： 1268-1273.
18	高博麟，陈慧，谢书港，等. 分布式电驱动车车速及路面附着系数融合估计［J］. 汽车工程， 2016， 38（2）： 216-220.
	GAO B L， CHEN H， XIE S G， et al. Velocity and road friction coefficient fusion estimation of distributed electric drive vehicle［J］. Automotive Engineering， 2016， 38（2）： 216-220.
19	CHENG Q， CORREA-VICTORINO A， CHARARA A. A new nonlinear observer using unscented Kalman filter to estimate sideslip angle， lateral tire road forces and tire road friction coefficient［C］. 2011 IEEE Intelligent Vehicles Symposium， Proceedings. IEEE， 2011： 709-714.
20	李刚，解瑞春，卫绍元，等. 基于双容积卡尔曼滤波的车辆状态与路面附着系数估计［J］. 中国科学：技术科学， 2015， 45（4）： 403-414.
	LI G， XIE R C， WEI S Y， et al. Estimation of vehicle state and road tire friction coefficient based on dual cubature Kalman filter［J］. Sci Sin Tech， 2015， 45（4）： 403-414.
21	熊璐，金达，冷搏，等. 考虑复杂激励条件的分布式驱动电动汽车路面附着系数自适应估计方法［J］. 机械工程学报， 2020， 56（18）： 123-133.
	XIONG L， JIN D， LENG B， et al. Adaptive tire-road friction estimation method for distributed drive electric vehicles considering multiple road excitations［J］. Journal of Mechanical Engineering， 2020， 56（18）： 123-133.
22	王世峰，都凯悦，孟颖，等. 基于机器学习的车辆路面类型识别技术研究［J］. 兵工学报， 2017， 38（8）： 1642-2648.
	WANG S F， DU K Y， MENG Y， et al. Machine learning-based road terrain recognition for land vehicles［J］. Acta Armamentarii， 2017， 38（8）： 1642-2648.
23	LENG B， JIN D， XIONG L， et al. Estimation of tire-road peak adhesion coefficient for intelligent electric vehicles based on camera and tire dynamics information fusion［J］. Mechanical Systems and Signal Processing， 2021， 150： 1-15.
24	熊璐，金达，冷搏，等. 视觉辅助的分布式驱动电动汽车路面估计方法［J］. 同济大学学报， 2019， 47（S1）： 99-103.
	XIONG L， JIN D， LENG B， et al. Road friction estimation method for distributed driving electric vehicle based on machine vision assistance［J］. Journal of Tongji University， 2019， 47（S1）： 99-103.
25	庹文坤. 多源信息融合的路面附着系数估计算法研究［D］. 石家庄：石家庄铁道大学， 2021.
	TUO W K. Research on algorithm of road-tire adhesion coefficient estimation based on multi-source information fusion［D］. Shijiazhuang ：Shijiazhuang Tiedao University， 2021.
26	LENG B， JIN D， HOU X C， at al. Tire-road peak adhesion coefficient estimation method based on fusion of vehicle dynamics and machine vision［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（11）： 21740-21752.
27	脱王捷. 主动安全相关的路面状况识别［D］. 重庆：重庆理工大学， 2020.
	TUO W J. Recognition method of road condition for active safety system［D］. Chongqing ：Chongqing University of Technology， 2020.
28	刘惠. 视觉与动力学信息融合的智能车辆路面附着系数估计［D］. 长春：吉林大学， 2021.
	LIU H. Vision and dynamics information fusion based road adhesion coefficient estimation for intelligent vehicles［D］. Changchun：Jilin University， 2021.
29	何中正. 基于多传感器数据融合的路面附着系数估计［D］. 长春：吉林大学， 2021.
	HE Z Z. Road friction coefficient estimation based on multi-sensor and data fusion［D］. Changchun：Jilin University， 2021.
30	PACEJKA H B， BAKKER E. The magic formula tyre model［J］. Vehicle System Dynamics， 1992， 21（S1）： 1-18.
31	CARPENTER J， CLIFFORDY P， FEARNHEAD P. Improved particle filter for nonlinear problems［J］. IEE Proceedings-Radar， Sonar and Navigation， 1999， 146（1）： 2-7.
32	CHEN L C， ZHU Y， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］. The European Conference on Computer Vision. ECCV， 2018： 801-818.
33	SANDLER M， HOWARD A， ZHU M， et al. MobileNetV2： inverted residuals and linear bottlenecks［C］. 2018 IEEE/CVF Computer Society Conference on Computer Vision and Pattern Recognition. IEEE， 2018： 4510-4520.
34	LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998， 86（11）： 2278-2323.
35	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90.
36	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. Computer Science， 2014： 1-14.
37	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016： 770-778.
38	HOWARD A G， ZHU M， CHEN B， et al. MobileNets： efficient convolutional neural networks for mobile vision applications［J］. arXiv preprint arXiv：1704.04861， 2017： 1-9.
39	HOWARD A， SANDLER M， CHEN B， et al. Searching for mobilenetv3［C］. 2019 IEEE International Conference on Computer Vision. IEEE， 2019： 1314-1324.
40	ZHANG X， ZHOU X， LIN M， et al. ShuffleNet： an extremely efficient convolutional neural network for mobile devices［C］. 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE， 2018： 6848-6856.
41	MA N， ZHANG X， ZHENG H T， et al. Shufflenet V2： practical guidelines for efficient CNN architecture design［C］. The European Conference on Computer Vision. ECCV， 2018： 116-131.
42	龚标. 典型交通事故形态车辆行驶速度技术鉴定［M］. 北京：中国标准出版社， 2006.
	GONG B. The speed technical evaluation for vehicles involved in representative road accidents［M］. Beijing： Standards Press of China， 2006.
43	余志生. 汽车理论［M］. 5版. 北京：机械工业出版社， 2009.
	YU Z S. Automotive theory［M］. 5th ed.Beijing： China Machine Press， 2009.
44	CORDTS M， OMRAN M， RAMOS S， et al. The cityscapes dataset for semantic urban scene understanding［C］. 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE， 2016： 3213-3223.
45	NEUHOLD G， OLLMANN T， BULO S R， et al. The mapillary vistas dataset for semantic understanding of street scenes［C］. 2017 IEEE International Conference on Computer Vision. IEEE， 2017： 5000-5009.
46	YU F， XIAN W， CHEN Y， et al. BDD100K： a diverse driving video database with scalable annotation tooling［J］. Arxiv， 2018： 1-16.

输入	操作	扩张因子	输出通道数	模块重复次数	步长
224²×32	Conv2d		32	1	2
112²×32	Bottleneck	1	16	1	1
112²×16	Bottleneck	6	24	2	2
56²×24	Bottleneck	6	32	3	2
28²×32	Bottleneck	6	64	4	2
14²×64	Bottleneck	6	96	3	1
14²×96	Bottleneck	6	160	3	2
7²×160	Bottleneck	6	320	1	1
7²×320	Conv2d 1×1		1 280	1	1
7²×1280	Avgpool 7×7			1
1×1×1280	Conv2d 1×1		k

路面类型	路面附着系数范围
干燥沥青	0.6-0.9
潮湿沥青	0.45-0.6
积雪沥青	0.1-0.45

软硬件名称	配置
操作系统	Windows11（64位）
编程语言	Python3.9
开发框架	PyTorch
CPU	AMD R7-6800HS
GPU	NVIDA3050
CUDA	11.7

参数	数值
整车质量m/kg	1 230
质心到前轴距离a/m	1.040
质心到后轴距离b/m	1.560
转动惯量I_z /（kg?m²）	1 343.1
车轮滚动半径R_w/mm	298
车轮转动惯量J_w/（kg?m²）	0.6

[1]	Zuliang Jin,Hanbing Wei,Liu Zheng,Lu Lou,Guofeng Zheng. Lightweight Semantic Segmentation Method Based on Local Window Cross Attention [J]. Automotive Engineering, 2023, 45(9): 1617-1625.
[2]	Wenguang Wu,Shuangyue Tian,Zhiyong Zhang,Bin Zhang. Research on Semantic Segmentation of Uneven Features of Unpaved Road [J]. Automotive Engineering, 2023, 45(8): 1468-1478.
[3]	Xia Zhao,Zhao Li,Rui Fu,Zhenzhen Ge,Chang Wang. Real-Time Detection of Distracted Driving Behavior Based on Deep Convolution-Tokens Dimensionality Reduction Optimized Visual Transformer Model [J]. Automotive Engineering, 2023, 45(6): 974-988.
[4]	Xinrong Zhang,Xin Wang,Xinle Gong,Jin Huang,Dan Huang,Pengxing Wang. Segmented Identification Method of Tire-Road Friction Coefficient for Intelligent Vehicles [J]. Automotive Engineering, 2023, 45(10): 1923-1932.
[5]	Runhui Huang,Likun Hu,Mingfang Su,Daye Xu,Aoran Chen. Semantic Segmentation Method of LiDAR Point Cloud Based on 3D Conical Grid [J]. Automotive Engineering, 2022, 44(8): 1173-1182.
[6]	Runze Feng,Kun Jiang,Weiguang Yu,Diange Yang. Chinese Traffic Sign Recognition Based on Two-stage Classification Algorithm [J]. Automotive Engineering, 2022, 44(3): 434-441.
[7]	Dafang Wang,Lei Liu,Jiang Cao,Gang Zhao,Wenshuo Zhao,Wei Tang. Semantic Segmentation Method of Autonomous Driving Images Based on Atrous Spatial Pyramid Pooling [J]. Automotive Engineering, 2022, 44(12): 1818-1824.
[8]	Dafang Wang,Hai Shang,Jiang Cao,Tao Wang,Xiangteng Xia,Yulin Han. Semantic Segmentation Method of Point Cloud in Automatic Driving Scene Based on Self-attention Mechanism [J]. Automotive Engineering, 2022, 44(11): 1656-1664.
[9]	Shengjie Huang,Manjiang Hu,Yunshui Zhou,Zhouping Yin,Xiaohui Qin,Yougang Bian,Qianqian Jia. Vehicle Visual SLAM in Dynamic Scenes Based on Semantic Segmentation and Motion Consistency Constraints [J]. Automotive Engineering, 2022, 44(10): 1503-1510.
[10]	Hai Yu,Junjun Deng,Zhenpo Wang,Fengchun Sun. Inverter Fault Diagnosis Method Based on Convolutional Neural Network [J]. Automotive Engineering, 2022, 44(1): 142-152.
[11]	Xiangteng Xia,Dafang Wang,Jiang Cao,Gang Zhao,Jingming Zhang. Semantic Segmentation Method of On-board Lidar Point Cloud Based on Sparse Convolutional Neural Network [J]. Automotive Engineering, 2022, 44(1): 26-35.
[12]	He Huang,Kun Wu,Xinrui Li,Jun Wang,Huifeng Wang,Feng Ru. A Multi⁃feature Particle Filter Vehicle Tracking Algorithm Based on Improved Moth⁃flaming Optimization [J]. Automotive Engineering, 2021, 43(9): 1322-1327.
[13]	Ying Jin,Xinyong Qiao,Cheng Gu,Hao Guo,Chuming Ning. A Method for Fault Diagnosis of Fuel Injector of Diesel Engine Based on Res⁃CNN and Fuel Pressure Wave [J]. Automotive Engineering, 2021, 43(6): 943-951.
[14]	Hai Wang,Baixiang Cai,Yingfeng Cai,Ze Liu,Kai Sun,Long Chen. Detection of Water⁃covered and Wet Areas on Road Pavement Based on Semantic Segmentation Network [J]. Automotive Engineering, 2021, 43(4): 485-491.
[15]	Wang Hai, Wang Kuan, Cai Yingfeng, Liu Ze, Chen Long. Traffic Sign Recognition Based on Improved Cascade Convolution Neural Network [J]. Automotive Engineering, 2020, 42(9): 1256-1262.

Tire-Road Friction Estimation Method Based on Image Recognition and Dynamics Fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 46

Related Articles 15

Metrics

Comments

Recommended 10