融合Lite-HRNet的Yolo v5双模态自动驾驶小目标检测方法

doi:10.19562/j.chinasae.qcgc.2022.10.005

Abstract

Abstract:

This paper proposes a Yolo v5 network fused with Lite-HRNet to solve the problem of missed detection in the current target detection algorithm for autonomous driving field when detecting small and dense targets. Firstly， in order to obtain high-resolution feature detection maps， Lite-HRNet is used as the backbone network of Yolo v5 to enhance the detection of small and dense objects. In order to improve the detection performance in dark scenes， the infrared image and the visible light image are dynamically weighted to give full play to the complementary advantages of the visible light image and the infrared image. Because of the sufficient feature fusion of the backbone network， in order to speed up the detection speed， the feature fusion structure in the detection layer is cancelled. Secondly， $α$ -EIoU is used as the bounding box loss function in order to speed up the convergence and improve the regression accuracy. At the same time， the bisecting K-means algorithm is used for clustering to select more appropriate anchor boxes for the data set， and the small target data augmentation algorithm is used for sample expansion of the dataset. Finally， a comparative test with Yolo v5 on the flir dataset is conducted. According to the experimental results， the average detection accuracy of this algorithm is 7.64% higher than Yolo v5， and the missed detection rate of small targets and dense targets is significantly reduced.

Key words: automatic driving, object detection, infrared image, Yolo v5, small target, Lite-HRNet

Zilong Liu,Xiangfei Shen. Small Target Detection Method for Dual-Modal Autonomous Driving with Yolo v5 and Lite-HRNet Fusion[J].Automotive Engineering, 2022, 44(10): 1511-1520.

Figures/Tables 15

References 24

1	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014： 580-587.
2	GIRSHICK R. Fast r-cnn［C］.Proceedings of the IEEE International Conference on Computer Vision. 2015： 1440-1448.
3	REN S， HE K， GIRSHICK R， et al. Faster r-cnn： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis & Machine Intelligence， 2017， 39（6）：1137-1149.
4	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot multiBox detector［C］. European Conference on Computer Vision. Springer， Cham， 2016： 21-37.
5	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection［J］. IEEE， 2016： 779-788.
6	REDMON J， FARHADI A. Yolo9000： better， faster， stronger［C］. IEEE Conference on Computer Vision & Pattern Recognition， 2017：6517-6525.
7	REDMON J， FARHADI A. Yolov3： an incremental improvement［J］. Arxiv Preprint Arxiv：， 2018.
8	ZHANG J， ZHANG L， LIU T， et al. Yolso： you only look small object［J］. Journal of Visual Communication and Image Representation， 2021， 81： 103348.
9	YU C， XIAO B， GAO C， et al. Lite-hrnet： a lightweight high-resolution network［C］. Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 10440-10450.
10	WANG C Y， LIAO H Y M， WU Y H， et al. Cspnet： a new backbone that can enhance learning capability of cnn［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops， 2020： 390-391.
11	YU J， JIANG Y， WANG Z， et al. Unitbox： an advanced object detection network［C］.Proceedings of the 24th ACM International Conference on Multimedia， 2016： 516-520.
12	REZATOFIGHI H， TSOI N， GWAK J Y， et al. Generalized intersection over union： a metric and a loss for bounding box regression［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 658-666.
13	ZHENG Z， WANG P， LIU W， et al. Distance-iou loss： faster and better learning for bounding box regression［C］.Proceedings of the AAAI Conference on Artificial Intelligence， 2020， 34（7）： 12993-13000.
14	ZHANG Y F， REN W， ZHANG Z， et al. Focal and efficient iou loss for accurate bounding box regression［J］. Arxiv Preprint Arxiv：2101. 08158， 2021.
15	HE J， ERFANI S， MA X， et al. alpha-IoU： a family of power intersection over union losses for bounding box regression［J］. Advances in Neural Information Processing Systems， 2021， 34.
16	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation［C］. Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019： 5693-5703.
17	ZHANG X， ZHOU X， LIN M， et al. Shufflenet： an extremely efficient convolutional neural network for mobile devices［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018： 6848-6856.
18	CHEN Q， WANG Y， YANG T， et al. You only look one-level feature［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021： 13039-13048.
19	KISANTAL M， WOJNA Z， MURAWSKI J， et al. Augmentation for small object detection［J］. Arxiv Preprint Arxiv：， 2019.
20	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft coco： common objects in context［C］. European Conference on Computer Vision. Springer， Cham， 2014： 740-755.
21	WONG JAHA. ALGORITHM AS 136： a K-means clustering algorithm［J］. Journal of the Royal Statistical Society， 1979， 28（1）：100-108.
22	GE Z， LIU S， WANG F， et al. Yolox： exceeding yolo series in 2021［J］. Arxiv Preprint Arxiv：， 2021.
23	ZHU X， LYU S， WANG X， et al. Tph-Yolo5： improved Yolov5 based on transformer prediction head for object detection on drone-captured scenarios［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021： 2778-2788.
24	QIAO S， CHEN L C， YUILLE A. Detectors： detecting objects with recursive feature pyramid and switchable atrous convolution［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 10213-10224.

[1]	LI Huan, HUANG Ying, ZHANG Fu-Jun, ZHAO Yu, GE Yan-Wu. [J]. , 2016, 38(12): 1420 -1426 .
[2]	HAO Han, WANG Si-南, LI Xiao, LIU Zong-Wei, ZHAO Fu-Quan. [J]. , 2017, 39(1): 1 -8 .
[3]	XU Cheng-Shan, JIANG Fa-Chao, SONG Sen-Nan, TIAN Guang-Yu. [J]. , 2017, 39(1): 9 -14 .
[4]	JIANG Ting, DAI Bing. [J]. , 2016, 38(12): 1459 -1466 .
[5]	ZHANG Yi-Bing, 吕Jian-Jun , YUAN Guang-Hui. [J]. , 2016, 38(12): 1467 -1470 .
[6]	ZHANG Lei, ZHANG Jin-Qiu, LUO Tao, BI Zhan-Dong, YAO Jun. [J]. , 2016, 38(12): 1494 -1499 .
[7]	CUI An, LI Bin, WANG Xue-Liang, ZHANG Shi-Zhan. [J]. , 2016, 38(12): 1521 -1525 .
[8]	CHU Liang, ZHAO Di, LI Wen-Hui. [J]. , 2017, 39(1): 61 -65 .
[9]	FENG Chong, ZHANG Dong-Hao, LUO Yu-Gong, LI Ke-Qiang. [J]. , 2017, 39(1): 66 -72 .
[10]	HAN Xiao-Mei, LIN Xue-Dong, LI De-Gang. [J]. , 2017, 39(1): 23 -27 .

模型	输入	GFLOPs	模型大小/MB
替换前	608×608×3	38.5	137.4
替换后	608×608×3	17.6	76.2

算法	R/%	P/%	mAP@0.5/%	FPS/（帧·s^-1）
Yolo v5	83.93	79.23	82.59	62.51
本文算法	92.67	87.54	90.23	61.89

类别	R/%		P/%		AP/%
类别	A	B	A	B	A	B
person	88.72	96.34	84.27	92.21	87.83	94.57
car	89.06	94.82	83.29	88.37	88.57	92.81
bus	82.93	89.47	78.53	83.76	80.13	86.73
truck	80.61	86.51	72.04	78.62	78.82	84.48
bicycle	82.74	95.04	79.39	93.06	81.52	93.71
dog	79.51	93.81	77.86	89.19	78.68	89.07

算法	改进点						mAP@0.5/%	FPS/（帧·s^-1）
算法	A	B	C	D	E	F	mAP@0.5/%	FPS/（帧·s^-1）
Yolo v5							82.59	62.51
算法1	√						84.27	63.03
算法2		√					87.05	56.82
算法3			√				80.67	67.39
算法4				√			83.91	61.09
算法5					√		82.72	63.48
算法6						√	84.69	62.73
本文算法	√	√	√	√	√	√	90.23	61.89

算法	mAP@0.5/%	FPS/（帧·s^-1）
Faster RCNN	78.71	12.49
SSD	67.39	37.02
Yolo v4	83.62	60.39
YoloX	84.13	61.45
TPH-Yolo v5	88.41	58.07
DetectoRS	91.44	9.73
本文算法	90.23	61.89

Small Target Detection Method for Dual-Modal Autonomous Driving with Yolo v5 and Lite-HRNet Fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 24

Related Articles 15

Metrics

Comments

Recommended 10

[1]	Feng Gao,Defu Feng,Qiuxia Hu. Accelerating Technologies of Numerical Optimization for Motion Planning Designed by Nonlinear Model Predictive Control [J]. Automotive Engineering, 2023, 45(8): 1438-1447.
[2]	Tao Lu,Xin Jin,Yifei Liao,Shengjie Huang,Yilin Yang,Guotao Xie,Xiaohui Qin. Visual SLAM Based on Jacobian Null-space Marginalization [J]. Automotive Engineering, 2023, 45(8): 1457-1467.
[3]	Wenguang Wu,Shuangyue Tian,Zhiyong Zhang,Bin Zhang. Research on Semantic Segmentation of Uneven Features of Unpaved Road [J]. Automotive Engineering, 2023, 45(8): 1468-1478.
[4]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[5]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[6]	Yanyan Chen,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Efficient Automatic Driving Instance Segmentation Method Based on Detection [J]. Automotive Engineering, 2023, 45(4): 541-550.
[7]	Manjiang Hu,Binjie Mou,Zeyu Yang,Yougang Bian,Xiaohui Qin,Biao Xu. A Hybrid A* Path Planning Method Based on DBSCAN and Dichotomy [J]. Automotive Engineering, 2023, 45(3): 341-349.
[8]	Jie Hu,Haoyan Liu,Minchao Zhang,Zhihao Zhang,Qi Zhu,Ruipeng Chen,Man Luo. Research on the Decision and Planning System of Automated Valet Parking Based on Finite State Machine [J]. Automotive Engineering, 2023, 45(2): 243-252.
[9]	Qin Shi,Yafang Chen,Teng Cheng,Qiang Zhang,Wenchong Wang,Benyi Shi. Unsupervised Monocular Infrared Image Depth Estimation Based on Local Plane Guidance Layer [J]. Automotive Engineering, 2023, 45(12): 2291-2298.
[10]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[11]	Jie Hu,Boyuan Xu,Zongquan Xiong,Minjie Chang,Di Guo,Lihao Xie. Cross-Domain Object Detection Algorithm Based on Multi-scale Mask Classification Domain Adaptive Network [J]. Automotive Engineering, 2022, 44(9): 1327-1338.
[12]	Liang Chu,Cheng Chang,Xu Wang,Di Zhao,Yanwu Xu. Research on Brake-by-Wire System and Pressure Control Algorithm for Advanced Automatic Driving [J]. Automotive Engineering, 2022, 44(3): 308-318.
[13]	Desheng Xie,Youchun Xu,Feng Lu,Shiju Pan. Real-time Detection of 3D Objects Based on Multi-Sensor Information Fusion [J]. Automotive Engineering, 2022, 44(3): 340-349.
[14]	Linyan Wang,Huijun Zhang,Hongyu Hu. Effect of Driving Experience on Take-Over Performance of L3 Automatic Driving [J]. Automotive Engineering, 2022, 44(10): 1521-1526.
[15]	Xiaohui Qin,Zhewen Wang,Tao Pang,Weiqing Shi,Ning Sun,Manjiang Hu. Vehicle Positioning Method Based on Tight Coupling of Vehicle Model in Enclosed Environments [J]. Automotive Engineering, 2021, 43(9): 1328-1335.