融合Lite-HRNet的Yolo v5双模态自动驾驶小目标检测方法

doi:10.19562/j.chinasae.qcgc.2022.10.005

摘要/Abstract

摘要：

针对目前自动驾驶领域的目标检测算法在对道路小目标和密集目标进行检测的时候出现漏检的问题，提出一种融合Lite-HRNet的Yolo v5网络。首先为了获得高分辨率的特征检测图将Lite-HRNet作为Yolo v5的主干网络，以增强对小目标及密集目标的检测。为提升暗光场景下的检测性能，将红外图像与可见光图像进行动态权值融合，充分发挥可见光图像与红外图像的互补优势。由于主干网络进行了充分的特征融合，为加快检测速度取消在检测层中的特征融合结构。其次为了加快收敛速度和提高回归精度采用α-EIoU作为边界框损失函数，同时为选取针对数据集更合适的先验框，使用二分K-means算法进行聚类，并且使用小目标数据增强算法对数据集进行样本扩充。最后在flir数据集上进行对比测试，根据实验结果，提出的算法比Yolo v5在平均精度上提高了7.64%，小目标和密集目标的漏检率明显减少。

关键词: 自动驾驶, 目标检测, 红外图像, Yolo v5, 小目标, Lite-HRNet

Abstract:

This paper proposes a Yolo v5 network fused with Lite-HRNet to solve the problem of missed detection in the current target detection algorithm for autonomous driving field when detecting small and dense targets. Firstly， in order to obtain high-resolution feature detection maps， Lite-HRNet is used as the backbone network of Yolo v5 to enhance the detection of small and dense objects. In order to improve the detection performance in dark scenes， the infrared image and the visible light image are dynamically weighted to give full play to the complementary advantages of the visible light image and the infrared image. Because of the sufficient feature fusion of the backbone network， in order to speed up the detection speed， the feature fusion structure in the detection layer is cancelled. Secondly， $α$ -EIoU is used as the bounding box loss function in order to speed up the convergence and improve the regression accuracy. At the same time， the bisecting K-means algorithm is used for clustering to select more appropriate anchor boxes for the data set， and the small target data augmentation algorithm is used for sample expansion of the dataset. Finally， a comparative test with Yolo v5 on the flir dataset is conducted. According to the experimental results， the average detection accuracy of this algorithm is 7.64% higher than Yolo v5， and the missed detection rate of small targets and dense targets is significantly reduced.

Key words: automatic driving, object detection, infrared image, Yolo v5, small target, Lite-HRNet

刘子龙,沈祥飞. 融合Lite-HRNet的Yolo v5双模态自动驾驶小目标检测方法[J]. 汽车工程, 2022, 44(10): 1511-1520.

Zilong Liu,Xiangfei Shen. Small Target Detection Method for Dual-Modal Autonomous Driving with Yolo v5 and Lite-HRNet Fusion[J]. Automotive Engineering, 2022, 44(10): 1511-1520.

图/表 15

图1

图2

图3

图4

表1

图5

图6

图7

图8

图9

表2

表3

图10

表4

表5

参考文献 24

1	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014： 580-587.
2	GIRSHICK R. Fast r-cnn［C］.Proceedings of the IEEE International Conference on Computer Vision. 2015： 1440-1448.
3	REN S， HE K， GIRSHICK R， et al. Faster r-cnn： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis & Machine Intelligence， 2017， 39（6）：1137-1149.
4	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot multiBox detector［C］. European Conference on Computer Vision. Springer， Cham， 2016： 21-37.
5	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection［J］. IEEE， 2016： 779-788.
6	REDMON J， FARHADI A. Yolo9000： better， faster， stronger［C］. IEEE Conference on Computer Vision & Pattern Recognition， 2017：6517-6525.
7	REDMON J， FARHADI A. Yolov3： an incremental improvement［J］. Arxiv Preprint Arxiv：， 2018.
8	ZHANG J， ZHANG L， LIU T， et al. Yolso： you only look small object［J］. Journal of Visual Communication and Image Representation， 2021， 81： 103348.
9	YU C， XIAO B， GAO C， et al. Lite-hrnet： a lightweight high-resolution network［C］. Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 10440-10450.
10	WANG C Y， LIAO H Y M， WU Y H， et al. Cspnet： a new backbone that can enhance learning capability of cnn［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops， 2020： 390-391.
11	YU J， JIANG Y， WANG Z， et al. Unitbox： an advanced object detection network［C］.Proceedings of the 24th ACM International Conference on Multimedia， 2016： 516-520.
12	REZATOFIGHI H， TSOI N， GWAK J Y， et al. Generalized intersection over union： a metric and a loss for bounding box regression［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 658-666.
13	ZHENG Z， WANG P， LIU W， et al. Distance-iou loss： faster and better learning for bounding box regression［C］.Proceedings of the AAAI Conference on Artificial Intelligence， 2020， 34（7）： 12993-13000.
14	ZHANG Y F， REN W， ZHANG Z， et al. Focal and efficient iou loss for accurate bounding box regression［J］. Arxiv Preprint Arxiv：2101. 08158， 2021.
15	HE J， ERFANI S， MA X， et al. alpha-IoU： a family of power intersection over union losses for bounding box regression［J］. Advances in Neural Information Processing Systems， 2021， 34.
16	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation［C］. Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019： 5693-5703.
17	ZHANG X， ZHOU X， LIN M， et al. Shufflenet： an extremely efficient convolutional neural network for mobile devices［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018： 6848-6856.
18	CHEN Q， WANG Y， YANG T， et al. You only look one-level feature［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021： 13039-13048.
19	KISANTAL M， WOJNA Z， MURAWSKI J， et al. Augmentation for small object detection［J］. Arxiv Preprint Arxiv：， 2019.
20	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft coco： common objects in context［C］. European Conference on Computer Vision. Springer， Cham， 2014： 740-755.
21	WONG JAHA. ALGORITHM AS 136： a K-means clustering algorithm［J］. Journal of the Royal Statistical Society， 1979， 28（1）：100-108.
22	GE Z， LIU S， WANG F， et al. Yolox： exceeding yolo series in 2021［J］. Arxiv Preprint Arxiv：， 2021.
23	ZHU X， LYU S， WANG X， et al. Tph-Yolo5： improved Yolov5 based on transformer prediction head for object detection on drone-captured scenarios［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021： 2778-2788.
24	QIAO S， CHEN L C， YUILLE A. Detectors： detecting objects with recursive feature pyramid and switchable atrous convolution［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 10213-10224.

模型	输入	GFLOPs	模型大小/MB
替换前	608×608×3	38.5	137.4
替换后	608×608×3	17.6	76.2

算法	R/%	P/%	mAP@0.5/%	FPS/（帧·s^-1）
Yolo v5	83.93	79.23	82.59	62.51
本文算法	92.67	87.54	90.23	61.89

类别	R/%		P/%		AP/%
类别	A	B	A	B	A	B
person	88.72	96.34	84.27	92.21	87.83	94.57
car	89.06	94.82	83.29	88.37	88.57	92.81
bus	82.93	89.47	78.53	83.76	80.13	86.73
truck	80.61	86.51	72.04	78.62	78.82	84.48
bicycle	82.74	95.04	79.39	93.06	81.52	93.71
dog	79.51	93.81	77.86	89.19	78.68	89.07

算法	改进点						mAP@0.5/%	FPS/（帧·s^-1）
算法	A	B	C	D	E	F	mAP@0.5/%	FPS/（帧·s^-1）
Yolo v5							82.59	62.51
算法1	√						84.27	63.03
算法2		√					87.05	56.82
算法3			√				80.67	67.39
算法4				√			83.91	61.09
算法5					√		82.72	63.48
算法6						√	84.69	62.73
本文算法	√	√	√	√	√	√	90.23	61.89

算法	mAP@0.5/%	FPS/（帧·s^-1）
Faster RCNN	78.71	12.49
SSD	67.39	37.02
Yolo v4	83.62	60.39
YoloX	84.13	61.45
TPH-Yolo v5	88.41	58.07
DetectoRS	91.44	9.73
本文算法	90.23	61.89