基于检测的高效自动驾驶实例分割方法

doi:10.19562/j.chinasae.qcgc.2023.04.002

Abstract

Abstract:

The instance segmentation algorithm based on deep learning has achieved excellent performance in large-scale general scenarios. However， the segmentation of multi-objective instances for complex traffic scenes is still challenging， especially in the balance between high accuracy and fast inference speed， which is crucial to driving safety of intelligent vehicles. In view of this， based on the real-time algorithm Orienmask， a multi-head segmentation framework is proposed based on the one-stage detection method. Specifically， the proposed framework comprises of a backbone， a feature fusion module and a multi-head mask construction module. Firstly， complete high-dimensional feature maps are obtained by adding residual structures to the backbone.Secondly， in order to generate discriminative feature representations， the feature pyramid module is reconstructed by introducing in self-calibrate convolutions and the information propagation path is improved by global attention mechanism， so as to further optimize the feature fusion module of the proposed framework. Finally， a multi-head mask construction mechanism is proposed to significantly improve the segmentation performance of different targets by refining the size distribution of instances in the traffic scenes. The proposed algorithm has been tested and validated on the open-source dataset BDD100k， and has achieved an average intersection ratio of 23.3% and 19.4% （mAP@0.5：0.95） on bounding boxes and segmentation masks， respectively. Compared with the baseline， the average index are increased by 5.2 % and 2.2 %. At the same time， the road experiment on the self-built real-vehicle platform also proves that the proposed algorithm can adapt to actual driving environments and meet the demands of real-time segmentations.

Key words: autonomous vehicles, deep Learning, object detection, instance segmentation

Yanyan Chen,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Efficient Automatic Driving Instance Segmentation Method Based on Detection[J].Automotive Engineering, 2023, 45(4): 541-550.

Figures/Tables 13

References 39

1	范丽丽，赵宏伟，赵浩宇，等. 基于深度卷积神经网络的目标检测研究综述［J］. 光学精密工程， 2020， 28： 1152-1164.
	FAN Lili， ZHAO Hongwei， ZHAO Haoyu， et al. Survey of target detection based ondeep convolutional neural networks［J］. Optics and Precision Engineering， 2020，28： 1152-1164.
2	GARCIA-GARCIA A， ORTS-ESCOLANO S， OPREA S， et al. A review on deep learning techniques applied to semantic segmentation ［J］. arXiv：170406857. 2017.
3	王海，蔡柏湘，蔡英凤，等. 基于语义分割网络的路面积水与湿滑区域检测［J］. 汽车工程， 2021， 43： 485-491.
	WANG Hai， CAI Baixiang， CAI Yingfeng， et al. Detection of water⁃covered and wet areas on road pavement based on semantic segmentation network［J］. Automotive Engineering， 2021， 43： 485-491.
4	CAI Y， DAI L， WANG H， et al. Pedestrian motion trajectory prediction in intelligent driving from far shot first-person perspective video ［J］. IEEE Transactions on Intelligent Transportation Systems， 2021， 23（6）： 5298-5313.
5	GU W， BAI S， KONG L. A review on 2D instance segmentation based on deep neural networks ［J］. Image Vision Computing， 2022： 104401.
6	苏丽，孙雨鑫，苑守正. 基于深度学习的实例分割研究综述［J］. 智能系统学报， 2021， 17： 16-31.
	SU Li， SUN Yuxin， YUAN Shouzheng. A survey of instance sementation research based on deep learning［J］.CAAI Transactions on Intelligent Systems， 2021，17： 16-31.
7	GAO N， SHAN Y， WANG Y， et al. SSAP： single-shot instance segmentation with affinity pyramid［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2019.
8	DE BRABANDERE B， NEVEN D， VAN GOOL L. Semantic instance segmentation with a discriminative loss function ［J］. arXiv：170802551. 2017.
9	LIU S， JIA J， FIDLER S， et al. SGN： sequential grouping networks for instance segmentation［C］. Proceedings of the IEEE International Conference on Computer Vision， F， 2017.
10	XIE E， SUN P， SONG X， et al. PolarMask： single shot instance segmentation with polar representation［C］. Proceedings of the Proceedings of the IEEE/CVF conference On Computer Vision And Pattern Recognition， F， 2020.
11	DONG B， ZENG F， WANG T， et al. SOLQ： segmenting objects by learning queries ［J］. Advances in Neural Information Processing Systems， 2021， 34.
12	WANG X， KONG T， SHEN C， et al. SOLO： segmenting objects by locations［C］. Proceedings of the European Conference on Computer Vision， F， 2020. Springer.
13	WANG X， ZHANG R， KONG T， et al. Solov2： dynamic and fast instance segmentation ［J］. Advances in Neural Information Processing Systems， 2020， 33： 17721-17732.
14	TIAN Z， SHEN C， CHEN H， et al. FCOS： fully convolutional one-stage object detection［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2019.
15	REDMON J， FARHADI A. Yolov3： an incremental improvement ［J］. arXiv： 180402767 2018.
16	BOCHKOVSKIY A， WANG C Y， LIAO H Y M J A P A. Yolov4： optimal speed and accuracy of object detection ［J］. arXiv：200410934. 2020.
17	蔡英凤，张田田，王海，等. 基于实例分割和自适应透视变换算法的多车道线检测［J］. 东南大学学报（自然科学版）， 2020， 50： 775-781.
	CAI Yingfeng， ZHANG Tiantian，WANG Hai， et al. Multi-lane detection based on instance segmentation and adaptive perspective transformation［J］. Journal of Southeast University （Natural Science Edition）， 2020，50：775-781.
18	TIAN Z， SHEN C， CHEN H. Conditional convolutions for instance segmentation［C］. Proceedings of the European Conference on Computer Vision， F， 2020. Springer.
19	HE K， GKIOXARI G， DOLLáR P， et al. Mask R-CNN［C］. Proceedings of the IEEE International Conference on Computer Vision （ICCV）， F March 01， 2017， 2017.
20	HUANG Z， HUANG L， GONG Y， et al. Mask scoring R-CNN［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2019.
21	KIRILLOV A， WU Y， HE K， et al. PointRend： image segmentation as rendering［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
22	BOLYA D， ZHOU C， XIAO F， et al. YOLACT： real-time instance segmentation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2019.
23	BOLYA D， ZHOU C， XIAO F， et al. YOLACT++： better real-time instance segmentation ［J］. IEEE Transactions on Pattern Analysis Machine Intelligence， 2020.
24	HU J， CAO L， LU Y， et al. ISTR： end-to-end instance segmentation with transformers ［J］. arXiv：210500637. 2021.
25	FANG Y， YANG S， WANG X， et al. Instances as queries［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2021.
26	GU W， BAI S， KONG L. A review on 2D instance segmentation based on deep neural networks ［J］. Image and Vision Computing， 2022， 120： 104401.
27	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks ［J］. Advances in Neural Information Processing Systems， 2015， 28.
28	CAI Z， VASCONCELOS N. Cascade R-CNN： high quality object detection and instance segmentation ［J］. IEEE Transactions on Pattern Analysis Machine Intelligence， 2019， 43（5）： 1483-1498.
29	CHEN H， SUN K， TIAN Z， et al. BlendMask： top-down meets bottom-up for instance segmentation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
30	DU W， XIANG Z， CHEN S， et al. Real-time instance segmentation with discriminative orientation maps［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2021.
31	LIN T Y， DOLLáR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， F， 2017.
32	LIU J J， HOU Q， CHENG M M， et al. Improving convolutional networks with self-calibrated convolutions［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
33	LIU S， QI L， QIN H， et al. Path aggregation network for instance segmentation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， F， 2018.
34	CAI Y， LUAN T， GAO H， et al. YOLOv4-5D： an effective and efficient object detector for autonomous driving［J］. IEEE Transactions on Instrumentation and Measurement， 2021， 70： 1-13.
35	YU F， XIAN W， CHEN Y， et al. BDD100K： a diverse driving video database with scalable annotation tooling ［J］. arXiv：1805.0487，201.
36	ZHAO H， SHI J， QI X， et al. Pyramid scene parsing network［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， F， 2017.
37	JAIN A K. Data clustering： 50 years beyond K-means ［J］. Pattern Recognition Letters， 2010， 31（8）： 651-66.
38	DENG J， DONG W， SOCHER R， et al. ImageNet： a large-scale hierarchical image database［C］. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition， F， 2009.
39	CAO Y， XU J， LIN S， et al. GCNet： non-local networks meet squeeze-excitation networks and beyond［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops， F， 2019.

[1]	CHEN Wu-Wei, DENG Shu-Chao, HUANG He, XIE You-Hao. [J]. , 2016, 38(12): 1488 -1493 .
[2]	PENG Qian-Lei, GUI Liang-Jin, FAN Zi-Jie. [J]. , 2016, 38(12): 1500 -1507 .
[3]	DONG Zhu-Rong, ZHANG Xin, HU Song-Hua, QIU Hao. [J]. , 2017, 39(1): 79 -85 .
[4]	SHI Hai-Min, YU Xiao-Li, LU Guo-Dong, HUANG Yu-Qi, LIU Zhen-Tao, HUANG Rui. [J]. , 2017, 39(1): 102 -106 .
[5]	CAO Li-Bo, HU Yuan, YAN Ling-Bo, PENG Yu, SHI Xiang-南. [J]. , 2017, 39(2): 174 -180 .
[6]	WANG Bin, LI Tie, ZHANG Xiao-Qing, GUO Tao, SHI Qi. [J]. , 2017, 39(3): 256 -261 .
[7]	LI Jun, SU Yan-Zhao, KUI Han-Bing, HU Ming-Hui, ZHANG Sheng-Gen, LIU Heng-Shuo. [J]. , 2017, 39(3): 296 -303 .
[8]	LIN Xin-You, MO Li-Ping, LUO Yong, ZHANG Shao-Bo. [J]. , 2017, 39(4): 369 -375 .
[9]	CHU Liang, LIU Da-Liang, LIU Hong-Wei, CAI Jian-Wei, ZHAO Di. [J]. , 2017, 39(4): 471 -479 .
[10]	YANG Zheng-Jun, FU Bing-Zheng, YIN Hang, TAN Jian-Wei, ZHOU Xiao-Yan, GE Yun-Shan. [J]. , 2017, 39(5): 497 -502 .

层数	锚框0	锚框1	锚框2
第1层	（8，10）	（14，12）	（17，20）
第2层	（14，37）	（36，15）	（25，29）
第3层	（34，44）	（67，30）	（47，69）
第4层	（78，98）	（127，175）	（271，337）

方法	行人	骑行者	轿车	货车	巴士	火车	摩托	自行车	mAP@0.5：0.95 （seg）	mAP@0.5：0.95 （box）	FPS
Mask R-CNN	27.6	6.3	43.9	20.8	23.1	0.0	2.0	5. 9	16.2	22.3	14.3
Cascade Mask	28.8	7.3	45.4	24.7	27.1	0.0	9.9	5.6	18.6	25.9	13.2
GCNet	28.0	4.35	43.9	22.4	20.9	0.0	5.09	4.16	16.1	22.4	13.9
YoLACT									15.4	18.5	20.0
Solov2-Lite	19.2	7.2	35.9	19.9	26.8	0.0	15.6	5.4	16.2		36.7
baseline	16.3	5.3	38.4	25.2	26.2	0.0	22.4	3.9	17.2	18.1	38.56
Ours	19.2	4.0	43.3	30.8	26.8	11.9	16.1	3.3	19.4	23.3	27.7

方法	CSP	FPN （Sconv）	PANet （Sconv）	Multi-head	mAP@0.5：0.95 （box）	mAP@0.5 （box）	mAP@0.5：0.95 （seg）	mAP@0.5 （seg）
baseline					18.1	35.3	17.2	31.9
+	√				18.2	35.3	17.2	31.9
+	√	√			18.6	35.9	17.7	32.7
+	√	√	√		20.8	38.4	18.4	34.2
+	√	√	√	√	23.3	41.0	19.4	36.0

[1]	Xinke Fu,Yingfeng Cai,Long Chen,Hai Wang,Qingchao Liu. Decision-Making for Autonomous Driving in Uncertain Environment [J]. Automotive Engineering, 2024, 46(2): 211-221.
[2]	Ze Gao, Zunkang Chu, Jiasheng Shi, Fu Lin, Weixiong Rao, Haiyan Yu. Research on Fast Prediction Method of Stress Field of Automotive Parts Based on Graph Network [J]. Automotive Engineering, 2024, 46(1): 170-178.
[3]	Weiguo Liu,Zhiyu Xiang,Rui Liu,Guodong Li,Zixu Wang. Research on End-to-End Vehicle Motion Planning Method Based on Deep Learning [J]. Automotive Engineering, 2023, 45(8): 1343-1352.
[4]	Cheng Lin, Bowen Wang, Lü Peiyuan, Xinle Gong, Xiao Yu. Research on Motion Planning and Cooperative Control for Autonomous Vehicles with Lane Change Gaming Maneuvers Under the Curved Road [J]. Automotive Engineering, 2023, 45(7): 1099-1111.
[5]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[6]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[7]	Fengchong Lan,Jikai Chen,Jiqing Chen,Xinping Jiang,Zihan Li,Wei Pan. Research on Lithium Battery Remaining Useful Life Prediction Method Driven by Real Vehicle Data [J]. Automotive Engineering, 2023, 45(2): 175-182.
[8]	Zhifei Wu,Shoubiao Li. Lane Detection Algorithm Based on Instance Segmentation [J]. Automotive Engineering, 2023, 45(2): 263-272.
[9]	Lü Ying,Xu Qi,Qiuzheng Liu,Xinyu Wang,Guoying Chen. Path Tracking Control Method with Steering Lag for Autonomous Vehicles [J]. Automotive Engineering, 2023, 45(12): 2234-2241.
[10]	Linhui Li,Xinliang Zhang,Yifan Fu,Jing Lian,Jiaxu Ma. Research on Visible Light and Infrared Post-Fusion Detection Based on TC-YOLOv7 Algorithm [J]. Automotive Engineering, 2023, 45(12): 2280-2290.
[11]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[12]	Long Chen,Chen Yang,Yingfeng Cai,Hai Wang,Yicheng Li. Pedestrian Crossing Intention Prediction Method Based on Multimodal Feature Fusion [J]. Automotive Engineering, 2023, 45(10): 1779-1790.
[13]	Xiaojun Zhang,Jingzhe Xi,Yanlei Shi,Anlu Yuan. Lightweight YOLOv7-R Algorithm for Road-Side View Target Detection [J]. Automotive Engineering, 2023, 45(10): 1833-1844.
[14]	Jie Hu,Yuanjie Li,Hao Geng,Huangzheng Geng,Xiong Guo,Hongwei Yi. Construction of Vehicle Fault Knowledge Graph Based on Deep Learning [J]. Automotive Engineering, 2023, 45(1): 52-60.
[15]	Fengchong Lan,Yingjie Liu,Jiqing Chen,Zhaolin Liu. Study on Motion Planning of Autonomous Vehicles in Cut-in Scenes Based on Dynamic Game Algorithm [J]. Automotive Engineering, 2023, 45(1): 9-19.

Efficient Automatic Driving Instance Segmentation Method Based on Detection

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 39

Related Articles 15

Metrics

Comments

Recommended 10