基于实例激活图的自动驾驶实时实例分割算法

doi:10.19562/j.chinasae.qcgc.2025.04.003

Abstract

Abstract:

Instance segmentation algorithms based on deep learning are capable of helping intelligent vehicles to obtain accurate perception information. However， due to the limitation of manufacturing cost， the computing resources on intelligent vehicles are usually limited. In order to obtain high-precision recognition and segmentation under limited computing resources， the algorithm itself is required to make full use of the extracted features. Meanwhile， although the one-stage instance segmentation algorithm has a relative fast inference speed， it has poor performance in accuracy. To this end， structural improvement based on the one-stage instance segmentation algorithm SparseInst is conducted to enhance the model’s utilization of effective features. Specifically， firstly， residual connection is added inside the basic building block of the backbone. Secondly， a three-scale feature fusion module is designed to overcome the problem of indirect interaction of cross-scale features in the encoder. A decoupled instance activation module is designed to enhance the model's ability to learn instance features. In addition， the improved algorithm makes full use of detail features to refine the mask features to improve the quality of the generated masks. Finally， the kernel is used to initialize the score of the target object， which improves the utilization rate of the extracted features. The improved algorithm surpasses similar algorithms in mask accuracy on multiple datasets and has strong real-time performance. To further verify the effectiveness of the improved algorithm， experiments using data collected from a real vehicle platform are conducted. When the input image resolution is 640×480， the model inference speed reaches 54 FPS， and the instance mask is segmented accurately.

Key words: autonomous driving, deep learning, real-time instance segmentation, feature utilization

Qirui Qin,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Real-Time Instance Segmentation Algorithm for Autonomous Driving Based on Instance Activation Maps[J].Automotive Engineering, 2025, 47(4): 614-624.

Figures/Tables 20

References 38

1	王海，李洋，蔡英凤，等. 基于激光雷达的 3D 实时车辆跟踪［J］. 汽车工程， 2021， 43（7）： 1013-1021.
	WANG Hai， LI Yang， CAI Yingfeng， et al. 3D real⁃time vehicle tracking based on lidar ［J］. Automotive Engineering， 2021， 43（7）： 1013-1021.
2	武志斐，李守彪. 基于实例分割的车道线检测算法［J］. 汽车工程， 2023， 45（2）： 263-272.
	WU Zhifei， LI Shoubiao. Lane detection algorithm based on instance segmentation ［J］. Automotive Engineering， 2023， 45（2）： 263-272.
3	陈妍妍，王海，蔡英凤，等. 基于检测的高效自动驾驶实例分割方法［J］. 汽车工程， 2023， 45（4）： 541-550.
	CHEN Yanyan， WANG Hai， CAI Yingfeng， et al. Efficient automatic driving instance segmentation method based on detection ［J］. Automotive Engineering， 2023， 45（4）： 541-550.
4	HE K， GKIOXARI G， DOLLáR P， et al. Mask R-CNN［C］. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision， F， 2017.
5	ZHANG G， LU X， TAN J， et al. RefineMask： towards high-quality instance segmentation with fine-grained features［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2021.
6	WEN Q， YANG J， YANG X， et al. PatchDCT： patch refinement for high quality instance segmentation［C］. Proceedings of the The Eleventh International Conference on Learning Representations， F， 2022.
7	WANG X， KONG T， SHEN C， et al. SOLO： segmenting objects by locations［C］. Proceedings of the Computer Vision-ECCV 2020： 16th European Conference， Glasgow， UK， August 23-28， 2020， Proceedings， Part XVIII 16， F， 2020. Springer.
8	WANG X， ZHANG R， KONG T， et al. SOLOv2： dynamic and fast instance segmentation ［J］. Advances in Neural Information Processing Systems， 2020， 33： 17721-17732.
9	BOLYA D， ZHOU C， XIAO F， et al. Yolact： real-time instance segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2019.
10	TIAN Z， SHEN C， CHEN H. Conditional convolutions for instance segmentation［C］. Proceedings of the Computer Vision-ECCV 2020： 16th European Conference， Glasgow， UK， August 23-28， 2020， Proceedings， Part I 16， F， 2020. Springer.
11	CHEN Y， DAI X， LIU M， et al. Dynamic convolution： attention over convolution kernels［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
12	CHENG B， MISRA I， SCHWING A G， et al. Masked-attention mask transformer for universal image segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2022.
13	JAIN J， LI J， CHIU M T， et al. OneFormer： one transformer to rule universal image segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2023.
14	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［J］. Advances in Neural Information Processing Systems， 2017， 30.
15	CHENG T， WANG X， CHEN S， et al. Sparse instance activation for real-time instance segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2022.
16	YU F， CHEN H， WANG X， et al. BDD100K： a diverse driving dataset for heterogeneous multitask learning［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
17	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， F， 2016.
18	BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： optimal speed and accuracy of object detection ［J］. arXiv preprint arXiv：， 2020.
19	WANG W， XIE E， LI X， et al. PVT v2： improved baselines with pyramid vision transformer ［J］. Computational Visual Media， 2022， 8（3）： 415-424.
20	WANG C， HE W， NIE Y， et al. Gold-YOLO： efficient object detector via gather-and-distribute mechanism ［J］. Advances in Neural Information Processing Systems， 2024， 36.
21	HOU Q， ZHOU D， FENG J. Coordinate attention for efficient mobile network design［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2021.
22	YU Z， ZHAO C， WANG Z， et al. Searching central difference convolutional networks for face anti-spoofing［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
23	ZHU X， HU H， LIN S， et al. Deformable convnets v2： more deformable， better results［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2019.
24	CARION N， MASSA F， SYNNAEVE G， et al. End-to-end object detection with transformers［C］. Proceedings of the European Conference on Computer Vision， F， 2020. Springer.
25	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［C］. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision， F， 2017.
26	MILLETARI F， NAVAB N， AHMADI S A. V-Net： fully convolutional neural networks for volumetric medical image segmentation［C］. Proceedings of the 2016 Fourth International Conference on 3D Vision （3DV）， F， 2016. IEEE.
27	ZHENG S， LU C， NARASIMHAN S G. TPSeNCE： towards artifact-free realistic rain generation for deraining and object detection in rain［C］. Proceedings of the Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， F， 2024.
28	CAESAR H， BANKITI V， LANG A H， et al. nuScenes： a multimodal dataset for autonomous driving［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
29	SUN P， KRETZSCHMAR H， DOTIWALLA X， et al. Scalability in perception for autonomous driving： Waymo open dataset［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
30	KINGMA D P， BA J. Adam： a method for stochastic optimization ［J］. arXiv preprint arXiv：， 2014.
31	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： common objects in context［C］. Proceedings of the Computer Vision-ECCV 2014： 13th European Conference， Zurich， Switzerland， September 6-12， 2014， Proceedings， Part V 13， F， 2014. Springer.
32	LEE Y， PARK J. CenterMask： real-time anchor-free instance segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2020.
33	HE J， LI P， GENG Y， et al. Fastinst： a simple query-based model for real-time instance segmentation［C］. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， F， 2023.
34	LYU C， ZHANG W， HUANG H， et al. RTMDet： an empirical study of designing real-time object detectors ［J］. arXiv preprint arXiv：， 2022.
35	YANG R， SONG L， GE Y， et al. BoxSnake： polygonal instance segmentation with box supervision［C］. Proceedings of the Proceedings of the IEEE/CVF International Conference on Computer Vision， F， 2023.
36	LI W， LIU W， ZHU J， et al. Box2Mask： box-supervised instance segmentation via level-set evolution ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2024.
37	JOCHER G， CHAURASIA A， QIU J. Ultralytics YOLOv8 ［Z］. 2023.
38	JOCHER G， QIU J. Ultralytics YOLO11 ［Z］. 2024.

Backbone	mAP（Mask）	FPS
ResNet50	19.3	34.6
CSPDarknet53	19.5	32.5
PVTv2-B1	20.8	20.1

算法	BDD100K		R-BDD		nuImages		Waymo
算法	mAP（Mask）	FPS	mAP（Mask）	FPS	mAP（Mask）	FPS	FPS
Yolact	19.5	26.0	14.1	26.0	34.7	23.7	19.2
Centermask	19.4	23.9	13.8	23.9	34.9	21.7	16.6
CondInst	19.9	22.1	14.8	22.1	34.4	18.8	14.1
SOLOv2	20.3	23.8	16.4	23.8	35.2	21.2	15.9
RTMDet-Ins	21.7	24.2	18.0	24.2	35.6	22.3	17.3
Boxsnake	18.3	14.1	13.9	14.1	32.1	11.9	6.1
YOLOv8-seg	17.7	30.0	14.5	30.1	35.3	28.0	22.1
FastInst	21.6	25.7	18.6	25.5	36.7	23.2	19.0
Box2mask	20.2	16.6	14.2	16.6	34.8	13.1	7.2
YOLOv11-seg	19.8	33.0	15.8	33.0	35.7	29.6	23.7
Baseline	19.3	34.6	16.0	34.5	35.1	31.6	25.4
Ours	22.4	31.0	19.5	31.0	37.7	28.8	22.5

连接方法	mAP（Mask）
Baseline	19.3
Case I	19.6
Case Ⅱ	18.7
Case Ⅲ	18.5

Branch Ⅰ	Branch Ⅱ	Branch Ⅲ	mAP（Mask）
3×3 Conv			19.3
3×3 Conv	3×3 Conv		19.9
3×3 Conv	5×5 Conv		20.3
1×1 Conv	3×3 Conv	5×5 Conv	19.9

Improved ResNet50	TFFM	Decouple IAM	MDRM	Initializing score with kernel	mAP （Mask）	FPS
					19.3	34.6
√					19.6	34.5
√	√				20.0	33.2
√	√	√			21.0	32.7
√	√	√	√		21.6	31.3
√	√	√	√	√	22.4	31.0

Real-Time Instance Segmentation Algorithm for Autonomous Driving Based on Instance Activation Maps

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 38

Related Articles 15

Metrics

Comments

Recommended 0

[1]	Jinhui Suo, Xiaowei Wang, Peiwen Jiang, Chi Ding, Ming Gao, Yougang Bian. Domain Adaptive Visual Object Detection for Autonomous Driving Based on Multi-granularity Relation Reasoning [J]. Automotive Engineering, 2025, 47(2): 201-210.
[2]	Jiangkun Li,Ruixue Zong,Weiwen Deng,Ying Wang,Juan Ding. Directed Graph-Based Method for Evaluating Similarity in Urban Intersection Scenarios [J]. Automotive Engineering, 2025, 47(1): 23-34.
[3]	Daofei Li,Hao Pan. Application of Scenario Complexity Evaluation in Trajectory Prediction and Automated Driving Decision-Making [J]. Automotive Engineering, 2024, 46(9): 1556-1563.
[4]	Hai Wang,Jianguo Li,Yingfeng Cai,Long Chen. A LiDAR-Based Dynamic Driving Scene Multi-task Segmentation Network [J]. Automotive Engineering, 2024, 46(9): 1608-1616.
[5]	Jianan Zhang,Zhaozheng Hu,Jie Meng,Huahua Hu,Jie Zuo. Distributed Simulation Platform Architecture and Application of Autonomous Driving for Vehicle-Road-Map Collaboration [J]. Automotive Engineering, 2024, 46(8): 1335-1345.
[6]	Le Tao,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Point Cloud for Autonomous Driving Scenarios [J]. Automotive Engineering, 2024, 46(7): 1208-1218.
[7]	Linhui Li,Yifan Fu,Ting Wang,Xuecheng Wang,Jing Lian. Trajectory Prediction Method Enhanced by Self-supervised Pretraining [J]. Automotive Engineering, 2024, 46(7): 1219-1227.
[8]	Hai Wang,Guirong Zhang,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-modal Data Mining Algorithm for Corner Case of Automatic Driving Road Scene [J]. Automotive Engineering, 2024, 46(7): 1239-1248.
[9]	Zhigang Yang,Yujing Li,Chao Xia,Mengjia Wang,Lei Yu. A High Time-Resolution Reconstruction on the Automotive Turbulent Wake Based on LSTM-POD [J]. Automotive Engineering, 2024, 46(7): 1302-1313.
[10]	Hai Wang,Yuxuan Ding,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-class Multi-target Tracking Algorithm Combining Motion Speed and Appearance Features in Driving Scenarios [J]. Automotive Engineering, 2024, 46(6): 956-964.
[11]	Jing Huang,Xiangzhen Liu,Xiaoyang Deng,Ran Chen. Research on Intelligent Vehicle Trajectory Planning Based on Multimodal Trajectory Prediction [J]. Automotive Engineering, 2024, 46(6): 965-974.
[12]	Fuxing Yao,Chao Sun,Yungang Lan,Bing Lu,Bo Wang,Haiyang Yu. A Lane Change Decision Method for Intelligent Connected Vehicles Based on Mixture of Expert Model [J]. Automotive Engineering, 2024, 46(5): 882-892.
[13]	Mengfan Li,Zhongxiang Feng,Weihua Zhang,Jingyu Li. Study on Driver's Visual Transfer Characteristics During the Takeover Process of Human-Computer Co-driving Mode [J]. Automotive Engineering, 2024, 46(5): 795-804.
[14]	Ting Chikit,Yafei Wang,Yichen Zhang,Mingyu Wu,Yile Wang. Energy-Saving Planning Method for Autonomous Driving Mining Trucks Based on Composite Dynamic Sampling [J]. Automotive Engineering, 2024, 46(4): 588-595.
[15]	Hongyi Liang,Jikai Chen,Wanli Liu,Fengchong Lan,Bingda Mo,Jiqing Chen. Prediction of the Remaining Useful Life of Real Vehicle Lithium Batteries by Fusion of K-means Clustering and Sequence Decomposition [J]. Automotive Engineering, 2024, 46(4): 634-642.