基于稀疏卷积神经网络的车载激光雷达点云语义分割方法

doi:10.19562/j.chinasae.qcgc.2022.01.004

Abstract

Abstract:

The semantic segmentation of point cloud obtained by on-board lidar scanning is one of the important means for ensuring driving safety and enhancing the driver's understanding of surrounding environment. Due to memory limitation and the sparse characteristics of large-scale point cloud scenes， directly continue to apply the traditional neural network approach to the large-scale point cloud scenario does not work well. Therefore， taking the advantage of the sparseness of large-scale point cloud， sparse convolutional neural network （sparse CNN） is used to extract the characteristics of voxel cloud in this paper. With consideration of the information loss cause by density inconsistence in point-wise processing sub-branch suppressed point cloud data， additional 3D-CA and 3D-SA modules are designed to improve the characteristics extraction of sparse CNN. The results of experiment show that compared with traditional convolutional neural network method and point cloud projection on plane method， the sparse CNN method for the semantic segmentation of large scale point cloud can have 4.1% and 3.4% higher IOU respectively， demonstrating the effectiveness of the method adopted.

Key words: autonomous driving, point cloud, semantic segmentation, sparse convolution neural network

Xiangteng Xia,Dafang Wang,Jiang Cao,Gang Zhao,Jingming Zhang. Semantic Segmentation Method of On-board Lidar Point Cloud Based on Sparse Convolutional Neural Network[J].Automotive Engineering, 2022, 44(1): 26-35.

Figures/Tables 12

立方网格边长/m	MIOU/%	网格数	非空点数
0.02	56.854	$1.250 × 1010$	120 875
0.04	58.413	$1.563 × 109$	98 960
0.05	60.800	$8.000 × 108$	89 660
0.06	58.840	$4.461 × 108$	81 156
0.08	57.183	$1.953 × 108$	67 603

算法	MIOU	road	sidewalk	parking	other-ground	building	car	truck	bicycle	motorcycle
$P o i n t N e t + +$	20.1	$72.0$	$41.8$	$18.7$	$5.6$	$62.3$	$53.7$	$0.9$	$1.9$	$0.2$
$R a n d L A N e t$	53.9	$90.7$	$73.7$	$60.3$	$20.4$	$86.9$	$94.2$	$40.1$	$26.0$	$25.8$
$K P C o n v$	58.8	$90.3$	$72.7$	$61.3$	$31.5$	$90.5$	$95.0$	$33.4$	$30.2$	$42.5$
$R a n g e N e t + +$	52.2	$91.8$	$75.2$	$65.0$	$27.8$	$87.4$	$91.4$	$25.7$	$25.7$	$34.4$
$S q u e e z e S e g V 3$	55.9	$91.7$	$74.8$	$63.4$	$26.4$	$89.0$	$92.5$	$29.6$	$38.7$	$36.5$
$S a l s a N e x t$	59.5	$91.7$	$75.8$	$63.7$	$29.1$	$90.2$	$91.9$	$38.9$	$48.3$	$38.6$
$F u s i o n N e t [26]$	61.3	$91.8$	$77.1$	$68.8$	$30.8$	$92.5$	$95.3$	$41.8$	$47.5$	$37.7$
$K P R N e t$	63.1	$93.2$	$80.6$	$73.9$	$30.2$	$91.7$	$95.5$	$23.6$	$54.1$	$47.9$
文中算法	62.9	$90.0$	$74.1$	$65.4$	$14.8$	$91.7$	$96.6$	$46.6$	$48.0$	$47.1$
算法	other-vehicle	vegetation	trunk	terrain	person	bicyclist	motorcyclist	fence	pole	traffic sign
$P o i n t N e t + +$	$0.2$	$46.5$	$13.8$	$30.0$	$0.9$	$1.0$	$0.0$	$16.9$	$6.0$	$8.9$
$R a n d L A N e t$	$38.9$	$81.4$	$61.3$	$66.8$	$49.2$	$48.2$	$7.2$	$56.3$	$49.2$	$47.7$
$K P C o n v$	$44.3$	$84.8$	$69.2$	$69.1$	$61.5$	$61.6$	$11.8$	$64.2$	$56.4$	$47.4$
$R a n g e N e t + +$	$23.0$	$80.5$	$55.1$	$64.6$	$38.3$	$38.8$	$4.8$	$58.6$	$47.9$	$55.9$
$S q u e e z e S e g V 3$	$33.0$	$82.0$	$58.7$	$65.4$	$45.6$	$46.2$	$20.1$	$59.4$	$49.6$	$58.9$
$S a l s a N e x t$	$31.9$	$81.8$	$63.6$	$66.5$	$60.2$	$59.0$	$19.4$	$64.2$	$54.3$	$62.1$
$F u s i o n N e t [26]$	$34.5$	$84.5$	$69.8$	$68.5$	$59.5$	$56.8$	$11.9$	$69.4$	$60.4$	$66.5$
$K P R N e t$	$42.6$	$85.7$	$69.8$	$71.2$	$65.9$	$65.0$	$16.5$	$68.4$	$58.7$	$64.1$
文中算法	$49.2$	$85.7$	$71.8$	$70.2$	$63.7$	$61.1$	$25.2$	$67.6$	$61.8$	$64.5$

References 25

1	何汉. 三维点云语义分割技术研究［D］.成都：电子科技大学，2020.
	HE H. Research on semantic segmentation of 3D point cloud［D］. Chengdu： University of Electronic Science and Technology of China， 2020.
2	GUO Y， WANG H， HU Q， et al. Deep learning for 3D point clouds： a survey［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， IEEE， 2020.
3	SU H， MAJI S， KALOGERAKIS E， et al. Multi-view convolutional neural networks for 3D shape recognition［C］. Proceedings of the IEEE international conference on computer vision. 2015： 945-953.
4	WU B， WAN A， YUE X， et al. SqueezeSeg： convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud［C］. 2018 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2018： 1887-1893.
5	IANDOLA F N， HAN S， MOSKEWICZ M W， et al. SqueezeNet： AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size［J］. arXiv preprint arXiv：， 2016.
6	WU B， ZHOU X， ZHAO S， et al. SqueezeSegV2： improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud［C］. 2019 International Conference on Robotics and Automation （ICRA）. IEEE， 2019： 4376-4382.
7	XU C， WU B， WANG Z， et al. SqueezeSegV3： Spatially-adaptive convolution for efficient point-cloud segmentation［C］. European Conference on Computer Vision. Springer， Cham， 2020： 1-19.
8	CORTINHAL T， TZELEPIS G， AKSORY E， et al. SalsaNext： fast， uncertainty-aware semantic segmentation of lidar point clouds for autonomous driving［J］. arXiv preprint arXiv：， 2020.
9	HUANG J， YOU S. Point cloud labeling using 3D Convolutional neural network［C］. 2016 23rd International Conference on Pattern Recognition （ICPR）. IEEE， 2016： 2670-2675.
10	MENG H， GAO L， LAI Y， et al. VV-Net： voxel vae net with group convolutions for point cloud segmentation［C］. 2019 IEEE/CVF International Conference on Computer Vision. IEEE， 2019： 8500-8508.
11	GRAHAM B， ENGELCKE M， MAATEN L. 3D semantic segmentation with submanifold sparse convolutional networks［C］. 2018 IEEE conference on computer vision and pattern recognition. IEEE， 2018： 9224-9232.
12	CHOY C， GWAK J， SAVARESE S， et al. 4D spatio-temporal convnets： minkowski convolutional neural networks［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE，2019： 3075-3084.
13	ZHU X， ZHOU H， WANG T， et al. Cylindrical and asymmetrical 3D convolution networks for lidar segmentation［C］. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2021： 9939：9948.
14	QI C R， SU H， MO K， et al. PointNet： deep learning on point sets for 3d classification and segmentation［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2017： 77-85.
15	QI C R， YI L， SU H， et al. PointNet++： deep hierarchical feature learning on point sets in a metric space［C］. 31st International Conference on Neural Information Processing Systems. Curran Associates Inc：， 2017： 5099-5108.
16	THOMAS H， R.QI C， DESCHAUD J E， et al. KPConv： flexible and deformable convolution for point clouds［C］. 2019 IEEE/CVF International Conference on Computer Vision. IEEE， 2019： 6411-6420.
17	HU Q， YANG B， XIE L， et al. Randla-net： efficient semantic segmentation of large-scale point clouds［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2020： 11108-11117.
18	LIU Z， TANG H， LIN Y， et al. Point-voxel cnn for efficient 3D deep learning［J］. arXiv preprint arXiv：， 2019.
19	KOCHANOV D， NEJADASL F K， BOOIJ O， et al. KPRNet： improving projection-based LiDAR semantic segmentation［J］. arXiv preprint arXiv：， 2020.
20	SANDLER M， HOWARD A， ZHU M， et al. MobilenetV2： Inverted residuals and linear bottlenecks［C］. 2018 IEEE conference on computer vision and pattern recognition. 2018： 4510-4520.
21	WOO S， PARK J， LEE J Y， et al. Cbam： Convolutional block attention module［C］. 2018 European conference on computer vision （ECCV）. IEEE，2018： 3-19.
22	BEHLEY J， GARBADE M， MILIOTO A， et al. Semantickitti： a dataset for semantic scene understanding of Lidar sequences［C］. 2019 IEEE/CVF International Conference on Computer Vision. IEEE， 2019： 9297-9307.
23	PAN Y， GAO B， MEI J， et al. SemanticPOSS： a point cloud dataset with large quantity of dynamic instances［C］. 2020 IEEE Intelligent Vehicles Symposium. Las Vegas， NV， USA： IEEE， 2020： 687-693.
24	LOSHCHILOV I， HUTTER F. Decoupled weight decay regularization［C］. 2019 International Conference on Learning Representations. New Orleans. Springer International Publishing， 2019 ： 453-461.
25	YAN X， GAO J， LI J， et al. Sparse single sweep LiDAR point cloud segmentation via learning contextual shape priors from scene completion［J］. arXiv preprint arXiv：， 2020.

[1]	XU Cheng-Shan, JIANG Fa-Chao, SONG Sen-Nan, TIAN Guang-Yu. [J]. , 2017, 39(1): 9 -14 .
[2]	ZHANG Yi-Bing, 吕Jian-Jun , YUAN Guang-Hui. [J]. , 2016, 38(12): 1467 -1470 .
[3]	RONG Bing, XIAO Pan, ZHOU Jian-Wen, LIU Yi-Lu. [J]. , 2017, 39(2): 214 -219 .
[4]	WANG Zhen, CUI Ya-Hui-.Liu-Kai, XU Lin. [J]. , 2017, 39(3): 304 -310 .
[5]	GUAN Xin, XU Liang, GUO Rui, WANG Bo, SONG Hai-Yan. [J]. , 2017, 39(3): 311 -316 .
[6]	JIN Geng-Ri, GUAN Xin, ZHAN Jun, GUO Rui. [J]. , 2017, 39(3): 364 -368 .
[7]	XIONG Xing-Wang, GAO Jun-Hua, ZHOU Tao, YU Jin-Tao. [J]. , 2017, 39(4): 381 -385 .
[8]	CAO Li-Feng, ZHOU Shou-Qin, XIE Xiao-Peng. [J]. , 2017, 39(4): 401 -406 .
[9]	Liu Zhiqiang & Pu Xian. An Experimental Study on Anti-lock Braking of Continuous Regenerative Braking System in Electric Vehicles[J]. , 2018, 40(7): 804 .
[10]	. Algorithm Research of Parking Space Detection Based on Images[J]. , 0, (): 0 .

网络模型	MIOU/%
网络基本层	61.7
网络基本层+3D-CA	62.1
网络基本层+3D-SA	62.4
网络基本层+3D-CA+3D-SA	62.9

算法	MIOU/%	行人/%	汽车/%	骑手/%
RandLANet	54.7	69.2	68.4	26.7
KPConv	61.7	77.3	78.6	29.4
SqueezeSegV2	35.2	48.0	48.3	9.4
RangeNet++	39.2	57.3	55.7	4.6
MINet	46.1	62.4	63.8	12.1
JS3CNet^［25］	66.3	80.0	79.8	39.1
文中算法	67.5	83.9	83.6	35.1

[1]	Teng Cheng,Hao Ni,Qiang Zhang,Wenchong Wang,Qin Shi. Two-Stage Multimodal Fusion Networks Based on Virtual Point Clouds [J]. Automotive Engineering, 2024, 46(2): 222-229.
[2]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[3]	Yanli Ma, Qin Qin, Fangqi Dong, Yining Lou. Takeover Risk Assessment Model Based on Risk Field Theory Under Different Cognitive Secondary Tasks [J]. Automotive Engineering, 2024, 46(1): 9-17.
[4]	Zuliang Jin,Hanbing Wei,Liu Zheng,Lu Lou,Guofeng Zheng. Lightweight Semantic Segmentation Method Based on Local Window Cross Attention [J]. Automotive Engineering, 2023, 45(9): 1617-1625.
[5]	Weiguo Liu,Zhiyu Xiang,Weiping Liu,Daoxin Qi,Zixu Wang. Research on Vehicle Control Algorithm Based on Distributed Reinforcement Learning [J]. Automotive Engineering, 2023, 45(9): 1637-1645.
[6]	Ming Wang,Xiaolin Tang,Kai Yang,Guofa Li,Xiaosong Hu. A Motion Planning Method for Autonomous Vehicles Considering Prediction Risk [J]. Automotive Engineering, 2023, 45(8): 1362-1372.
[7]	Wenguang Wu,Shuangyue Tian,Zhiyong Zhang,Bin Zhang. Research on Semantic Segmentation of Uneven Features of Unpaved Road [J]. Automotive Engineering, 2023, 45(8): 1468-1478.
[8]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[9]	Lei Zhang, Keren Guan, Xiaolin Ding, Pengyu Guo, Zhenpo Wang, Fengchun Sun. Tire-Road Friction Estimation Method Based on Image Recognition and Dynamics Fusion [J]. Automotive Engineering, 2023, 45(7): 1222-1234.
[10]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[11]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[12]	Zhongsheng Chen,Chaolin Li,Wang Zuo,Xinglin Hou. Study on Improved Point Cloud Registration Algorithm Enhanced by Double Down-sampling [J]. Automotive Engineering, 2023, 45(4): 572-578.
[13]	Qingyang Huang,Xiaoping Jin,Yikang Zhang. Analysis of Drivers' Driving Posture Change Rule Under the Condition of Automatic Driving Level Improvement [J]. Automotive Engineering, 2023, 45(3): 382-392.
[14]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[15]	Pulei Xu,Yingfeng Cai,Yubo Lian,Xiaoqiang Sun,Hai Wang,Long Chen,Yilin Zhong. AFS/DYC Coordinated Control of Intelligent Vehicles Based on Improved Hierarchical Extensibility Theory [J]. Automotive Engineering, 2023, 45(1): 20-31.

Semantic Segmentation Method of On-board Lidar Point Cloud Based on Sparse Convolutional Neural Network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 25

Related Articles 15

Metrics

Comments

Recommended 10