基于自注意力机制的自动驾驶场景点云语义分割方法

doi:10.19562/j.chinasae.qcgc.2022.11.004

Abstract

Abstract:

Semantic segmentation of vehicle lidar scene point cloud is the basic work of automatic driving environment perception. In view of the insufficient ability to extract local features and difficulty to capture the global context information of the existing processing method of point cloud in large-scale automatic driving scene， the local and global self-attention encoders are designed based on the self-attention mechanism and the feature aggregation module is built for feature extraction. The experimental results show that compared with RandLA-Net， also adopting local feature aggregation， the method proposed can increase the MIoU by 5.7 percentage points on SemanticKITTI dataset， and adding local self-attention encoder also raises the segmentation accuracy of small targets such as vehicles and pedestrians by more than 2 percentage points.

Key words: semantic segmentation, large-scale point cloud, self-attention mechanism

Dafang Wang,Hai Shang,Jiang Cao,Tao Wang,Xiangteng Xia,Yulin Han. Semantic Segmentation Method of Point Cloud in Automatic Driving Scene Based on Self-attention Mechanism[J].Automotive Engineering, 2022, 44(11): 1656-1664.

Figures/Tables 9

References 30

1	姜枫，顾庆，郝慧珍，等.基于内容的图像分割方法综述［J］.软件学报，2017，28（1）：160-183.DOI：10.13328/j.cnki.jos.005136.
	JIANG F， GU Q， HAO H Z， et al. Survey on content-based image segmentation methods［J］. Journal of Software， 2017， 28（1）： 160-183.
2	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］. Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition， 2015： 3431-3440.
3	GUO Y， WANG H， HU Q， et al. Deep learning for 3D point clouds： a survey［J］. IEEE transactions on Pattern Analysis and Machine Intelligence， 2020， 43（12）： 4338-4364.
4	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer， Cham， 2015： 234-241.
5	HE K， GKIOXARI G， DOLLÁR P， et al. Mask r-cnn［C］. Proceedings of the IEEE International Conference on Computer Vision， 2017： 2961-2969.
6	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. Deeplab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFS［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 40（4）： 834-848.
7	SU H， MAJI S， KALOGERAKIS E， et al. Multi-view convolutional neural networks for 3D shape recognition［C］. Proceedings of the IEEE International Conference on Computer Vision， 2015： 945-953.
8	WU B， WAN A， YUE X， et al. Squeezeseg： convolutional neural nets with recurrent crf for real-time road-object segmentation from 3D lidar point cloud［C］.2018 IEEE International Conference on Robotics and Automation （ICRA）. IEEE， 2018： 1887-1893.
9	IANDOLA F N， HAN S， MOSKEWICZ M W， et al. SqueezeNet： AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size［J］. arXiv preprint arXiv：， 2016.
10	WU B， ZHOU X， ZHAO S， et al. Squeezesegv2： improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud［C］.2019 International Conference on Robotics and Automation （ICRA）. IEEE， 2019： 4376-4382.
11	XU C， WU B， WANG Z， et al. Squeezesegv3： spatially-adaptive convolution for efficient point-cloud segmentation［C］.European Conference on Computer Vision. Springer， Cham， 2020： 1-19.
12	MILIOTO A， VIZZO I， BEHLEY J， et al. Rangenet++： fast and accurate lidar semantic segmentation［C］.2019 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2019： 4213-4220.
13	ÇIÇEK Ö， ABDULKADIR A， LIENKAMP S S， et al. 3D U-Net： learning dense volumetric segmentation from sparse annotation［C］.International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer， Cham， 2016： 424-432.
14	TCHAPMI L， CHOY C， ARMENI I， et al. Segcloud： semantic segmentation of 3D point clouds［C］.2017 International Conference on 3D Vision （3DV）. IEEE， 2017： 537-547.
15	MENG H Y， GAO L， LAI Y K， et al. VV-net： voxel vae net with group convolutions for point cloud segmentation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019： 8500-8508.
16	ZHOU H， ZHU X， SONG X， et al. Cylinder3D： an effective 3D framework for driving-scene lidar semantic segmentation［J］. arXiv preprint arXiv：， 2020.
17	WANG F， ZHUANG Y， GU H， et al. Octreenet： a novel sparse 3-D convolutional neural network for real-time 3-D outdoor scene analysis［J］. IEEE Transactions on Automation Science and Engineering， 2019， 17（2）： 735-747.
18	QI C R， SU H， MO K， et al. Pointnet： deep learning on point sets for 3D classification and segmentation［C］.Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition， 2017： 652-660.
19	QI C R， YI L， SU H， et al. Pointnet++： deep hierarchical feature learning on point sets in a metric space［J］. Advances in Neural Information Processing Systems， 2017， 30.
20	JIANG M， WU Y， ZHAO T， et al. Pointsift： a sift-like network module for 3D point cloud semantic segmentation［J］. arXiv preprint arXiv：， 2018.
21	LI Y， BU R， SUN M， et al. Pointcnn： convolution on X-transformed points［J］. Advances in Neural Information Processing Systems， 2018， 31.
22	THOMAS H， QI C R， DESCHAUD J E， et al. Kpconv： flexible and deformable convolution for point clouds［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 6411-6420.
23	LIU Z， TANG H， LIN Y， et al. Point-voxel cnn for efficient 3D deep learning［J］. Advances in Neural Information Processing Systems， 2019， 32.
24	XU J， ZHANG R， DOU J， et al. Rpvnet： a deep and efficient range-point-voxel fusion network for lidar point cloud segmentation［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 16024-16033.
25	HU Q， YANG B， XIE L， et al. Randla-net： efficient semantic segmentation of large-scale point clouds［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 11108-11117.
26	ZHAO H， JIANG L， JIA J， et al. Point transformer［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 16259-16268.
27	BEHLEY J， GARBADE M， MILIOTO A， et al. Semantickitti： a dataset for semantic scene understanding of lidar sequences［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 9297-9307.
28	PAN Y， GAO B， MEI J， et al. Semanticposs： a point cloud dataset with large quantity of dynamic instances［C］.2020 IEEE Intelligent Vehicles Symposium （IV）. IEEE， 2020： 687-693.
29	GEIGER A， LENZ P， STILLER C， et al. Vision meets robotics： the KITTI dataset［J］. The International Journal of Robotics Research， 2013， 32（11）： 1231-1237.
30	ZHANG F， FANG J， WAH B， et al. Deep fusionnet for point cloud semantic segmentation［C］.European Conference on Computer Vision. Springer， Cham， 2020： 644-663.

[1]	SHEN Zhe, WANG Yi-Gang, YANG Zhi-Gang, LI Fang-Xu. [J]. , 2016, 38(12): 1440 -1445 .
[2]	ZENG Bi-Qiang, GAO Ji-Dong, PENG Wei, SUN Zhen-Dong. [J]. , 2016, 38(12): 1446 -1451 .
[3]	LIU Hui, WANG Xiao-Jie, XIANG Chang-Le. [J]. , 2016, 38(12): 1483 -1487 .
[4]	CHEN Wu-Wei, DENG Shu-Chao, HUANG He, XIE You-Hao. [J]. , 2016, 38(12): 1488 -1493 .
[5]	PENG Qian-Lei, GUI Liang-Jin, FAN Zi-Jie. [J]. , 2016, 38(12): 1500 -1507 .
[6]	DONG Zhu-Rong, ZHANG Xin, HU Song-Hua, QIU Hao. [J]. , 2017, 39(1): 79 -85 .
[7]	SHI Hai-Min, YU Xiao-Li, LU Guo-Dong, HUANG Yu-Qi, LIU Zhen-Tao, HUANG Rui. [J]. , 2017, 39(1): 102 -106 .
[8]	QIN Chao-Ju, YANG Zhen-Zhong, ZHANG Wei-Zheng, SONG Li-Ye, YUAN Yan-Peng. [J]. , 2017, 39(2): 133 -137 .
[9]	ZHANG Guan-Jun, ZHAO Xin-Feng, CAO Li-Bo. [J]. , 2017, 39(2): 150 -158 .
[10]	CAO Li-Bo, HU Yuan, YAN Ling-Bo, PENG Yu, SHI Xiang-南. [J]. , 2017, 39(2): 174 -180 .

网络结构	MIoU/%
RandLA-Net	53.9
1LSAE、1GSAE	56.2
1GSAE、1LSAE	37.5
2LSAE、1GSAE	59.6
3LSAE、1GSAE	57.3
1LASE、2GSAE	54.9

算法	MIoU/%	汽车	行人	骑手
RandLA-Net	53.9	66.2	70.1	25.5
+LSAE	56.5	68.8	72.5	28.3
+GSAE	48.5	55.7	68.3	21.4
+FAM	56.8	69.2	72.7	28.4

邻近点个数k	MIoU/%	汽车	行人	骑手
8	52.4	64.3	70.2	22.6
16	56.5	68.8	72.5	28.3
32	56.1	67.4	73.3	27.5
64	53.9	65.8	71.0	24.9

算法	消耗时间/s	MIoU/%
KPConv	656	58.8
RandLA-Net	149	53.9
本文	177	59.6

算法	MIoU/%	道路	人行道	停车场	其他地面	建筑物	汽车	货车	自行车	摩托车
PointNet++	20.1	72.0	41.8	18.7	5.6	62.3	53.7	0.9	1.9	0.2
SqueezeSegV2	39.7	88.6	67.6	45.8	17.7	73.7	81.8	13.4	18.5	17.9
RangeNet++	52.2	91.8	75.2	65.0	27.8	87.4	91.4	25.7	25.7	34.4
KPConv	58.8	90.3	72.7	61.3	31.5	90.5	95.0	33.4	30.2	42.5
FusionNet^［30］	61.3	91.8	77.1	68.8	30.8	92.5	95.3	41.8	47.5	37.7
RandLA-Net	53.9	90.7	73.7	60.3	20.4	86.9	94.2	40.1	26.0	25.8
本文	59.6	91.1	74.5	62.2	28.5	90.6	96.9	43.5	32.3	33.8
算法	其他车辆	植被	树干	地形	行人	自行车手	摩托车手	栅栏	电线杆	交通标志
PointNet++	0.2	46.5	13.8	30.0	0.9	1.0	0.0	16.9	6.0	8.9
SqueezeSegV2	14.0	71.8	35.8	60.2	20.1	25.1	3.9	41.1	20.2	36.3
RangeNet++	23.0	80.5	55.1	64.6	38.3	38.8	4.8	58.6	47.9	55.9
KPConv	44.3	84.8	69.2	69.1	61.5	61.6	11.8	64.2	56.4	47.4
FusionNet	34.5	84.5	69.8	68.5	59.5	56.8	11.9	69.4	60.4	66.5
RandLA-Net	38.9	81.4	61.3	66.8	49.2	48.2	7.2	56.3	49.2	47.7
本文	43.2	83.9	65.4	68.7	64.8	58.8	12.5	67.3	57.6	57.3

Semantic Segmentation Method of Point Cloud in Automatic Driving Scene Based on Self-attention Mechanism

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 30

Related Articles 8

Metrics

Comments

Recommended 10

[1]	Zuliang Jin,Hanbing Wei,Liu Zheng,Lu Lou,Guofeng Zheng. Lightweight Semantic Segmentation Method Based on Local Window Cross Attention [J]. Automotive Engineering, 2023, 45(9): 1617-1625.
[2]	Wenguang Wu,Shuangyue Tian,Zhiyong Zhang,Bin Zhang. Research on Semantic Segmentation of Uneven Features of Unpaved Road [J]. Automotive Engineering, 2023, 45(8): 1468-1478.
[3]	Lei Zhang, Keren Guan, Xiaolin Ding, Pengyu Guo, Zhenpo Wang, Fengchun Sun. Tire-Road Friction Estimation Method Based on Image Recognition and Dynamics Fusion [J]. Automotive Engineering, 2023, 45(7): 1222-1234.
[4]	Runhui Huang,Likun Hu,Mingfang Su,Daye Xu,Aoran Chen. Semantic Segmentation Method of LiDAR Point Cloud Based on 3D Conical Grid [J]. Automotive Engineering, 2022, 44(8): 1173-1182.
[5]	Dafang Wang,Lei Liu,Jiang Cao,Gang Zhao,Wenshuo Zhao,Wei Tang. Semantic Segmentation Method of Autonomous Driving Images Based on Atrous Spatial Pyramid Pooling [J]. Automotive Engineering, 2022, 44(12): 1818-1824.
[6]	Shengjie Huang,Manjiang Hu,Yunshui Zhou,Zhouping Yin,Xiaohui Qin,Yougang Bian,Qianqian Jia. Vehicle Visual SLAM in Dynamic Scenes Based on Semantic Segmentation and Motion Consistency Constraints [J]. Automotive Engineering, 2022, 44(10): 1503-1510.
[7]	Xiangteng Xia,Dafang Wang,Jiang Cao,Gang Zhao,Jingming Zhang. Semantic Segmentation Method of On-board Lidar Point Cloud Based on Sparse Convolutional Neural Network [J]. Automotive Engineering, 2022, 44(1): 26-35.
[8]	Hai Wang,Baixiang Cai,Yingfeng Cai,Ze Liu,Kai Sun,Long Chen. Detection of Water⁃covered and Wet Areas on Road Pavement Based on Semantic Segmentation Network [J]. Automotive Engineering, 2021, 43(4): 485-491.