基于激光雷达点云的动态驾驶场景多任务分割网络

doi:10.19562/j.chinasae.qcgc.2024.09.008

摘要/Abstract

摘要：

在自动驾驶场景理解任务中进行准确的可行驶区域以及动静态物体分割对于后续的局部运动规划和运动控制至关重要。然而当前基于激光雷达点云的通用语义分割方法并不能在车端边缘计算设备上实现实时且鲁棒的预测，且不能预测当前时刻的物体运动状态。为解决该问题本文提出一种可行驶区域及动静态物体多任务分割网络MultiSegNet。该网络利用激光雷达输出的深度图及处理后得到的残差图像作为编码空间特征和运动特征的表征输入到网络用于特征学习，从而避免直接处理无序高密度点云。针对深度图在不同方向视角内目标分布数量差异较大的特点，本文提出了变分辨率分组输入策略。该方法能在降低网络计算量的同时提高网络的分割精度。为适配不同尺度目标所需要的卷积感受野尺寸本文提出了深度值引导的分层空洞卷积模块。同时本文为有效关联并融合不同时域下物体的空间位置和姿态信息提出了时空运动特征增强网络。为验证所提出MultiSegNet的有效性，本文在大规模点云驾驶场景数据集SemanticKITTI及nuScenes上进行验证。结果表明：可行驶区域、静态物体和动态物体的分割IoU分别达到98%、97%和70%，性能优于主流网络，且在边缘计算设备上实现实时推理。

关键词: 无人驾驶, 激光雷达, 多任务点云分割网络, 动态物体分割

Abstract:

In autonomous driving scene understanding task， accurate segmentation of drivable areas， dynamic and static objects is essential for subsequent local motion planning and motion control. However， the current general semantic segmentation method based on lidar point cloud cannot achieve real-time and robust prediction on vehicle-end edge computing devices， and cannot predict the motion state of objects at the current moment. In order to solve this problem， a multi-task segmentation network MultiSegNet for driving areas and dynamic and static objects is proposed in this paper. The network uses the depth map output by the lidar and the processed residual image as the representation of the encoded spatial features and motion features to input to the network for feature learning， so as to avoid directly processing disordered high-density point clouds. For the large difference in the number of target distributions in different directions of the depth map， a variable resolution grouping input strategy is proposed， which can reduce the amount of network computation and improve the segmentation accuracy of the network. In order to adapt to the size of the convolutional receptive field required for targets at different scales， a depth-value-guided hierarchical dilated convolution module is proposed. At the same time， in order to effectively correlate and fuse the spatial position and attitude information of objects in different time domains， a spatiotemporal motion feature enhancement network is proposed. The effectiveness of the proposed MultiSegNet is verified on the large-scale point cloud driving scene datasets SemanticKITTI and nuScenes. The results show that the segmentation IoU of driving area， static object and dynamic object reaches 98%， 97% and 70%， respectively， which is better than that of mainstream networks， with real-time inference realized on edge computing devices.

Key words: autonomous driving, lidar, multi-task point cloud segmentation network, dynamic object segmentation

王海,李建国,蔡英凤,陈龙. 基于激光雷达点云的动态驾驶场景多任务分割网络[J]. 汽车工程, 2024, 46(9): 1608-1616.

Hai Wang,Jianguo Li,Yingfeng Cai,Long Chen. A LiDAR-Based Dynamic Driving Scene Multi-task Segmentation Network[J]. Automotive Engineering, 2024, 46(9): 1608-1616.

图/表 9

图1

图2

图3

图4

表 1

表 2

表3

表4

图5

参考文献 20

1	MILIOTO A， VIZZO I， BEHLEY J， et al. RangeNet++： fast and accurate lidar semantic segmentation［C］. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2019： 4213-4220.
2	CHEN X， LI S， MERSCH B， et al. Moving object segmentation in 3D LiDAR data： a learning-based approach exploiting sequential data［J］. IEEE Robotics and Automation Letters， 2021， 6（4）： 6529-6536.
3	蔡英凤，王海，陈小波，等. 驾驶辅助系统基于融合显著性的行人检测算法［J］. 汽车工程， 2015， 37（10）： 1215-1220.
	CAI Yingfeng， WANG Hai， CHEN Xiaobo， et al. A pedestrian detection algorithm based on fusion saliency for driving assistance systems ［J］. Automotive Engineering， 2015， 37 （10）： 1215-1220.
4	LANG A H， VORA S， CAESAR H， et al. Pointpillars： fast encoders for object detection from point clouds［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 12697-12705.
5	MATURANA D， SCHERER S. VoxNet： a 3D convolutional neural network for real-time object recognition［C］. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）. IEEE， 2015： 922-928.
6	WANG S， ZHU J， ZHANG R. Meta-RangeSeg： lidar sequence semantic segmentation using multiple feature aggregation［J］. IEEE Robotics and Automation Letters， 2022， 7（4）： 9739-9746.
7	KONG L， LIU Y， CHEN R， et al. Rethinking range view representation for lidar segmentation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2023： 228-240.
8	BEHLEY J， GARBADE M， MILIOTO A， et al. SemanticKITTI： a dataset for semantic scene understanding of lidar sequences［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 9297-9307.
9	CAESAR H， BANKITI V， LANG A H， et al. NuScenes： a multimodal dataset for autonomous driving［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 11621-11631.
10	FAN L， XIONG X， WANG F， et al. Rangedet： in defense of range view for lidar-based 3D object detection［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 2918-2927.
11	SUN J， DAI Y， ZHANG X， et al. Efficient spatial-temporal information fusion for LiDAR-based 3D moving object segmentation［J］. arXiv preprint arXiv：， 2022.
12	TANG H， LIU Z， ZHAO S， et al. Searching efficient 3D architectures with sparse point-voxel convolution［C］. European Conference on Computer Vision. Springer， 2020： 685-702.
13	BERMAN M， TRIKI A R， BLASCHKO M B. The lovász-softmax loss： a tractable surrogate for the optimization of the intersection-over-union measure in neural networks［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 4413-4421.
14	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［C］. Proceedings of the IEEE International Conference on Computer Vision， 2017： 2980-2988.
15	ZHANG Z. Improved adam optimizer for deep neural networks［C］. 2018 IEEE/ACM 26th International Symposium on Quality of Service （IWQoS）. IEEE， 2018： 1-2.
16	CORTINHAL T， TZELEPIS G， ERDAL AKSOY E. SalsaNext： fast， uncertainty-aware semantic segmentation of LiDAR point clouds［C］. International Symposium on Visual Computing. Springer， 2020： 207-222.
17	ZHU X， ZHOU H， WANG T， et al. Cylindrical and asymmetrical 3D convolution networks for lidar segmentation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 9939-9948.
18	ZHANG Y， ZHOU Z， DAVID P， et al. PolarNet： an improved grid representation for online lidar point clouds semantic segmentation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 9601-9610.
19	YAN X， GAO J， ZHENG C， et al. 2DPASS： 2D priors assisted semantic segmentation on lidar point clouds［C］. Computer Vision-ECCV 2022： 17th European Conference， Tel Aviv， Israel， October 23-27， 2022， Proceedings， Part XXVIII. Springer， 2022： 677-695.
20	CHOY C， GWAK J， SAVARESE S. 4D Spatio-Temporal ConvNets： Minkowski convolutional neural networks［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 3075-3084.

项目	Baseline	SANet	DLCM	VRInput	Free Space/%	Static Object/%	Moving Object/%	mIoU/%	性能提升/%
（a）	√				97.6	96.3	66.6	86.8
（b）	√	√			97.8	96.9	67.6	87.4	0.6
（c）	√		√		97.6	97.0	67.9	87.5	0.7
（d）	√			√	97.8	96.7	68.0	87.5	0.7
（e）	√	√	√		98.0	97.3	68.4	87.9	1.1
（f）	√	√	√	√	98.1	97.8	69.3	88.4	1.6

项目	Convolution Dilatation Rate	Free Space/%	Static Object/%	Moving Object/%	Mean IoU /%
（a）	2	97.6	96.5	67.6	87.2
（b）	3	98.1	96.8	68.4	87.7
（c）	4	97.7	96.6	67.7	87.3
（d）	5	97.6	96.7	67.5	87.2

方法	Modality	Free Space /%	Static Object /%	Moving Object /%	mIoU /%	推理耗时 /ms
Cylinder3D^［17］	Voxel-based	97.8	95.3	60.6	84.5	170
PolarNet^［18］	BEV-based	97.1	95.7	62.3	85.0	62.43
LMNet^［2］	RV-based	96.4	95.2	68.4	86.6	14.26
2DPASS^［19］	RGB&RV-based	98.0	96.5	61.8	85.4	62.85
4DMOS^［20］	RV-based	97.3	95.7	68.6	87.4	51.23
MotionSeg3D^［11］	RV-based	97.2	96.1	69.3	87.8	50.27
Ours	RV-based	98.4	97.5	70.6	88.8	46.34

方法	Modality	Free Space /%	Static Object /%	Moving Object /%	mIoU /%	推理耗时 /ms
Cylinder3D^［17］	Voxel-based	98.0	95.8	60.9	84.9	170
PolarNet^［18］	BEV-based	97.3	95.9	62.5	85.2	62.49
LMNet^［2］	RV-based	96.7	95.4	68.6	87.0	14.23
2DPASS^［19］	RGB&RV-based	98.4	96.9	61.8	85.7	62.87
4DMOS^［20］	RV-based	97.6	95.8	68.9	87.5	51.26
MotionSeg3D^［11］	RV-based	97.8	96.6	69.7	88.2	50.24
Ours	RV-based	98.8	98.0	70.8	89.2	46.37

[1]	金立生,张洪瑜,郭柏苍. 基于特征增稳的混合固态激光雷达目标检测[J]. 汽车工程, 2024, 46(6): 1015-1024.
[2]	伍文广,田双岳,张志勇,金斌,邱增华. 凸凹不平道路几何参数识别和模型重构方法研究[J]. 汽车工程, 2023, 45(2): 273-284.
[3]	李骏,万文星,郝三强,秦武,刘霏霏. 复杂路况下无人驾驶路径跟踪模型预测控制研究[J]. 汽车工程, 2022, 44(5): 664-674.
[4]	夏祥腾,王大方,曹江,赵刚,张京明. 基于稀疏卷积神经网络的车载激光雷达点云语义分割方法[J]. 汽车工程, 2022, 44(1): 26-35.
[5]	张晓龙,江昆,孙恺,刘茂林,邵振雷,肖鹏川,杨殿阁. 激光雷达的内参建模与点云修正方法[J]. 汽车工程, 2021, 43(8): 1228-1237.
[6]	王海,李洋,蔡英凤,孙恺,陈龙. 基于激光雷达的3D实时车辆跟踪[J]. 汽车工程, 2021, 43(7): 1013-1021.
[7]	龚章鹏,王国业,于是. 基于体素网络的道路场景多类目标识别算法[J]. 汽车工程, 2021, 43(4): 469-477.
[8]	罗玉涛,秦瀚. 基于稀疏彩色点云的自动驾驶汽车3D目标检测方法[J]. 汽车工程, 2021, 43(4): 492-500.
[9]	姜武华,周松林,王其东,陈无畏,陈佳佳. 基于自适应多特征融合的路沿检测与跟踪方法研究[J]. 汽车工程, 2021, 43(12): 1762-1770.
[10]	赵治国,王鹏,陈晓蓉,梁凯冲. 转向避撞工况下装载激光雷达车辆的障碍物跟踪[J]. 汽车工程, 2021, 43(11): 1611-1619.
[11]	石健,成前,金立生,胡耀光,蒋晓蓓,郭柏苍,王武宏. 面向视频结构化的细粒度车辆检测分类模型[J]. 汽车工程, 2021, 43(10): 1427-1434.
[12]	熊莹, 毛雪松. 无人驾驶车通过高速公路闸口区的方法研究[J]. 汽车工程, 2020, 42(9): 1263-1269.
[13]	陈龙, 解云鹏, 蔡英凤, 孙晓强, 滕成龙, 邹凯. 极限工况下无人驾驶车辆稳定跟踪控制^*[J]. 汽车工程, 2020, 42(8): 1016-1026.
[14]	蔡英凤, 邰康盛, 王海, 李祎承, 陈龙. 无人驾驶汽车周边车辆行为识别算法研究^*[J]. 汽车工程, 2020, 42(11): 1464-1472.
[15]	彭晓燕, 谢浩, 黄晶. 无人驾驶汽车局部路径规划算法研究^*[J]. 汽车工程, 2020, 42(1): 1-10.