地下矿道无人车可行驶区域检测算法

doi:10.19562/j.chinasae.qcgc.2024.11.008

Abstract

Abstract:

The detection of free space in underground mine tunnels is the key sensing technology for underground mining autonomous driving systems. However， the characteristics of low illumination and complex working environment inside the tunnels bring great challenges to this task. In view of this， in this paper an algorithm for detecting free space in underground mine tunnels is proposed. Firstly， a dual-branch feature extraction backbone network is proposed to solve the problem of difficulty in extracting image features caused by the degradation of tunnel details. Secondly， for the problem of incomplete detection of drivable areas in underground mining tunnels， an adaptive multi-scale atrous spatial pyramid pooling feature enhancement module is proposed. Finally， a dual-branch channel attention mechanism fusion module is developed to solve the problem of inaccurate boundary extraction in the underground mine tunnels. The experiments are conducted on a self-made dataset specifically designed for underground mine tunnels. The results show that the proposed algorithm surpasses other existing methods such as Deeplabv3+， UNet， DDRNet-23， and PIDNet， with an increase of 2.07， 2.39， 1.87， and 1.92 percentage points in terms of MIoU scores， and 1.78， 2.45， 1.84， and 1.86 in terms of mAcc scores， respectively. The effectiveness of the proposed algorithm has been validated through its successful application in real mine tunnel scenarios， particularly for underground mining autonomous driving vehicles.

Key words: autonomous driving, unmanned mine truck, free space detection, semantic segmentation

Zhijun Chen,Chaowei Wang,Chaozhong Wu,Chuang Qian,Huaizhu Wu,Guangjun Shen. Algorithm for Detecting Free Space in Underground Mine Tunnels for Autonomous Vehicles[J].Automotive Engineering, 2024, 46(11): 2017-2027.

Figures/Tables 22

References 27

1	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2015： 3431-3440.
2	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015： 18th International Conference， Munich， Germany， October 5-9， 2015， proceedings， part III 18. Springer International Publishing， 2015： 234-241.
3	ZHOU Z， RAHMAN SIDDIQUEE M M， TAJBAKHSH N， et al. Unet++： a nested u-net architecture for medical image segmentation［C］. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support： 4th International Workshop， DLMIA 2018， and 8th International Workshop， ML-CDS 2018， Held in Conjunction with MICCAI 2018， Granada， Spain， September 20， 2018， Proceedings 4. Springer International Publishing， 2018： 3-11.
4	HUANG H， LIN L， TONG R， et al. UNet 3+： a full-scale connected uNet for medical image segmentation［C］. ICASSP 2020-2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2020： 1055-1059.
5	POUDEL R P K， LIWICKI S， CIPOLLA R. Fast-SCNN： fast semantic segmentation network［J］. arXiv preprint arXiv：， 2019.
6	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. Semantic image segmentation with deep convolutional nets and fully connected crfs［J］. arXiv preprint arXiv：， 2014.
7	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. DeepLab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 40（4）： 834-848.
8	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation［J］. arXiv preprint arXiv：， 2017.
9	CHEN L C， ZHU Y， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］. Proceedings of the European Conference on Computer Vision （ECCV）， 2018： 801-818.
10	ZHAO H， SHI J， QI X， et al. Pyramid scene parsing network［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2017： 2881-2890.
11	STRUDEL R， GARCIA R， LAPTEV I， et al. Segmenter： transformer for semantic segmentation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 7262-7272.
12	CHENG B， MISRA I， SCHWING A G， et al. Masked-attention mask transformer for universal image segmentation［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2022： 1290-1299.
13	ZHAO W， RAO Y， LIU Z， et al. Unleashing text-to-image diffusion models for visual perception［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2023： 5729-5739.
14	PASZKE A， CHAURASIA A， KIM S， et al. ENet： a deep neural network architecture for real-time semantic segmentation［J］. arXiv preprint arXiv：， 2016.
15	HOWARD A G， ZHU M， CHEN B， et al. MobileNets： efficient convolutional neural networks for mobile vision applications［J］. arXiv preprint arXiv：， 2017.
16	SANDLER M， HOWARD A， ZHU M， et al. MobileNetV2： inverted residuals and linear bottlenecks［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 4510-4520.
17	HOWARD A， SANDLER M， CHU G， et al. Searching for mobileNetV3［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 1314-1324.
18	GAMAL M， SIAM M， ABDEL-RAZEK M. ShuffleSeg： real-time semantic segmentation network［J］. arXiv preprint arXiv：， 2018.
19	ZHANG X， ZHOU X， LIN M， et al. ShuffleNet： an extremely efficient convolutional neural network for mobile devices［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 6848-6856.
20	MA N， ZHANG X， ZHENG H T， et al. ShuffleNet V2： practical guidelines for efficient CNN architecture design［C］. Proceedings of the European Conference on Computer Vision （ECCV）， 2018： 116-131.
21	YU C， WANG J， PENG C， et al. BiseNet： bilateral segmentation network for real-time semantic segmentation［C］. Proceedings of the European Conference on Computer Vision （ECCV）， 2018： 325-341.
22	PAN H， HONG Y， SUN W， et al. Deep dual-resolution networks for real-time and accurate semantic segmentation of traffic scenes［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 24（3）： 3448-3460.
23	XU J， XIONG Z， BHATTACHARYYA S P. PIDNet： a real-time semantic segmentation network inspired by PID controllers［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2023： 19529-19539.
24	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 770-778.
25	HE K， ZHANG X， REN S， et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（9）： 1904-1916.
26	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 7132-7141.
27	SHRIVASTAVA A， GUPTA A， GIRSHICK R. Training region-based object detectors with online hard example mining［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 761-769.

[1]	ZHOU Wei, ZHANG Cheng-Ning, LI Jun-Qiu. [J]. , 2016, 38(12): 1407 -1414 .
[2]	HAO Han, WANG Si-南, LI Xiao, LIU Zong-Wei, ZHAO Fu-Quan. [J]. , 2017, 39(1): 1 -8 .
[3]	XU Cheng-Shan, JIANG Fa-Chao, SONG Sen-Nan, TIAN Guang-Yu. [J]. , 2017, 39(1): 9 -14 .
[4]	JIANG Ting, DAI Bing. [J]. , 2016, 38(12): 1459 -1466 .
[5]	ZHANG Yi-Bing, 吕Jian-Jun , YUAN Guang-Hui. [J]. , 2016, 38(12): 1467 -1470 .
[6]	LIU Hui, WANG Xiao-Jie, XIANG Chang-Le. [J]. , 2016, 38(12): 1483 -1487 .
[7]	CHEN Wu-Wei, DENG Shu-Chao, HUANG He, XIE You-Hao. [J]. , 2016, 38(12): 1488 -1493 .
[8]	PENG Qian-Lei, GUI Liang-Jin, FAN Zi-Jie. [J]. , 2016, 38(12): 1500 -1507 .
[9]	LI Ling, MA Li, MOU Yu, XU Chao, LI Wen-Ru, SHI Shu-Ming. [J]. , 2016, 38(12): 1508 -1514 .
[10]	GAO Ji-Dong, GAO Bo-Lin, XIE Shu-Gang. [J]. , 2016, 38(12): 1515 -1520 .

层名	输出特征图尺寸（H×W×C）		网络层主要结构
L1	128×128×128		Inverted Residual Block×1，stride=2
L2	64×64×256		Inverted Residual Block×1，stride=2
L3	32×32×64		Inverted Residual Block×4
L4	16×16×128	32×32×64	bottleneck block×2	大核bottleneck block×2
L4	DBFAM		DBFAM模块×1
L5	8×8×256	32×32×96	bottleneck block×2	大核bottleneck block×2
L5	DBFAM		DBFAM模块×1
L6	4×4×384	32×32×128	bottleneck block×2	bottleneck block×2
	SAPPM-L	SAPPM-H	SAPPM-L×1	SAPPM-H×1
	DBFAM		DBFAM

方法	池化核大小	填充	步长
SAPPM-L	［1，1，3，5，global］	［0，1，2，4，0］	［1，1，2，4，global］
SAPPM-H	［1，5，9，13，global］	［0，2，4，6，0］	［1，1，2，4，global］

方法	MIoU（normal）	mAcc（normal）
SegNet	87.31	89.65
Fast-scnn	93.21	95.65
Deeplabv3+-r18+ FCN	93.97	96.26
Deeplabv3+-50+ FCN	94.03	96.31
MobileNetv3+ASPP	93.57	95.89
UNet + FCN	93.71	95.66
DDRNet-23	94.23	96.25
PIDNet-S	94.18	96.23
DDMNet（ours）	96.10	98.09

方法	MIoU（hard）	mAcc（hard）
SegNet	82.11	84.81
Fast-scnn	86.41	89.23
Deeplabv3+-r18+ FCN	89.88	91.86
Deeplabv3+-50+ FCN	90.83	92.15
MobileNetv3+ASPP	89.14	91.29
UNet + FCN	89.67	91.85
DDRNet-23	91.32	93.10
PIDNet-S	91.04	93.14
DDMNet（ours）	93.30	95.22

工况	MIoU（hard）	mAcc（hard）
长直道	96.94	98.64
弯道	95.68	97.74
斜坡	96.17	96.21
会车区域	95.30	95.42
上料卸料区	93.78	94.83

Algorithm for Detecting Free Space in Underground Mine Tunnels for Autonomous Vehicles

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 22

References 27

Related Articles 15

Metrics

Comments

Recommended 10

方法	Float	Param	FPS
SegNet	2.860G	29.5M	4.22
Fast-scnn	0.232G	1.4M	11.84
Deeplabv3+-r18+ FCN	13.565G	12.3M	10.71
Deeplabv3+-50+ FCN	44.121G	41.2M	5.39
MobileNetv3+ASPP	3.487G	4.4M	14.67
UNet + FCN	50.738G	30.0M	9.33
DDRNet-23	1.14G	20.1M	11.72
PIDNet-S	1.821G	7.6M	8.08
DDMNet（ours）	1.731G	14.6M	12.24

实验方法	MDFNet	SAPPM	DBFAM	MIoU/%	FPS
DDRNet				94.23	11.72
MDFNet	√			95.35	12.40
MDFNet+SAPPM	√	√		95.70	12.32
MDFNet+DBFAM	√		√	95.59	12.20
MDFNet+SAPPM+DBFAM	√	√	√	96.10	12.24

[1]	Daofei Li,Hao Pan. Application of Scenario Complexity Evaluation in Trajectory Prediction and Automated Driving Decision-Making [J]. Automotive Engineering, 2024, 46(9): 1556-1563.
[2]	Hai Wang,Jianguo Li,Yingfeng Cai,Long Chen. A LiDAR-Based Dynamic Driving Scene Multi-task Segmentation Network [J]. Automotive Engineering, 2024, 46(9): 1608-1616.
[3]	Jianan Zhang,Zhaozheng Hu,Jie Meng,Huahua Hu,Jie Zuo. Distributed Simulation Platform Architecture and Application of Autonomous Driving for Vehicle-Road-Map Collaboration [J]. Automotive Engineering, 2024, 46(8): 1335-1345.
[4]	Le Tao,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Point Cloud for Autonomous Driving Scenarios [J]. Automotive Engineering, 2024, 46(7): 1208-1218.
[5]	Linhui Li,Yifan Fu,Ting Wang,Xuecheng Wang,Jing Lian. Trajectory Prediction Method Enhanced by Self-supervised Pretraining [J]. Automotive Engineering, 2024, 46(7): 1219-1227.
[6]	Hai Wang,Yuxuan Ding,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-class Multi-target Tracking Algorithm Combining Motion Speed and Appearance Features in Driving Scenarios [J]. Automotive Engineering, 2024, 46(6): 956-964.
[7]	Jing Huang,Xiangzhen Liu,Xiaoyang Deng,Ran Chen. Research on Intelligent Vehicle Trajectory Planning Based on Multimodal Trajectory Prediction [J]. Automotive Engineering, 2024, 46(6): 965-974.
[8]	Fuxing Yao,Chao Sun,Yungang Lan,Bing Lu,Bo Wang,Haiyang Yu. A Lane Change Decision Method for Intelligent Connected Vehicles Based on Mixture of Expert Model [J]. Automotive Engineering, 2024, 46(5): 882-892.
[9]	Mengfan Li,Zhongxiang Feng,Weihua Zhang,Jingyu Li. Study on Driver's Visual Transfer Characteristics During the Takeover Process of Human-Computer Co-driving Mode [J]. Automotive Engineering, 2024, 46(5): 795-804.
[10]	Ting Chikit,Yafei Wang,Yichen Zhang,Mingyu Wu,Yile Wang. Energy-Saving Planning Method for Autonomous Driving Mining Trucks Based on Composite Dynamic Sampling [J]. Automotive Engineering, 2024, 46(4): 588-595.
[11]	Yiwei Zhou,Mo Xia,Bing Zhu. Multimodal Vehicle Trajectory Prediction Methods Considering Multiple Traffic Participants in Urban Road Scenarios [J]. Automotive Engineering, 2024, 46(3): 396-406.
[12]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[13]	Xiaolin Tang,Lu Gan,Guofa Li,Keqiang Li,Wenbo Chu. Large Model Alignment Technology for Autonomous Driving: A Review [J]. Automotive Engineering, 2024, 46(11): 1937-1951.
[14]	Song Gao,Jianglin Zhou,Bolin Gao,Jian Lu,He Wang,Yueyun Xu. Pedestrian Trajectory Prediction Method Based on Multi-information Fusion Network [J]. Automotive Engineering, 2024, 46(11): 1973-1982.
[15]	Hailun Zhang,Guangwei Wang,Qingwen Meng,Qing Xu,Jianqiang Wang,Keqiang Li. An Online Semi-supervised Hybrid Approach for Vehicle Behavior Perception at Intersections [J]. Automotive Engineering, 2024, 46(11): 1993-2004.