基于空洞空间池化金字塔的自动驾驶图像语义分割方法

doi:10.19562/j.chinasae.qcgc.2022.12.003

Abstract

Abstract:

If a vehicle can accurately and quickly understand the semantics of people and vehicles on the road， it can guide the obstacle avoidance and path planning to a large extent. The existing semantic segmentation methods based on deep learning need a tradeoff between segmentation speed and segmentation accuracy. In this paper， based on the existing semantic segmentation network， the multi-scale semantic information of image can be obtained by adding an atrous spatial pyramid pooling structure after the reference network of feature extraction. Experimental results show that modules A_ASPP_1 and A_ASPP_2 proposed can effectively segment images of common people and various vehicles in automatic driving scenes. Compared with BiSeNet， two corresponding improved network structures have 2.1 and 1.2 percentage points higher mean intersection over union of training results respectively， though with a little lower segmentation speed.

Key words: semantic segmentation, autonomous driving, atrous spatial pyramid pooling

Dafang Wang,Lei Liu,Jiang Cao,Gang Zhao,Wenshuo Zhao,Wei Tang. Semantic Segmentation Method of Autonomous Driving Images Based on Atrous Spatial Pyramid Pooling[J].Automotive Engineering, 2022, 44(12): 1818-1824.

Figures/Tables 12

类型	输出尺寸
输入图片	$3 × 448 × 448$
3 × 3 × 3 Conv	$64 × 224 × 224$
4 × down	$3 × 112 × 112$
8 × down	$3 × 56 × 56$
16 × down	$256 × 28 × 28$
32 × down	$512 × 14 × 14$
ASPP	$512 × 14 × 14$
torch.mean	$512 × 1 × 1$
conv_bn_relu	$128 × 1 × 1$

类型	输出尺寸
ARM+conv_bn_relu+上采样	$128 × 14 × 14$
上采样+ conv_bn_relu+ARM	$128 × 28 × 28$
上采样+ conv_bn_relu+FFM	$128 × 56 × 56$
conv_bn_relu+上采样	$1 × 448 × 448$

参数	数值
图片输入尺寸	448 $×$ 448
损失函数	Cross entropy
优化器	SGD
Batch size	16
迭代轮数	1 300

References 16

1	孙璐，陈洪海. 最大类间方差法在图像分割中的应用［J］. 煤炭技术， 2008， 27（7）： 144-145.
	SUN Lu， CHEN Honghai. Application of maximum interclass variance method in image segmentation［J］. Coal Technology， 2008， 27（7）： 144-145.
2	黄长专，王彪，杨忠. 图像分割方法研究［J］. 计算机技术与发展， 2009， 19（6）： 76-79.
	HUANG Changzhuan， WANG Biao， YANG Zhong. Research on image segmentation method［J］. Computer Technology and Development，2009， 19（6）： 76-79.
3	杨晖，曲秀杰. 图像分割方法综述［J］. 电脑开发与应用， 2005， 18（3）： 21-23.
	YANG Hui， QU Xiujie. A review of image segmentation methods［J］. Computer Development and Application，2005， 18（3）： 21-23.
4	PAPADEAS I， TSOCHATZIDIS L， AMANATIADIS A， et al. Real-time semantic image segmentation with deep learning for autonomous driving： a survey［J］. Applied Sciences， 2021， 11（19）： 8802.
5	PASZKE A， CHAURASIA A， KIM S， et al. Enet： a deep neural network architecture for real-time semantic segmentation［J］. arXiv Preprint arXiv：， 2016.
6	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv Preprint arXiv：， 2014.
7	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］ .Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 770-778.
8	SZEGEDY C， LIU W， JIA Y， et al. Going deeper with convolutions［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2015： 1-9.
9	YU C， GAO C， WANG J， et al. Bisenet v2： bilateral network with guided aggregation for real-time semantic segmentation［J］. International Journal of Computer Vision， 2021， 129（11）： 3051-3068.
10	YAN S， WU C， WANG L， et al. Ddrnet： depth map denoising and refinement for consumer depth cameras using cascaded cnns［C］ .Proceedings of the European Conference on Computer Vision （ECCV）， 2018： 151-167.
11	JIN W D， XU J， CHENG M M， et al. Icnet： intra-saliency correlation network for co-saliency detection［J］. Advances in Neural Information Processing Systems， 2020， 33： 18749-18759.
12	ZHAO H， SHI J， QI X， et al. Pyramid scene parsing network［C］.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2017： 2881-2890.
13	WANG K， LIEW J H， ZOU Y， et al. Panet： few-shot image semantic segmentation with prototype alignment［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 9197-9206.
14	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. Deeplab： semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected crfs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 40（4）： 834-848.
15	YANG M， YU K， ZHANG C， et al. Denseaspp for semantic segmentation in street scenes［C］ .Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 3684-3692.
16	BOONMA P， SUZUKI J. BiSNET： a biologically-inspired middleware architecture for self-managing wireless sensor networks［J］. Computer Networks， 2007， 51（16）： 4599-4616.

[1]	SHEN Zhe, WANG Yi-Gang, YANG Zhi-Gang, LI Fang-Xu. [J]. , 2016, 38(12): 1440 -1445 .
[2]	ZENG Bi-Qiang, GAO Ji-Dong, PENG Wei, SUN Zhen-Dong. [J]. , 2016, 38(12): 1446 -1451 .
[3]	LIU Hui, WANG Xiao-Jie, XIANG Chang-Le. [J]. , 2016, 38(12): 1483 -1487 .
[4]	CHEN Wu-Wei, DENG Shu-Chao, HUANG He, XIE You-Hao. [J]. , 2016, 38(12): 1488 -1493 .
[5]	PENG Qian-Lei, GUI Liang-Jin, FAN Zi-Jie. [J]. , 2016, 38(12): 1500 -1507 .
[6]	DONG Zhu-Rong, ZHANG Xin, HU Song-Hua, QIU Hao. [J]. , 2017, 39(1): 79 -85 .
[7]	SHI Hai-Min, YU Xiao-Li, LU Guo-Dong, HUANG Yu-Qi, LIU Zhen-Tao, HUANG Rui. [J]. , 2017, 39(1): 102 -106 .
[8]	QIN Chao-Ju, YANG Zhen-Zhong, ZHANG Wei-Zheng, SONG Li-Ye, YUAN Yan-Peng. [J]. , 2017, 39(2): 133 -137 .
[9]	ZHANG Guan-Jun, ZHAO Xin-Feng, CAO Li-Bo. [J]. , 2017, 39(2): 150 -158 .
[10]	CAO Li-Bo, HU Yuan, YAN Ling-Bo, PENG Yu, SHI Xiang-南. [J]. , 2017, 39(2): 174 -180 .

语义类别	网络模块	mIoU/%
行人	A_ASPP_1	80.6
行人	A_ASPP_2	80.4
自行车	A_ASPP_1	39.4
自行车	A_ASPP_2	37.6
公交	A_ASPP_1	92.6
公交	A_ASPP_2	92.3
摩托车	A_ASPP_1	79.4
摩托车	A_ASPP_2	80.4
汽车	A_ASPP_1	86.7
汽车	A_ASPP_2	86.6

模型	Backbone	mIoU/%	mPA/%	fps
FCN	VGG16	62.2
SegNet	VGG16	66.9
BiSeNet	Resnet18	69.7	91.1	88
A_ASPP_1	Resnet18	71.8	91.9	80
A_ASPP_2	Resnet18	70.9	91.7	79

[1]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[2]	Yanli Ma, Qin Qin, Fangqi Dong, Yining Lou. Takeover Risk Assessment Model Based on Risk Field Theory Under Different Cognitive Secondary Tasks [J]. Automotive Engineering, 2024, 46(1): 9-17.
[3]	Zuliang Jin,Hanbing Wei,Liu Zheng,Lu Lou,Guofeng Zheng. Lightweight Semantic Segmentation Method Based on Local Window Cross Attention [J]. Automotive Engineering, 2023, 45(9): 1617-1625.
[4]	Weiguo Liu,Zhiyu Xiang,Weiping Liu,Daoxin Qi,Zixu Wang. Research on Vehicle Control Algorithm Based on Distributed Reinforcement Learning [J]. Automotive Engineering, 2023, 45(9): 1637-1645.
[5]	Ming Wang,Xiaolin Tang,Kai Yang,Guofa Li,Xiaosong Hu. A Motion Planning Method for Autonomous Vehicles Considering Prediction Risk [J]. Automotive Engineering, 2023, 45(8): 1362-1372.
[6]	Wenguang Wu,Shuangyue Tian,Zhiyong Zhang,Bin Zhang. Research on Semantic Segmentation of Uneven Features of Unpaved Road [J]. Automotive Engineering, 2023, 45(8): 1468-1478.
[7]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[8]	Lei Zhang, Keren Guan, Xiaolin Ding, Pengyu Guo, Zhenpo Wang, Fengchun Sun. Tire-Road Friction Estimation Method Based on Image Recognition and Dynamics Fusion [J]. Automotive Engineering, 2023, 45(7): 1222-1234.
[9]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[10]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[11]	Qingyang Huang,Xiaoping Jin,Yikang Zhang. Analysis of Drivers' Driving Posture Change Rule Under the Condition of Automatic Driving Level Improvement [J]. Automotive Engineering, 2023, 45(3): 382-392.
[12]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[13]	Pulei Xu,Yingfeng Cai,Yubo Lian,Xiaoqiang Sun,Hai Wang,Long Chen,Yilin Zhong. AFS/DYC Coordinated Control of Intelligent Vehicles Based on Improved Hierarchical Extensibility Theory [J]. Automotive Engineering, 2023, 45(1): 20-31.
[14]	Runhui Huang,Likun Hu,Mingfang Su,Daye Xu,Aoran Chen. Semantic Segmentation Method of LiDAR Point Cloud Based on 3D Conical Grid [J]. Automotive Engineering, 2022, 44(8): 1173-1182.
[15]	Zhenhai Gao,Xiangtong Yan,Fei Gao. A Decision-making Method for Longitudinal Autonomous Driving Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2022, 44(7): 969-975.

Semantic Segmentation Method of Autonomous Driving Images Based on Atrous Spatial Pyramid Pooling

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 16

Related Articles 15

Metrics

Comments

Recommended 10