基于特征融合的多层次多模态目标检测

doi:10.19562/j.chinasae.qcgc.2021.11.005

Abstract

Abstract:

For the problems of low robustness of the environment perception and identification difficulty of small targets of autonomous driving in complex environment， a multi-level and multi-modal fusion method based on feature fusion is proposed in this paper. Firstly， the image and point cloud modal information are mapped to the same dimension， and the hierarchical features of different size targets are extracted. On this basis， the multi-modal multi-level feature fusion is carried out. Then， six comparative experiments are designed to verify the effectiveness of each module. Finally， the Waymo data set and NIO real car data are used for training and testing. The test results show that the detection MAP value of the network is improved by 23.1% compared with that of YOLO V3.

Key words: autonomous driving, environmental perception, hierarchical feature fusion, multimodal fusion, small target detection

Teng Cheng,Lei Sun,Dengchao Hou,Qin Shi,Junning Zhang,Jiong Chen,He Huang. Multi-level and Multi-modal Target Detection Based on Feature Fusion[J].Automotive Engineering, 2021, 43(11): 1602-1610.

Figures/Tables 20

References 16

1	王宝锋，齐志权，马国成，等. 一种基于雷达和机器视觉信息融合的车辆识别方法［J］. 汽车工程， 2015， 37（6）：674-678，736.
	WANG B F， QI Z Q， MA G C， et al. Vehicle detection based on information fusion of radar and machine vision［J］. Automotive Engineering， 2015， 37（6）：674-678，736.
2	张燕咏，张莎，张昱，等. 基于多模态融合的自动驾驶感知及计算［J］. 计算机研究与发展， 2020（9）：1781-1799.
	ZHANG Y Y， ZHANG S， ZHANG Y， et al. Multi-modality fusion perception and computing in autonomous driving［J］. Journal of Computer Research and Development， 2020（9）：1781-1799.
3	张新钰，邹镇洪，李志伟，等. 面向自动驾驶目标检测的深度多模态融合技术［J］. 智能系统学报，2020，15（4）：758-771.
	ZHANG X Y， ZOU Z H， LI Z W， et al. Deep multi-modal fusion in object detection for autonomous driving［J］. Caai Transactions on Intelligent Systems，2020，15（4）：758-771.
4	郭磊，李克强，王建强，等. 一种基于特征的车辆检测方法［J］. 汽车工程，2006，28（11）：1031-1035.
	GUO L， LI K Q， WANG J Q， et al. A feature-based vehicle detection method［J］. Automotive Engineering， 2006，28（11）：1031-1035.
5	常昕，陈晓冬，张佳琛，等.基于激光雷达和相机信息融合的目标检测及跟踪［J］.光电工程，2019，46（7）：91-101.
	CHANG X， CHEN X D， ZHANG J C， et al. An object detection and tracking algorithm based on LiDAR and camera information fusion［J］. Opto-Electronic Engineering，2019，46（7）：91-101.
6	孙宁，秦洪懋，张利，等.基于多传感器信息融合的车辆目标识别方法［J］.汽车工程，2017，39（11）：1310-1315.
	SUN N， QIN H M， ZHANG L， et al. Vehicle target recognition based on multi-sensor information fusion［J］. Automotive Engineering， 2017，39（11）：1310-1315.
7	吴愿，薛培林，殷国栋，等. 基于特征融合的双模态低辨识度目标识别［J］. 中国机械工程，2021，32（10）：1205-1212，1221.
	WU Y， XUE P L， YIN G D， et al. Low-identification dual target recognition based on feature fusion［J］. China Mechanical Engineering，2021，32（10）：1205-1212，1221.
8	KIM T ， GHOSH J . Robust detection of non-motorized road users using deep learning on optical and LIDAR data［C］. 2016 IEEE 19th International Conference on Intelligent Transportation Systems （ITSC）. IEEE， 2016.
9	金彦亮，葛飞扬. 基于FPN融合的全时段自动驾驶中行人检测［J］. 工业控制计算机，2020，33（6）：59-60，63.
	JIN Y L， GE F Y. Pedestrian detection in full－time automatic driving based on fpn fusion architecture［J］. Industrial Control Computer，2020，33（6）：59-60，63.
10	LIN T Y， DOLLAR P， GIRSHICK R ， et al. Feature pyramid networks for object detection［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE Computer Society， 2017.
11	谢文亮，朱丹，佟新鑫. 一种基于视觉注意的小目标检测方法［J］. 计算机工程与应用， 2013， 49（12）：125-128.
	XIE W L， ZHU D， TONG X X. Small target detection method based on visual attention［J］. Computer Engineering and Applications， 2013， 49（12）：125-128.
12	DOU J， XUE J， FANG J. SEG-VoxelNet for 3D vehicle detection from RGB and LiDAR data［C］. 2019 International Conference on Robotics and Automation （ICRA）. 2019.
13	廖家才，曹立波，夏家豪，等. 基于核相关滤波和运动模型的多目标轨迹跟踪［J］. 汽车工程，2019，41（10）：1179-1188.
	LIAO J C， CAO L B， XIA J H， et al. Multi-target trajectory tracking based on kernelized correlation filtering and motion mode［J］. Automotive Engineering， 2019，41（10）：1179-1188.
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2016.
15	SUN C， SHRIVASTAVA A， SINGH S， et al. Revisiting unreasonable effectiveness of data in deep learning era［C］. IEEE Computer Society. IEEE Computer Society， 2017.
16	HE Y， XU D，WU L， et al. LFFD： a light and fast face detector for edge devices［J］.arXiv Preprint，arXiv：，2019.

[1]	XU Cheng-Shan, JIANG Fa-Chao, SONG Sen-Nan, TIAN Guang-Yu. [J]. , 2017, 39(1): 9 -14 .
[2]	ZHANG Yi-Bing, 吕Jian-Jun , YUAN Guang-Hui. [J]. , 2016, 38(12): 1467 -1470 .
[3]	JIA Ji-De, WU Chun-Zhi, JIA Xiang-Yu, REN Gang, HAN Jia-Jia. [J]. , 2017, 39(1): 97 -101 .
[4]	RONG Bing, XIAO Pan, ZHOU Jian-Wen, LIU Yi-Lu. [J]. , 2017, 39(2): 214 -219 .
[5]	WANG Zhen, CUI Ya-Hui-.Liu-Kai, XU Lin. [J]. , 2017, 39(3): 304 -310 .
[6]	GUAN Xin, XU Liang, GUO Rui, WANG Bo, SONG Hai-Yan. [J]. , 2017, 39(3): 311 -316 .
[7]	JIN Geng-Ri, GUAN Xin, ZHAN Jun, GUO Rui. [J]. , 2017, 39(3): 364 -368 .
[8]	XIONG Xing-Wang, GAO Jun-Hua, ZHOU Tao, YU Jin-Tao. [J]. , 2017, 39(4): 381 -385 .
[9]	CAO Li-Feng, ZHOU Shou-Qin, XIE Xiao-Peng. [J]. , 2017, 39(4): 401 -406 .
[10]	Chen Wenqiang, Xiong Hui, Li Keqiang, Li Xiaofei& Zhang Dezhao. Concurrent Pedestrian and Cyclist Detection Based on Deep Neural Networks[J]. , 2018, 40(6): 726 .

项目	AP/汽车	AP/行人	AP/自行车手	MAP
YOLOV3	57.93%	43.14%	25.11%	42.06%
调整先验框	55.65%	43.96%	30.61%	43.40%
增大尺度	62.44%	46.25%	30.25%	46.31%
数据增强	63.72%	48.41%	31.03%	47.72%
扩大分辨率	65.91%	50.08%	33.16%	49.72%
多模态融合	67.32%	53.74%	34.29%	51.79%

[1]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[2]	Yanli Ma, Qin Qin, Fangqi Dong, Yining Lou. Takeover Risk Assessment Model Based on Risk Field Theory Under Different Cognitive Secondary Tasks [J]. Automotive Engineering, 2024, 46(1): 9-17.
[3]	Weiguo Liu,Zhiyu Xiang,Weiping Liu,Daoxin Qi,Zixu Wang. Research on Vehicle Control Algorithm Based on Distributed Reinforcement Learning [J]. Automotive Engineering, 2023, 45(9): 1637-1645.
[4]	Ming Wang,Xiaolin Tang,Kai Yang,Guofa Li,Xiaosong Hu. A Motion Planning Method for Autonomous Vehicles Considering Prediction Risk [J]. Automotive Engineering, 2023, 45(8): 1362-1372.
[5]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[6]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[7]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[8]	Qingyang Huang,Xiaoping Jin,Yikang Zhang. Analysis of Drivers' Driving Posture Change Rule Under the Condition of Automatic Driving Level Improvement [J]. Automotive Engineering, 2023, 45(3): 382-392.
[9]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.
[10]	Teng Cheng,Dengchao Hou,Qiang Zhang,Qin Shi,Ligang Guo. Research on Multi-modal Late Fusion Framework Based on D-S Evidence Theory [J]. Automotive Engineering, 2023, 45(10): 1815-1823.
[11]	Pulei Xu,Yingfeng Cai,Yubo Lian,Xiaoqiang Sun,Hai Wang,Long Chen,Yilin Zhong. AFS/DYC Coordinated Control of Intelligent Vehicles Based on Improved Hierarchical Extensibility Theory [J]. Automotive Engineering, 2023, 45(1): 20-31.
[12]	Runhui Huang,Likun Hu,Mingfang Su,Daye Xu,Aoran Chen. Semantic Segmentation Method of LiDAR Point Cloud Based on 3D Conical Grid [J]. Automotive Engineering, 2022, 44(8): 1173-1182.
[13]	Zhenhai Gao,Xiangtong Yan,Fei Gao. A Decision-making Method for Longitudinal Autonomous Driving Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2022, 44(7): 969-975.
[14]	Dafang Wang,Jingdong Du,Jiang Cao,Mei Zhang,Gang Zhao. Research on Style Transfer Network for Autonomous Driving Data Generation [J]. Automotive Engineering, 2022, 44(5): 684-690.
[15]	Jie Hu,Minchao Zhang,Wencai Xu,Ruinan Chen,Xinkai Zhong,Linglei Zhu. Parallel Parking Trajectory Planning for Autonomous Vehicles [J]. Automotive Engineering, 2022, 44(3): 330-339.

Multi-level and Multi-modal Target Detection Based on Feature Fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 16

Related Articles 15

Metrics

Comments

Recommended 10