面向自动驾驶道路场景中异常案例的多模态数据挖掘算法

doi:10.19562/j.chinasae.qcgc.2024.07.011

Abstract

Abstract:

The development of visual perception technology based on deep learning is beneficial for the advancement of environment perception technology in automatic driving systems. However， for corner cases of autonomous driving scenario， there are still some problems in the current perception model. This is because the ability of the perception model based on deep learning depends on the distribution of the training dataset. Especially when categories in the driving scene never appear in the training set， the perception system is often fragile. Therefore， identifying unknown categories and extreme scenarios remains a challenge for the safety of automatic driving perception technology. From the perspective of processing data sets， in this paper a novel multimodal automatic corner case mining process called "Corner Case Mining Pipeline （CCMP）" is proposed. In order to verify the effectiveness of "CCMP"， the concern case subset "Waymo-Anomaly" on the basis of Waymo open datasets is established， with a total of 3 200 images， each of which will contain the corner case scene defined in the text. Then based on the private data set Waymo-Anomaly， it is proved that the recall rate of "CCMP" corner case mining can reach 91.7%. In addition， the effectiveness of object detectors targeting long-tailed distributions in datasets containing corner case is experimentally verified. Ultimately， the authenticity of the automatic driving perception model in the real world is expected to improve from the perspective of datasets processing.

Key words: autonomous vehicles, deep learning, object detection, corner case

Hai Wang,Guirong Zhang,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-modal Data Mining Algorithm for Corner Case of Automatic Driving Road Scene[J].Automotive Engineering, 2024, 46(7): 1239-1248.

Figures/Tables 12

References 33

1	范丽丽，赵宏伟，赵浩宇，等. 基于深度卷积神经网络的目标检测研究综述［J］. 光学精密工程， 2020， 28： 1152-1164.
	FAN Lili， ZHAO Hongwei， ZHAO Haoyu， et al. Survey of target detection based ondeep convolutional neural networks［J］. Optics and Precision Engineering， 2020，28： 1152-1164.
2	王海，蔡柏湘，蔡英凤，等. 基于语义分割网络的路面积水与湿滑区域检测［J］. 汽车工程， 2021， 43（4）： 485-491.
	WANG Hai， CAI Baixiang， CAI Yingfeng， et al. Detection of water⁃covered and wet areas on road pavement based on semantic segmentation network［J］. Automotive Engineering， 2021， 43（4）： 485-491.
3	WONG K， WANG S， REN M， et al. Identifying unknown instances for autonomous driving［C］. Conference on Robot Learning. PMLR， 2020： 384-393.
4	DU X， WANG Z， CAI M， et al. VOS： learning what you don't know by virtual outlier synthesis［J］. arXiv preprint arXiv：， 2022.
5	WANG H， CHEN Y， CAI Y， et al. SFNet-N： an improved SFNet algorithm for semantic segmentation of low-light autonomous driving road scenes［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（11）： 21405-21417.
6	KENDALL A， BADRINARAYANAN V， CIPOLLA R. Bayesian SegNet： model uncertainty in deep convolutional encoder-decoder architectures for scene understanding［J］. arXiv preprint arXiv：， 2015.
7	GRCIC M， BEVANDIC P， KALAFATIC Z， et al. Dense anomaly detection by robust learning on synthetic negative data［J］. arXiv preprint arXiv：， 2021.
8	OHGUSHI T， HORIGUCHI K， YAMANAKA M. Road obstacle detection method based on an autoencoder with semantic segmentation［C］. Proceedings of the Asian Conference on Computer Vision， 2020.
9	VOJIR T， ŠIPKA T， ALJUNDI R， et al. Road anomaly detection by partial image reconstruction with segmentation coupling［C］.Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 15651-15660.
10	YU F， CHEN H， WANG X， et al. BDD100K： a diverse driving dataset for heterogeneous multitask learning［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 2636-2645.
11	SUN P， KRETZSCHMAR H， DOTIWALLA X， et al. Scalability in perception for autonomous driving： Waymo open dataset［C］.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 2446-2454.
12	ZHANG Y， KANG B， HOOI B， et al. Deep long-tailed learning： a survey［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2023.
13	CHAN R， ROTTMANN M， GOTTSCHALK H. Entropy maximization and meta classification for out-of-distribution detection in semantic segmentation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 5128-5137.
14	BAUR C， WIESTLER B， ALBARQOUNI S， et al. Deep autoencoding models for unsupervised anomaly segmentation in brain MR images［C］. Brainlesion： Glioma， Multiple Sclerosis， Stroke and Traumatic Brain Injuries： 4th International Workshop， BrainLes 2018， Held in Conjunction with MICCAI 2018， Granada， Spain， September 16， 2018， Revised Selected Papers， Part I 4. Springer International Publishing， 2019： 161-169.
15	BREITENSTEIN J， TERMÖHLEN J A， LIPINSKI D， et al. Corner cases for visual perception in automated driving： some guidance on detection approaches［J］. arXiv preprint arXiv：， 2021.
16	LI B， YAO Y， TAN J， et al. Equalized focal loss for dense long-tailed object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2022： 6990-6999.
17	WANG T， ZHU Y， ZHAO C， et al. Adaptive class suppression loss for long-tail object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 3103-3112.
18	CONTRIBUTORS M M S. MMSegmentation： openMMLab semantic segmentation toolbox and benchmark［EB/OL］. http：//github.com/open-mmlab/mmsegmentation，2020.
19	HE K， ZHANG X， REN S， et al. Identity mappings in deep residual networks［C］. Computer Vision-ECCV 2016： 14th European Conference， Amsterdam， The Netherlands， October 11-14， 2016， Proceedings， Part IV 14. Springer International Publishing， 2016： 630-645.
20	WANG C， DU P， WU H， et al. A cucumber leaf disease severity classification method based on the fusion of DeepLabV3+ and U-Net［J］. Computers and Electronics in Agriculture， 2021， 189： 106373.
21	FISCHLER M A， BOLLES R C. Random sample consensus： a paradigm for model fitting with applications to image analysis and automated cartography［J］. Communications of the ACM， 1981， 24（6）： 381-395.
22	MOORE T， STOUCH D. A generalized extended Kalman filter implementation for the robot operating system［C］. Intelligent Autonomous Systems 13： Proceedings of the 13th International Conference IAS-13. Springer International Publishing， 2016： 335-348.
23	CAI Z， VASCONCELOS N. Cascade R-CNN： delving into high quality object detection［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 6154-6162.
24	WOO S， DEBNATH S， HU R， et al. Convnext v2： co-designing and scaling convnets with masked autoencoders［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2023： 16133-16142.
25	GE Z， LIU S， WANG F， et al. YOLOX： exceeding YOLO series in 2021［J］. arXiv preprint arXiv：， 2021.
26	BIFFI L J， MITISHITA E， LIESENBERG V， et al. ATSS deep learning-based approach to detect apple fruits［J］. Remote Sensing， 2020， 13（1）： 54.
27	SUN P， ZHANG R， JIANG Y， et al. Sparse R-CNN： end-to-end object detection with learnable proposals［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 14454-14463.
28	GUPTA A， NARAYAN S， JOSEPH K J， et al. OW-DETR： open-world detection transformer［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2022： 9235-9244.
29	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： common objects in context［C］. Computer Vision-ECCV 2014： 13th European Conference， Zurich， Switzerland， September 6-12， 2014， Proceedings， Part V 13. Springer International Publishing， 2014： 740-755.
30	CHEN K， WANG J， PANG J， et al. MMDetection： open mmlab detection toolbox and bench mark［J］. arXiv preprint arXiv：， 2019.
31	LI K， CHEN K， WANG H， et al. CODA： a real-world road corner case dataset for object detection in autonomous driving［C］. European Conference on Computer Vision. Cham： Springer Nature Switzerland， 2022： 406-423.
32	ZHANG H， CHANG H， MA B， et al. Cascade RetinaNet： maintaining consistency for single-stage object detection［J］. arXiv preprint arXiv：， 2019.
33	GIRSHICK R. Fast R-CNN［C］. Proceedings of the IEEE International Conference on Computer Vision， 2015： 1440-1448.

算法	数据集	车辆（AP）	行人（AP）	骑行人（AP）	所有类别（AP）	未知类（AR₅₀）
YOLOX	Waymo	58.2	43.8	34.0	41.3
RetinaNet ^［32］		56.6	42.1	33.7	40.5
Faster R-CNN ^［33］		59.5	44.9	36.1	42.0
Cascade R-CNN		63.7	46.3	40.5	43.9
Sparse RCNN		56.1	41.0	30.6	39.9
DDetr		60.1	42.3	35.3	42.7
YOLOX	Waymo-Anomaly	58.3	43.8	34.0	41.3	34.5
RetinaNet		56.9	42.1	33.6	40.5	30.1
Faster R-CNN		62.1	46.0	36.2	43.1	29.8
Cascade R-CNN		66.5	47.8	40.5	45.2	29.5
Sparse RCNN		56.8	41.3	30.6	40.0	28.1
DDetr		61.5	42.5	35.9	43.8	35.0

Model	AP_r	AP_c	AP_f	mAP
RetinaNet	13.3	33.6	58.6	42.3
Faster R-CNN	12.1	36.0	60.1	43.9
RetinaNet+EKF	20.3	34.9	58.8	44.1
Faster R-CNN + EFL	16.1	36.7	60.1	44.2
Faster R-CNN + ACSL	18.8	37.2	60.9	44.7

[1]	Le Tao,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Point Cloud for Autonomous Driving Scenarios [J]. Automotive Engineering, 2024, 46(7): 1208-1218.
[2]	Lisheng Jin,Hongyu Zhang,Baicang Guo. Semi Solid-State LiDAR Object Detection Algorithm Enhanced by Feature Stability Enhancement [J]. Automotive Engineering, 2024, 46(6): 1015-1024.
[3]	Guodong Wang,Li Liu,Yu Meng,Haiping Du,Guoxing Bai,Qing Gu. Research on Collision Avoidance Limit of Autonomous Vehicles [J]. Automotive Engineering, 2024, 46(6): 985-994.
[4]	Hongyi Liang,Jikai Chen,Wanli Liu,Fengchong Lan,Bingda Mo,Jiqing Chen. Prediction of the Remaining Useful Life of Real Vehicle Lithium Batteries by Fusion of K-means Clustering and Sequence Decomposition [J]. Automotive Engineering, 2024, 46(4): 634-642.
[5]	Xinke Fu,Yingfeng Cai,Long Chen,Hai Wang,Qingchao Liu. Decision-Making for Autonomous Driving in Uncertain Environment [J]. Automotive Engineering, 2024, 46(2): 211-221.
[6]	Ze Gao, Zunkang Chu, Jiasheng Shi, Fu Lin, Weixiong Rao, Haiyan Yu. Research on Fast Prediction Method of Stress Field of Automotive Parts Based on Graph Network [J]. Automotive Engineering, 2024, 46(1): 170-178.
[7]	Weiguo Liu,Zhiyu Xiang,Rui Liu,Guodong Li,Zixu Wang. Research on End-to-End Vehicle Motion Planning Method Based on Deep Learning [J]. Automotive Engineering, 2023, 45(8): 1343-1352.
[8]	Cheng Lin, Bowen Wang, Lü Peiyuan, Xinle Gong, Xiao Yu. Research on Motion Planning and Cooperative Control for Autonomous Vehicles with Lane Change Gaming Maneuvers Under the Curved Road [J]. Automotive Engineering, 2023, 45(7): 1099-1111.
[9]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[10]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[11]	Yanyan Chen,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Efficient Automatic Driving Instance Segmentation Method Based on Detection [J]. Automotive Engineering, 2023, 45(4): 541-550.
[12]	Fengchong Lan,Jikai Chen,Jiqing Chen,Xinping Jiang,Zihan Li,Wei Pan. Research on Lithium Battery Remaining Useful Life Prediction Method Driven by Real Vehicle Data [J]. Automotive Engineering, 2023, 45(2): 175-182.
[13]	Lü Ying,Xu Qi,Qiuzheng Liu,Xinyu Wang,Guoying Chen. Path Tracking Control Method with Steering Lag for Autonomous Vehicles [J]. Automotive Engineering, 2023, 45(12): 2234-2241.
[14]	Linhui Li,Xinliang Zhang,Yifan Fu,Jing Lian,Jiaxu Ma. Research on Visible Light and Infrared Post-Fusion Detection Based on TC-YOLOv7 Algorithm [J]. Automotive Engineering, 2023, 45(12): 2280-2290.
[15]	Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels [J]. Automotive Engineering, 2023, 45(11): 2082-2091.

A Multi-modal Data Mining Algorithm for Corner Case of Automatic Driving Road Scene

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 33

Related Articles 15

Metrics

Comments

Recommended 0