基于特征和标签联合分布匹配的智能驾驶跨域自适应目标检测

doi:10.19562/j.chinasae.qcgc.2023.11.009

Abstract

Abstract:

Current cross-domain adaptive object detection methods primarily focus on reducing domain shift by learning domain-invariant features through feature distribution alignment. However， they often overlook label distribution shift issues caused by variations in object combinations and class imbalances in real-world scenarios， resulting in poor generalization performance. To address this， this paper proposes a novel domain adaptive object detection algorithm that simultaneously aligns domain distributions at both feature and label levels. Firstly， an image-level classification embedding module is introduced to enhance the transferability and discriminability of global features through contrastive learning. Next， a class-level distribution alignment module is presented to achieve inter-domain multimodal structure alignment through multi-level feature alignment. Finally， an enhanced consistency regularization module is proposed to achieve cross-domain label distribution alignment through region-based consistency regularization. Experimental results across multiple datasets demonstrate that the proposed domain alignment algorithm effectively improves semantic consistency both before and after cross-domain data adaptation. This provides a valuable solution for the effective deployment of autonomous vehicles in cross-domain scenarios.

Key words: autonomous vehicles, autonomous driving, object detection, domain adaptation

Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels[J].Automotive Engineering, 2023, 45(11): 2082-2091.

Figures/Tables 11

References 42

1	陈文强，熊辉，李克强，等. 基于深度神经网络的行人及骑车人联合检测［J］. 汽车工程， 2018， 40（6）： 726-733.
	CHEN W Q， XIONG L， LI K Q， et al. Joint detection of pedestrians and cyclists based on deep neural network［J］. Automotive Engineering， 2018， 40（6）： 726-733.
2	冯润泽，江昆，于伟光，等. 基于两阶段分类算法的中国交通标志牌识别［J］. 汽车工程， 2023，44（3）：434-441.
	FENG R Z， JIANG K， YU W G， et al. Chinese traffic sign recognition based on two-stage classification algorithm［J］. Automotive Engineering， 2023，44（3）：434-441.
3	谢德胜，徐友春，陆峰，等. 基于多传感器信息融合的3维目标实时检测［J］. 汽车工程， 2022， 44（3）： 11.
	XIE D S， XU Y C， LU F， et al. Real-time detection of 3D targets based on multi-sensor information fusion［J］. Automotive Engineering， 2022， 44（3）： 11.
4	CHEN Y， LI W， SAKARIDIS C， et al. Domain adaptive faster R-CNN for object detection in the wild［C］. Proceedings of the IEEE conference on computer vision and pattern recognition， 2018： 3339-3348.
5	王大方，杜京东，曹江，等. 面向自动驾驶数据生成的风格迁移网络研究［J］. 汽车工程， 2022， 44（5）： 684-690.
	WANG D F， DU J D， CAO J， et al. Research on style transfer network for autonomous driving data generation［J］. Automotive Engineering， 2022， 44（5）： 684-690.
6	胡杰，徐博远，熊宗权，等. 基于多尺度掩码分类域自适应网络的跨域目标检测算法［J］. 汽车工程， 2022， 44（9）： 1327-1338.
	HU J， XU B Y， XIONG Z Q， et al. Cross-domain target detection algorithm based on multi-scale mask classification domain adaptive network［J］. Automotive Engineering， 2022， 44（9）： 1327-1338.
7	LI B， WANG Y， CHE T， et al. Rethinking distributional matching based domain adaptation［J］. arXiv preprint arXiv：， 2020.
8	CAO Y， LONG M， WANG J. Unsupervised domain adaptation with distribution matching machines［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2018， 32（1）.
9	CAI T， GAO R， LEE J， et al. A theory of label propagation for subpopulation shift［C］. International Conference on Machine Learning. PMLR， 2021： 1170-1182.
10	LIU X， GUO Z， LI S， et al. Adversarial unsupervised domain adaptation with conditional and label shift： infer， align and iterate［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 10367-10376.
11	KUMAR A， RAGHUNATHAN A， JONES R， et al. Fine-tuning can distort pretrained features and underperform out-of-distribution［J］. arXiv preprint arXiv：， 2022.
12	WU Y， WINSTON E， KAUSHIK D， et al. Domain adaptation with asymmetrically-relaxed distribution alignment［C］. International Conference on Machine Learning. PMLR， 2019： 6872-6881.
13	TACHET DES COMBES R， ZHAO H， WANG Y X， et al. Domain adaptation with conditional distribution matching and generalized label shift［J］. Advances in Neural Information Processing Systems， 2020， 33： 19276-19289.
14	ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［C］. Proceedings of the IEEE International Conference on Computer Vision， 2017： 2223-2232.
15	LIU F， ZHANG X， WAN F， et al. Domain contrast for domain adaptive object detection［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2021， 32（12）： 8227-8237.
16	DENG Z， KONG Q， AKIRA N， et al. Hierarchical contrastive adaptation for cross-domain object detection［J］. Machine Vision and Applications， 2022， 33（4）： 62.
17	HUANG W， YI M， ZHAO X. Towards the generalization of contrastive self-supervised learning［J］. arXiv preprint arXiv：， 2021.
18	OORD A， LI Y， VINYALS O. Representation learning with contrastive predictive coding［J］. arXiv preprint arXiv：， 2018.
19	YANG G， XIA H， DING M， et al. Bi-directional generation for unsupervised domain adaptation［C］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020， 34（4）： 6615-6622.
20	HSU H K， YAO C H， TSAI Y H， et al. Progressive domain adaptation for object detection［C］. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， 2020： 749-757.
21	GAO L， HU H M， LI M. A progressive domain adaptation for object detection via coarse-grained foreground guidance［C］. 2022 IEEE 24th International Workshop on Multimedia Signal Processing （MMSP）. IEEE， 2022： 1-6.
22	PRABHU V， KHARE S， KARTIK D， et al. Sentry： selective entropy optimization via committee consistency for unsupervised domain adaptation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 8558-8567.
23	CORDTS M， OMRAN M， RAMOS S， et al. The cityscapes dataset for semantic urban scene understanding［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 3213-3223.
24	SAKARIDIS C， DAI D， VAN GOOL L. Semantic foggy scene understanding with synthetic data［J］. International Journal of Computer Vision， 2018， 126（9）： 973-992.
25	JOHNSON-ROBERSON M， BARTO C， MEHTA R， et al. Driving in the matrix： can virtual worlds replace human-generated annotations for real world tasks？［J］. arXiv preprint arXiv：， 2016.
26	EVERINGHAM M， VAN GOOL L， WILLIAMS C K I， et al. The pascal visual object classes （VOC） challenge［J］. International Journal of Computer Vision， 2010， 88（2）： 303-338.
27	INOUE N， FURUTA R， YAMASAKI T， et al. Cross-domain weakly-supervised object detection through progressive domain adaptation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 5001-5009.
28	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv：， 2014.
29	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 770-778.
30	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. Advances in Neural Information Processing Systems， 2015， 28.
31	ZHU X， PANG J， YANG C， et al. Adapting object detectors via selective cross-domain alignment［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 687-696.
32	HE Z， ZHANG L. Multi-adversarial faster-rcnn for unrestricted object detection［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 6668-6677.
33	SAITO K， USHIKU Y， HARADA T， et al. Strong-weak distribution alignment for adaptive object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 6956-6965.
34	XU C D， ZHAO X R， JIN X， et al. Exploring categorical regularization for domain adaptive object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 11724-11733.
35	CHEN C， ZHENG Z， DING X， et al. Harmonizing transferability and discriminability for adapting object detectors［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 8869-8878.
36	ZHANG Y， WANG Z， MAO Y. RPN prototype alignment for domain adaptive object detector［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 12425-12434.
37	WANG H， LIAO S， SHAO L. AFAN： augmented feature alignment network for cross-domain object detection［J］. IEEE Transactions on Image Processing， 2021， 30： 4046-4056.
38	XU Q， LI Y， WANG G， et al. Skip-layer network with optimization method for domain adaptive detection［J］. Plos One， 2022， 17（3）： e0263748.
39	YU F， WANG D， CHEN Y， et al. SC-UDA： style and content gaps aware unsupervised domain adaptation for object detection［C］. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， 2022： 382-391.
40	LIU D， ZHANG C， SONG Y， et al. Decompose to adapt： cross-domain object detection via feature disentanglement［J］. IEEE Transactions on Multimedia， 2022， 25： 1333-1344.
41	TIAN K， ZHANG C， WANG Y， et al. Multi-level consistency regularization for domain adaptive object detection［J］. Neural Computing and Applications， 2023： 1-16.
42	VAN DER MAATEN L， HINTON G. Visualizing data using t-SNE［J］. Journal of Machine Learning Research， 2008， 9（11）.

[1]	HAO Han, WANG Si-南, LI Xiao, LIU Zong-Wei, ZHAO Fu-Quan. [J]. , 2017, 39(1): 1 -8 .
[2]	ZONG Yi-Qi, GU Zheng-Qi, LUO Ze-Min, JIANG Cai-Mao, ZHANG Qi-Dong, YANG Zhen-Dong. [J]. , 2016, 38(12): 1427 -1433 .
[3]	ZHANG Ying-Chao, ZHAN Da-Peng, ZHAO Jing, ZHANG Zhe, LI Jie. [J]. , 2016, 38(12): 1434 -1439 .
[4]	JIANG Ting, DAI Bing. [J]. , 2016, 38(12): 1459 -1466 .
[5]	CAO Li-Bo, HU Yuan, YAN Ling-Bo, PENG Yu, SHI Xiang-南. [J]. , 2017, 39(2): 174 -180 .
[6]	YU Zhuo-Ping, LENG Bo. [J]. , 2017, 39(3): 243 -248 .
[7]	LI Jun, SU Yan-Zhao, KUI Han-Bing, HU Ming-Hui, ZHANG Sheng-Gen, LIU Heng-Shuo. [J]. , 2017, 39(3): 296 -303 .
[8]	. Research on Defrosting Prediction Method based on Evaporating Temperature in Mobile Air-Conditioning System[J]. , 0, (): 0 .
[9]	. Simulation and Test of Vibration and Noise for the Drive Axle[J]. , 0, (): 0 .
[10]	SUN Bin-Bin, GAO Song-.Wang-Peng-Wei-.Li-Jun-Wei-.Li-Yan-Qiang. [J]. , 2017, 39(4): 386 -393 .

方法	person	rider	car	truck	bus	train	motor	bicycle	mAP/ %
Source^［30］	24.1	33.1	34.3	4.1	22.3	3.0	15.3	26.5	20.3
DA-Faster^［4］	25.0	31.0	40.5	22.1	35.3	20.2	20.0	27.1	27.6
SCDA^［31］	33.5	38.0	48.5	26.5	39.0	23.3	28.0	33.6	33.8
MAF^［32］	28.2	39.5	43.9	23.8	39.9	33.3	29.2	33.9	34.0
SWDA^［33］	29.9	42.3	43.5	24.5	36.2	32.6	30.0	35.3	34.3
ICR-CCR^［34］	32.9	43.8	49.2	27.2	45.1	36.4	30.3	34.6	37.4
HTCN^［35］	33.2	47.5	47.9	31.6	47.4	40.9	32.3	37.1	39.8
RPN-PR^［36］	33.3	45.6	50.5	30.4	43.6	42.0	29.7	36.8	39.0
AFAN^［37］	42.5	44.6	57.0	26.4	48.0	28.3	33.2	37.1	39.6
SLNO^［38］	33.4	44.1	49.3	24.9	42.3	29.3	30.1	37.1	36.3
SC-UDA^［39］	38.5	43.7	56.0	27.1	43.8	29.7	31.2	39.5	38.7
DDF^［40］	37.2	46.3	51.9	24.7	43.9	34.2	33.3	40.8	39.1
FLDMN	33.4	45.4	50.9	29.9	55.4	38.3	33.4	36.5	40.4

方法	AP on Car/ %
Source^［30］	34.6
DA-Faster^［4］	39.0
MAF^［32］	41.1
SWDA^［33］	40.1
HTCN^［35］	42.5
SLNO^［38］	42.7
FLDMN	43.0

方法	aero	bird	botl	car	chai	tabl	hrs	prsn	shep	train	bike
Source^［30］	35.6	24.3	20.0	32.8	30.6	13.8	36.8	48.7	16.5	22.9	52.5
DA-Faster^［4］	15.0	12.4	19.8	23.2	22.1	10.6	19.6	34.6	1.0	19.7	34.6
SWDA^［33］	26.2	32.6	38.5	37.1	34.8	17.0	33.8	61.6	9.3	54.1	48.5
ICR-CCR^［34］	28.7	31.8	40.1	36.6	38.7	17.6	33.3	61.3	22.3	49.1	55.3
HTCN^［35］	33.6	34.0	45.6	39.8	39.7	21.1	39.1	63.0	19.3	50.2	58.9
DC^［15］	47.1	38.8	46.6	52.6	39.1	31.7	34.9	67.8	22.8	44.9	53.2
HCDN^［16］	35.0	33.6	45.7	41.7	45.2	22.3	38.0	66.0	27.0	52.1	62.5
CROD^［41］	45.0	31.4	44.3	42.5	50.2	22.0	36.5	58.9	8.3	56.1	62.4
FLDMN	39.4	36.3	35.6	44.2	46.8	34.6	40.0	64.9	21.6	65.4	65.6
方法	boat	bus	cat	cow	dog	mbike	plnt	sofa	tv	mAP/ %
Source^［30］	23.0	43.9	10.7	11.7	6.0	45.9	41.9	7.3	32.0	27.8
DA-Faster^［4］	11.9	21.1	3.1	26.3	10.0	39.4	29.3	17.1	24.8	19.8
SWDA^［33］	33.7	54.3	18.6	58.3	12.5	65.5	52.0	24.9	49.1	38.1
ICR-CCR^［34］	26.0	63.6	9.4	49.3	14.1	74.3	46.3	24.3	44.3	38.3
HTCN^［35］	23.4	57.0	12.0	51.3	20.1	72.8	43.1	30.1	51.8	40.3
DC^［15］	37.0	45.8	14.5	48.4	23.7	87.0	54.0	23.8	51.0	43.2
HCDN^［16］	30.6	64.5	12.7	55.6	20.6	86.8	50.0	29.7	47.9	43.4
CROD^［41］	30.5	61.4	21.1	59.1	20.3	47.7	46.7	33.7	58.4	41.8
FLDMN	38.6	71.4	15.9	61.2	21.8	89.9	46.6	29.3	56.1	46.3

方法	person	rider	car	truck	bus	train	mcyc	bicy	mAP/%
Source^［30］	24.1	33.1	34.3	4.1	22.3	3.0	15.3	26.5	20.3
ICR-CCR^［34］	29.7	37.3	43.6	20.8	37.3	12.8	25.7	31.7	29.9
Variant1	33.0	43.9	50.9	29.0	45.1	31.2	35.6	36.2	38.1
Variant2	33.3	44.1	51.1	28.6	47.0	38.6	33.7	37.2	39.2
FLDMN	33.4	45.4	50.9	29.9	55.4	38.3	33.4	36.5	40.4

方法	aero	bird	botl	car	chai	tabl	hrs	prsn	shep	train	bike
Source^［30］	35.6	24.3	20.0	32.8	30.6	13.8	36.8	48.7	16.5	22.9	52.5
DA-Faster^［4］	15.0	12.4	19.8	23.2	22.1	10.6	19.6	34.6	1.0	19.7	34.6
FLDMN-F	37.7	39.0	37.7	41.7	45.3	23.8	35.6	63.8	26.0	63.6	64.6
FLDMN	39.4	36.3	35.6	44.2	46.8	34.6	40.0	64.9	21.6	65.4	65.6
方法	boat	bus	cat	cow	dog	bike	plnt	sofa	tv	mAP/%
Source^［30］	23.0	43.9	10.7	11.7	6.0	45.9	41.9	7.3	32.0	27.8
DA-Faster^［4］	11.9	21.1	3.1	26.3	10.0	39.4	29.3	17.1	24.8	19.8
FLDMN-F	38.4	64.4	14.7	60.1	18.2	79.0	51.0	31.8	54.8	44.6
FLDMN	38.6	71.4	15.9	61.2	21.8	89.9	46.6	29.3	56.1	46.3

Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 42

Related Articles 15

Metrics

Comments

Recommended 10

[1]	Xinke Fu,Yingfeng Cai,Long Chen,Hai Wang,Qingchao Liu. Decision-Making for Autonomous Driving in Uncertain Environment [J]. Automotive Engineering, 2024, 46(2): 211-221.
[2]	Xiaocong Zhao,Shiyu Fang,Zirui Li,Jian Sun. Extraction and Application of Key Utility Term for Social Driving Interaction [J]. Automotive Engineering, 2024, 46(2): 230-240.
[3]	Yanli Ma, Qin Qin, Fangqi Dong, Yining Lou. Takeover Risk Assessment Model Based on Risk Field Theory Under Different Cognitive Secondary Tasks [J]. Automotive Engineering, 2024, 46(1): 9-17.
[4]	Weiguo Liu,Zhiyu Xiang,Weiping Liu,Daoxin Qi,Zixu Wang. Research on Vehicle Control Algorithm Based on Distributed Reinforcement Learning [J]. Automotive Engineering, 2023, 45(9): 1637-1645.
[5]	Ming Wang,Xiaolin Tang,Kai Yang,Guofa Li,Xiaosong Hu. A Motion Planning Method for Autonomous Vehicles Considering Prediction Risk [J]. Automotive Engineering, 2023, 45(8): 1362-1372.
[6]	Cheng Lin, Bowen Wang, Lü Peiyuan, Xinle Gong, Xiao Yu. Research on Motion Planning and Cooperative Control for Autonomous Vehicles with Lane Change Gaming Maneuvers Under the Curved Road [J]. Automotive Engineering, 2023, 45(7): 1099-1111.
[7]	Dongyu Zhao, Shuen Zhao. Autonomous Driving 3D Object Detection Based on Cascade YOLOv7 [J]. Automotive Engineering, 2023, 45(7): 1112-1122.
[8]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.
[9]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[10]	Yanyan Chen,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Efficient Automatic Driving Instance Segmentation Method Based on Detection [J]. Automotive Engineering, 2023, 45(4): 541-550.
[11]	Qingyang Huang,Xiaoping Jin,Yikang Zhang. Analysis of Drivers' Driving Posture Change Rule Under the Condition of Automatic Driving Level Improvement [J]. Automotive Engineering, 2023, 45(3): 382-392.
[12]	Lü Ying,Xu Qi,Qiuzheng Liu,Xinyu Wang,Guoying Chen. Path Tracking Control Method with Steering Lag for Autonomous Vehicles [J]. Automotive Engineering, 2023, 45(12): 2234-2241.
[13]	Long Chen,Chen Yang,Yingfeng Cai,Hai Wang,Yicheng Li. Pedestrian Crossing Intention Prediction Method Based on Multimodal Feature Fusion [J]. Automotive Engineering, 2023, 45(10): 1779-1790.
[14]	Pulei Xu,Yingfeng Cai,Yubo Lian,Xiaoqiang Sun,Hai Wang,Long Chen,Yilin Zhong. AFS/DYC Coordinated Control of Intelligent Vehicles Based on Improved Hierarchical Extensibility Theory [J]. Automotive Engineering, 2023, 45(1): 20-31.
[15]	Fengchong Lan,Yingjie Liu,Jiqing Chen,Zhaolin Liu. Study on Motion Planning of Autonomous Vehicles in Cut-in Scenes Based on Dynamic Game Algorithm [J]. Automotive Engineering, 2023, 45(1): 9-19.