基于特征和标签联合分布匹配的智能驾驶跨域自适应目标检测

doi:10.19562/j.chinasae.qcgc.2023.11.009

摘要/Abstract

摘要：

现有的跨域自适应目标检测器主要通过特征分布匹配学习域不变特征来减小域偏移。然而，它们忽略了现实世界中因目标检测对象组合不同和类不平衡等引起的标签分布偏移问题，导致这些方法泛化性较差。为解决这一问题，本文提出一种基于特征和标签联合分布匹配的域自适应目标检测算法，以同时在特征和标签级别上显式对齐域分布。首先，提出一个图像级分类嵌入模块，通过对比学习增强全局特征可迁移性和可判别性。然后，提出类级分布对齐模块，通过多级特征对齐实现域间多模态结构对齐。最后，提出增强一致性正则化模块，通过区域一致性正则化实现跨域标签分布对齐。在多个数据集上的实验表明，所提出的域对齐算法能有效地促进跨域数据迁移前后语义一致性，为智能汽车跨视觉域应用提供了一种有效的解决方案。

关键词: 智能驾驶, 智能汽车, 目标检测, 域自适应

Abstract:

Current cross-domain adaptive object detection methods primarily focus on reducing domain shift by learning domain-invariant features through feature distribution alignment. However， they often overlook label distribution shift issues caused by variations in object combinations and class imbalances in real-world scenarios， resulting in poor generalization performance. To address this， this paper proposes a novel domain adaptive object detection algorithm that simultaneously aligns domain distributions at both feature and label levels. Firstly， an image-level classification embedding module is introduced to enhance the transferability and discriminability of global features through contrastive learning. Next， a class-level distribution alignment module is presented to achieve inter-domain multimodal structure alignment through multi-level feature alignment. Finally， an enhanced consistency regularization module is proposed to achieve cross-domain label distribution alignment through region-based consistency regularization. Experimental results across multiple datasets demonstrate that the proposed domain alignment algorithm effectively improves semantic consistency both before and after cross-domain data adaptation. This provides a valuable solution for the effective deployment of autonomous vehicles in cross-domain scenarios.

Key words: autonomous vehicles, autonomous driving, object detection, domain adaptation

刘正发,吴亚,刘佩根,顾荣琦,陈广. 基于特征和标签联合分布匹配的智能驾驶跨域自适应目标检测[J]. 汽车工程, 2023, 45(11): 2082-2091.

Zhengfa Liu,Ya Wu,Peigen Liu,Rongqi Gu,Guang Chen. Cross-Domain Object Detection for Intelligent Driving Based on Joint Distribution Matching of Features and Labels[J]. Automotive Engineering, 2023, 45(11): 2082-2091.

图/表 11

图1

图2

图3

图4

表1

表2

表3

表4

表5

图5

图6

参考文献 42

1	陈文强，熊辉，李克强，等. 基于深度神经网络的行人及骑车人联合检测［J］. 汽车工程， 2018， 40（6）： 726-733.
	CHEN W Q， XIONG L， LI K Q， et al. Joint detection of pedestrians and cyclists based on deep neural network［J］. Automotive Engineering， 2018， 40（6）： 726-733.
2	冯润泽，江昆，于伟光，等. 基于两阶段分类算法的中国交通标志牌识别［J］. 汽车工程， 2023，44（3）：434-441.
	FENG R Z， JIANG K， YU W G， et al. Chinese traffic sign recognition based on two-stage classification algorithm［J］. Automotive Engineering， 2023，44（3）：434-441.
3	谢德胜，徐友春，陆峰，等. 基于多传感器信息融合的3维目标实时检测［J］. 汽车工程， 2022， 44（3）： 11.
	XIE D S， XU Y C， LU F， et al. Real-time detection of 3D targets based on multi-sensor information fusion［J］. Automotive Engineering， 2022， 44（3）： 11.
4	CHEN Y， LI W， SAKARIDIS C， et al. Domain adaptive faster R-CNN for object detection in the wild［C］. Proceedings of the IEEE conference on computer vision and pattern recognition， 2018： 3339-3348.
5	王大方，杜京东，曹江，等. 面向自动驾驶数据生成的风格迁移网络研究［J］. 汽车工程， 2022， 44（5）： 684-690.
	WANG D F， DU J D， CAO J， et al. Research on style transfer network for autonomous driving data generation［J］. Automotive Engineering， 2022， 44（5）： 684-690.
6	胡杰，徐博远，熊宗权，等. 基于多尺度掩码分类域自适应网络的跨域目标检测算法［J］. 汽车工程， 2022， 44（9）： 1327-1338.
	HU J， XU B Y， XIONG Z Q， et al. Cross-domain target detection algorithm based on multi-scale mask classification domain adaptive network［J］. Automotive Engineering， 2022， 44（9）： 1327-1338.
7	LI B， WANG Y， CHE T， et al. Rethinking distributional matching based domain adaptation［J］. arXiv preprint arXiv：， 2020.
8	CAO Y， LONG M， WANG J. Unsupervised domain adaptation with distribution matching machines［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2018， 32（1）.
9	CAI T， GAO R， LEE J， et al. A theory of label propagation for subpopulation shift［C］. International Conference on Machine Learning. PMLR， 2021： 1170-1182.
10	LIU X， GUO Z， LI S， et al. Adversarial unsupervised domain adaptation with conditional and label shift： infer， align and iterate［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 10367-10376.
11	KUMAR A， RAGHUNATHAN A， JONES R， et al. Fine-tuning can distort pretrained features and underperform out-of-distribution［J］. arXiv preprint arXiv：， 2022.
12	WU Y， WINSTON E， KAUSHIK D， et al. Domain adaptation with asymmetrically-relaxed distribution alignment［C］. International Conference on Machine Learning. PMLR， 2019： 6872-6881.
13	TACHET DES COMBES R， ZHAO H， WANG Y X， et al. Domain adaptation with conditional distribution matching and generalized label shift［J］. Advances in Neural Information Processing Systems， 2020， 33： 19276-19289.
14	ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［C］. Proceedings of the IEEE International Conference on Computer Vision， 2017： 2223-2232.
15	LIU F， ZHANG X， WAN F， et al. Domain contrast for domain adaptive object detection［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2021， 32（12）： 8227-8237.
16	DENG Z， KONG Q， AKIRA N， et al. Hierarchical contrastive adaptation for cross-domain object detection［J］. Machine Vision and Applications， 2022， 33（4）： 62.
17	HUANG W， YI M， ZHAO X. Towards the generalization of contrastive self-supervised learning［J］. arXiv preprint arXiv：， 2021.
18	OORD A， LI Y， VINYALS O. Representation learning with contrastive predictive coding［J］. arXiv preprint arXiv：， 2018.
19	YANG G， XIA H， DING M， et al. Bi-directional generation for unsupervised domain adaptation［C］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020， 34（4）： 6615-6622.
20	HSU H K， YAO C H， TSAI Y H， et al. Progressive domain adaptation for object detection［C］. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， 2020： 749-757.
21	GAO L， HU H M， LI M. A progressive domain adaptation for object detection via coarse-grained foreground guidance［C］. 2022 IEEE 24th International Workshop on Multimedia Signal Processing （MMSP）. IEEE， 2022： 1-6.
22	PRABHU V， KHARE S， KARTIK D， et al. Sentry： selective entropy optimization via committee consistency for unsupervised domain adaptation［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2021： 8558-8567.
23	CORDTS M， OMRAN M， RAMOS S， et al. The cityscapes dataset for semantic urban scene understanding［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 3213-3223.
24	SAKARIDIS C， DAI D， VAN GOOL L. Semantic foggy scene understanding with synthetic data［J］. International Journal of Computer Vision， 2018， 126（9）： 973-992.
25	JOHNSON-ROBERSON M， BARTO C， MEHTA R， et al. Driving in the matrix： can virtual worlds replace human-generated annotations for real world tasks？［J］. arXiv preprint arXiv：， 2016.
26	EVERINGHAM M， VAN GOOL L， WILLIAMS C K I， et al. The pascal visual object classes （VOC） challenge［J］. International Journal of Computer Vision， 2010， 88（2）： 303-338.
27	INOUE N， FURUTA R， YAMASAKI T， et al. Cross-domain weakly-supervised object detection through progressive domain adaptation［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2018： 5001-5009.
28	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv：， 2014.
29	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2016： 770-778.
30	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. Advances in Neural Information Processing Systems， 2015， 28.
31	ZHU X， PANG J， YANG C， et al. Adapting object detectors via selective cross-domain alignment［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 687-696.
32	HE Z， ZHANG L. Multi-adversarial faster-rcnn for unrestricted object detection［C］. Proceedings of the IEEE/CVF International Conference on Computer Vision， 2019： 6668-6677.
33	SAITO K， USHIKU Y， HARADA T， et al. Strong-weak distribution alignment for adaptive object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2019： 6956-6965.
34	XU C D， ZHAO X R， JIN X， et al. Exploring categorical regularization for domain adaptive object detection［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 11724-11733.
35	CHEN C， ZHENG Z， DING X， et al. Harmonizing transferability and discriminability for adapting object detectors［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2020： 8869-8878.
36	ZHANG Y， WANG Z， MAO Y. RPN prototype alignment for domain adaptive object detector［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition， 2021： 12425-12434.
37	WANG H， LIAO S， SHAO L. AFAN： augmented feature alignment network for cross-domain object detection［J］. IEEE Transactions on Image Processing， 2021， 30： 4046-4056.
38	XU Q， LI Y， WANG G， et al. Skip-layer network with optimization method for domain adaptive detection［J］. Plos One， 2022， 17（3）： e0263748.
39	YU F， WANG D， CHEN Y， et al. SC-UDA： style and content gaps aware unsupervised domain adaptation for object detection［C］. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision， 2022： 382-391.
40	LIU D， ZHANG C， SONG Y， et al. Decompose to adapt： cross-domain object detection via feature disentanglement［J］. IEEE Transactions on Multimedia， 2022， 25： 1333-1344.
41	TIAN K， ZHANG C， WANG Y， et al. Multi-level consistency regularization for domain adaptive object detection［J］. Neural Computing and Applications， 2023： 1-16.
42	VAN DER MAATEN L， HINTON G. Visualizing data using t-SNE［J］. Journal of Machine Learning Research， 2008， 9（11）.

方法	person	rider	car	truck	bus	train	motor	bicycle	mAP/ %
Source^［30］	24.1	33.1	34.3	4.1	22.3	3.0	15.3	26.5	20.3
DA-Faster^［4］	25.0	31.0	40.5	22.1	35.3	20.2	20.0	27.1	27.6
SCDA^［31］	33.5	38.0	48.5	26.5	39.0	23.3	28.0	33.6	33.8
MAF^［32］	28.2	39.5	43.9	23.8	39.9	33.3	29.2	33.9	34.0
SWDA^［33］	29.9	42.3	43.5	24.5	36.2	32.6	30.0	35.3	34.3
ICR-CCR^［34］	32.9	43.8	49.2	27.2	45.1	36.4	30.3	34.6	37.4
HTCN^［35］	33.2	47.5	47.9	31.6	47.4	40.9	32.3	37.1	39.8
RPN-PR^［36］	33.3	45.6	50.5	30.4	43.6	42.0	29.7	36.8	39.0
AFAN^［37］	42.5	44.6	57.0	26.4	48.0	28.3	33.2	37.1	39.6
SLNO^［38］	33.4	44.1	49.3	24.9	42.3	29.3	30.1	37.1	36.3
SC-UDA^［39］	38.5	43.7	56.0	27.1	43.8	29.7	31.2	39.5	38.7
DDF^［40］	37.2	46.3	51.9	24.7	43.9	34.2	33.3	40.8	39.1
FLDMN	33.4	45.4	50.9	29.9	55.4	38.3	33.4	36.5	40.4

方法	AP on Car/ %
Source^［30］	34.6
DA-Faster^［4］	39.0
MAF^［32］	41.1
SWDA^［33］	40.1
HTCN^［35］	42.5
SLNO^［38］	42.7
FLDMN	43.0

方法	aero	bird	botl	car	chai	tabl	hrs	prsn	shep	train	bike
Source^［30］	35.6	24.3	20.0	32.8	30.6	13.8	36.8	48.7	16.5	22.9	52.5
DA-Faster^［4］	15.0	12.4	19.8	23.2	22.1	10.6	19.6	34.6	1.0	19.7	34.6
SWDA^［33］	26.2	32.6	38.5	37.1	34.8	17.0	33.8	61.6	9.3	54.1	48.5
ICR-CCR^［34］	28.7	31.8	40.1	36.6	38.7	17.6	33.3	61.3	22.3	49.1	55.3
HTCN^［35］	33.6	34.0	45.6	39.8	39.7	21.1	39.1	63.0	19.3	50.2	58.9
DC^［15］	47.1	38.8	46.6	52.6	39.1	31.7	34.9	67.8	22.8	44.9	53.2
HCDN^［16］	35.0	33.6	45.7	41.7	45.2	22.3	38.0	66.0	27.0	52.1	62.5
CROD^［41］	45.0	31.4	44.3	42.5	50.2	22.0	36.5	58.9	8.3	56.1	62.4
FLDMN	39.4	36.3	35.6	44.2	46.8	34.6	40.0	64.9	21.6	65.4	65.6
方法	boat	bus	cat	cow	dog	mbike	plnt	sofa	tv	mAP/ %
Source^［30］	23.0	43.9	10.7	11.7	6.0	45.9	41.9	7.3	32.0	27.8
DA-Faster^［4］	11.9	21.1	3.1	26.3	10.0	39.4	29.3	17.1	24.8	19.8
SWDA^［33］	33.7	54.3	18.6	58.3	12.5	65.5	52.0	24.9	49.1	38.1
ICR-CCR^［34］	26.0	63.6	9.4	49.3	14.1	74.3	46.3	24.3	44.3	38.3
HTCN^［35］	23.4	57.0	12.0	51.3	20.1	72.8	43.1	30.1	51.8	40.3
DC^［15］	37.0	45.8	14.5	48.4	23.7	87.0	54.0	23.8	51.0	43.2
HCDN^［16］	30.6	64.5	12.7	55.6	20.6	86.8	50.0	29.7	47.9	43.4
CROD^［41］	30.5	61.4	21.1	59.1	20.3	47.7	46.7	33.7	58.4	41.8
FLDMN	38.6	71.4	15.9	61.2	21.8	89.9	46.6	29.3	56.1	46.3

方法	person	rider	car	truck	bus	train	mcyc	bicy	mAP/%
Source^［30］	24.1	33.1	34.3	4.1	22.3	3.0	15.3	26.5	20.3
ICR-CCR^［34］	29.7	37.3	43.6	20.8	37.3	12.8	25.7	31.7	29.9
Variant1	33.0	43.9	50.9	29.0	45.1	31.2	35.6	36.2	38.1
Variant2	33.3	44.1	51.1	28.6	47.0	38.6	33.7	37.2	39.2
FLDMN	33.4	45.4	50.9	29.9	55.4	38.3	33.4	36.5	40.4

方法	aero	bird	botl	car	chai	tabl	hrs	prsn	shep	train	bike
Source^［30］	35.6	24.3	20.0	32.8	30.6	13.8	36.8	48.7	16.5	22.9	52.5
DA-Faster^［4］	15.0	12.4	19.8	23.2	22.1	10.6	19.6	34.6	1.0	19.7	34.6
FLDMN-F	37.7	39.0	37.7	41.7	45.3	23.8	35.6	63.8	26.0	63.6	64.6
FLDMN	39.4	36.3	35.6	44.2	46.8	34.6	40.0	64.9	21.6	65.4	65.6
方法	boat	bus	cat	cow	dog	bike	plnt	sofa	tv	mAP/%
Source^［30］	23.0	43.9	10.7	11.7	6.0	45.9	41.9	7.3	32.0	27.8
DA-Faster^［4］	11.9	21.1	3.1	26.3	10.0	39.4	29.3	17.1	24.8	19.8
FLDMN-F	38.4	64.4	14.7	60.1	18.2	79.0	51.0	31.8	54.8	44.6
FLDMN	38.6	71.4	15.9	61.2	21.8	89.9	46.6	29.3	56.1	46.3