面向非匹配语义数据的高保真度图像翻译方法

doi:10.19562/j.chinasae.qcgc.2025.10.007

Abstract

Abstract:

In the field of intelligent driving environment perception， image translation models often fail when there is a significant semantic mismatch between source and target domains， leading to semantic inversion and detail degradation. To address this challenge， in this paper a high-fidelity image translation method tailored for asymmetric domain data is proposed. Based on the diffusion-model generator structure， a multi-adaptive skip connection （MASC） module and a high-dimensional vector consistency loss （HVC loss） are proposed. The MASC module combines dynamic normalization and attention mechanism to adaptively process semantic and style information in skip connection， while the HVC loss constrains semantic mapping relationship in high-dimensional symbolic space. Compared to the optimal results， the proposed model reduces the FID and KID scores by 15.36 and 0.003 4 on RainSurface， and by 1.44 and 0.000 8 on public datasets， respectively.

Key words: image translation, deep learning, data augmentation, diffusion models

Zhuo Li,Libo Cao,Jiacai Liao,Haowei Cui,Yue Zhang. High-Fidelity Unsupervised Image Translation for Mismatched Semantic Data[J].Automotive Engineering, 2025, 47(10): 1914-1922.

Figures/Tables 14

References 30

[1]	LIN C T， HUANG S W， WU Y Y， et al. GAN-based day-to-night image style transfer for nighttime vehicle detection ［J］. IEEE Transactions on Intelligent Transportation Systems， 2021， 22（2）： 951-963.
[2]	CHEN X， LI H， LI M Q， et al. Learning a sparse transformer network for effective image deraining ［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， Vancouver， CANADA， F Jun 17-24， 2023.
[3]	CHEN Y C， LIN Y Y， YANG M H， et al. CrDoCo： pixel-level domain transfer with cross-domain consistency ［C］. Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， Long Beach， CA， F Jun 16-20， 2019.
[4]	GOODFELLOW I J， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial nets ［C］. Proceedings of the 28th Conference on Neural Information Processing Systems （NIPS）， Montreal， CANADA， F Dec 08-13， 2014.
[5]	NICHOL A， DHARIWAL P. Improved denoising diffusion probabilistic models ［C］. Proceedings of the International Conference on Machine Learning （ICML）， Electr Network， F Jul 18-24， 2021.
[6]	ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks ［C］. Proceedings of the 16th IEEE International Conference on Computer Vision （ICCV）， Venice， ITALY， F Oct 22-29， 2017.
[7]	LIU M Y， BREUEL T， KAUTZ J. Unsupervised image-to-image translation networks ［C］. Proceedings of the 31st Annual Conference on Neural Information Processing Systems （NIPS）， Long Beach， CA， F Dec 04-09， 2017.
[8]	HUANG X， LIU M Y， BELONGIE S， et al. Multimodal unsupervised image-to-image translation ［C］. Proceedings of the 15th European Conference on Computer Vision （ECCV）， Munich， GERMANY， F Sep 08-14， 2018.
[9]	JIA Z W， YUAN B D， WANG K K， et al. Semantically robust unpaired image translation for data with unmatched semantics statistics ［C］. Proceedings of the 18th IEEE/CVF International Conference on Computer Vision （ICCV）， Electr Network， F Oct 11-17， 2021.
[10]	PARK T， EFROS A A， ZHANG R， et al. Contrastive learning for unpaired image-to-image translation ［C］. Proceedings of the Computer Vision-ECCV 2020： 16th European Conference， Glasgow， UK， August 23–28， 2020， Proceedings， Part IX 16， F， 2020. Springer.
[11]	LEE H Y， LI Y H， LEE T H， et al. Progressively unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation ［J］. Sensors， 2023， 23（15）.
[12]	JIANG Y F， CHANG S Y， WANG Z Y. TransGAN： two pure transformers can make one strong GAN， and that can scale up ［C］. Proceedings of the 35th Annual Conference on Neural Information Processing Systems （NeurIPS）， Electr Network， F Dec 06-14， 2021.
[13]	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］. Proceedings of the 31st Annual Conference on Neural Information Processing Systems （NIPS）， Long Beach， CA， F Dec 04-09， 2017.
[14]	THEISS J， LEVERETT J， KIM D， et al. Unpaired image translation via vector symbolic architectures ［C］. Proceedings of the 17th European Conference on Computer Vision （ECCV）， Tel Aviv， ISRAEL， F Oct 23-27， 2022.
[15]	HO J， JAIN A， ABBEEL P. Denoising diffusion probabilistic models ［J］. Advances in Neural Information Processing Systems， 2020， 33： 6840-6851.
[16]	SASAKI H， WILLCOCKS C G， BRECKON T P. UNIT-DDPM： UNpaired image translation with denoising diffusion probabilistic models ［J］. arXiv， 2021.
[17]	ZHAO M， BAO F， LI C X， et al. EGSDE： unpaired image-to-image translation via energy-guided stochastic differential equations ［C］. Proceedings of the 36th Conference on Neural Information Processing Systems （NeurIPS）， Electr Network， F Nov 28-Dec 09， 2022.
[18]	WU C H， DE LA TORRE F， IEEE. A latent space of stochastic diffusion models for zero-shot image editing and guidance ［C］； Proceedings of the IEEE/CVF International Conference on Computer Vision （ICCV）， Paris， FRANCE， F Oct 02-06， 2023.
[19]	PARMAR G， PARK T， NARASIMHAN S， et al. One-step image translation with text-to-image models ［J］. arXiv preprint arXiv：， 2024.
[20]	DHARIWAL P， NICHOL A. Diffusion models beat GANs on image synthesis ［C］. Proceedings of the 35th Annual Conference on Neural Information Processing Systems （NeurIPS）， Electr Network， F Dec 06-14， 2021.
[21]	KINGMA D P， WELLING M. Auto-encoding variational bayes ［Z］. Banff， Canada. 2013.
[22]	ROMBACH R， BLATTMANN A， LORENZ D， et al. High-resolution image synthesis with latent diffusion models ［C］. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， New Orleans， LA， F Jun 18-24， 2022.
[23]	HU E J， SHEN Y， WALLIS P， et al. LoRA： low-rank adaptation of large language models ［J］. ICLR， 2022， 1（2）： 3.
[24]	RADFORD A， KIM J W， HALLACY C， et al. Learning transferable visual models from natural language supervision ［C］. Proceedings of the International Conference on Machine Learning （ICML）， Electr Network， F Jul 18-24， 2021.
[25]	CORDTS M， OMRAN M， RAMOS S， et al. The cityscapes dataset for semantic urban scene understanding ［C］. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）， Seattle， WA， F Jun 27-30， 2016.
[26]	RICHTER S R， VINEET V， ROTH S， et al. Playing for data： ground truth from computer games ［C］. Proceedings of the 14th European Conference on Computer Vision （ECCV）， Amsterdam， NETHERLANDS， F Oct 08-16， 2016.
[27]	HENSEL M， RAMSAUER H， UNTERTHINER T， et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium ［C］. Proceedings of the 31st Annual Conference on Neural Information Processing Systems （NIPS）， Long Beach， CA， F Dec 04-09， 2017.
[28]	SUTHERLAND J， ARBEL M， GRETTON A. Demystifying MMD GANS ［C］. Proceedings of the International Conference for Learning Representations， F， 2018.
[29]	CHERIAN A， SULLIVAN A. Sem-GAN： semantically-consistent image-to-image translation ［C］. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision （wacv）， F， 2019， IEEE.
[30]	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation ［J］. arXiv preprint arXiv：， 2017.

模型	FID	KID	SSIM
Cyclegan	166.680 9	0.124 0	0.608 7
SRUNIT	200.837 1	0.171 9	0.534 6
CUT	192.218 1	0.148 8	0.517 4
img2img-turbo	122.002 4	0.072 1	0.861 9
VSAIT	57.443 3	0.018 3	0.950 8
Ours	42.079 1	0.014 9	0.935 6

模型	FID	KID	SSIM
Cyclegan	185.165 3	0.185 2	0.426 5
SRUNIT	90.966 7	0.108 8	0.544 0
CUT	96.709 7	0.121 0	0.538 8
img2img-turbo	146.526 5	0.176 5	0.456 2
VSAIT	54.078 3	0.060 3	0.859 5
Ours	52.632 3	0.059 5	0.883 5

模型	FID	KID	SSIM
w/o HVC loss & MASC	88.561 2	0.025 6	0.732 6
w/o HVC loss	80.156 2	0.023 3	0.753 3
w/o MASC	67.516 9	0.019 6	0.856 5
Ours	42.079 1	0.014 9	0.935 6

模型	mIoU	PA
w/o HVC	0.637 4	0.381 5
Ours	0.917 5	0.823 2

High-Fidelity Unsupervised Image Translation for Mismatched Semantic Data

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 30

Related Articles 15

Metrics

Comments

Recommended 0

[1]	Qinyu Sun,Hang Zhou,Rui Fu,Chang Wang,Tao Huang,Junfeng Yang,Yunhao Wang. Prediction of Lane Change Intention Based on Driver's Cognitive-Making Space [J]. Automotive Engineering, 2025, 47(8): 1468-1478.
[2]	Kai Gao,Xinyu Liu,Lin Hu,Xiangming Huang,Tiefang Zou,Peng Liu. Vehicle Trajectory Prediction with Spatial-Temporal Interaction Based on Sparse Attention [J]. Automotive Engineering, 2025, 47(5): 809-819.
[3]	Chenyu Liu,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Camera and Radar Fusion for Autonomous Driving Scenarios [J]. Automotive Engineering, 2025, 47(5): 829-838.
[4]	Qirui Qin,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Real-Time Instance Segmentation Algorithm for Autonomous Driving Based on Instance Activation Maps [J]. Automotive Engineering, 2025, 47(4): 614-624.
[5]	Yue Han,Yingfeng Cai,Long Chen,Xiaoqiang Sun,Hai Wang,Ze Liu,Zhongyu Rao. Vehicle Trajectory Prediction Method Based on Dynamic Attention and Goal-Guided Mechanism [J]. Automotive Engineering, 2025, 47(10): 1861-1871.
[6]	Yupeng Lin,Li Ma,Tingting Lan,Rui Fu. Research on Motion Sickness Assessment Model Based on Multi-dimensional Parameter Fusion of Human and Vehicle by BiLSTM-Transformer [J]. Automotive Engineering, 2025, 47(10): 1885-1894.
[7]	Le Tao,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Point Cloud for Autonomous Driving Scenarios [J]. Automotive Engineering, 2024, 46(7): 1208-1218.
[8]	Hai Wang,Guirong Zhang,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-modal Data Mining Algorithm for Corner Case of Automatic Driving Road Scene [J]. Automotive Engineering, 2024, 46(7): 1239-1248.
[9]	Zhigang Yang,Yujing Li,Chao Xia,Mengjia Wang,Lei Yu. A High Time-Resolution Reconstruction on the Automotive Turbulent Wake Based on LSTM-POD [J]. Automotive Engineering, 2024, 46(7): 1302-1313.
[10]	Hongyi Liang,Jikai Chen,Wanli Liu,Fengchong Lan,Bingda Mo,Jiqing Chen. Prediction of the Remaining Useful Life of Real Vehicle Lithium Batteries by Fusion of K-means Clustering and Sequence Decomposition [J]. Automotive Engineering, 2024, 46(4): 634-642.
[11]	Qin Shi,Zhiwei Li,Teng Cheng,Qiang Zhang,Wenchong Wang. Intrusion Detection Framework for CAN Networks Based on Evidence Deep Learning [J]. Automotive Engineering, 2024, 46(11): 2039-2045.
[12]	Qiuyan Zhang,Ze Cheng,Xu Liu. RUL Estimation Method for Lithium-ion Batteries Based on Multi-dimensional and Multi-scale Features [J]. Automotive Engineering, 2024, 46(10): 1897-1903.
[13]	Ze Gao, Zunkang Chu, Jiasheng Shi, Fu Lin, Weixiong Rao, Haiyan Yu. Research on Fast Prediction Method of Stress Field of Automotive Parts Based on Graph Network [J]. Automotive Engineering, 2024, 46(1): 170-178.
[14]	Weiguo Liu,Zhiyu Xiang,Rui Liu,Guodong Li,Zixu Wang. Research on End-to-End Vehicle Motion Planning Method Based on Deep Learning [J]. Automotive Engineering, 2023, 45(8): 1343-1352.
[15]	Jiahao Zhao,Zhiquan Qi,Zhifeng Qi,Hao Wang,Lei He. Calculation of Heading Angle of Parallel Large Vehicle Based on Tire Feature Points [J]. Automotive Engineering, 2023, 45(6): 1031-1039.