面向车载异构平台的多帧视图变换全景影像拼接算法

doi:10.19562/j.chinasae.qcgc.2025.08.010

摘要/Abstract

摘要：

汽车辅助驾驶技术中的全景影像功能能够有效消除驾驶员视线盲区，降低事故发生率，然而现有的车载全景影像系统大多存在画面质量不佳、流畅度差、低实时性等问题。本文针对其中图像边缘质量和算法实时性问题重点优化，提出了一种基于多帧视图变换的全景影像拼接算法。首先为解决鱼眼相机校正图边缘清晰度和分辨率降低问题，提出使用VDSR模型对校正图进行重建；其次提出基于多帧的视图变换算法，在完成图像视角变换的同时实现图像拼接，降低算法执行时间；并且通过运动补偿将相邻帧鱼眼图像映射到全景图像中，提高图像质量。车载异构平台上进行的整车实验结果表明，相比单帧视图变换，应用本文方法生成的全景图像在PSNR和SSIM指标上各提升了2.27 dB和0.022，图像输出帧率达到30 FPS，在保证实时性的同时明显提升了全景图像的主观视觉效果。

关键词: 全景图像拼接, 视图变换, 图像超分辨率, 图像融合

Abstract:

Surround View System in ADAS （Advances Driver Assistance System） can effectively eliminate the driver's blind spots and reduce the accident rate. However， most existing vehicular panoramic imaging systems suffer from issues such as poor image quality， lack of smoothness， and low real-time performance. In this paper focusing on optimizing the image edge quality and algorithm real-time performance， a panoramic image-stitching algorithm based on multi-frame view transformation is proposed. Firstly， to solve the problem of reduced edge clarity and resolution of the fisheye camera correction image， the VDSR model is proposed to reconstruct the correction image. Secondly， a multi-frame-based view transformation algorithm is proposed， which accomplishes image perspective transformation while simultaneously performing image stitching， thereby reducing the algorithm's execution time. Additionally， motion compensation is applied to map adjacent frame fisheye images into the panoramic image， enhancing image quality. The vehicle experiments conducted on the in-vehicle heterogeneous platform show that compared with the single-frame view transformation， the panoramic images generated by the method proposed in this paper have improved by 2.27 dB and 0.022 in PSNR and SSIM indicators respectively， with the image output frame rate reaching 30 FPS， significantly enhancing the subjective visual quality of the panoramic images while ensuring real-time performance.

Key words: panoramic image stitching, perspective transformation, image super-resolution, image fusion

陈丽娟,汪凯鑫,吴磊,卢佳伟,吴德. 面向车载异构平台的多帧视图变换全景影像拼接算法[J]. 汽车工程, 2025, 47(8): 1534-1545.

Lijuan Chen,Kaixin Wang,Lei Wu,Jiawei Lu,De Wu. Panoramic Image Stitching Algorithm with Multi-frame-based View Transformation for Vehicle Heterogeneous Platform[J]. Automotive Engineering, 2025, 47(8): 1534-1545.

图/表 25

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

表1

表2

图15

图16

图17

表3

车载鱼眼相机透视投影变化矩阵"

方向	数值
$H f r o n t$	-0.096 791 310 802 662 51	-1.324 900 827 817 967 00	1 032.902 587 175 856
	-0.011 786 273 784 901 07	-0.769 927 900 924 278 10	576.515 257 108 657
	-0.000 019 929 715 829 61	-0.001 381 943 648 780 73	1
$H l e f t$	-0.015 833 665 415 653 93	-1.679 149 307 450 583 00	1 017.469 314 252 892
	0.215 775 028 685 574 10	-1.274 077 570 049 654 00	498.899 617 230 729
	-0.000 017 825 061 558 85	-0.001 775 184 404 021 12	1
$H b a c k$	0.138 266 179 918 707 90	-1.519 831 656 083 429 00	821.089 590 448 981
	-0.006 967 935 709 695 03	-1.531 523 269 370 587 00	926.775 300 466 658
	-0.000 007 501 204 166 16	-0.001 582 974 563 210 589	1
$H r i g h t$	-0.006 329 831 636 902 95	-1.782 753 041 151 986 00	889.456 257 268 860
	-0.245 557 952 355 265 80	-1.321 407 134 875 273 00	947.248 246 257 102
	-0.000 009 560 011 809 37	-0.001 836 388 848 579 45	1

表3

图18

图19

图20

表4

表5

参考文献 28

[1]	王向军，白皓月，吴凡璐，等. 基于改进球面透视投影的鱼眼图像畸变校正方法［J］. 图学学报， 2018， 39（1）： 43-49.
	WANG X J， BAI H Y， WU F L， et al. Fisheye lens distortion correction method based on improved spherical perspective project-tion［J］. Journal of Graphics， 2018， 39（1）： 43-49.
[2]	LARAQUI A， AZMI K， LARAQUI M. Image stitching based onparametric transformations［C］. Proceedings of the 2019 International Conference on Intelligent Systems and Advanced Computing Sciences. Rabat： IEEE， 2019： 1-8.
[3]	ZHU M L， YU X M. Multi-feature fusion algorithm in VR pan-oramic image detail enhancement processing［J］. IEEE Access， 2020， 8： 142055-142064.
[4]	赵三峰，谢明，陈玉明. 基于逆向投影的全景泊车系统设计与实现［J］. 计算机工程与应用， 2017， 53（23）： 267-270.
	ZHAO S F， XIE M， CHEN Y M. Design and implementation ofaround view parking system based on reverse projection algorithm［J］. Computer Engineering and Applications， 2017， 53（23）： 267-270.
[5]	ZHANG Z Y. A flexible new technique for camera calibration［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence， 2000， 22（11）： 1330-1334.
[6]	GAO J， KIM S J， BROWN M S， et al. Constructing image pan-oramas using dual-homography warping［C］. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition.Colorado Springs： IEEE， 2011： 49-56.
[7]	ZHANG F， LIU F. Parallax-tolerant image stitching［C］. Proceedings of the 2014 IEEE Conference on Computer Vision and Patt-ern Recognition. Columbus： IEEE， 2014： 1354-1361.
[8]	FU M Y， LIANG H， ZHU C H， et al. Image stitching techniqu-es applied to plane or 3-D models： a review［J］. IEEE Sensors Journal， 2023， 23（8）： 8060-8079.
[9]	LOWE D G. Distinctive image features from scale-invariant keyp-oints［J］. International Journal of Computer Vision， 2004， 60（2）： 91-110.
[10]	BAY H， ESS A， TUYTELAARS T， et al. Speeded-up robust fe-atures［J］. Computer Vision and Image Understanding， 2008， 110（3）：404-417.
[11]	RUBLEE E， RABAUD V， KONOLIGE K， et al. ORB： an effi-cient alternative to SIFT or SURF［C］. Proceedings of the 2011 IEEE International Conference on Computer Vision. Barcelona： IEEE， 2011： 2564-2571.
[12]	朱新峰，宋健. 轻量级图像超分辨率研究综述［J］. 计算机工程与应用， 2024， 60（16）： 49-60.
	ZHU X F，SONG J. Review of research on light weight image super-resolution［J］. Computer Engineering and Applications，2024，60（16）： 49-60.
[13]	LIANG J， CAO J， SUN G， et al. SwinIR： image restoration using Swin Transformer［C］. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans： IEEE， 2022： 1833-1844.
[14]	SUN B， ZHANG Y， JIANG S， et al. Hybrid pixel-unshuffled ne-twork for lightweight image super-resolution［C］. Proceedings of the 37th AAAI Conference on Artificial Intelligence. Washington D.C.： AAAI， 2023： 2375-2383.
[15]	DONG C， LOY C C， HE K， et al. Image super-resolution usingdeep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（2）： 295-307.
[16]	KIM J， LEE J K， LEE K M. Accurate image super-resolution u-sing very deep convolutional networks［C］. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas： IEEE， 2016： 1646-1654.
[17]	程德强，王培杰，董彦强，等. 基于多尺度空间注意力引导的图像超分辨率重建网络［J］. 北京航空航天大学学报， 2023.
	CHENG Deqiang， WANG Peijie， DONG Yanqiang， et al. Image super-resolution reconstruction based on multi-scale spatial attention guidance network［J］. Journal of Beijing University of Aeronautics and Astronautics， 2023.
[18]	WANG Z， BOVIK A C， SHEIKH H R， et al. Image quality as-sessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing， 2004， 13： 600-612.
[19]	HAO J， XIE J M， ZHANG J Y， et al. A stronger stitching algo-rithm for fisheye images based on deblurring and registration［C］. Proceedings of the 2023 IEEE Sensors Conference. Boston： IEEE，2023： 1-4.
[20]	王长波，高岩. 3D计算机图形学［M］. 北京：机械工业出版社， 2010： 49-50.
	WANG C B， GAO Y. 3D computer graphics［M］. Beijing： ChinaMachine Press， 2010： 49-50.
[21]	SONG D Y， UM G M， LEE K， et al. End-to-end image stitchi-ng network via multi-homography estimation［C］. Proceedings of the 2021 IEEE International Conference on Signal Processing. Sa-n Diego： IEEE， 2021： 763-767.
[22]	WANG Z Y， ZHANG S Y， FANG Z J， et al. Image stitching algorithm based on improved Harris corners extraction［C］. Procee-dings of the International Conference on Information Science andControl Engineering. Shanghai： China Computer Federation， 2020：714-718.
[23]	FISCHLER M A， BOLLES R C. Random sample consensus： a paradigm for model fitting with applications to image analysis and automated cartography［J］. Communications of the ACM， 1981， 24（6）： 381-395.
[24]	CHAND R K， KUMAR B， RANJEET M， et al. Implementation of panoramic image stitching using Python［C］. Proceedings of the 2023 International Conference on Computing Communication and Networking Technologies. Agartala： IEEE， 2023： 1-7.
[25]	MENG M， LIU S. High-quality panorama stitching based on asy-mmetric bidirectional optical flow［C］. Proceedings of the 2020 International Conference on Computational Intelligence and Applications. Beijing： IEEE， 2020： 118-122.
[26]	YANH S， KIM Y， JEONG J. Fine edge-preserving technique fordisplay devices［J］. IEEE Transactions on Consumer Electronics， 2008， 54（4）： 1761-1769.
[27]	DOSOVITSKIY A， FISCHER P， ILG E， et al. Optical flow esti-mation： from classical to deep learning approaches［J］. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence， 2023， 45（7）： 8234-8251.
[28]	XU X H. Multi-focus image fusion algorithm based on rough setand neural network［J］. IEEE Sensors Journal， 2020， 20（20）： 11967-11974.

项目	X/mm	Y/mm	Z/mm	Yaw	Pitch	Roll
前置摄像头	3 911.8	0	584.5	0°	16°	0°
后置摄像头	-1 030.2	50	576.6	0°	35°	0°
左置摄像头	2 134.7	1 044.3	1 011.7	90°	38°	0°
右置摄像头	2 107.7	-1 032.3	1 012.1	-90°	38°	0°

内参矩阵			畸变参数
959.018 127	0.0	768.257 690	0.073 492
0.0	457.107 941	456.914 093	-0.002 394
0.0	0.0	1.0	-0.001 477
			-0.000 430

组别	PSNR/dB		SSIM
组别	SFVT	MFVT	SFVT	MFVT
第1组	33.26	34.95	0.919 5	0.940 2
第2组	31.27	34.27	0.901 1	0.924 8
第3组	34.46	36.78	0.927 8	0.945 1
第4组	33.15	35.24	0.911 2	0.937 6
平均值	33.04	35.31	0.914 9	0.936 9