MSF-Diffuser：BEV下基于扩散模型的多传感器自适应融合自动驾驶方法

doi:10.19562/j.chinasae.qcgc.2025.06.011

Abstract

Abstract:

Autonomous driving algorithms are a major research focus in the field of intelligent vehicles. Currently， to achieve panoramic autonomous driving， most domestic approaches use multi-sensor fusion. However， existing solutions face problems such as low sensor utilization and unreasonable fusion strategies. For these problems， in this paper， an autonomous driving framework based on multi-sensor fusion （camera+LiDAR+Radar） under a bird's-eye view （BEV） is proposed. In this framework， dual encoding based on point and velocity is used， coupled with feature interaction to extract millimeter-wave radar point cloud features， thereby enhancing the utilization of millimeter-wave radar information and facilitating subsequent fusion. In the fusion module， LSTM is used to store the features from multiple modalities as well as the fused BEV features， which allows for the calculation of feature consistency loss between different modalities and continuity loss for the fused BEV features and historical frames， leading to smoother and more precise feature fusion. Finally， the diffusion model is introduced and the Multi-modal U-Net is proposed for denoising， which improves the robustness of trajectory planning. Extensive experiments are conducted using the CARLA simulator on the authoritative Longest-06 benchmark and Town-05 Long benchmark， getting a DS （Driving Score） of 73.80 $±$ 1.01 and 73.7 $±$ 1.3 respectively. The results show that the proposed approach achieves better panoramic autonomous driving with superior performance and flexibility compared to existing methods.

Key words: autonomous driving, multi-sensor fusion, feature interaction, diffusion model

Mingchen Wang,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. MSF-Diffuser: A Multi-sensor Adaptive Fusion Autonomous Driving Method Based on Diffusion Model Under BEV[J].Automotive Engineering, 2025, 47(6): 1122-1132.

Figures/Tables 13

方法	传感器	DS	IS	RC
Transfuser	C+L	47.30 $±$ 5.72	0.50 $±$ 0.06	93.38 $±$ 1.2
LAV	C+L	32.74 $±$ 1.45	0.51 $±$ 0.02	70.36 $±$ 3.14
Interfuser	C+L	53.2 $±$ 2.6	0.72 $±$ 0.03	73.6 $±$ 1.4
Transfuser++	C+L	70 $±$ 2.0	0.74 $±$ 0.02	95.0 $±$ 2.0
Thinktwice	C+L	61.3	0.81	73.0
BEVFuser	C+L+R	73.67 $±$ 1.29	0.76 $±$ 0.02	96.02 $±$ 0.8
本文方法	C+L+R	73.80 $±$ 1.01	0.78 $±$ 0.02	94.61 $±$ 0.5

方法	传感器	DS	IS	RC
VAD	C	30.31	0.40	75.20
Transfuser	C+L	51.2 $±$ 1.5	0.66 $±$ 0.01	81.1 $±$ 1.20
LAV	C+L	46.3 $±$ 2.5	0.67 $±$ 0.03	68.7 $±$ 3.4
Interfuser	C+L	68.3 $±$ 1.9	0.71 $±$ 0.01	95.0 $±$ 2.9
Transfuser++	C+L	71.0 $±$ 2.0	0.71 $±$ 0.02	100.0
Thinktwice	C+L	65.0 $±$ 1.7	0.69 $±$ 0.05	95.5 $±$ 2.0
ReasonNet	C+L	73.2 $±$ 1.9	0.76 $±$ 0.03	95.9 $±$ 2.3
BEVFuser	C+L+R	73.4 $±$ 2.0	0.75 $±$ 0.01	98.19 $±$ 0.73
本文方法	C+L+R	73.7 $±$ 1.3	0.76 $±$ 0.01	97.0 $±$ 0.69

扩散模型	DS
	71.98 $±$ 2.03
√	73.80 $±$ 1.01

References 22

1	JIANG B， CHEN S， XU Q， et al. VAD： vectorized scene representation for efficient autonomous driving［J］. arXiv， 2023.
2	LE D T， SHI H， CAI J， et al. DifFUSER： diffusion model for robust multi-sensor fusion in 3D object detection and BEV segmentation［J］. arXiv， 2024.
3	YANG J， GAO S， QIU Y， et al. GenAD： generalized predictive model for autonomous driving［J］. arXiv， 2024.
4	BAI X， HU Z， ZHU X， et al. TransFusion： robust LiDAR-camera fusion for 3D object detection with transformers［J/OL］. http：//arxiv.org/pdf/2203.11496.
5	CHEN S， SUN P， SONG Y， et al. DiffusionDet： diffusion model for object detection［C］.2023 IEEE/CVF International Conference on Computer Vision （ICCV）. Paris， France： IEEE， 2023： 19773-19786.
6	LIU Z， CAI Y， WANG H， et al. Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（7）： 6640-6653.
7	LIN Z， LIU Z， XIA Z， et al. RCBEVDet： radar-camera fusion in bird’s eye view for 3D object detection［J］. arXiv， 2024.
8	SHAO H， WANG L， CHEN R， et al. ReasonNet： end-to-end driving with temporal and global reasoning［C］.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， BC， Canada： IEEE， 2023： 13723-13733.
9	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780.
10	CHITTA K， PRAKASH A， JAEGER B， et al. TransFuser： imitation with transformer-based sensor fusion for autonomous driving［J］. arXiv， 2022.
11	HU Y， YANG J， CHEN L， et al. Planning-oriented autonomous driving［J/OL］. https：//arxiv.org/pdf/2212.10156
12	YANG C， CHEN Y， TIAN H， et al. BEVFormer v2： adapting modern image backbones to bird’s-eye-view recognition via perspective supervision［C］.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， BC， Canada： IEEE， 2023： 17830-17839.
13	WANG H， CHEN Z， CAI Y， et al. Voxel-RCNN-Complex： an effective 3-D point cloud object detector for complex traffic conditions［J］. IEEE Transactions on Instrumentation and Measurement， 2022， 71： 1-12.
14	WANG H， TAO L， PENG Y， et al. Voxel RCNN-HA： a point cloud multiobject detection algorithm with hybrid anchors for autonomous driving［J］. IEEE Transactions on Transportation Electrification， 2024， 10（3）： 7286-7296.
15	YAN J， LIU Y， SUN J， et al. Cross modal transformer： towards fast and robust 3D object detection［J］. arXiv， 2023.
16	WANG H， QIU M， CAI Y， et al. Sparse U-PDP： a unified multi-task framework for panoptic driving perception［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（10）： 11308-11320.
17	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［J］. arXiv， 2015.
18	JAEGER B， CHITTA K， GEIGER A. Hidden biases of end-to-end driving models［J］. arXiv， 2023.
19	SHAO H， WANG L， CHEN R， et al. Safety-enhanced autonomous driving using interpretable sensor fusion transformer［J］. arXiv， 2022.
20	HU C， ZHENG H， LI K， et al. FusionFormer： a multi-sensory fusion in bird’s-eye-view and temporal consistent transformer for 3D object detection［J］. arXiv， 2023.
21	CHEN D， KRÄHENBÜHL P. Learning from all vehicles［J］. arXiv， 2022.
22	JIA X， WU P， CHEN L， et al. Think twice before driving： towards scalable decoders for end-to-end autonomous driving［J］. arXiv， 2023.

[1]	Chenyu Liu,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Camera and Radar Fusion for Autonomous Driving Scenarios [J]. Automotive Engineering, 2025, 47(5): 829-838.
[2]	Qirui Qin,Hai Wang,Yingfeng Cai,Long Chen,Yicheng Li. Real-Time Instance Segmentation Algorithm for Autonomous Driving Based on Instance Activation Maps [J]. Automotive Engineering, 2025, 47(4): 614-624.
[3]	Jinhui Suo, Xiaowei Wang, Peiwen Jiang, Chi Ding, Ming Gao, Yougang Bian. Domain Adaptive Visual Object Detection for Autonomous Driving Based on Multi-granularity Relation Reasoning [J]. Automotive Engineering, 2025, 47(2): 201-210.
[4]	Jiangkun Li,Ruixue Zong,Weiwen Deng,Ying Wang,Juan Ding. Directed Graph-Based Method for Evaluating Similarity in Urban Intersection Scenarios [J]. Automotive Engineering, 2025, 47(1): 23-34.
[5]	Daofei Li,Hao Pan. Application of Scenario Complexity Evaluation in Trajectory Prediction and Automated Driving Decision-Making [J]. Automotive Engineering, 2024, 46(9): 1556-1563.
[6]	Hai Wang,Jianguo Li,Yingfeng Cai,Long Chen. A LiDAR-Based Dynamic Driving Scene Multi-task Segmentation Network [J]. Automotive Engineering, 2024, 46(9): 1608-1616.
[7]	Jianan Zhang,Zhaozheng Hu,Jie Meng,Huahua Hu,Jie Zuo. Distributed Simulation Platform Architecture and Application of Autonomous Driving for Vehicle-Road-Map Collaboration [J]. Automotive Engineering, 2024, 46(8): 1335-1345.
[8]	Le Tao,Hai Wang,Yingfeng Cai,Long Chen. Multi-object Detection Algorithm Based on Point Cloud for Autonomous Driving Scenarios [J]. Automotive Engineering, 2024, 46(7): 1208-1218.
[9]	Linhui Li,Yifan Fu,Ting Wang,Xuecheng Wang,Jing Lian. Trajectory Prediction Method Enhanced by Self-supervised Pretraining [J]. Automotive Engineering, 2024, 46(7): 1219-1227.
[10]	Luxing Li,Chao Wei. Multi-sensor Data Fusion for Intelligent Vehicles Based on Tripartite Graph Matching [J]. Automotive Engineering, 2024, 46(7): 1228-1238.
[11]	Hai Wang,Yuxuan Ding,Tong Luo,Meng Qiu,Yingfeng Cai,Long Chen. A Multi-class Multi-target Tracking Algorithm Combining Motion Speed and Appearance Features in Driving Scenarios [J]. Automotive Engineering, 2024, 46(6): 956-964.
[12]	Jing Huang,Xiangzhen Liu,Xiaoyang Deng,Ran Chen. Research on Intelligent Vehicle Trajectory Planning Based on Multimodal Trajectory Prediction [J]. Automotive Engineering, 2024, 46(6): 965-974.
[13]	Mengfan Li,Zhongxiang Feng,Weihua Zhang,Jingyu Li. Study on Driver's Visual Transfer Characteristics During the Takeover Process of Human-Computer Co-driving Mode [J]. Automotive Engineering, 2024, 46(5): 795-804.
[14]	Fuxing Yao,Chao Sun,Yungang Lan,Bing Lu,Bo Wang,Haiyang Yu. A Lane Change Decision Method for Intelligent Connected Vehicles Based on Mixture of Expert Model [J]. Automotive Engineering, 2024, 46(5): 882-892.
[15]	Ting Chikit,Yafei Wang,Yichen Zhang,Mingyu Wu,Yile Wang. Energy-Saving Planning Method for Autonomous Driving Mining Trucks Based on Composite Dynamic Sampling [J]. Automotive Engineering, 2024, 46(4): 588-595.

MSF-Diffuser: A Multi-sensor Adaptive Fusion Autonomous Driving Method Based on Diffusion Model Under BEV

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 22

Related Articles 15

Metrics

Comments

Recommended 0

方法	DS
PointNet	58.03
PointNet++	64.27
Voxel RCNN	60.59
本文方法	73.80

特征一致性损失和连续性损失	DS
	64.58
√	73.80