基于改进TD3算法的混动飞行汽车空地协调能量管理策略研究

doi:10.19562/j.chinasae.qcgc.2025.11.004

Abstract

Abstract:

For severe power fluctuation， challenges in SOC and temperature control， frequent engine start-stop， and poor fuel economy in hybrid flying vehicles under air-ground coordination， in this paper a series connected architecture based on a turboshaft engine-generator set and battery collaborative power supply is constructed. An improved TD3 strategy （KI-TD3） is proposed， guided by structural priors. By incorporating the prior information of the engine’s economic working zone， positive working point reward， dynamic exploration mechanism， and action space restriction method are constructed to enhance the performance of the strategy. The simulation results show that the KI-TD3 strategy achieves better power distribution and battery control. Compared to standard TD3， it ensures more accurate SOC convergence to the target value of 0.25， stabilizes the temperature rise， concentrates engine operation in efficient zones， and cuts fuel use by 3.5%. Compared to DP， it further reduces fuel use by 5.2%， suppresses start-stop and power spikes， and keeps operation near minimum BSFC， significantly improving economy.

Key words: hybrid flying vehicle, structural prior knowledge, energy management, fuel economy

Zhongkai Luan,Yukun Shen,Wanzhong Zhao,Chunyan Wang,Jianhao Zhou,Pengchang Jiang. Research on Energy Management Strategy for Hybrid Electric Flying Vehicles Based on an Improved TD3 Algorithm[J].Automotive Engineering, 2025, 47(11): 2093-2102.

Figures/Tables 14

算法：KI-TD3

1：初始化：Critic网络 $Q θ 1$ 、 $Q θ 2$ ，Actor网络 $π ?$ ，网格参数分别为 $θ 1$ 、 $θ 2$ 、 $?$ ，目标网络 $Q θ 1'$ 、 $Q θ 2'$ 、 $π ?'$ 网格参数 $θ 1' ← θ 1$ 、 $θ 2' ← θ 2$ 、 $?' ← ?$ ，清空经验回放池，加载发动机经济工作点集合 $s e x p$ ，设置最大探索噪声 $σ m a x$ ，缩放因子 $β$ 、动作空间 $A g l o b a l$ 、 $A e x p$ ，设步数计数器 $t = 0$

2：for episode=1：M do

3：初始化环境状态 $s t = 0$ ，初始化动作空间 $A t = A g l o b a l$

4：for t=1：T

5：计算当前状态 $s t$ 到 $s e x p$ 的最小距离 $d t$

6：计算探索噪声标准差 $σ t = σ m a x ? e - β d t$

7：规范动作 $a t ← c l i p (π ? (s t | ?) + N (0, σ t 2), A t)$ ，动态探索并限制可行空间

8：执行动作a_t，获得奖励r_t 和下一状态s_t₊₁

9：根据 $s e x p$ 中最近工作点，修正奖励r_t

10：存储转移 $(s t, a t, r t, s t + 1)$ 至经验池中

11：每过d步，进行一次参数更新

12：从经验池中采样N条数据 $(s i, a i, r i, s i + 1)$

13：生成目标动作 $a ? t + 1 ← π ?' (s t + 1) + c l i p (N (0, σ m a x 2))$

14：更新目标价值函数： $y t = r t + γ · m i n i ∈ {1,2} Q θ i' (s t + 1, a ? t + 1)$

15：更新Critic网络参数最小化损失：

$L (θ i) = N - 1 ∑ (y t - Q θ i (s t, a t)) 2$ ， $i ∈ 1,2$

16：策略梯度方法更新Actor网络：

$? ? J (?) = N - 1 ∑ [? ? Q θ 1 (s t, π ? (s t))]$

17：软更新目标网络： $?' = (1 - τ) ?' + τ ?$ 、 $θ i' = (1 - τ) θ i' + θ i$

18：动态动作空间调整： $A t = (1 - γ t) ? A g l o b a l + γ t ? A e x p$

19：end for

20：end for

References 24

[1]	张新钰，荣松松，李骏，等.智能飞行汽车关键技术及发展趋势［J］. 中国科学：技术科学， 2024， 54（4）： 601-624.
	ZHANG X Y， RONG S S， LI J， et al. State-of-the-art and technical trends of intelligent flying cars［J］. Sci Sin Tech， 2024， 54（4）： 601-624.
[2]	张扬军，钱煜平，诸葛伟林，等.飞行汽车的研究发展与关键技术［J］. 汽车安全与节能学报， 2020， 11（1）： 1-16.
	ZHANG Y J， QIAN Y P， ZHUGE W L， et al. Progress and key technologies of flying cars［J］. J Automot Saf Energy， 2020， 11（1）： 1-16.
[3]	张钰锋，李京阳，宝音贺西. 飞行汽车混动系统动力电池和增程器的协调控制［J］. 科学技术与工程， 2024， 24（16）： 6943-6950.
	ZHANG Y F， LI J Y， BAOYIN H X. Coordination and control of power batteries and processors in the flight car hybrid system［J］. Science Technology and Engineering， 2024， 24（16）： 6943-6950.
[4]	WANFG W， CHEN Y， YANG C， et al. An efficient optimal sizing strategy for a hybrid electric air-ground vehicle using adaptive spiral optimization algorithm［J］. Journal of Power Sources， 2022， 517： 230704.
[5]	WEI Z， MA Y， XIAN C， et al. Power prediction‐based model predictive control for energy management in land and air vehicle with turboshaft engine［J］. Complexity， 2021， 2021（1）： 2953241.
[6]	BIRADAR A， DEBITETTO P， PHAN L， et al. Hybrid-electric powered aerospace systems and the battery energy density revolution［C］. 2018 IEEE Aerospace Conference. IEEE， 2018： 1-6.
[7]	施德华，袁超，汪少华，等. 考虑驾驶风格的混合动力汽车强化学习能量管理策略［J］. 西安交通大学学报， 2024， 58（10）： 51-62.
	SHI D H， YUAN C， WANG S H， et al. Reinforcement learning-based energy management strategy considering driving style for hybrid electric vehicle［J］. J Xi'an Jiaotong Univ， 2024， 58（10）： 51-62.
[8]	FENG Y， DONG Z. Optimal energy management with balanced fuel economy and battery life for large hybrid electric mining truck［J］. Journal of Power Sources， 2020， 454： 227948.
[9]	YANG C， DU X， WANG W， et al. Variable optimization domain-based cooperative energy management strategy for connected plug-in hybrid electric vehicles［J］. Energy， 2024， 290： 130206.
[10]	李石磊，叶清，袁志民，等. 在线深度强化学习探索策略生成方法综述［J］. 机器人， 2024， 46（6）： 753-768.
	LI S L， YE Q， YUANG Z M， et al. Exploratory policy generation methods in on-line deep reinforcement learning： a survey［J］. Robot， 2024， 46（6）： 753-768.
[11]	丁世飞，杜威，张健，等. 多智能体深度强化学习研究进展［J］. 计算机学报， 2024， 47（7）： 1547-1567.
	DING S F， DU W， ZHANG J， et al. Research progress of multi-agent deep reinforcement learning［J］. J Comput Res Dev， 2024， 47（7）： 1547-1567.
[12]	HAN D， MULYANA B， STANKOVIC V， et al. A survey on deep reinforcement learning algorithms for robotic manipulation［J］. Sensors， 2023， 23（7）： 3762.
[13]	STEKOLSHCHIK R. Some approaches used to overcome overestimation in Deep Reinforcement Learning algorithms［J］. arXiv preprint arXiv：， 2020.
[14]	WANG H， HE H， BAI Y， et al. Parameterized deep Q-network based energy management with balanced energy economy and battery life for hybrid electric vehicles［J］. Applied Energy， 2022， 320： 119270.
[15]	王姝，韩琪，赵轩，等. 融合机器视觉的插电式混合动力汽车预测型能量管理策略［J］. 汽车工程， 2025， 47（4）： 625-635.
	WANG S， HAN Q， ZHAO X， et al. Predictive energy management strategy of plug-in hybrid electric vehicle with computer vision［J］. Automot Eng， 2025， 47（4）： 625-635.
[16]	TANG X， ZHANG J， PI D， et al. Battery health-aware and deep reinforcement learning-based energy management for naturalistic data-driven driving scenarios［J］. IEEE Transactions on Transportation Electrification， 2021， 8（1）： 948-964.
[17]	DENG K， LIU Y， HAI D， et al. Deep reinforcement learning based energy management strategy of fuel cell hybrid railway vehicles considering fuel cell aging［J］. Energy Conversion and Management， 2022， 251： 115030.
[18]	RUAN S， MA Y， WEI Z， et al. Data-driven cooperative differential game based energy management strategy for hybrid electric propulsion system of a flying car［J］. IEEE Transactions on Intelligent Transportation Systems， 2024.
[19]	RUAN S， MA Y， WEI Z， et al. Hierarchical control strategy for the hybrid electric propulsion system of a flying car with engine start-stop system and dynamic coordination［J］. IEEE Transactions on Transportation Electrification， 2023， 10（3）： 4904-4918.
[20]	YANG C， LU Z， WANG W， et al. An efficient intelligent energy management strategy based on deep reinforcement learning for hybrid electric flying car［J］. Energy， 2023， 280： 128118.
[21]	WEI Z， MA Y， YANG N， et al. Reinforcement learning based power management integrating economic rotational speed of turboshaft engine and safety constraints of battery for hybrid electric power system［J］. Energy， 2023， 263： 125752.
[22]	MOHAMED O， KHALIL A. Progress in modeling and control of gas turbine power generation systems： a survey［J］. Energies， 2020， 13（9）： 2358.
[23]	YANG Z P， ZHANG X Y， LI J K. Modeling and simulation of heavy duty gas turbine［C］. 2021 6th International Conference on Power and Renewable Energy （ICPRE）. IEEE， 2021： 1495-1500.
[24]	JI F， ZHANG X， DU F， et al. Experimental and numerical investigation on micro gas turbine as a range extender for electric vehicle［J］. Applied Thermal Engineering， 2020， 173： 115236.

参数	数值
车辆质量M/kg	1 800.00
重力加速度g/（m·s^-2）	9.81
滚动阻力系数f	0.018
阻力系数C_d	0.51
正面面积A_a/m²	7.80
车轮半径r/m	0.39
垂直面积A_u/m²	32.00
水平飞行迎风面积A/m²	12.40
转动阻力系数C_Q	0.002 6
旋翼半径R/m	0.90
旋翼设计转速/（r·min^-1）	2 400
转子电机额定功率P_R/kW	120
轮毂电机额定功率P_W/kW	50

参数	数值
单节容量C_cell/（A·h）	4.8
单节质量m_cell/g	68
串联节数N_s	192
并联节数N_p	15
单节电压范围U_min~max/V	3.0~4.2
持续最大放电倍率	5C
瞬时最大放电倍率	8C
持续最大充电倍率	3C
瞬时最大充电倍率	4C

项目	DP	TD3	KI-TD3
启停次数	16	5	6
燃油消耗/kg	69.8	68.6	66.2

Research on Energy Management Strategy for Hybrid Electric Flying Vehicles Based on an Improved TD3 Algorithm

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 24

Related Articles 15

Metrics

Comments

Recommended 0

[1]	Jing Lian,Peng Yang,Linhui Li,Yafu Zhou,Xuesong Sun. Energy Management Strategy Based on Plug-In Fuel Cell Buses [J]. Automotive Engineering, 2025, 47(7): 1305-1316.
[2]	Shu Wang,Qi Han,Xuan Zhao,Penghui Xie. Predictive Energy Management Strategy of Plug-in Hybrid Electric Vehicle with Computer Vision [J]. Automotive Engineering, 2025, 47(4): 625-635.
[3]	Yong Cui,Jun Liu,Chao Wang,Yibao Wang,Jianwei Liu,Linchuan Zhang. Study on the Effect of Low Viscosity Lubricating Oil on Fuel Economy and Durability of Heavy Commercial Diesel Engine [J]. Automotive Engineering, 2024, 46(5): 935-944.
[4]	Ting Chikit,Yafei Wang,Yichen Zhang,Mingyu Wu,Yile Wang. Energy-Saving Planning Method for Autonomous Driving Mining Trucks Based on Composite Dynamic Sampling [J]. Automotive Engineering, 2024, 46(4): 588-595.
[5]	Yaxiong Wang,Yiying Fan,Kai Ou,Zhongbao Wei,Jiujun Zhang. Research Progress on Traffic Information-Integrated Energy Management for Fuel Cell Vehicles [J]. Automotive Engineering, 2024, 46(12): 2314-2328.
[6]	Jinxia Liu,Zhihao Liang,Qiang Wang,Mingxing Wen. Research on the Equivalent Fuel Consumption Minimizing Energy Management Strategy of Through-the-Road Vehicle [J]. Automotive Engineering, 2023, 45(2): 285-292.
[7]	Chunyang Qi,Chuanxue Song,Shixin Song,Liqiang Jin,Da Wang,Feng Xiao. Research on Energy Management Strategy for Hybrid Electric Vehicles Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2023, 45(10): 1954-1964.
[8]	Shaobo Xie,Pengcheng Qu,Jiacheng Li,Huiqing Wang,Kun Lang. Study on Coordinated Control of Speed Planning and Energy Management for Connected Hybrid Electric Truck in Vehicle Following Scene [J]. Automotive Engineering, 2022, 44(8): 1136-1143.
[9]	Xiaodong Wei,Bo Liu,Jianghao Leng,Xingyu Zhou,Chao Sun,Fengchun Sun. Research on Eco-driving of Fuel Cell Vehicles via Convex Optimization [J]. Automotive Engineering, 2022, 44(6): 851-858.
[10]	Yong Chen,Changyin Wei,Xiaoyu Li,Yanlin Li,Caixia Liu,Xiaozhe Lin. Research on Fuzzy Energy Management Strategy for Extended-Range Electric Vehicles with Driving Condition Identification [J]. Automotive Engineering, 2022, 44(4): 514-524.
[11]	Ximing Cheng,Wei Hu,Jun Zhai,Ronghua Luo,Pan Zhang,Ye Xu. Study on the Efficiency of Low-voltage Electric System in Battery Electric Vehicles [J]. Automotive Engineering, 2022, 44(4): 601-608.
[12]	Zhe Zhang,Haitao Ding,Niaona Zhang,Konghui Guo. Economic Cruising Speed Planning of Intelligent Network Connected Electric Vehicle [J]. Automotive Engineering, 2022, 44(4): 609-616.
[13]	Guogui You,Junshou Su,Tairong Chen. Research on Fuel Economy Enhancement of a Heavy Semi-trailer Tractor [J]. Automotive Engineering, 2022, 44(2): 171-178.
[14]	Junqiu Li,Jiwei Liu,Chaofeng Zhu. Research on Energy Management of Hybrid Electric Vehicle Based on Time-Varying Chassis Configuration [J]. Automotive Engineering, 2022, 44(12): 1866-1876.
[15]	Guogui You,Jianwei Lu,Junshou Su,Mingyong Luan,Shuai Huang. Research on Driving Cycles of Heavy Semi-trailer Tractor and Optimization of Its Powertrain System [J]. Automotive Engineering, 2022, 44(11): 1735-1745.