基于效率优化的混合动力车辆强化学习能量管理策略研究

doi:10.19562/j.chinasae.qcgc.2021.07.012

Abstract

Abstract:

Taking the power split hybrid electric vehicle as the object ，this paper establishes the model for calculating the system comprehensive efficiency and proposes an efficiency optimization based energy management strategy with reinforcement learning. Firstly， the efficiency model of key components and the efficiency model of coupling mechanism are established. Based on the general structure of stepless speed regulation of composite transmission， the influence law of power splitting coefficient on the efficiency is analyzed， and the system comprehensive efficiency model is further constructed. Then with efficiency optimization as the goal， an energy management strategy based on reinforcement learning is proposed. Simulation comparisons are implemented， and the results show that the proposed strategy can achieve excellent fuel economy while maintaining battery SOC within a smaller fluctuation range. Finally， a test bench is built and the test results prove the correctness of the established efficiency model and the effectiveness of the proposed energy management strategy.

Key words: hybrid electric vehicle, efficiency optimization, energy management strategy, reinforcement learning

Ningkang Yang,Lijin Han,Hui Liu,Xin Zhang. Research on Efficiency Optimization Based Energy Management Strategy for a Hybrid Electric Vehicle with Reinforcement Learning[J].Automotive Engineering, 2021, 43(7): 1046-1056.

Figures/Tables 25

特征	$k ε 1$	$k ε 2$	$k ε$
$ε 1 < 1, ε 2 < 1$	$1 - ε 1 + ε 2$	$1 + ε 1 - ε 2$	$ε 12 + ε 22 + 1 - ε 1 - ε 2$
$???????? ε 2 > 1 α a 1 > α', α a 2 > α'$	$2 - ε 1$	$ε 1$	$ε 12 - ε 1 + ε 2$
$???????? ε 1 > 1 α a 1 > α', α a 2 > α'$	$ε 2$	$2 - ε 2$	$ε 22 + ε 1 - ε 2$
$α a 1 < α' α a 2 > α'$	2	0	$ε 1 + ε 2$
$α a 1 > α' α a 2 < α'$	0	2	$ε 1 + ε 2$

算法参数： $α ∈ 0,1$ ， $γ ∈ (0,1)$ ，策略b，循环次数N。
对于所有的 $s ∈ S, ? a ∈ A$ ，任意初始化 $Q (s, a)$
对于N=1，2，…，循环：对于每一幕，循环：
S_t←当前状态
根据S_t，依照策略π选择动作A_t 执行动作A_t，计算奖励R_t，观察状态S_t+1
更新 $Q (S t, A t)$ 依照 $Q (S t, A t) ← Q (S t, A t) + α R t + γ m a x a Q (S t + 1, a) - Q (S t, A t)$
直至终结状态
直至所有 $Q (s, a)$ 收敛

References 19

1	MASHADI B， EMADI S. Dual⁃mode power⁃split transmission for hybrid electric vehicles［J］. IEEE Transactions on Vehicular Technology，2010，59（7）： 3223-3232.
2	KIM N D， KIM J M， KIM H S. Control strategy for a dual⁃mode electromechanical， infinitely variable transmission for hybrid electric vehicles［J］. Journal of Automobile Engineering， 2008， 222（9）： 1587-1601.
3	朱元，田光宇，等. 行星齿轮结构的混合动力汽车的系统效率［J］. 汽车工程， 2004， 26（3）： 260-265.
	ZHU Y， TIAN G Y，et al. A research on the system efficiency of a hybrid electric vehicle with power⁃split powertrain［J］. Automotive Engineering， 2004， 26（3）： 260-265.
4	阚振广，马彪. 车辆传动装置功率损失建模计算［J］. 车辆与动力技术， 2003（2）： 25-28.
	KAN Z G， MA B. Modelling and calculation of vehicle transmission watt loss［J］. Vehicle & Power Technology， 2003（2）： 25-28.
5	TROVÃO J P， PEREIRINHA P G， JORGE H M， et al. A multi⁃level energy management system for multi⁃source electric vehicles⁃an integrated rule⁃based meta⁃heuristic approach［J］. Applied Energy， 2013， 105： 304-318.
6	YU H， TARSITANO D， HU X， et al. Real time energy management strategy for a fast charging electric urban bus powered by hybrid energy storage system［J］. Energy， 2016， 112： 322-331.
7	PATIL R M， FILIPI Z H， FATHY K. Comparison of supervisory control strategies for series plug⁃in hybrid electric vehicle powertrains through dynamic programming［J］. IEEE Transactions on control systems technology， 2013， 22（2）： 502-509.
8	CHEN Z， MI C， XIONG R， et al. Energy management of a power⁃split plug⁃in hybrid electric vehicle based on genetic algorithm and quadratic programming［J］. Journal of Power Sources， 2014， 248（7）： 416-426.
9	CHEN S Y， WU C H， HUNG Y H， et al. Optimal strategies of energy management integrated with transmission control for a hybrid electric vehicle using dynamic particle swarm optimization［J］. Energy，2018， 160： 154-170.
10	ZOU Y， LIU T， SUN F C， et al. Comparative study of dynamic programming and Pontryagin’s minimum principle on energy management for a parallel hybrid electric vehicle［J］. Energies， 2013， 6（4）： 2305-2318.
11	LIU H， LI X， WANG W， et al. Adaptive equivalent consumption minimisation strategy and dynamic control allocation⁃based optimal power management strategy for four⁃wheel drive hybrid electric vehicles［J］. Proceedings of the Institution of Mechanical Engineers， Part D： Journal of Automobile Engineering，2019， 233（12）： 3125-3146.
12	LI X， HAN L， LIU H， et al. Real⁃time optimal energy management strategy for a dual⁃mode power⁃split hybrid electric vehicle based on an explicit model predictive control algorithm［J］. Energy，2019，172： 1161-1178.
13	XU B， et al. Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle［J］. Applied Energy， 2020， 259： 114200.
14	WU Y， TAN H， PENG J， et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series⁃parallel plug⁃in hybrid electric bus［J］. Applied Energy， 2019， 247： 454-466.
15	姚建初，陈义保，周济，等. 齿轮传动啮合效率计算方法的研究［J］.机械工程学报， 2001， 37（11）： 18-21.
	YAO J C， CHEN Y B， ZHOU J， et al. Study on a method for calculating gearing meshing efficiency ［J］. Journal of Mechanical Engineering， 2001， 37（11）： 18-21.
16	HU J B， PENG Z X， YUAN S H. Drag torque prediction model for the wet clutches［J］. Chinese Journal of Mechanical Engineering， 2009， 22（2）： 238-243.
17	ROST U. Das Viskositaets-Temperaturverhalten von Schmieroelen ［J］. Erdoel und Kohle， 8： 650-651.
18	郭宏榆，姜久春，王吉松，等. 功率型锂离子动力电池的内阻特性［J］. 北京交通大学学报， 2011， 35（5）.
	GUO H Y， JIANG J C， WANG J S， et al. Characteristic on internal resistance of lithium⁃ion power battery［J］. Journal of Beijing Jiaotong University， 2011， 35（5）.
19	SUTTON R S， BARTO A G. Reinforcement learning： an introduction［M］. MIT Press， 2018.

策略	最终SOC	等效油耗/L	相对值/%	综合效率
基于规则	0.644 1	4.733	100	0.279 9
基于强化学习	0.607 3	4.327	91.42	0.306 2

试验设备	相关参数
发动机	涡轮增压柴油发动机（BF04M1013FC），120 kW/（2 300 r·min^-1），4.76L
电机	永磁同步电机（A、B相同），额定功率60 kW，额定转速2 600 r/min，峰值功率110 kW，峰值转速5 500 r/min，峰值转矩400 N·m
电池组	三元聚合锂离子电池，电压360 V，容量36 A·h
加载转置	电涡流测功机，440 kW，6 500 r/min
转速转矩传感器	型号：HX901，量程：0~2 000 N·m，0~5 000 r/min
直流电流传感器	型号：LHB?300A，量程：0~300 A
直流电压传感器	型号：KDV?1 000 V，量程：0~1 000 V
综合控制器	RapidECU S1，芯片型号：MPC5674F，32位，160 MHz，4通道，4 MB

效率误差	传统方法	本文方法
平均绝对误差	0.038 9	0.010 1
最大绝对误差	0.052 2	0.017 9

方法	最终SOC	等效油耗/L	偏差/%
试验	0.604 4	4.810	10.04
仿真	0.607 3	4.327

[1]	Weiguo Liu,Zhiyu Xiang,Weiping Liu,Daoxin Qi,Zixu Wang. Research on Vehicle Control Algorithm Based on Distributed Reinforcement Learning [J]. Automotive Engineering, 2023, 45(9): 1637-1645.
[2]	Lisheng Jin,Guangde Han,Xianyi Xie,Baicang Guo,Guofeng Liu,Wentao Zhu. Review of Autonomous Driving Decision-Making Research Based on Reinforcement Learning [J]. Automotive Engineering, 2023, 45(4): 527-540.
[3]	Yapeng Li,Xiaolin Tang,Xiaosong Hu. Study on Eco-driving of PHEVS Based on Hierarchical Control Strategy [J]. Automotive Engineering, 2023, 45(4): 551-560.
[4]	Jinxia Liu,Zhihao Liang,Qiang Wang,Mingxing Wen. Research on the Equivalent Fuel Consumption Minimizing Energy Management Strategy of Through-the-Road Vehicle [J]. Automotive Engineering, 2023, 45(2): 285-292.
[5]	Jie Li,Xiaodong Wu,Min Xu,Yonggang Liu. Reinforcement Learning Based Multi-objective Eco-driving Strategy in Urban Scenarios [J]. Automotive Engineering, 2023, 45(10): 1791-1802.
[6]	Chunyang Qi,Chuanxue Song,Shixin Song,Liqiang Jin,Da Wang,Feng Xiao. Research on Energy Management Strategy for Hybrid Electric Vehicles Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2023, 45(10): 1954-1964.
[7]	Zhenhai Gao,Xiangtong Yan,Fei Gao. A Decision-making Method for Longitudinal Autonomous Driving Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2022, 44(7): 969-975.
[8]	Jiangkun Li,Weiwen Deng,Bingtao Ren,Wenqi Wang,Juan Ding. Automatic Driving Edge Scene Generation Method Based on Scene Dynamics and Reinforcement Learning [J]. Automotive Engineering, 2022, 44(7): 976-986.
[9]	Yong Chen,Changyin Wei,Xiaoyu Li,Yanlin Li,Caixia Liu,Xiaozhe Lin. Research on Fuzzy Energy Management Strategy for Extended-Range Electric Vehicles with Driving Condition Identification [J]. Automotive Engineering, 2022, 44(4): 514-524.
[10]	Zhichao Liu,Tianlei Zheng,Shaohui Liu,Huiming Gong. Research on Energy Consumption and Driving Range Evaluation Method of Plug-in Hybrid Electric Passenger Car [J]. Automotive Engineering, 2022, 44(2): 161-170.
[11]	Dongjian Song,Bing Zhu,Jian Zhao,Jiayi Han,Yanchen Liu. Human-Like Behavior Decision-Making of Intelligent Vehicles Based on Driving Behavior Generation Mechanism [J]. Automotive Engineering, 2022, 44(12): 1797-1808.
[12]	Junqiu Li,Jiwei Liu,Chaofeng Zhu. Research on Energy Management of Hybrid Electric Vehicle Based on Time-Varying Chassis Configuration [J]. Automotive Engineering, 2022, 44(12): 1866-1876.
[13]	Dehua Shi,Xiangwei Rong,Shaohua Wang,Kaimei Zhang,Long Chen,Chun Li. Research on the Optimal Dynamic Coordinated Control of Power Split Hybrid Electric Vehicles with Dual-Clutch Collaboration [J]. Automotive Engineering, 2022, 44(12): 1877-1888.
[14]	Lijun Qian,Jian Chen,Bing Wu,Liang Xuan,Chen Chen,Liangliang Chen. Eco⁃driving Control for Hybrid Electric Vehicle Platoon with Consideration of Driver Operation Error [J]. Automotive Engineering, 2021, 43(7): 1037-1045.
[15]	Shaobo Xie,Kangkang Zhang,Qiankun Zhang,Huiran Luo. Study on Energy Management Strategy for Parallel Plug⁃in Hybrid Electric Vehicles Considering Battery Electric⁃Thermal⁃Depth⁃of⁃Discharge [J]. Automotive Engineering, 2021, 43(6): 791-798.

Research on Efficiency Optimization Based Energy Management Strategy for a Hybrid Electric Vehicle with Reinforcement Learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 25

References 19

Related Articles 15

Metrics

Comments

Recommended 10