基于自适应动态规划的HEV能量管理研究综述*

doi:10.19562/j.chinasae.qcgc.2020.11.006

Abstract

Abstract: Hybrid electric vehicles integrate the both advantages of fuel vehicles and battery electric vehicles, which can not only effectively reduce fuel consumption while meeting power demand, but also achieve long-distance driving. The energy management strategy of hybrid electric vehicles can achieve better power distribution, so that the engine and electric motor can work in high efficiency regions. Current energy management strategies can be divided into three categories: rule-based, optimization-based and artificial intelligence-based control strategies. In this paper, the adaptive dynamic programming (ADP) algorithm in artificial intelligence-based category is summarized, the structural development, and the algorithm implementation and the application to energy management strategy research of ADP algorithm are analyzed, with its present inadequacies and future development trends pointed out.

Key words: hybrid electric vehicle, energy management strategy, adaptive dynamic programming, research review

Jin Hui, Zhang Zihao. Review of Research on HEV Energy Management Based on Adaptive Dynamic Programming[J].Automotive Engineering, 2020, 42(11): 1490-1496.

References

[1] IEA. CO₂ emissions from fuel combustion [R]. 2019 Highlights, IEA, Paris. DOI: https://webstore. iea. org/co2-emissions-from-fuel-combustion-2019-highlights.
[2] 安锋. 中国传统燃油汽车退出时间表研究[R]. 能源与交通创新中心(iCET),2019.
AN Feng. A study on China’s timetable for phasing-out traditional ice-vehicles[R]. Innovation Center For Energy and Transportation(iCET),2019.
[3] 袁忠. 纯电动汽车的发展及其关键技术[J]. 时代汽车,2019(18):67-68.
YUAN Zhong. The development of pure electric vehicle and its key technologies[J]. New Energy Automobile, 2019(18):67-68.
[4] 邱晨曦. 氢燃料电池汽车的发展与展望[J]. 汽车实用技术,2019(23):25-27,48.
QIU Chenxi. Development and prospect of hydrogen fuel cell vehicles[J]. Automobile Applied Technology, 2019(23):25-27,48.
[5] 王磊. 一种混联式混合动力客车能量管理及模式切换协调控制研究 [D]. 上海:上海交通大学,2013.
WANG Lei. Research on energy management strategy and mode transition control for a series-parallel hybrid electric bus[D]. Shanghai:Shanghai Jiao Tong University, 2013.
[6] SABRI M F M , DANAPALASINGAM K A, RAHMAT M F. A review on hybrid electric vehicles architecture and energy management strategies[J]. Renewable and Sustainable Energy Reviews,2016,53:1433-1442.
[7] ALI A M , DIRK S. Realtime application of progressive optimal search and adaptive dynamic programming in multi-source HEVs[C]. ASME 2017 Dynamic Systems and Control Conference, 2017:1-10.
[8] DU Guodong, ZOU Yuan, ZHANG Xudong, et al. Intelligent energy management for hybrid electric tracked vehicles using online reinfo-rcement learning[J]. Applied Energy,2019,251.
[9] XIONG Rui, CHEN Huan, WANG Chun, et al. Towards a smarter hybrid energy storage system based on battery and ultracapacitor - acritical review on topology and energy management[J]. Journal of Cleaner Production,2018,202: 1228-1240.
[10] LIN Chanchiao, PENG Huei, GRIZZLE J W. Power management strategy for a parallel hybrid electric truck[J]. IEEE Transctions onControl Systems Technology, 2003, 6(11): 839-849.
[11] CHRENKO D, GAN S, GUTENKUNST C, et al. Novel classification of control strategies for hybrid electric vehicles[C]. 2015 IEEE Vehicle Power and Propulsion Conference (VPPC), Montreal, QC, 2015:1-6.
[12] 刘洋. 基于自适应动态规划控制策略的混合动力电动汽车能量管理研究[D]. 长春:长春工业大学,2018.
LIU Yang. Research on energy management strategy for hybrid electric vehicle based on adaptive dynamic programming algorithm[D]. Changchun :Changchun University of Technology, 2018.
[13] BELLMAN R. Dynamic programming [M]. Courier Corporation, 2013.
[14] 徐萍萍,邹伟,宋建国,等. 基于动态规划的PHEV全程能量管理策略研究[J]. 高技术通讯, 2010, 20(10):1096-1100.
XU Pingping, ZOU Wei, SONG Jianguo, et al. Research on trip based optimal power management for plug-in electric vehicles[J]. High Technology Letters, 2010, 20(10):1096-1100.
[15] 张静,于浩. 基于等效燃油消耗最小算法的并联式混合动力卡车控制策略[J]. 科学技术与工程,2019,19(18):302-308.
ZHANG Jing, YU Hao. Control strategy of parallel hybrid electric truck based on ECMS algorithm[J]. Science Technology and Engineering, 2019,19(18):302-308.
[16] 申彩英. 串联混合动力汽车能量优化管理策略研究[D]. 天津:天津大学,2010.
SHEN Caiying. Study on energy optimal management for series hybrid electric vehicles[D]. Tianjin :Tianjin University,2010.
[17] 席利贺,张欣,吴建政,等. 基于动态规划与神经网络的增程式电动汽车能量管理策略研究[J]. 公路交通科技,2018,35(9):128-136.
XI Lihe, ZHANG Xin, WU Jianzheng, et al. Study on energy management strategy for extended-range electric vehicle based on dynamic programming and neural network[J]. Journal of Highway and Transportation Research and Development, 2018,35(9):128-136.
[18] SUTTON R S, BARTO A G. Reinforcement learning:an introduction[M]. The MIT Press, 2018.
[19] 张化光,张欣,罗艳红,等. 自适应动态规划综述[J]. 自动化学报, 2013,39 (4):303-311.
ZHANG Huaguang, ZHANG Xin, LUO Yanhong, et al. An overview of research on adaptive dynamic programming[J]. Acta Automatica Sinica, 2013,39 (4):303-311.
[20] WEI Q, LIU D, LEWIS F L, et al. Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids[J]. IEEE Transactions on Industrial Electronics,2017,64(5): 4110-4120.
[21] WU Nan, WANG Honglei. Deep learning adaptive dynamic programming for real time energy management and control strategy of microgrid[J]. Journal of Cleaner Production, 2018,204:1169- 1177.
[22] ZENG Peng, LI Hepeng , HE Haibo, et al. Dynamic energy management of a microgrid using approximate dynamic programming and deep recurrent neural network learning[J]. IEEE Transactions on Smart Grid, 2019,10(4):4435-4245.
[23] ZHU Yuanheng , ZHAO Dongbin , LI Xiangjun , et al. Control-limited adaptive dynamic programming for multi-battery energy storage systems[J]. IEEE Transactions on Smart Grid, 2019,10(4):4235-4244.
[24] LI G, GORGES D. Ecological adaptive cruise control and energy management strategy for hybrid electric vehicles based on heuristic dynamic programming[J]. IEEE Transactions on Intelligent Tra-nsportation Systems, 2019,20(9):3526-3535.
[25] 刘念,刘春生,孙景亮. 自适应动态规划算法在飞行器追逃中的应用[J]. 飞行力学,2016,34(6):45-48,62.
LIU Nian, LIU Chunsheng, SUN Jingliang. Application of adaptive dynamic programming algorithm in the pursuit-evasion of aircraft[J]. Flight Dynamics, 2016,34(6):45-48,62.
[26] 陈燕妮,刘春生,孙景亮. 基于自适应最优控制的有限时间微分对策制导律[J]. 控制理论与应用,2019,36(6):877-884.
CHEN Yanni, LIU Chunsheng, SUN Jingliang. Finite-time differential guidance law based on adaptive optimal control[J]. Control Theory & Applications, 2019,36(6):877-884.
[27] 魏阿龙. 基于自适应动态规划的无人机编队最优协同控制研究[D]. 南京:南京航空航天大学,2018.
WEI Along. Research on adaptive dynamic programming-based optimal cooperative control of UAV formation[D]. Nanjing :Nanjing University of Aeronautics and Astronautics,2018.
[28] 孙景亮,刘春生. 基于自适应动态规划的导弹制导律研究综述[J]. 自动化学报,2017,43(7):1101-1113.
SUN Jingliang, LIU Chunsheng. An overview on the adaptive dynamic programming based missile guidance law[J]. Acta Automatica Sinica, 2017,43(7):1101-1113.
[29] 穆朝絮,张勇,余瑶,等. 基于自适应动态规划的航空航天飞行器鲁棒控制研究综述[J]. 空间控制技术与应用,2019,45(4):71-79.
MU Chaoxu, ZHANG Yong, YU Yao, et al. An overview on robust control of aviation and aerospace aircraft based on adaptive dynamic programming[J]. Aerospace Control and Application, 2019,45(4):71-79.
[30] 赵冬斌,刘德荣,易建强. 基于自适应动态规划的城市交通信号优化控制方法综述[J]. 自动化学报,2009,35(6):676-681.
ZHAO Dongbin, LIU Derong, YI Jianqiang. An overview on the adaptive dynamic programming based urban city trac signal optimal control[J]. Acta Automatica Sinica, 2009,35(6):676- 681.
[31] WERBOS P J. Advanced forecasting methods for globalcrisis warning and models of intelligence[J]. General Systems Yearbook,1977,22:25-38.
[32] HE Haibo, NI Zhen, FU Jian. A three-network architecture for online learning and optimization based on adaptive dynamic programming[J]. Neurocomputing,2011,78(1):3-13.
[33] PADHI R, UNNIKRISHNAN N, WANG X H, et al. A singlenetwork adaptive critic (SNAC) architecture for optimal control synthesis for aclass of nonlinear systems[J]. Neural Networks, 2006,19(10): 1648-1660.
[34] PADHI R, BALAKRISHNAN S N. Optimal management of beaver population using a reduced-order distributed parameter model and singlenetwork adaptive critics[J]. IEEE Transactions on Control Systems Technology, 2006, 14(4):628-640.
[35] WANG Feiyue, ZHANG Huaguang, LIU Derong. Adaptive dynamic programming: an Introduction[J]. IEEE Computational Intelligence Magazine, 2009, 4(2):39-47.
[36] 刘毅,章云. 基于值迭代的自适应动态规划的收敛条件[J]. 广东工业大学学报,2017,34(5):10-14.
LIU Yi, ZHANG Yun. Convergence condition of value-iteration based adaptive dynamic programming[J]. Journal of Guangdong University of Technology, 2017,34(5):10-14.
[37] LI W, XU G, WANG Z, et al. Dynamic energy management for hybrid electric vehicle based on approximate dynamic programming[C]. 2008 7th World Congress on Intelligent Control and Automa-tion, Chongqing, 2008:7864-7869.
[38] LIU T, ZOU Y, LIU D, et al. Reinforcement learning of adaptive energy management with transition probability for a hybrid electric tracked vehicle[J]. IEEE Transactions on Industrial Electronics,2015, 62(12):7837-7846.
[39] YIN Yanli, RAN Yan, ZHANG Liufeng, et al. An energy management strategy for a super-mild hybrid electric vehicle based on a dnown model of reinforcement learning[J]. Journal of Control Science and Engineering,2019:1-12.
[40] ZOU Y , LIU T , LIU D , et al. Reinforcement learning-based real -time energy management for a hybrid tracked vehicle[J]. Applied Energy, 2016, 171:372-382.
[41] LIU Teng, WANG Bo, YANG Chenglang. Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning[J]. Energy,2018,160: 544-555.
[42] DU Guodong, ZOU Yuan, ZHANG Xudong, et al. Intelligent energy management for hybrid electric tracked vehicles using online reinfor-cement learning[J]. Applied Energy,2019,251:1-16.
[43] LI W , XU G , XU Y . Online learning control for hybrid electricvehicle[J]. Chinese Journal of Mechanical Engineering, 2012, 25(1):98-106.
[44] LI G , Görges D. Fuel-efficient gear shift and power split strategy for parallel HEVs based on heuristic dynamic programming and neural networks[J]. IEEE Transactions on Vehicular Techn-ology,2019,68(10): 9519-9528.
[45] LI G, Görges D. Energy management strategy for paral-lel hybrid electric vehicles based on approximate dynamic progra-mming and velocity forecast[J]. Journal of the Franklin Institute,2019,356(16):9502-9523.
[46] 蔡岗. 动作依赖型自适应动态规划方法的实时控制应用研究[D]. 南宁:广西大学, 2009.
CAI Gang. The applied research of real-time controller via action- depended approximate dynamic programming method[D]. Nanning:Guangxi University, 2009.

[1]	WANG Dan, LIU Zhong-Chang, TIAN Jing, HAN Yong-Qiang, TAN Man-Zhi. [J]. , 2016, 38(12): 1415 -1419 .
[2]	KONG Xiang-Dong, ZHANG Zhen-Dong, XIE Nai-Liu, CHENG Qiang, YIN Cong-Bo. [J]. , 2016, 38(12): 1471 -1476 .
[3]	LI Ling, MA Li, MOU Yu, XU Chao, LI Wen-Ru, SHI Shu-Ming. [J]. , 2016, 38(12): 1508 -1514 .
[4]	GAO Ji-Dong, GAO Bo-Lin, XIE Shu-Gang. [J]. , 2016, 38(12): 1515 -1520 .
[5]	QIN Xiao-Hui, WANG Jian-Qiang, XIE Bo-Yuan, HU Man-Jiang, LI Ke-Qiang. [J]. , 2017, 39(1): 73 -78 .
[6]	SU Chang, FU Li-Ming, WEI Jun, CAO Yue, HUANG Lei, LI Shuo. [J]. , 2017, 39(1): 35 -40 .
[7]	ZHANG Jian, ZHOU Dao-Kui, TONG Rui, QI Hao-Chen. [J]. , 2017, 39(1): 47 -51 .
[8]	ZHENG Wei. [J]. , 2017, 39(1): 107 -112 .
[9]	SHAO Shi-Ping, HUANG Ju-Hua, CAO Ming. [J]. , 2017, 39(2): 200 -205 .
[10]	Chen-Xin, FENG Xiao, SHEN Chuan-Liang, WANG Shuo, HU Cui-Song, LI Yan-Yang, YANG Chang-Hai-　. [J]. , 2017, 39(2): 206 -213 .

Review of Research on HEV Energy Management Based on Adaptive Dynamic Programming

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 10

[1]	Yapeng Li,Xiaolin Tang,Xiaosong Hu. Study on Eco-driving of PHEVS Based on Hierarchical Control Strategy [J]. Automotive Engineering, 2023, 45(4): 551-560.
[2]	Jinxia Liu,Zhihao Liang,Qiang Wang,Mingxing Wen. Research on the Equivalent Fuel Consumption Minimizing Energy Management Strategy of Through-the-Road Vehicle [J]. Automotive Engineering, 2023, 45(2): 285-292.
[3]	Chunyang Qi,Chuanxue Song,Shixin Song,Liqiang Jin,Da Wang,Feng Xiao. Research on Energy Management Strategy for Hybrid Electric Vehicles Based on Inverse Reinforcement Learning [J]. Automotive Engineering, 2023, 45(10): 1954-1964.
[4]	Yong Chen,Changyin Wei,Xiaoyu Li,Yanlin Li,Caixia Liu,Xiaozhe Lin. Research on Fuzzy Energy Management Strategy for Extended-Range Electric Vehicles with Driving Condition Identification [J]. Automotive Engineering, 2022, 44(4): 514-524.
[5]	Zhichao Liu,Tianlei Zheng,Shaohui Liu,Huiming Gong. Research on Energy Consumption and Driving Range Evaluation Method of Plug-in Hybrid Electric Passenger Car [J]. Automotive Engineering, 2022, 44(2): 161-170.
[6]	Junqiu Li,Jiwei Liu,Chaofeng Zhu. Research on Energy Management of Hybrid Electric Vehicle Based on Time-Varying Chassis Configuration [J]. Automotive Engineering, 2022, 44(12): 1866-1876.
[7]	Dehua Shi,Xiangwei Rong,Shaohua Wang,Kaimei Zhang,Long Chen,Chun Li. Research on the Optimal Dynamic Coordinated Control of Power Split Hybrid Electric Vehicles with Dual-Clutch Collaboration [J]. Automotive Engineering, 2022, 44(12): 1877-1888.
[8]	Lijun Qian,Jian Chen,Bing Wu,Liang Xuan,Chen Chen,Liangliang Chen. Eco⁃driving Control for Hybrid Electric Vehicle Platoon with Consideration of Driver Operation Error [J]. Automotive Engineering, 2021, 43(7): 1037-1045.
[9]	Ningkang Yang,Lijin Han,Hui Liu,Xin Zhang. Research on Efficiency Optimization Based Energy Management Strategy for a Hybrid Electric Vehicle with Reinforcement Learning [J]. Automotive Engineering, 2021, 43(7): 1046-1056.
[10]	Shaobo Xie,Kangkang Zhang,Qiankun Zhang,Huiran Luo. Study on Energy Management Strategy for Parallel Plug⁃in Hybrid Electric Vehicles Considering Battery Electric⁃Thermal⁃Depth⁃of⁃Discharge [J]. Automotive Engineering, 2021, 43(6): 791-798.
[11]	Changhui Wang,Zemin Liu,Yanfei Li,Shijin Shuai,Huifang Shao,Yinhui Wang. An Experimental Study on Emissions and Operation Characteristics of a Range Extender Under Charge Sustaining Mode [J]. Automotive Engineering, 2021, 43(5): 667-674.
[12]	Jie Hu,Ming Wang,Di Liu,Fuwu Yan,Kai Cao. Optimization of Control Strategy for Hybrid Power System Based on Traffic Information [J]. Automotive Engineering, 2021, 43(5): 675-682.
[13]	Hao Zhang,Qinhao Fan,Wei Wang,Jin Huang,Zhi Wang. Reinforcement Learning Based Energy Management Strategy for Hybrid Electric Vehicles Using Multi⁃mode Combustion [J]. Automotive Engineering, 2021, 43(5): 683-691.
[14]	Long Chen,Zhixing Chen,Xing Xu,Feng Wang,Yingfeng Cai,Tao Zhang. Energy Management Strategy of Plug⁃in Hybrid Electric Vehicle with Engine Start and Stop Optimization [J]. Automotive Engineering, 2021, 43(3): 313-322.
[15]	Xiaohua Zeng,Hongxu Chen,Dafeng Song,Chen Cui,Zhanjiang Li,Yuanguang Jiang. Braking Control Strategy Based on Optimal Regenerative Braking Torque Curve of Motor [J]. Automotive Engineering, 2021, 43(2): 162-170.