机器人驾驶车辆深度强化学习换挡策略*

doi:10.19562/j.chinasae.qcgc.2020.11.004

摘要/Abstract

摘要： 为提高机器人驾驶车辆的动力性能,提出了一种基于深度神经网络强化学习的机器人驾驶车辆偏向动力性的换挡策略。首先建立了车辆纵向动力学模型、驾驶机器人运动学模型、用马尔可夫过程描述的机器人驾驶车辆换挡策略强化学习模型,并以车速、加速度和油门开度为状态变量、挡位为动作变量,设计了状态空间、动作空间和奖惩机制。然后通过深度神经网络强化学习算法求解了机器人驾驶车辆动力性换挡策略。最后对比本文中提出策略与其他换挡策略的仿真与试验结果,验证了提出策略的有效性。

关键词: 驾驶机器人, 换挡策略, 深度强化学习, 神经网络

Abstract: In order to improve the power performance of the robot-driven vehicle, a gearshifting strategy for robot-driven vehicles in favour of power performanceis proposed based on deep neural network reinforcement learning. Firstly the vehicle longitudinal dynamics model, the kinematics model of the robot driver and the reinforcement learning model of the gearshifting strategy for the robot-driven vehicles described with Markov process are constructed, and the state space, the action space and the reward and punishment mechanism are designed with the vehicle speed, acceleration and throttle opening as the state variables and the gear position as the action variable. Then the gearshifting strategy for the robot-driven vehicles in favour of power performance is solved by using deep neural network reinforcement learning algorithm. The comparison between the simulation and test results with the strategy proposed in this paper and those with other strategyie verifies the effectiveness of the proposed strategy

Key words: robot driver, gearshifting strategy, deep reinforcement learning, neural network

周楠, 陈刚. 机器人驾驶车辆深度强化学习换挡策略^*[J]. 汽车工程, 2020, 42(11): 1473-1481.

Zhou Nan, Chen Gang. Gearshifting Strategy for Robot-driven Vehicles Based on Deep Reinforcement Learning[J]. Automotive Engineering, 2020, 42(11): 1473-1481.

参考文献

[1] WONG N, CHAMBERS C, STOL K, et al. Development of a robotic driver for autonomous vehicle following[J]. International Journal of Intelligent Systems Technologies and Applications,2010,8(1-4):276-287.
[2] HIRATA N, MIZUTANI N, MATSUI H, et al. Fuel consumption in a driving test cycle by robotic driver considering system dynamics[C]. IEEE International Conference on Robotics and Automation (ICRA), Seattle, Washington, USA,2015:3374-3379.
[3] CHEN Gang, ZHANG Weigong. Hierarchical coordinated control method for unmanned robot applied to automotive test[J]. IEEE Transactions on Industrial Electronics,2016,63(2):1039-1051.
[4] CHEN Gang, ZHANG Weigong. Digital prototyping design of electromagnetic unmanned robot applied to automotive test[J]. Robotics and Computer-Integrated Manufacturing,2015,32:54-64.
[5] 张为公,陈晓冰.汽车驾驶机器人关键技术[J]. 江苏大学学报(自然科学版),2005,26(1):20-23.
[6] 陈刚,张为公,常思勤.汽车驾驶机器人模糊神经网络换挡控制方法[J]. 农业机械学报,2011,42(6):6-11.
[7] MIAO C, LIU H, ZHU G G. Three-parameter transmission gear-shifting schedule for improved fuel economy[J]. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering,2018,232(4):521-533.
[8] 陈清洪,秦大同.自动变速汽车神经网络三参数换挡策略[J]. 控制理论与应用,2010,27(11):1580-1584.
[9] ZHU T, WU Y, WEI Z, et al. A new approach of shift schedule optimization for AMT vehicle based on optimal theory[C]. Proceedings of the International Conference on Artificial Intelligence and Robotics and the International Conference on Automation, Control and Robotics Engineering. Kitakyushu, Japan,2016,10:1-5.
[10] 李浩,吴训成,张珏成.基于变载荷的最佳动力性AMT换挡规律修正[J]. 控制工程,2015,22(1):50-54.
[11] YIN X, LU H, ZHAO X, et al. Performance evaluation approach improvement for individualized gearshift schedule optimization[C]. SAE Paper 2016-01-1147.
[12] NGO V D, COLIN NAVARRETE J A, HOFMAN T, et al. Optimal gear shift strategies for fuel economy and drivability[J]. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering,2013,227(10):1398-1413.
[13] LEI Y, FU Y, LIU K, et al. Research on optimal gearshift strategy for stepped automatic transmission based on vehicle power demand[C]. SAE Paper 2017-01-1108.
[14] 段艳杰,吕宜生,张杰,等.深度学习在控制领域的研究现状与展望[J]. 自动化学报,2016,42(5):643-654.
[15] 薛金林,张为公,龚宗洋.基于强化学习神经网络的车速跟踪控制[J]. 测控技术,2007,26(7):36-38.
[16] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature,2015,518(7540):529-533.
[17] YANG Z, MERRICK K, JIN L, et al. Hierarchical deep reinforcement learning for continuous action control[J]. IEEE Transactions on Neural Networks and Learning Systems,2018,29(11):5174-5184.
[18] ZHANG T, KAHN G, LEVINE S, et al. Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search[C]. IEEE International Conference on Robotics and Automation (ICRA). Stockholm, Sweden,2016:528-535.
[19] 夏光,徐波涛,唐希雯,等. 基于质量与坡度辨识的汽车自动变速器换挡控制研究[J]. 汽车工程,2018,40(3):305-312.
[20] 陈刚,张为公, 龚宗洋,等.用于汽车驾驶机器人的车辆性能自学习方法[J]. 中国机械工程,2010,21(4):491-495.
[21] 国家环境保护总局. 轻型汽车污染物排放限值及测量方法(中国Ⅲ、Ⅳ阶段):GB 18352.3—2005[S]. 北京:中国标准出版社,2005.

机器人驾驶车辆深度强化学习换挡策略^*

Gearshifting Strategy for Robot-driven Vehicles Based on Deep Reinforcement Learning

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	王彦鑫,李海岩,崔世海,贺丽娟,吕文乐. 基于BP神经网络的儿童乘员头部损伤预测模型及评估参数研究[J]. 汽车工程, 2024, 46(2): 329-336.
[2]	胡明辉,朱广曜,刘长贺,唐国峰. 考虑迟滞特性的卡尔曼滤波和门控循环单元神经网络的锂离子电池SOC联合估计[J]. 汽车工程, 2023, 45(9): 1688-1701.
[3]	管欣,仲昭辉,詹军,奚腾龙,叶昊,高深圳,成健,廖世辉,蔡均. 基于卷积神经网络的汽车操纵稳定性试验类型识别方法[J]. 汽车工程, 2023, 45(9): 1765-1771.
[4]	王明,唐小林,杨凯,李国法,胡晓松. 考虑预测风险的自动驾驶车辆运动规划方法[J]. 汽车工程, 2023, 45(8): 1362-1372.
[5]	陈国强,申正义,孙利,支梦帆,李彤. 基于BP神经网络优化遗传算法的智能座舱感性意象预测[J]. 汽车工程, 2023, 45(8): 1479-1488.
[6]	张雷, 关可人, 丁晓林, 郭鹏宇, 王震坡, 孙逢春. 基于图像识别与动力学融合的路面附着系数估计方法[J]. 汽车工程, 2023, 45(7): 1222-1234.
[7]	韩勇,林旭洁,黄红武,蔡鸿瑜,罗金镕,李燕婷. 典型汽车碰撞事故场景中行人运动轨迹预测方法[J]. 汽车工程, 2023, 45(6): 1022-1030.
[8]	赵霞,李朝,付锐,葛振振,王畅. 基于深度卷积-Tokens降维优化视觉Transformer的分心驾驶行为实时检测[J]. 汽车工程, 2023, 45(6): 974-988.
[9]	何智成,杜磊浩,周恩临,覃高峰,黄晋. 基于改进连续型Hopfield神经网络的CAN总线负载率优化[J]. 汽车工程, 2023, 45(12): 2338-2347.
[10]	李捷,吴晓东,许敏,刘永刚. 基于强化学习的城市场景多目标生态驾驶策略[J]. 汽车工程, 2023, 45(10): 1791-1802.
[11]	梁旺,秦兆博,陈亮,边有钢,胡满江. 基于改进BP神经网络的智能车纵向控制方法[J]. 汽车工程, 2022, 44(8): 1162-1172.
[12]	毕贵红,谢旭,蔡子龙,骆钊,陈臣鹏,赵鑫. 动态条件下基于深度学习的锂电池容量估计[J]. 汽车工程, 2022, 44(6): 868-878.
[13]	张哲雨,吕超,李景行,熊光明,吴绍斌,龚建伟. 基于车辆视角数据的行人轨迹预测与风险等级评定[J]. 汽车工程, 2022, 44(5): 675-683.
[14]	石琴,刘鑫,应贺烈,王铭伟,贺泽佳,贺林. 电液线控制动系统压力反步控制算法研究[J]. 汽车工程, 2022, 44(5): 747-755.
[15]	陈勇,魏长银,李晓宇,李彦林,刘彩霞,林霄喆. 融合工况识别的增程式电动汽车模糊能量管理策略研究[J]. 汽车工程, 2022, 44(4): 514-524.