Abstract:

The development of end-to-end autonomous driving algorithms has become a hot topic in current autonomous driving technology research and development. Classic reinforcement learning algorithms leverage information such as vehicle state and environmental feedback to train the vehicle for driving， through trial-and-error learning to obtain the best strategy， so as to achieve the development of end-to-end autonomous driving algorithms. However， there is still the problem of low development efficiency. The article proposes an asynchronous distributed reinforcement learning framework to address the inefficiency and high complexity problems in training RL algorithms in virtual simulation environment， establishes intra and inter process multi-agent parallel Soft Actor-Critic （SAC） distributed training framework on the Carla simulator to accelerate online RL training. Additionally， to achieve rapid model training and deployment， the article proposes a distributed model training and deployment system architecture based on Cloud-OTA， which mainly consists of an Over-the-Air Technology （OTA） platform， a cloud-based distributed training platform， and an on-vehicle computing platform. On this basis， the paper establishes an Autoware-Carla integrated validation framework based on ROS to improve model reusability and reduce migration and deployment cost. The experimental results show that compared with various mainstream autonomous driving methods， the method proposed in this paper has a faster training speed qualitatively， which can effectively cope with dense traffic flow and improve the adaptability of end-to-end autonomous driving strategies to unknown scenes， and reduce the time and resources required for experimentation in actual environment.

Key words: reinforcement learning, distributed system, multi-agent, autonomous driving, Carla, vehicle control