Reinforcement Learning Based End-to-End Autonomous Driving Decision-Making Method by Combining Image and Monocular Depth Features

  Objectives  Existing deep reinforcement learning (DRL)based end-to-end autonomous driving decision-making method is low robustness to noise, which would lead to safety problem. It is difficult to infer the optimal decision accurately by relying solely on the image features when facing with the complex scenes.

  Methods  An end-to-end decision-making model based on dueling deep Q-network(Dueling DQN) is established to improve the ability of decision evaluation and improve the robustness of the model. It obtains the current state according to the observed data, and outputs discrete quantities for controlling the vehicle (including throttle, steering and brake). The monocular depth feature is extracted accurately in a self-supervised learning manner, and which is combined with the image features for better representation of the current state.

  Results  The proposed method is tested in a simulation environment. (1) The comparison results with the state-of-the-art A3C model show that our Dueling DQN-based model is more robustness. (2) The comparison results with the image feature-based model show that combining the image and depth features is more beneficial to improve the decision-making accuracy.

  Conclusions  Training an agent with Dueling DQN is beneficial to alleviate the security risks caused by making different decision when facing similar scenes. Training an agent together with image features and depth features is beneficial to enhance the agent̓s ability of environment perception, and improve the decision-making accuracy.