Keywords: 
• spatial cognition /
• optimal route planning /
• hierarchical reinforcement learning /
• pre-learning /
• real-time learning

Abstract: Against the contradictions between model-driven route planning and the diversity of human cognitive preferences for spatial cognition oriented optimal routes,we present a kind of interactive route planning approach based on hierarchical reinforcement learning.In this approach,optimal route criterias are translated into immediate rewards of turning decisions at intersections,and optimal route policies with maximal cumulative rewards can be found through a two-stage learning process.The first pre-learning stage automatically identifies some nodes in road network as subgoals and constructs corresponding subtasks containing local optimal route policies for achieving the subgoals.The second real-time learning stage focuses on efficiently updating the Q values of every available state-action pair using predefined policies,and tracing the optimal routes according to Q values.The experimental results show that our proposed approach learns effectively enough and ensures the routes found close to global optimal ones.