Comparative Study of Q-Learning and SARSA Algorithms for UAV Path Planning in 3D Environments

Abstract

Path planning processes in 3D environments with obstacles are essential for ensuring the successful navigation of Unmanned Aerial Vehicles (UAVs). Recently, the artificial intelligence theory presents increasingly effective algorithms and tools to address such a robotics challenge. This paper presents a comparative study of two most commonly used Reinforcement Learning (RL) algorithms, namely Q-learning and SARSA learning. Through numerical experimentations under different navigation scenarios with varying levels of complexity and increasing numbers of static obstacles, key performance metrics like path’s length, total actions number, average reward, execution time and collision avoidance are considered and evaluated. The demonstrative results indicate that the Q-learning algorithm well addressed the challenges of UAVs’ path planning in static environments. The performance of such an algorithm has proven to be satisfactory and more promising for UAVs’ navigation in complex configuration spaces when compared to the SARSA algorithm.