A Q-Learning Approach to Model-Free Infinite Horizon Control For Linear Time Delay Systems

Abstract

In this paper, an online Q-learning algorithm is proposed to address the infinite-horizon guaranteed cost control problem for linear time delay systems with completely unknown dynamics. The developed approach leverages a Lyapunov-Krasovskii functional as the state value function and integrates guaranteed cost control principles. Specifically, based on Bessel Legendre integral inequality, a Q-function tailored for handling guaranteed cost control in time delay systems is formulated. Furthermore, an integral reinforcement learning method based on an actor/critic approximator framework is used to dynamically estimate the Q-function parameters. Finally, the proposed approach is successfully applied to an interconnected power system.