RLReC: Towards Reinforcement Learning-based Dynamic Design of Reconfiguration Control

Abstract

The design of Reconfigurable Control Systems (RCS) is a tremendous task since that their increasing need for reconfiguration imposes the anticipation of all potential runtime changes at design time, which is in most cases infeasible. To deal with the high dynamicity of RCS at design-time we propose a novel design approach, RLReC (Reinforcement Learning-based Reconfiguration Control) allowing for the improvement of reconfiguration knowledge through online (runtime) exploration. This approach combines the benefits of meta-modelling with Reinforcement Learning (RL) principals for reconfiguration knowledge modelling in order to handle early and effective exploration of various design choices. In this work, the reconfiguration controller is an autonomous software agent designed as a RL agent (RLRA) that handles both offline (exploitation) and online (exploration) learning through interactions with the controlled system at runtime. In particular, the RLRA has to select reconfiguration rules that were not selected before, which is known as exploration. The main aim of this proposal is to cope with design-time unplanned changes by giving the RLRA the ability of improving its knowledge through online learning feedback. We illustrate the use of our contributions with a running example from the manufacturing domain.