Physics-informed deep reinforcement learning for robust self-adaptive control of next-generation grid-connected multilevel inverters

Abstract

This paper proposes a physics-informed deep reinforcement learning (PI-DRL) framework for robust and self-adaptive control of a grid-connected five-level neutral-point-clamped (NPC) multilevel inverter. The proposed approach combines a TD3-based actor-critic architecture with power-electronic knowledge incorporated through the state representation, reward function, and an explicit safety mechanism. The controller simultaneously considers grid-current tracking, total harmonic distortion (THD), dc-link voltage regulation, capacitor-voltage balancing, switching effort, and safe operation under disturbed grid conditions. A comprehensive MATLAB/Simulink study compares the proposed PI-DRL controller with conventional PI, model predictive control (MPC), sliding-mode control (SMC), and standard DRL. The evaluation considers nominal operation as well as grid-voltage sag, grid-frequency deviation, filter-inductance variation, unbalanced grid-voltage conditions, and nonlinear-load operation. Under nominal conditions, PI-DRL achieves a THD of 2.1%, a settling time of 7 ms, an overshoot of 3%, a steady-state current error of 0.8%, and a dc-link voltage ripple of 6 V. Under disturbed conditions, the proposed controller maintains the lowest THD and power-tracking error among the investigated strategies, reaching THD values of 3.01% under the unbalanced grid condition and 3.47% under nonlinear loading. A reward-term ablation study further indicates that the capacitor-balance, harmonic, and safety components contribute to voltage regulation, waveform quality, and safe control behavior, respectively. Overall, the simulation results indicate that embedding physical objectives into the DRL framework improves robustness and generalization across the tested operating conditions.